Document exec restore hooks (#2896)

Signed-off-by: Andrew Reed <andrew@replicated.com>

site/content/docs/main/api-types/restore.md

@@ -73,8 +73,9 @@ spec:
   # to restore from. If specified, and BackupName is empty, Velero will
   # restore from the most recent successful backup created from this schedule.
   scheduleName: my-scheduled-backup-name
-  # Actions to perform during or post restore. The only hook currently supported is
-  # adding an init container to a pod before it can be restored. Optional.
+  # Actions to perform during or post restore. The only hooks currently supported are
+  # adding an init container to a pod before it can be restored and executing a command in a
+  # restored pod's container. Optional.
   hooks:
     # Array of hooks that are applicable to specific resources. Optional.
     resources:
@@ -98,9 +99,10 @@ spec:
         matchLabels:
           app: velero
           component: server
-      # An array of hooks to run during or after restores. Currently only "init" hooks are supported.
+      # An array of hooks to run during or after restores. Currently only "init" and "exec" hooks
+      # are supported.
       postHooks:
-      # The type of the hook. This must be "init".
+      # The type of the hook. This must be "init" or "exec".
       - init:
           # An array of container specs to be added as init containers to pods to which this hook applies to.
           initContainers:
@@ -126,6 +128,27 @@ spec:
             - /bin/ash
             - -c
             - echo -n "DEADFEED" >> /restores/pvc2-vm/deadfeed
+      - exec:
+          # The container name where the hook will be executed. Defaults to the first container.
+          # Optional.
+          container: foo
+          # The command that will be executed in the container. Required.
+          command:
+          - /bin/bash
+          - -c
+          - "psql < /backup/backup.sql"
+          # How long to wait for a container to become ready. This should be long enough for the
+          # container to start plus any preceding hooks in the same container to complete. The wait
+          # timeout begins when the container is restored and may require time for the image to pull
+          # and volumes to mount. If not set the restore will wait indefinitely. Optional.
+          waitTimeout: 5m
+          # How long to wait once execution begins. Defaults to 30 seconds. Optional.
+          execTimeout: 1m
+          # How to handle execution failures. Valid values are `Fail` and `Continue`. Defaults to
+          # `Continue`. With `Continue` mode, execution failures are logged only. With `Fail` mode,
+          # no more restore hooks will be executed in any container in any pod and the status of the
+          # Restore will be `PartiallyFailed`. Optional.
+          onError: Continue
 # RestoreStatus captures the current status of a Velero restore. Users should not set any data here.
 status:
   # The current phase. Valid values are New, FailedValidation, InProgress, Completed, PartiallyFailed, Failed.

site/content/docs/main/restore-hooks.md

@@ -154,4 +154,106 @@ and
 }
 ```
 
+## Exec Restore Hooks
+
+Use an Exec Restore hook to execute commands in a restored pod's containers after they start.
+
+There are two ways to specify `Exec` restore hooks:
+1. Specifying exec restore hooks in annotations
+1. Specifying exec restore hooks in the restore spec
+
+If a pod has the annotation `post.hook.restore.velero.io/command` then that is the only hook that will be executed in the pod.
+No hooks from the restore spec will be executed in that pod.
+
+### Specifying Exec Restore Hooks As Pod Annotations
+
+Below are the annotations that can be added to a pod to specify exec restore hooks:
+* `post.hook.restore.velero.io/container`
+    * The container name where the hook will be executed. Defaults to the first container. Optional.
+* `post.hook.restore.velero.io/command`
+    * The command that will be executed in the container. Required.
+* `post.hook.restore.velero.io/on-error`
+    * How to handle execution failures. Valid values are `Fail` and `Continue`. Defaults to `Continue`. With `Continue` mode, execution failures are logged only. With `Fail` mode, no more restore hooks will be executed in any container in any pod and the status of the Restore will be `PartiallyFailed`. Optional.
+* `post.hook.restore.velero.io/exec-timeout`
+    * How long to wait once execution begins. Defaults to 30 seconds. Optional.
+* `post.hook.restore.velero.io/wait-timeout`
+    * How long to wait for a container to become ready. This should be long enough for the container to start plus any preceding hooks in the same container to complete. The wait timeout begins when the container is restored and may require time for the image to pull and volumes to mount. If not set the restore will wait indefinitely. Optional.
+
+#### Example
+
+Use the below commands to add annotations to the pods before taking a backup.
+
+```bash
+$ kubectl annotate pod -n <POD_NAMESPACE> <POD_NAME> \
+    post.hook.restore.velero.io/container=postgres \
+    post.hook.restore.velero.io/command='["/bin/bash", "-c", "psql < /backup/backup.sql"]' \
+    post.hook.restore.velero.io/wait-timeout=5m \
+    post.hook.restore.velero.io/exec-timeout=45s \
+    post.hook.restore.velero.io/on-error=Continue
+```
+
+### Specifying Exec Restore Hooks in Restore Spec
+
+Exec restore hooks can also be specified using the `RestoreSpec`.
+Please refer to the documentation on the [Restore API Type][1] for how to specify hooks in the Restore spec.
+
+#### Multiple Exec Restore Hooks Example
+
+Below is an example of specifying restore hooks in  a `RestoreSpec`.
+When using the restore spec it is possible to specify multiple hooks for a single pod, as this example demonstrates.
+
+All hooks applicable to a single container will be executed sequentially in that container once it starts.
+The ordering of hooks executed in a single container follows the order of the restore spec.
+In this example, the `pg_isready` hook is guaranteed to run before the `psql` hook because they both apply to the same container and the `pg_isready` hook is defined first.
+
+If a pod has multiple containers with applicable hooks, all hooks for a single container will be executed before executing hooks in another container.
+In this example, if the postgres container starts before the sidecar container, both postgres hooks will run before the hook in the sidecar.
+This means the sidecar container may be running for several minutes before its hook is executed.
+
+Velero guarantees that no two hooks for a single pod are executed in parallel, but hooks executing in different pods may run in parallel.
+
+
+```yaml
+apiVersion: velero.io/v1
+kind: Restore
+metadata:
+  name: r2
+  namespace: velero
+spec:
+  backupName: b2
+  excludedResources:
+  ...
+  includedNamespaces:
+  - '*'
+  hooks:
+    resources:
+    - name: restore-hook-1
+      includedNamespaces:
+      - app
+      postHooks:
+      - exec:
+          execTimeout: 1m
+          waitTimeout: 5m
+          onError: Fail
+          container: postgres
+          command:
+          - /bin/bash
+          - '-c'
+          - 'while ! pg_isready; do sleep 1; done'
+      - exec:
+          container: postgres
+          waitTimeout: 6m
+          execTimeout: 1m
+          command:
+          - /bin/bash
+          - '-c'
+          - 'psql < /backup/backup.sql'
+      - exec:
+          container: sidecar
+          command:
+          - /bin/bash
+          - '-c'
+          - 'date > /start'
+```
+
 [1]: api-types/restore.md

site/content/docs/v1.5.0-rc.1/api-types/restore.md

@@ -73,8 +73,9 @@ spec:
   # to restore from. If specified, and BackupName is empty, Velero will
   # restore from the most recent successful backup created from this schedule.
   scheduleName: my-scheduled-backup-name
-  # Actions to perform during or post restore. The only hook currently supported is
-  # adding an init container to a pod before it can be restored. Optional.
+  # Actions to perform during or post restore. The only hooks currently supported are
+  # adding an init container to a pod before it can be restored and executing a command in a
+  # restored pod's container. Optional.
   hooks:
     # Array of hooks that are applicable to specific resources. Optional.
     resources:
@@ -98,9 +99,10 @@ spec:
         matchLabels:
           app: velero
           component: server
-      # An array of hooks to run during or after restores. Currently only "init" hooks are supported.
+      # An array of hooks to run during or after restores. Currently only "init" and "exec" hooks
+      # are supported.
       postHooks:
-      # The type of the hook. This must be "init".
+      # The type of the hook. This must be "init" or "exec".
       - init:
           # An array of container specs to be added as init containers to pods to which this hook applies to.
           initContainers:
@@ -126,6 +128,27 @@ spec:
             - /bin/ash
             - -c
             - echo -n "DEADFEED" >> /restores/pvc2-vm/deadfeed
+      - exec:
+          # The container name where the hook will be executed. Defaults to the first container.
+          # Optional.
+          container: foo
+          # The command that will be executed in the container. Required.
+          command:
+          - /bin/bash
+          - -c
+          - "psql < /backup/backup.sql"
+          # How long to wait for a container to become ready. This should be long enough for the
+          # container to start plus any preceding hooks in the same container to complete. The wait
+          # timeout begins when the container is restored and may require time for the image to pull
+          # and volumes to mount. If not set the restore will wait indefinitely. Optional.
+          waitTimeout: 5m
+          # How long to wait once execution begins. Defaults to 30 seconds. Optional.
+          execTimeout: 1m
+          # How to handle execution failures. Valid values are `Fail` and `Continue`. Defaults to
+          # `Continue`. With `Continue` mode, execution failures are logged only. With `Fail` mode,
+          # no more restore hooks will be executed in any container in any pod and the status of the
+          # Restore will be `PartiallyFailed`. Optional.
+          onError: Continue
 # RestoreStatus captures the current status of a Velero restore. Users should not set any data here.
 status:
   # The current phase. Valid values are New, FailedValidation, InProgress, Completed, PartiallyFailed, Failed.

site/content/docs/v1.5.0-rc.1/restore-hooks.md

@@ -154,4 +154,106 @@ and
 }
 ```
 
+## Exec Restore Hooks
+
+Use an Exec Restore hook to execute commands in a restored pod's containers after they start.
+
+There are two ways to specify `Exec` restore hooks:
+1. Specifying exec restore hooks in annotations
+1. Specifying exec restore hooks in the restore spec
+
+If a pod has the annotation `post.hook.restore.velero.io/command` then that is the only hook that will be executed in the pod.
+No hooks from the restore spec will be executed in that pod.
+
+### Specifying Exec Restore Hooks As Pod Annotations
+
+Below are the annotations that can be added to a pod to specify exec restore hooks:
+* `post.hook.restore.velero.io/container`
+    * The container name where the hook will be executed. Defaults to the first container. Optional.
+* `post.hook.restore.velero.io/command`
+    * The command that will be executed in the container. Required.
+* `post.hook.restore.velero.io/on-error`
+    * How to handle execution failures. Valid values are `Fail` and `Continue`. Defaults to `Continue`. With `Continue` mode, execution failures are logged only. With `Fail` mode, no more restore hooks will be executed in any container in any pod and the status of the Restore will be `PartiallyFailed`. Optional.
+* `post.hook.restore.velero.io/exec-timeout`
+    * How long to wait once execution begins. Defaults to 30 seconds. Optional.
+* `post.hook.restore.velero.io/wait-timeout`
+    * How long to wait for a container to become ready. This should be long enough for the container to start plus any preceding hooks in the same container to complete. The wait timeout begins when the container is restored and may require time for the image to pull and volumes to mount. If not set the restore will wait indefinitely. Optional.
+
+#### Example
+
+Use the below commands to add annotations to the pods before taking a backup.
+
+```bash
+$ kubectl annotate pod -n <POD_NAMESPACE> <POD_NAME> \
+    post.hook.restore.velero.io/container=postgres \
+    post.hook.restore.velero.io/command='["/bin/bash", "-c", "psql < /backup/backup.sql"]' \
+    post.hook.restore.velero.io/wait-timeout=5m \
+    post.hook.restore.velero.io/exec-timeout=45s \
+    post.hook.restore.velero.io/on-error=Continue
+```
+
+### Specifying Exec Restore Hooks in Restore Spec
+
+Exec restore hooks can also be specified using the `RestoreSpec`.
+Please refer to the documentation on the [Restore API Type][1] for how to specify hooks in the Restore spec.
+
+#### Multiple Exec Restore Hooks Example
+
+Below is an example of specifying restore hooks in  a `RestoreSpec`.
+When using the restore spec it is possible to specify multiple hooks for a single pod, as this example demonstrates.
+
+All hooks applicable to a single container will be executed sequentially in that container once it starts.
+The ordering of hooks executed in a single container follows the order of the restore spec.
+In this example, the `pg_isready` hook is guaranteed to run before the `psql` hook because they both apply to the same container and the `pg_isready` hook is defined first.
+
+If a pod has multiple containers with applicable hooks, all hooks for a single container will be executed before executing hooks in another container.
+In this example, if the postgres container starts before the sidecar container, both postgres hooks will run before the hook in the sidecar.
+This means the sidecar container may be running for several minutes before its hook is executed.
+
+Velero guarantees that no two hooks for a single pod are executed in parallel, but hooks executing in different pods may run in parallel.
+
+
+```yaml
+apiVersion: velero.io/v1
+kind: Restore
+metadata:
+  name: r2
+  namespace: velero
+spec:
+  backupName: b2
+  excludedResources:
+  ...
+  includedNamespaces:
+  - '*'
+  hooks:
+    resources:
+    - name: restore-hook-1
+      includedNamespaces:
+      - app
+      postHooks:
+      - exec:
+          execTimeout: 1m
+          waitTimeout: 5m
+          onError: Fail
+          container: postgres
+          command:
+          - /bin/bash
+          - '-c'
+          - 'while ! pg_isready; do sleep 1; done'
+      - exec:
+          container: postgres
+          waitTimeout: 6m
+          execTimeout: 1m
+          command:
+          - /bin/bash
+          - '-c'
+          - 'psql < /backup/backup.sql'
+      - exec:
+          container: sidecar
+          command:
+          - /bin/bash
+          - '-c'
+          - 'date > /start'
+```
+
 [1]: api-types/restore.md