Trilio Site Recovery for Kubernetes
Guide

Custom Resource Reference

Complete API reference for all site recovery custom resources


Overview

This page is the complete API reference for all Trilio Site Recovery custom resources in the siterecovery.trilio.io/v1alpha1 API group. Every DR operation—protecting a VM, triggering a failover, running a test drill, or monitoring replication health—is declared through one of these custom resources and can be driven by kubectl, GitOps tooling, or the TSR web console. Understanding which resource lives on which cluster, what each field does, and which phase values to expect is essential for operating Site Recovery reliably.


Prerequisites

Before working with Site Recovery custom resources you need:

  • OpenShift 4.17 or later on the TSR-Manager cluster and both workload clusters (OpenShift 4.14–4.16 are within the supported floor but not validated for Tech Preview)
  • KubeVirt installed on both workload clusters (the kubevirt.io/v1 API must be served)
  • DRBD 9 kernel module (drbd, drbd_transport_tcp, drbd_transport_lb-tcp) loadable on every workload node that will host a protected VM
  • Three Kubernetes clusters: one TSR-Manager cluster and two workload clusters
  • TCP ports 7000–8000 inclusive open between primary and DR worker nodes for DRBD replication (one port per protected VM; port 8000 is mandatory)
  • kubectl configured to reach the TSR-Manager cluster
  • tsrctl supplied by Trilio with the release (contact Trilio support to obtain it)
  • oc (OpenShift CLI) for oc adm must-gather support bundle collection
  • OLM (Operator Lifecycle Manager) installed on the TSR-Manager cluster
  • Site Recovery operator installed via OLM (CatalogSource + OperatorGroup + Subscription for package site-recovery, channel tech-preview, catalog quay.io/triliovault/site-recovery-catalog) in trilio-site-recovery-system
  • A ProtectionZone CR in a dr-<name> namespace on the TSR-Manager cluster, which automatically deploys workload control planes and DRBD node agents to both workload clusters

Cluster placement: All TSR-Manager-side CRs (ProtectionZone, ProtectionGroup, ProtectionRequest, DRBDReplicationPolicy, FailoverRequest, TestFailover, Hook, ReplicationGroupStatus, RPOEvent) live in the dr-<zone-deployment-name> namespace on the TSR-Manager cluster. DRBDResource CRs live in the protected VM's namespace on each workload cluster. Running kubectl get drbdresource against the TSR-Manager cluster returns server doesn't have a resource type.


Installation

Site Recovery custom resources are installed automatically when you install the operator and create a ProtectionZone. You do not run helm install directly—the Helm charts are embedded in the operator binary and applied by the ProtectionZone controller.

Step 1 — Install the operator via OLM

Create the following objects on the TSR-Manager cluster in order:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: site-recovery-catalog
  namespace: trilio-site-recovery-system
spec:
  sourceType: grpc
  image: quay.io/triliodata/site-recovery-catalog:<release-tag>
  displayName: Trilio Site Recovery
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: site-recovery-og
  namespace: trilio-site-recovery-system
spec: {}
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: site-recovery
  namespace: trilio-site-recovery-system
spec:
  channel: stable
  name: site-recovery
  source: site-recovery-catalog
  sourceNamespace: trilio-site-recovery-system

Step 2 — Provision zone credentials

Use tsrctl (supplied by Trilio) to create a per-zone ServiceAccount and kubeconfig Secret on each workload cluster. Run this on a host that can reach both workload clusters:

# Create credentials on each workload cluster
tsrctl zone-credential create \
  --namespace trilio-site-recovery-access \
  --zone <zone-name> \
  --access-namespace trilio-site-recovery-access

This creates a ServiceAccount in trilio-site-recovery-access on each workload cluster and produces kubeconfig Secrets that the TSR-Manager references.

Step 3 — Create the zone namespace and kubeconfig Secrets on TSR-Manager

kubectl create namespace dr-<zone-name>

# Copy or create the kubeconfig Secrets for each workload cluster
kubectl create secret generic cluster1-kubeconfig \
  --from-file=kubeconfig=./cluster1.kubeconfig \
  -n dr-<zone-name>

kubectl create secret generic cluster2-kubeconfig \
  --from-file=kubeconfig=./cluster2.kubeconfig \
  -n dr-<zone-name>

Step 4 — Create a ProtectionZone CR

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: primary-dr-zone
  namespace: dr-primary-dr-zone
spec:
  displayName: "Primary DR Zone"
  description: "Two-cluster DRBD protection zone."
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

Apply it and wait for the zone to reach Available:

kubectl apply -f protectionzone.yaml
kubectl get pz -n dr-primary-dr-zone --watch

When the zone is Available, the ClustersValidated condition is True and the TSR-Manager webhooks will accept ProtectionGroup and ProtectionRequest creates. The ProtectionZone controller also automatically creates a DRBDReplicationPolicy named <zoneName>-default with isDefault: true and Protocol C—you do not create this yourself.


Configuration

The following describes the key fields for each custom resource. All resources use apiVersion: siterecovery.trilio.io/v1alpha1. Only spec fields that require explanation beyond their names are described in detail; refer to the field tables in the Usage section for complete field lists.

ProtectionZone

The top-level configuration object. Each zone occupies its own namespace (dr-<name>) on the TSR-Manager cluster, and each workload cluster may belong to only one zone at a time.

FieldDefaultNotes
spec.storageBackendMode— (required)Must be DRBD. The schema admits Linstor but the admission webhook rejects it: Linstor backend is not supported yet; use DRBD.
spec.clusters[].clusterID— (required)Must be cluster1 or cluster2. The two entries must use different slot values.
spec.clusters[].distributionNameRHEL-CoreOS-9Selects the DRBD kernel-module loader image. RHEL-CoreOS-10 is accepted by the schema but not validated in Tech Preview.
spec.clusters[].kubeconfigSecretRef.name— (required)Name of the Secret in the zone namespace containing the kubeconfig. Both name and key must be non-empty; the webhook rejects empty values even though the schema permits them.
spec.logging.level— (required when logging is set)One of error, info, debug, trace. Controls runtime log level of all zone components via the site-recovery-logging ConfigMap; takes effect within ~15 s without pod restarts.
spec.logging.controlPlaneLevelinherits levelOverride for TSR-Manager and workload control-plane pods only.
spec.logging.nodeAgentLevelinherits levelOverride for DRBD node agent pods only.
spec.nodeAgent.startupDeadlineSeconds120How long the node agent keeps the siterecovery.trilio.io/not-ready:NoSchedule taint on its node while DRBD resources converge. Range: 10–3600. Changing this value rolls the agent DaemonSet.
spec.hooks.jobTTLSeconds3600How long a succeeded hook Job is retained (seconds).
spec.hooks.failedJobRetentionSeconds604800How long a failed or timed-out hook Job is retained (7 days).

Immutable fields: spec.storageBackendMode and each cluster's name and clusterID are immutable once set. kubeconfigSecretRef and distributionName are mutable to support credential rotation and OpenShift upgrades.

ProtectionGroup

Groups one or more KubeVirt VMs for coordinated failover. Lives on the TSR-Manager cluster in the zone namespace.

FieldDefaultNotes
spec.virtualMachines[].name— (required)Name of the KubeVirt VM on the workload cluster.
spec.virtualMachines[].namespaceCR namespaceNamespace of the VM on the workload cluster.
spec.virtualMachines[].bootOrder—Lower values boot first during failover. VMs sharing a value boot in parallel. VMs without a value boot together in a final batch.
spec.desiredStaterunningWhether VMs in this group should be running or stopped on the active cluster.
spec.sla.rpoTargetSeconds0Target RPO in seconds. Feeds the replication monitor objective (ReplicationGroupStatus.spec.rpoObjectiveSeconds). 0 is the Protocol C synchronous guarantee.
spec.sla.rtoTargetSeconds— (required)Target RTO in seconds. Recorded for compliance reporting.
spec.hooks—List of hook bindings. Each entry references a Hook CR by hookName (same namespace).
spec.hooks[].name— (required)Binding identifier, embedded in Job names.
spec.hooks[].hookName— (required)Name of the referenced Hook CR in the same namespace.
spec.hooks[].type— (required)Lifecycle point: pre-failover, post-failover, pre-failback, post-failback, pre-test, post-test, test-verification.
spec.hooks[].runOntargetsource or target. source is only admitted for pre-failover and pre-failback bindings.
spec.hooks[].onFailureAbortAbort fails the operation; Continue records a warning and proceeds.
spec.hooks[].order0Lower runs first among hooks of the same type.
spec.vmIdentity.macAddressesPreservePreserve copies interface MACs verbatim to the DR copy; Clear lets KubeMacPool assign fresh MACs.

ProtectionRequest

Declares protection for a single KubeVirt VM. The spec.sourceCluster field is the failover lever—patching it to the DR cluster's slot triggers a role flip.

FieldDefaultNotes
spec.sourceCluster— (required)cluster1 or cluster2. The cluster currently running the VM. Patching this field triggers a planned failover for this VM.
spec.virtualMachine.name— (required)Name of the KubeVirt VM.
spec.virtualMachine.namespace— (required)Namespace of the VM on the workload cluster. Must be set explicitly; cannot be inferred from the PR's namespace.
spec.replicationConfig.replicationProtocolCOnly C (synchronous, RPO=0) is supported. A is rejected by the admission webhook: ReplicationProtocolA is not supported yet.
spec.replicationConfig.rpo0Must be 0 when using Protocol C. A non-zero value is rejected: rpo must be 0 when spec.replicationConfig.replicationProtocol is C.
spec.replicationConfig.drbdMetadataConfig.size128MiSize of the DRBD external-metadata PVC created per protected volume. Minimum 128Mi.
spec.replicationConfig.drbdMetadataConfig.storageClassNamecluster defaultStorageClass for metadata PVCs. Must bind immediately (WaitForFirstConsumer classes are rejected).
spec.standbyPlacement—Optional per-key edits to the DR VM's node-level scheduling rules (nodeSelector, nodeAffinity, tolerations). Immutable once set.

Phase values (in order): Pending → Validating → CreatingDRBD → Syncing → ReadyToActivate → Activating → Protected. On teardown: Unprotecting. On error: Failed.

FailoverRequest

Triggers a planned or unplanned failover of an entire ProtectionGroup. The controller patches each member ProtectionRequest's spec.sourceCluster and waits for the DRBD Primary role to confirm on the target side.

FieldDefaultNotes
spec.protectionGroupRef.name— (required)Name of the ProtectionGroup to fail over.
spec.targetCluster— (required)cluster1 or cluster2. The cluster to fail over TO.
spec.failoverTypeplannedplanned (graceful, waits for sync) or unplanned (force-promote, RTO-optimized).
spec.batchBootTimeoutSeconds300Max seconds to wait for one boot-order batch's VMs to reach Running.
spec.drainTimeoutSeconds300Accepted and range-checked (30–3600) but not consumed by the failover controller in Tech Preview. Has no effect on drain or failover behaviour.
spec.skipHooksfalseWhen true, all hooks are recorded as Skipped rather than run. Useful for incident bypass. Immutable.

State values: Pending, InProgress, Completed, Failed, Cancelled.

Phase values: Initializing → Validating → RunningPreHooks → StoppingSource → WaitingForLocks → RemovingTaints → StartingTarget → Monitoring → RunningPostHooks → Completed. On error: Failed.

Note: status.vmStatuses holds per-VM detail. There is no status.message field and no status.failedVMs field on FailoverRequest.

TestFailover

Triggers a non-disruptive DR validation. The controller creates an isolated test namespace (site-recovery-<protection-group>-<random>, reported in status.testNamespace) on the standby cluster, spins up VM copies from DRBD snapshots inside a network-fenced namespace, runs verification, and reports results without touching production VMs.

FieldDefaultNotes
spec.protectionGroupRef.name— (required)Name of the ProtectionGroup to test.
spec.cleanupPolicyAutomaticAutomatic cleans up test resources after retentionTime. Manual keeps resources until cleanupRequested: true or the CR is deleted.
spec.retentionTime2hHow long test resources are kept before automatic cleanup. Pattern: ^[0-9]+(s|m|h|d)$.
spec.cleanupRequestedfalseSet to true to trigger cleanup while keeping the TestFailover CR as a record. Can be set but never unset.
spec.schedulingCheckFailFail excludes VMs the DR scheduler cannot place and marks the test Failed. Warn records the verdict and drills everything.
spec.timeout—Optional wall-clock limit (e.g. 2h). No default is applied by design; omit for large protection groups.
spec.batchBootTimeoutSeconds300Max seconds to wait for one boot-order batch's test VMs to reach Running.
spec.skipHooksfalseWhen true, all hooks (including test-verification hooks) are recorded as Skipped.
spec.verification.reachability—Optional list of network probes to run after automatic boot/agent/filesystem checks.
spec.verification.reachability[].type— (required)tcp or http.
spec.verification.reachability[].port— (required)TCP or HTTP port to probe.
spec.verification.reachability[].vmName—Restricts probe to a named VM. Empty means all test VMs.
spec.verification.reachability[].endpoint/HTTP path. Valid only for type: http.
spec.verification.reachability[].expectedStatus200Expected HTTP status code. Valid only for type: http.
spec.verification.reachability[].timeout30sProbe timeout. Maximum 10m.

Phase values: Queued → PreparingEnvironment → CreatingSnapshots → CreatingVolumes → CreatingVMs → StartingVMs → Verifying → Succeeded. On error: Failed. Cleanup: CleaningUp → Cleaned.

Note: status.testNamespace is controller-generated and never user-provided. Do not construct, guess, or grep for it.

DRBDResource

A workload-cluster-side CR (one per VM per cluster) representing all of a VM's replicated disks as a single DRBD resource. Named drbd-<namespace>-<vm>-<hash8>. Created and managed by the TSR-Manager-side ProtectionRequest controller; you inspect it to check connection state, phase, and node placement.

FieldDefaultNotes
spec.side— (required)Source or Target. Immutable; does not change on failover.
spec.role— (required)Primary or Secondary. Set by the ProtectionRequest controller; changes on role flip.
spec.port— (required)TCP port for DRBD replication. Range: 7000–8000.
spec.nodeName—Node this copy is pinned to. Immutable once set.
spec.pausedfalseSuspends convergence (maintenance knob).
spec.replicationIntentEnabledSuspended renders the resource with no connection mesh (used during unplanned failover recovery).
spec.volumes[].volumeNumber— (required)Stable cross-copy join key (boot disk = 0). Max 15 volumes per resource.
spec.volumes[].pvcName— (required)Local backing PVC name.

Phase values: Pending, Provisioning, ReadyToActivate, Syncing, Consistent, Degraded, Failed, Deleting.

Note: DRBDResource is only registered on workload clusters. Running kubectl get dres against the TSR-Manager cluster returns server doesn't have a resource type.

DRBDReplicationPolicy

Defines DRBD replication parameters. The ProtectionZone controller automatically creates one policy named <zoneName>-default with isDefault: true and Protocol C in the zone namespace. You do not create this object.

FieldDefaultNotes
spec.drbdProtocolCWire protocol. Only C (synchronous) is supported in Tech Preview.
spec.replicationModeSynchronousSynchronous or Asynchronous. Only Synchronous is functional in Tech Preview.
spec.isDefaultfalseMarks this as the default policy for its namespace.
spec.clusters[].storageClassMappings—Maps source StorageClass names to target StorageClass names. If a source class is absent, it is used as-is.
spec.networkConfig.portRangestart: 7000, end: 7999Accepted by the schema but never read by the port allocator. The allocator always uses 7000–8000 inclusive.

ReplicationGroupStatus

Auto-created observability CR per ProtectionGroup, named <protection-group-name>-replication. Written by the replication monitor controller; you do not create or update it.

Status fieldNotes
status.rpoStatusMet, Degraded, Violated, or Unknown. Driven by spec.rpoObjectiveSeconds (set from ProtectionGroup.spec.sla.rpoTargetSeconds).
status.aggregated.outOfSyncBytesTotal out-of-sync bytes across all VMs in the group.
status.aggregated.connectionStateWorst-case connection state: Connected, Degraded, or Disconnected.
status.volumeReplicationStatusesOne entry per VM (not per disk). A 3-disk VM appears once.

Filter by label siterecovery.trilio.io/protection-group=<pg-name>. Field selectors on spec.* or status.* do not work on custom resources.

RPOEvent

Immutable audit record emitted by the replication monitor on each replication status transition. All fields are in spec (no status subresource). You do not create or update RPOEvents.

FieldNotes
spec.eventTypeOne of: SyncCompleted, RPODegraded, RPOViolated, ConnectionLost, ConnectionRestored, ResyncStarted, ResyncCompleted, VolumeAdded, VolumeRemoved, MetadataUndersized, ResizePending, ResizeComplete.
spec.violationReasonOne of: ConnectionLost, NetworkTimeout, SyncLagExceeded, ProtocolCOutOfSync, ResyncInProgress, Unknown.

Hook

Reusable, namespaced definition of a Kubernetes Job to execute at failover or test lifecycle points. Referenced by ProtectionGroup.spec.hooks[].hookName. The admission webhook prevents deletion or spec edits while an operation is using it. Hook has no short name; use kubectl get hooks.

FieldDefaultNotes
spec.jobTemplate.image— (required)Container image for the hook Job. Must be pullable from the executing workload cluster.
spec.timeoutSeconds300Per-attempt timeout in seconds.
spec.retries0Retry count after failure. Maximum 5.
spec.retryDelaySeconds5Pause between retry attempts in seconds.
spec.jobTemplate.env—Environment variables. fieldRef and resourceFieldRef sources are rejected at admission.
spec.jobTemplate.volumes—Permitted sources: secret, configMap, emptyDir, projected. hostPath is not representable.

Usage

All Site Recovery operations are performed by creating or patching custom resources on the TSR-Manager cluster, except DRBDResource inspection which is done on workload clusters.

Check zone health

# On the TSR-Manager cluster, in the zone namespace
kubectl get pz -n dr-<zone-name>
kubectl describe pz <zone-name> -n dr-<zone-name>

The ClustersValidated condition must be True before the webhooks will accept ProtectionGroup or ProtectionRequest creates.

Protect a virtual machine

  1. Create a ProtectionGroup declaring the VM(s):
kubectl apply -f protectiongroup.yaml -n dr-<zone-name>
kubectl get pg -n dr-<zone-name>
  1. Create a ProtectionRequest for each VM:
kubectl apply -f protectionrequest.yaml -n dr-<zone-name>
kubectl get pr -n dr-<zone-name> --watch

Watch the phase progress: Pending → Validating → CreatingDRBD → Syncing → ReadyToActivate → Activating → Protected.

  1. Check the protected VM label on the workload cluster:
# On the workload cluster
kubectl get vm -A -l siterecovery.trilio.io/protected-vm=true
  1. Inspect DRBD state on each workload cluster:
# On each workload cluster, in the VM's namespace
kubectl get dres -n <vm-namespace>
kubectl describe dres drbd-<namespace>-<vm>-<hash8> -n <vm-namespace>

Monitor replication health

# ReplicationGroupStatus is auto-created; inspect it by label
kubectl get rgs -n dr-<zone-name> -l siterecovery.trilio.io/protection-group=<pg-name>
kubectl get rgs <pg-name>-replication -n dr-<zone-name>

# View RPO events (audit trail)
kubectl get rpoe -n dr-<zone-name> -l siterecovery.trilio.io/protection-group=<pg-name>

Run a test failover

kubectl apply -f testfailover.yaml -n dr-<zone-name>
kubectl get tfo -n dr-<zone-name> --watch

The test namespace is reported in status.testNamespace. Do not construct or guess this value.

After the test completes, trigger cleanup without losing the record:

kubectl patch tfo <name> -n dr-<zone-name> \
  --type merge \
  -p '{"spec":{"cleanupRequested":true}}'

Execute a planned failover

kubectl apply -f failoverrequest-planned.yaml -n dr-<zone-name>
kubectl get fr -n dr-<zone-name> --watch

Every failover is initiated by creating a FailoverRequest. Failover does not happen automatically.

Execute an unplanned failover

kubectl apply -f failoverrequest-unplanned.yaml -n dr-<zone-name>

Unprotect a VM

Delete the ProtectionRequest. The controller moves the VM through the Unprotecting phase, removes DRBD resources, and restarts the VM against its original storage.

kubectl delete pr <name> -n dr-<zone-name>
kubectl get pr <name> -n dr-<zone-name> --watch

Rotate zone credentials

# Rotate and refresh RBAC if permissions have changed
tsrctl zone-credential rotate --refresh-rbac

# Rotate and revoke the previous token
tsrctl zone-credential rotate --revoke-previous

Adjust runtime log level

Edit the site-recovery-logging ConfigMap in the zone namespace (TSR-Manager) or workload namespace (trilio-site-recovery-system). Changes take effect within ~15 seconds without pod restarts.

kubectl patch configmap site-recovery-logging \
  -n dr-<zone-name> \
  --type merge \
  -p '{"data":{"level":"debug","level.control-plane":"trace"}}'

Accepted values: error, info, debug, trace. There is no warn level.


Examples

ProtectionZone — minimal DRBD zone

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: primary-dr-zone
  namespace: dr-primary-dr-zone
spec:
  displayName: "Primary DR Zone"
  description: "DRBD protection zone pairing primary and DR sites."
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

ProtectionZone — with debug logging and custom node-agent deadline

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: primary-dr-zone
  namespace: dr-primary-dr-zone
spec:
  displayName: "Primary DR Zone"
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig
  logging:
    level: info
    controlPlaneLevel: debug
  nodeAgent:
    startupDeadlineSeconds: 180
  hooks:
    jobTTLSeconds: 7200
    failedJobRetentionSeconds: 604800

ProtectionGroup — two VMs with boot ordering and SLA

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: app-tier
  namespace: dr-primary-dr-zone
spec:
  virtualMachines:
    - name: db-vm
      namespace: app-production
      bootOrder: 1
    - name: web-vm
      namespace: app-production
      bootOrder: 2
  desiredState: running
  sla:
    rpoTargetSeconds: 0
    rtoTargetSeconds: 300
  vmIdentity:
    macAddresses: Preserve

ProtectionRequest — protect a single VM

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-db-vm
  namespace: dr-primary-dr-zone
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: db-vm
    namespace: app-production
  replicationConfig:
    replicationProtocol: C
    rpo: 0
    drbdMetadataConfig:
      storageClassName: fast-block
      size: 128Mi

ProtectionRequest — with standby placement overrides

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-web-vm
  namespace: dr-primary-dr-zone
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: web-vm
    namespace: app-production
  replicationConfig:
    replicationProtocol: C
    rpo: 0
  standbyPlacement:
    nodeSelector:
      add:
        workload-type: dr-standby
      remove:
        - zone

FailoverRequest — planned failover to cluster2

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: planned-failover-20260801
  namespace: dr-primary-dr-zone
spec:
  protectionGroupRef:
    name: app-tier
    namespace: dr-primary-dr-zone
  targetCluster: cluster2
  failoverType: planned
  batchBootTimeoutSeconds: 300
  skipHooks: false

FailoverRequest — unplanned failover (skip hooks)

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: unplanned-failover-incident
  namespace: dr-primary-dr-zone
spec:
  protectionGroupRef:
    name: app-tier
    namespace: dr-primary-dr-zone
  targetCluster: cluster2
  failoverType: unplanned
  skipHooks: true

TestFailover — basic DR drill with automatic cleanup

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: drill-q3-2026
  namespace: dr-primary-dr-zone
spec:
  protectionGroupRef:
    name: app-tier
    namespace: dr-primary-dr-zone
  cleanupPolicy: Automatic
  retentionTime: 2h
  schedulingCheck: Fail

TestFailover — with TCP and HTTP reachability probes

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: drill-with-probes
  namespace: dr-primary-dr-zone
spec:
  protectionGroupRef:
    name: app-tier
    namespace: dr-primary-dr-zone
  cleanupPolicy: Manual
  schedulingCheck: Warn
  verification:
    reachability:
      - type: tcp
        port: 5432
        vmName: db-vm
        timeout: 30s
      - type: http
        port: 8080
        vmName: web-vm
        endpoint: /healthz
        expectedStatus: 200
        timeout: 30s

Hook — pre-failover quiesce script

apiVersion: siterecovery.trilio.io/v1alpha1
kind: Hook
metadata:
  name: db-quiesce
  namespace: dr-primary-dr-zone
spec:
  description: "Quiesce the database before failover"
  timeoutSeconds: 120
  retries: 2
  retryDelaySeconds: 10
  jobTemplate:
    image: quay.io/myorg/db-quiesce-tool:v1.2.3
    command: ["/scripts/quiesce.sh"]
    args: ["--timeout", "90"]
    env:
      - name: DB_HOST
        value: "db-vm.app-production.svc"

ProtectionGroup — binding a Hook

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: app-tier
  namespace: dr-primary-dr-zone
spec:
  virtualMachines:
    - name: db-vm
      namespace: app-production
    - name: web-vm
      namespace: app-production
  desiredState: running
  sla:
    rpoTargetSeconds: 0
    rtoTargetSeconds: 300
  hooks:
    - name: pre-failover-quiesce
      hookName: db-quiesce
      type: pre-failover
      runOn: source
      onFailure: Abort
      order: 0
      vms:
        - name: db-vm
          namespace: app-production

Inspect per-VM DRBD state (on the workload cluster)

# List all DRBDResources in the VM's namespace
kubectl get dres -n app-production

# Describe a specific DRBDResource
kubectl describe dres drbd-app-production-db-vm-a1b2c3d4 -n app-production

Check replication group status

# On the TSR-Manager cluster
kubectl get rgs app-tier-replication -n dr-primary-dr-zone

# Watch for RPO events
kubectl get rpoe -n dr-primary-dr-zone \
  -l siterecovery.trilio.io/protection-group=app-tier \
  --sort-by=.spec.timestamp

View per-VM failover status

kubectl get fr planned-failover-20260801 \
  -n dr-primary-dr-zone \
  -o jsonpath='{.status.vmStatuses}'

Troubleshooting

Zone stuck in Provisioning / ClustersValidated=False

Symptom: kubectl get pz -n dr-<zone-name> shows Status: Provisioning and ClustersValidated condition is False.

Likely causes and fixes:

ReasonMeaningFix
ClusterAlreadyClaimedA workload cluster is leased to another zone (the siterecovery.trilio.io/owner-zone-namespace-uid annotation on its trilio-site-recovery-system namespace belongs to a different zone).Remove the other zone or use different workload clusters.
InsufficientClusterPermissionsThe zone credential authenticates but fails the RBAC permission sweep. The condition message names the missing verbs.Run tsrctl zone-credential rotate --refresh-rbac to issue a new credential with current RBAC.
ClusterIdentityChangedThe credential now reaches a cluster whose kube-system Namespace UID differs from the UID recorded at first claim. No override exists.Restore the original cluster or re-create the zone.
KubeVirtNotInstalledThe workload cluster does not serve kubevirt.io/v1.Install KubeVirt on the workload cluster before creating the zone.
SCCPrerequisiteMissingThe privileged SCC or its ClusterRole does not exist on the workload cluster.Verify OpenShift is correctly installed on the workload cluster.

ProtectionRequest stuck in Syncing

Symptom: A ProtectionRequest has been in Syncing for an unexpectedly long time.

Likely cause: Initial block-level sync is still in progress. Check progress:

# On TSR-Manager
kubectl get pr <name> -n dr-<zone-name> -o jsonpath='{.status.slowestVolumeSyncProgress}'

# On the workload cluster
kubectl get dres drbd-<namespace>-<vm>-<hash8> -n <vm-namespace> \
  -o jsonpath='{.status.syncProgress}'

Fix: Wait for sync to complete. If progress is stalled at 0%, check DRBD node agent logs on both workload clusters:

# On the workload cluster — find the node agent pod for the VM's node
POD=$(kubectl get pods -n trilio-site-recovery-system \
  --field-selector spec.nodeName=<node-name> \
  -l app.kubernetes.io/component=drbd-node-agent \
  -o jsonpath='{.items[0].metadata.name}')
kubectl logs -n trilio-site-recovery-system $POD

Verify TCP ports 7000–8000 inclusive are open between primary and DR worker nodes. Port 8000 is mandatory; a firewall stopping at 7999 will silently prevent replication for one volume.


DRBDResource in Degraded phase

Symptom: kubectl get dres -n <vm-namespace> shows Degraded in the Phase column.

Likely cause: The DRBD connection to the peer cluster is lost or the DRBD role is in conflict.

Fix:

  1. Check the connectionState column—Disconnected or StandAlone indicates a network issue.
  2. Verify network connectivity on TCP 7000–8000 between the two cluster's worker nodes.
  3. Check if the ProtectionRequest on TSR-Manager has a Ready=False condition with reason RoleConflict:
kubectl describe pr <name> -n dr-<zone-name> | grep -A5 RoleConflict
  1. Check the DRBD node agent DaemonSet is healthy on both workload clusters:
kubectl get daemonset -n trilio-site-recovery-system \
  -l app.kubernetes.io/component=drbd-node-agent

Node agent applying not-ready taint longer than expected

Symptom: A node carries the siterecovery.trilio.io/not-ready:NoSchedule taint after startup for longer than spec.nodeAgent.startupDeadlineSeconds.

Likely cause: One or more DRBD resources on the node have not converged. After the deadline, the taint is removed anyway and the DRBDResource carries a NodeStartupGate=False condition with reason StartupDeadlineExceeded.

Fix: The condition clears itself when the resource recovers. Check what it is waiting for:

# On the workload cluster
kubectl describe dres <name> -n <vm-namespace> | grep -A5 NodeStartupGate

If the node-agent pod is failing to start, check its logs and verify DRBD module loading:

POD=$(kubectl get pods -n trilio-site-recovery-system \
  --field-selector spec.nodeName=<node-name> \
  -l app.kubernetes.io/component=drbd-node-agent \
  -o jsonpath='{.items[0].metadata.name}')
kubectl logs -n trilio-site-recovery-system $POD -c drbd-module-loader

FailoverRequest stuck in InProgress

Symptom: A FailoverRequest remains InProgress for an extended period.

Likely cause: A hook is running slowly or has failed; VMs on the source cluster are not stopping; the boot-order batch timeout has not elapsed.

Fix: Inspect the hook statuses and per-VM statuses:

kubectl get fr <name> -n dr-<zone-name> -o jsonpath='{.status.phase}'
kubectl get fr <name> -n dr-<zone-name> -o jsonpath='{.status.vmStatuses}'
kubectl get fr <name> -n dr-<zone-name> -o jsonpath='{.status.hookStatuses}'

If a hook is blocking and the incident is time-critical, delete the current FailoverRequest, create a new one with spec.skipHooks: true, and investigate the hook failure afterwards.


TestFailover fails with protectionGroupRef not found

Symptom: Creating a TestFailover returns an error containing protectionGroupRef.

Likely cause: The ProtectionGroup referenced does not exist in the namespace specified in spec.protectionGroupRef.namespace, or spec.protectionGroupRef.namespace was omitted.

Fix: Verify the ProtectionGroup exists on the TSR-Manager cluster and that protectionGroupRef.namespace is set to the correct zone namespace:

kubectl get pg -n dr-<zone-name>

Note that spec.protectionGroupRef.namespace must be set explicitly on a TestFailover; it cannot be inferred.


oc adm must-gather fails to collect

Symptom: oc adm must-gather exits with an error or produces an incomplete bundle.

Fix: Run must-gather from the TSR-Manager cluster with the Site Recovery image. Contact Trilio support for the pinned image reference:

# Run on the TSR-Manager cluster
oc adm must-gather --image=quay.io/triliovault/site-recovery-must-gather:<release-tag>

The entrypoint inside the image is /usr/bin/gather. Do not pass a -- <command> override. The collector reaches through to all zone workload clusters and assembles a single read-only archive.