Prerequisites

This page lists every requirement you must satisfy before installing Trilio Site Recovery (Tech Preview). Site Recovery protects KubeVirt VirtualMachines across two workload clusters by replicating their persistent volumes at the block level via DRBD, with all orchestration driven from a dedicated TSR-Manager cluster. Meeting these prerequisites ensures that cluster validation, DRBD module loading, and zone credential provisioning all succeed on the first attempt. Read this page before you proceed to installation.

Cluster topology

Site Recovery requires three separate Kubernetes clusters:

The TSR-Manager cluster is required, not optional. A ProtectionZone pairs exactly two workload clusters (cluster1 and cluster2); you cannot register more than two workload clusters per zone.

Platform and version requirements

Site Recovery is developed, tested, and supported on OpenShift. Upstream Kubernetes workload clusters are not supported in Tech Preview.

OpenShift version

Two ranges are defined by different sources. Use both to determine whether your version is supported:

Versions 4.14–4.16 fall within the OLM-installable range but are outside the validated range; they are not covered by Tech Preview testing. OpenShift 4.17 is the supported floor for the TSR-Manager cluster and both workload clusters. Versions beyond 4.20 are expected to work but have not been validated.

Kubernetes version (non-OpenShift, for reference)

Upstream Kubernetes clusters are not supported in Tech Preview. The charts do install on plain Kubernetes (Kubernetes 1.27 and later corresponds to the OpenShift 4.14 floor), but three capabilities are OpenShift-specific and fail silently or are absent on upstream Kubernetes:

• DRBD node agent admission — the privileged SCC binding is not created; you must admit the privileged, hostNetwork DaemonSet through your own pod security policy.
• Test failover network isolation — the default-deny NetworkPolicy targets the openshift-dns namespace; on a kube-dns cluster, test VMs lose name resolution without warning.
• TSR web console access — no Route, no OAuth proxy, and no supported browser access method is provided.

Single-Node OpenShift (SNO)

SNO is supported for both workload clusters and the TSR-Manager cluster. A protected VM is pinned to one node per cluster, so a single-node cluster satisfies that binding. Plan for the consequence: node loss on SNO means the whole cluster side is lost; recovery requires the node-loss recovery path followed by re-protection.

Required software on workload clusters

KubeVirt / OpenShift Virtualization

Required on both workload clusters. Each must serve the kubevirt.io/v1 API, including the VirtualMachine, VirtualMachineInstance, and VirtualMachineInstanceMigration kinds, as well as the virtualmachines/status subresource. If a workload cluster does not serve kubevirt.io/v1, zone validation fails with reason KubeVirtNotInstalled before any component is installed.

The TSR-Manager cluster does not need KubeVirt.

DRBD kernel module (drbd9)

Every workload node that will host a protected VM must be able to load the DRBD 9 kernel module. The DRBD node agent DaemonSet includes a privileged init container (drbd-module-loader) that loads the module automatically at pod start using a precompiled kernel module matched by kernel ABI. You do not need to load the module manually.

Required modules:

• drbd (with usermode_helper=disabled)
• drbd_transport_tcp
• drbd_transport_lb-tcp
• drbd_transport_rdma (optional)

Hard failure: if a foreign DRBD module is already loaded with usermode_helper enabled, the init container exits with an error. You must reboot the node after removing whatever loaded it.

Node operating system

Verify a node's kernel family:

oc get node <node> -o jsonpath='{.status.nodeInfo.kernelVersion}'
# .el9_* -> supported     .el10_* -> not validated

Mixed OS families within a single cluster are not supported. The ProtectionZone deploys exactly one module-loader image family per workload cluster.

Network requirements

DRBD replication ports

DRBD replication traffic flows directly between the worker nodes of the primary and DR clusters, not through the TSR-Manager cluster. You must open the following ports in both directions between every primary worker node and every DR worker node:

Important: Port 8000 is mandatory. A firewall rule that stops at 7999 will silently prevent replication on the last allocatable volume. The allocator iterates 7000–8000 inclusive.

With the Tech Preview scale limit of up to 1,000 protected VMs per deployment, up to 1,001 ports in this range may be used.

Round-trip latency

Site Recovery uses Protocol C (synchronous) replication exclusively. Protocol C is the only supported replication protocol in Tech Preview; Protocol A is rejected by the admission webhook. Protocol C acknowledges a write only after both the primary and DR sides confirm it, delivering RPO=0.

For Protocol C to maintain acceptable write performance:

• Round-trip latency between primary and DR worker nodes must be under 10 ms.

There is no protocol selection or RPO tuning available; RPO is always 0.

API server connectivity

The TSR-Manager cluster must be able to reach the API servers of both workload clusters. These connections carry CR management traffic (DRBDResource creation, VM mirroring, failover orchestration) and are authenticated by the zone kubeconfig Secrets you create with tsrctl.

Required tools

kubectl and oc

• kubectl — required to create and manage all Site Recovery custom resources (ProtectionZone, ProtectionGroup, ProtectionRequest, FailoverRequest, TestFailover, Hook).
• oc (OpenShift CLI) — required for the OpenShift-specific paths: oc adm must-gather for support bundle collection, and Route management. Do not use kubectl-only operation as a supported mode on OpenShift.

tsrctl

tsrctl is the only Site Recovery CLI tool. It manages zone kubeconfig credentials exclusively — creating, rotating, inspecting, and deleting the kubeconfig Secrets that grant the TSR-Manager controllers access to each workload cluster's API server. It does not drive DR operations (failover, protect, and so on); those are performed through CRDs.

tsrctl is supplied by Trilio with the release. Contact Trilio support for the download for your platform and release. It is a single self-contained binary that runs on the workstation you perform credential operations from — it is not installed on any cluster.

What each release ships

Each archive contains the tsrctl binary (tsrctl.exe on Windows) and LICENSE.

Linux and macOS

Trilio support provides an installer script with the release. It detects your platform and architecture, downloads the matching archive, verifies it against checksums.txt, and installs the binary to /usr/local/bin when that directory is writable, otherwise to ~/.local/bin. It accepts:

Windows

The installer script does not support Windows. Download tsrctl_<version>_windows_amd64.zip and checksums.txt for your release, verify the archive before extracting it, then place tsrctl.exe on your PATH:

# Verify before extracting — do not skip this step
Get-FileHash -Algorithm SHA256 tsrctl_<version>_windows_amd64.zip
# Compare the result against the matching line in checksums.txt

Confirm the install

tsrctl --version

Helm

Helm ≥ 3 is required on the operator side, but you do not run helm install commands directly. The three Site Recovery Helm charts are embedded in the ProtectionZone controller binary and are applied automatically when you create a ProtectionZone CR. There is no chart repository to add.

OLM (Operator Lifecycle Manager)

OLM is required to install Site Recovery on OpenShift. The OLM bundle (site-recovery, channel stable, catalog quay.io/triliodata/site-recovery-catalog) delivers the ProtectionZone controller, the TSR web console frontend, and the backend API as CSV-embedded Deployments.

Image registry access

The TSR-Manager cluster and both workload clusters must be able to pull images from quay.io/triliodata. The DRBD node agent and module loader use imagePullPolicy: Always, meaning every pod start (including after node reboots) triggers a pull. If the registry is unreachable at pod start time, DRBD on that node does not recover until the pull succeeds.

Do not configure egress access to any internal Trilio registry; only quay.io/triliodata is the customer-facing image location.

Clock synchronization

All three clusters (TSR-Manager, primary workload, DR workload) must have NTP/chrony running on every node. Site Recovery's protection guards tolerate minor skew, but a drifting clock causes indirect failures: service account tokens, kubeconfig tokens, and TLS certificates are time-sensitive, and every cross-cluster timestamp comparison (replication freshness, failover progress against its five-minute wall-clock budget) assumes the sides agree on the time.

RBAC and permissions

Zone credentials

The tsrctl zone-credential create command must be run with credentials that have cluster-admin rights on both workload clusters. It creates, on each workload cluster:

• A per-zone ServiceAccount in the trilio-site-recovery-access namespace.
• A ClusterRole with exactly the verified install-time and runtime permissions.
• A time-limited token (default one year) with the cluster CA embedded.

The resulting kubeconfig Secrets are stored on the TSR-Manager cluster and grant the orchestration controllers cross-cluster access. Treat these kubeconfigs as cluster-wide Secret readers, because the TSR-Manager mirrors the Secrets that protected VMs reference.

OpenShift SCC

The DRBD node agent DaemonSet runs as a privileged pod (privileged: true, hostNetwork: true, runs as root). On OpenShift, the ProtectionZone controller automatically binds the node agent's dedicated ServiceAccount to the stock system:openshift:scc:privileged ClusterRole. Zone validation checks that the privileged SCC and its ClusterRole exist before installing anything; a node agent rejected by SCC admission is reported as SCCAdmissionDenied on the zone's WorkloadStacksReady condition.

The control plane Deployment (admission webhooks and DRBD placement reconciler) runs under restricted-v2 and does not require a privileged SCC.

Scale limits (Tech Preview)

Constraints enforced by admission webhooks

The following are rejected at the API server — you will see an error, not a degraded state:

Pre-installation checklist

Before proceeding to installation, confirm each of the following:

• Three clusters available: one TSR-Manager, one primary workload, one DR workload
• All clusters running OpenShift 4.17 or later
• KubeVirt installed on both workload clusters
• TCP ports 7000–8000 inclusive open between all primary and DR worker nodes
• Round-trip latency under 10 ms between primary and DR worker nodes
• RHEL 9-based RHCOS (el9 kernels) on all workload nodes
• NTP/chrony running on all nodes of all three clusters
• kubectl and oc installed on your workstation
• tsrctl binary obtained from Trilio support
• OLM installed and functional on the TSR-Manager cluster
• Egress from all clusters to quay.io/triliodata permitted
• Cluster-admin credentials available for both workload clusters (for zone credential creation)

Installation follows two complementary paths that you complete in order:

1. Install the operator via OLM on the TSR-Manager cluster — this deploys the ProtectionZone controller, the TSR web console frontend, and the backend API.
2. Create a ProtectionZone CR — this triggers automatic deployment of the TSR-Manager control plane and the workload control plane stacks (including the DRBD node agent DaemonSet) on both workload clusters.

You never run helm install directly. The three Helm charts are embedded in the ProtectionZone controller binary and applied by the controller when the zone reaches Available.

Step 1 — Mint zone credentials for both workload clusters

Before the operator can reach your workload clusters, you need kubeconfig Secrets. Use tsrctl on each workload cluster with cluster-admin rights.

tsrctl is supplied by Trilio with the release. Contact Trilio support to obtain the binary, then make it executable.

Create the zone credential for each workload cluster. The zone-credential create subcommand accepts --namespace (the namespace on the workload cluster for the ServiceAccount, defaults to trilio-site-recovery-access), --access-namespace, --context, and --kubeconfig global flags:

# Create the credential for the primary (cluster1) workload cluster
tsrctl zone-credential create \
  --kubeconfig /path/to/cluster1-admin.kubeconfig

# Create the credential for the DR (cluster2) workload cluster
tsrctl zone-credential create \
  --kubeconfig /path/to/cluster2-admin.kubeconfig

Each invocation produces a kubeconfig file. Store these files; you will reference them in the ProtectionZone CR in Step 4.

Step 2 — Install the operator via OLM on the TSR-Manager cluster

Apply the following resources on the TSR-Manager cluster. They install the Site Recovery operator from the Trilio catalog.

2a — Create the namespace

oc --context tsrmanager apply -f - <<'EOF'
apiVersion: v1
kind: Namespace
metadata:
  name: trilio-site-recovery-system
EOF

2b — Create the CatalogSource

oc --context tsrmanager apply -f - <<'EOF'
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: site-recovery-catalog
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/triliodata/site-recovery-catalog
  displayName: Trilio Site Recovery
  publisher: Trilio Data, Inc.
EOF

2c — Create the OperatorGroup

oc --context tsrmanager apply -f - <<'EOF'
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: site-recovery-og
  namespace: trilio-site-recovery-system
spec: {}
EOF

2d — Create the Subscription

oc --context tsrmanager apply -f - <<'EOF'
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: site-recovery
  namespace: trilio-site-recovery-system
spec:
  channel: stable
  name: site-recovery
  source: site-recovery-catalog
  sourceNamespace: openshift-marketplace
EOF

2e — Verify operator pods are running

oc --context tsrmanager get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/part-of=site-recovery

Wait until the ProtectionZone controller pod and the TSR web console pods are Running.

Step 3 — Create the zone namespace and upload kubeconfig Secrets

All zone-scoped resources live in a namespace named dr-<deployment-name> on the TSR-Manager cluster.

# Choose a deployment name, e.g. "prod"
export ZONE_NAME=prod

# Create the zone namespace
oc --context tsrmanager create namespace dr-${ZONE_NAME}

# Upload the cluster1 kubeconfig Secret
oc --context tsrmanager create secret generic cluster1-kubeconfig \
  --from-file=kubeconfig=/path/to/cluster1-zone.kubeconfig \
  -n dr-${ZONE_NAME}

# Upload the cluster2 kubeconfig Secret
oc --context tsrmanager create secret generic cluster2-kubeconfig \
  --from-file=kubeconfig=/path/to/cluster2-zone.kubeconfig \
  -n dr-${ZONE_NAME}

Step 4 — Create the ProtectionZone CR

Creating the ProtectionZone triggers cluster validation, RBAC checks, and automated installation of the TSR-Manager and workload control plane stacks (including the DRBD node agent DaemonSet on both workload clusters).

oc --context tsrmanager apply -f - <<EOF
apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: ${ZONE_NAME}
  namespace: dr-${ZONE_NAME}
spec:
  displayName: "Production DR Zone"
  description: "Primary and DR site pairing for production VMs."
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr-site
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig
EOF

storageBackendMode is required and must be DRBD. The Linstor value is rejected at admission.

Step 5 — Monitor zone provisioning

Watch the ProtectionZone until it reaches Available phase and the ClustersValidated condition is True:

oc --context tsrmanager get pz ${ZONE_NAME} -n dr-${ZONE_NAME} -w

To inspect conditions in detail:

oc --context tsrmanager get pz ${ZONE_NAME} -n dr-${ZONE_NAME} -o yaml

Key conditions to check:

Once ClustersValidated is True, you can create ProtectionGroup and ProtectionRequest CRs.

Step 6 — Verify workload stacks on both workload clusters

Confirm the workload control plane and DRBD node agent DaemonSet are deployed on each workload cluster:

# On the primary workload cluster
oc --context cluster1 get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/part-of=site-recovery

# On the DR workload cluster
oc --context cluster2 get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/part-of=site-recovery

You should see both a workload control plane Deployment pod (app.kubernetes.io/component: workload-control-plane) and one DRBD node agent DaemonSet pod per worker node (app.kubernetes.io/component: drbd-node-agent).

Site Recovery configuration is split between the ProtectionZone CR (zone-level settings) and the site-recovery-logging ConfigMap (runtime log levels). Customers do not author Helm chart values directly; the ProtectionZone controller applies the embedded charts automatically.

ProtectionZone spec fields

All fields below are set on the ProtectionZone CR in the dr-<name> namespace on the TSR-Manager cluster.

Required fields

Optional fields

Immutable fields

Once a ProtectionZone is created, the following fields cannot be changed:

• spec.storageBackendMode
• spec.clusters[].name (renaming a cluster is rejected)
• spec.clusters[].clusterID (slot reassignment is rejected)

Mutable fields (may be changed after creation):

• spec.displayName, spec.description
• spec.clusters[].kubeconfigSecretRef (to rotate credentials)
• spec.clusters[].distributionName (to follow an OpenShift RHCOS upgrade)
• spec.logging.*, spec.nodeAgent.*, spec.hooks.*

Runtime log level (without pod restarts)

The site-recovery-logging ConfigMap controls runtime log levels for both the control plane manager and the DRBD node agent in its namespace. Both components poll it approximately every 15 seconds.

The ConfigMap is automatically created in each component's namespace when spec.logging.level is set on the ProtectionZone. You can also edit it directly:

# Set all zone components to debug on the TSR-Manager
oc --context tsrmanager patch configmap site-recovery-logging \
  -n dr-prod \
  --type merge \
  -p '{"data":{"level":"debug"}}'

# Set control plane to debug, node agent to info (on a workload cluster)
oc --context cluster1 patch configmap site-recovery-logging \
  -n trilio-site-recovery-system \
  --type merge \
  -p '{"data":{"level":"info","level.control-plane":"debug","level.drbd-node-agent":"info"}}'

Accepted values: error, info, debug, trace. There is no warn level. The trace level is available at runtime via the ConfigMap but not as a startup flag.

DRBDReplicationPolicy (managed automatically)

The ProtectionZone controller creates a default DRBDReplicationPolicy named <zoneName>-default with isDefault: true in the zone namespace when the zone is provisioned. You do not create or modify this object.

Key facts:

• Protocol C (synchronous) is the only supported protocol. Protocol A is rejected at the ProtectionRequest admission webhook.
• RPO is always 0 and is not tunable.
• DRBD ports are allocated from the fixed range 7000–8000 inclusive by the allocator in the operator; the networkConfig.portRange field in the policy is accepted but not read by the allocator.

ProtectionRequest replication configuration

When you protect a VM, the ProtectionRequest's spec.replicationConfig accepts:

Node agent startup taint

The DRBD node agent applies siterecovery.trilio.io/not-ready:NoSchedule to its own node at startup and holds it until all DRBD resources on that node converge (or until startupDeadlineSeconds expires). Workloads that do not tolerate this taint — including CSI node plugins and KubeVirt's virt-handler — wait at most startupDeadlineSeconds on any freshly restarted node.

Adjust the deadline on the ProtectionZone:

oc --context tsrmanager patch pz prod -n dr-prod \
  --type merge \
  -p '{"spec":{"nodeAgent":{"startupDeadlineSeconds":240}}}'

Changing this value triggers a rolling update of the DRBD node agent DaemonSet.

All Site Recovery operations are declared through Kubernetes custom resources. The workflow proceeds in this order: install and configure the control plane → provision zone credentials → create a ProtectionZone → protect VMs → monitor replication → run test failovers → execute failovers.

All TSR-Manager-side CRs (ProtectionZone, ProtectionGroup, ProtectionRequest, FailoverRequest, TestFailover, Hook, ReplicationGroupStatus, RPOEvent) live in the dr-<name> namespace on the TSR-Manager cluster. DRBDResource CRs live on each workload cluster in the VM's namespace.

Managing zone credentials

Zone credentials are kubeconfig Secrets that grant the TSR-Manager controllers access to workload cluster API servers. They are minted and rotated exclusively with tsrctl.

Inspect a credential

tsrctl zone-credential inspect

Rotate a credential (reissue token, preserve RBAC)

tsrctl zone-credential rotate

Rotate and refresh RBAC (after a controller upgrade adds new permissions)

tsrctl zone-credential rotate --refresh-rbac

Rotate and revoke the previous token

tsrctl zone-credential rotate --revoke-previous

Delete a credential

tsrctl zone-credential delete

Force deletion (use when the workload cluster is unreachable):

tsrctl zone-credential delete --force

The credential is bound to the kube-system Namespace UID of the workload cluster it was created for. It cannot be redirected to a different cluster. If the zone's ClustersValidated condition shows reason ClusterIdentityChanged, the credential now reaches a different cluster; the only resolution is to restore the original cluster or re-create the zone.

Checking zone health

# List all zones
oc --context tsrmanager get pz -A

# Inspect a specific zone
oc --context tsrmanager get pz prod -n dr-prod -o yaml

# Watch conditions
oc --context tsrmanager get pz prod -n dr-prod \
  -o jsonpath='{.status.conditions}' | python3 -m json.tool

The ClustersValidated condition must be True before you can create ProtectionGroup or ProtectionRequest CRs. The TSR-Manager admission webhooks reject those creates while it is not True.

Protecting a VM

Create a ProtectionRequest on the TSR-Manager cluster in the zone namespace:

oc --context tsrmanager apply -f - <<'EOF'
apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-web-vm
  namespace: dr-prod
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: web-vm
    namespace: production
EOF

Monitor phase progression:

oc --context tsrmanager get pr protect-web-vm -n dr-prod -w

Phases in order: Pending → Validating → CreatingDRBD → Syncing → ReadyToActivate → Activating → Protected.

The VM must be running when you protect it. The node it runs on becomes its permanent home on the source cluster for the lifetime of the protection. Live migration and cold migration of protected VMs are denied by admission webhooks.

To check sync progress:

oc --context tsrmanager get pr protect-web-vm -n dr-prod \
  -o jsonpath='{.status.syncProgress}'

Monitoring replication health

The replication monitor auto-creates a ReplicationGroupStatus (short name: rgs) named <pg-name>-replication for each ProtectionGroup:

oc --context tsrmanager get rgs -n dr-prod

oc --context tsrmanager get rgs my-pg-replication -n dr-prod -o yaml

Key status fields:

Inspect DRBDResource CRs on the workload clusters for per-VM detail:

# On the primary workload cluster, in the VM's namespace
oc --context cluster1 get dres -n production
oc --context cluster1 get dres drbd-production-web-vm-<hash8> -n production -o yaml

DRBDResource CRs are named drbd-<namespace>-<vm>-<hash8>. They exist only on workload clusters; running kubectl get drbdresource against the TSR-Manager cluster returns server doesn't have a resource type.

Executing a failover

Create a FailoverRequest on the TSR-Manager cluster. failoverType is planned (graceful, waits for sync) or unplanned (force-promote, RTO-optimized):

oc --context tsrmanager apply -f - <<'EOF'
apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: failover-to-dr
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: my-pg
  targetCluster: cluster2
  failoverType: planned
EOF

Monitor progress:

oc --context tsrmanager get fr failover-to-dr -n dr-prod -w

Per-VM status is in status.vmStatuses. The FailoverRequest has no message field and no failedVMs field at the top level.

Running a test failover (DR drill)

A TestFailover spins up isolated copies of protected VMs on the standby cluster from DRBD snapshots, inside a controller-generated namespace fenced by a default-deny NetworkPolicy. It does not touch production VMs. Supported in DRBD backend mode only.

oc --context tsrmanager apply -f - <<'EOF'
apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: drill-my-pg
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: my-pg
  cleanupPolicy: Automatic
  retentionTime: 2h
EOF

The test namespace is generated by the controller and reported in status.testNamespace. Do not try to predict or construct it.

oc --context tsrmanager get tfo drill-my-pg -n dr-prod -w

Phases: Queued → PreparingEnvironment → CreatingSnapshots → CreatingVolumes → CreatingVMs → StartingVMs → Verifying → Succeeded or Failed → CleaningUp → Cleaned.

Trigger early cleanup while keeping the record:

oc --context tsrmanager patch tfo drill-my-pg -n dr-prod \
  --type merge \
  -p '{"spec":{"cleanupRequested":true}}'

Example 1 — Minimal ProtectionZone CR

A DRBD zone pairing a primary and DR cluster. Both clusters run RHEL CoreOS 9.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: prod
  namespace: dr-prod
spec:
  displayName: "Production DR Zone"
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr-site
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

Example 2 — ProtectionZone with logging and node agent tuning

Sets runtime log level to debug for all components, with the node agent given 3 minutes to converge at startup.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: prod
  namespace: dr-prod
spec:
  displayName: "Production DR Zone"
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr-site
      clusterID: cluster2
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig
  logging:
    level: debug
  nodeAgent:
    startupDeadlineSeconds: 180

Example 3 — ProtectionRequest for a single VM

Protects web-vm in the production namespace, currently running on cluster1. Metadata PVCs use a specific StorageClass and 256 Mi of metadata space per volume.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-web-vm
  namespace: dr-prod
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: web-vm
    namespace: production
  replicationConfig:
    replicationProtocol: C
    rpo: 0
    drbdMetadataConfig:
      storageClassName: ocs-storagecluster-ceph-rbd
      size: 256Mi

Example 4 — ProtectionGroup with boot ordering and RPO target

Groups two VMs that must fail over together. The database VM boots first (lower bootOrder).

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: my-pg
  namespace: dr-prod
spec:
  desiredState: running
  sla:
    rpoTargetSeconds: 0
    rtoTargetSeconds: 300
  virtualMachines:
    - name: db-vm
      namespace: production
      bootOrder: 1
    - name: web-vm
      namespace: production
      bootOrder: 2

Example 5 — Planned failover to DR cluster

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: planned-failover
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: my-pg
  targetCluster: cluster2
  failoverType: planned

oc --context tsrmanager apply -f planned-failover.yaml
oc --context tsrmanager get fr planned-failover -n dr-prod -w

Example 6 — Unplanned failover (force-promote)

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: emergency-failover
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: my-pg
  targetCluster: cluster2
  failoverType: unplanned
  skipHooks: true

Example 7 — Test failover with HTTP reachability probe

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: drill-my-pg
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: my-pg
  cleanupPolicy: Manual
  retentionTime: 4h
  schedulingCheck: Fail
  verification:
    reachability:
      - type: http
        port: 8080
        endpoint: /healthz
        expectedStatus: 200
        timeout: 30s
        vmName: web-vm

Example 8 — Listing replication health

# List all ReplicationGroupStatus objects in the zone namespace
oc --context tsrmanager get rgs -n dr-prod

# Get the auto-created RGS for protection group "my-pg"
oc --context tsrmanager get rgs my-pg-replication -n dr-prod -o yaml

# List RPOEvents (immutable audit records) for the zone
oc --context tsrmanager get rpoe -n dr-prod

Example 9 — Rotating a zone credential

After obtaining the updated tsrctl binary from Trilio:

# Rotate the credential and refresh RBAC to pick up any new permissions
tsrctl zone-credential rotate --refresh-rbac

# Rotate and immediately revoke the previous token
tsrctl zone-credential rotate --revoke-previous

Example 10 — Collecting a support bundle

oc adm must-gather \
  --image=quay.io/triliodata/site-recovery-must-gather \
  -n dr-prod

# Collect with a tighter log window
oc adm must-gather \
  --image=quay.io/triliodata/site-recovery-must-gather \
  -n dr-prod \
  -- /usr/bin/gather --since 2h

Use the format below for each issue: Symptom, Likely cause, Fix.

Zone stuck in Provisioning phase; ClustersValidated is False

Symptom: oc get pz <name> -n dr-<name> shows Status: Provisioning and the ClustersValidated condition is False.

Likely causes and fixes:

DRBD node agent pods are not starting; WorkloadStacksReady is False with reason SCCAdmissionDenied

Symptom: The zone's WorkloadStacksReady condition is False with reason SCCAdmissionDenied. The node agent DaemonSet shows FailedCreate events.

Likely cause: The node agent's DaemonSet pod was rejected by OpenShift SCC admission. The node agent requires the privileged SCC.

Fix: Verify that the ClusterRoleBinding to system:openshift:scc:privileged exists for the node agent ServiceAccount:

oc --context cluster1 get clusterrolebinding -l app.kubernetes.io/part-of=site-recovery

If missing, the zone credential may lack the bind verb on system:openshift:scc:privileged. Rotate with --refresh-rbac:

tsrctl zone-credential rotate --refresh-rbac

ProtectionRequest stuck in Syncing; sync progress not advancing

Symptom: oc get pr <name> -n dr-<name> shows Phase: Syncing and Sync percentage is not increasing.

Likely causes:

1. Network connectivity between worker nodes — DRBD replication uses TCP ports 7000–8000. A blocked port silently stalls replication.

   Fix: Verify firewall rules allow TCP 7000–8000 inclusive in both directions between primary and DR worker nodes. Check that port 8000 is included.

2. DRBD module not loaded on a node — the drbd-module-loader init container may have failed.

   Fix: Check the init container logs on the node agent pod:

   Copy

   # Find the agent pod on the relevant node
   oc --context cluster1 get pods -n trilio-site-recovery-system \
     -l app.kubernetes.io/component=drbd-node-agent \
     --field-selector spec.nodeName=<node-name> -o name
   
   # Check the init container
   oc --context cluster1 logs <pod-name> -n trilio-site-recovery-system \
     -c drbd-module-loader
   

3. usermode_helper enabled on a pre-loaded module — the loader exits hard.

   Fix: Unload the module and reboot the node.

oc get dres against the TSR-Manager cluster returns an error

Symptom: oc --context tsrmanager get dres -A returns server doesn't have a resource type 'drbdresources'.

Cause: DRBDResource CRDs are installed only on workload clusters. They do not exist on the TSR-Manager cluster by design.

Fix: Run DRBDResource queries against the workload clusters:

oc --context cluster1 get dres -n <vm-namespace>
oc --context cluster2 get dres -n <vm-namespace>

ProtectionRequest shows Ready=False, reason: RoleConflict

Symptom: The ProtectionRequest's Ready condition is False with reason RoleConflict.

Cause: A DRBD role conflict was detected — both sides of the replication pair are attempting to be Primary simultaneously.

Fix: Inspect the DRBDResource CRs on both workload clusters to identify which side holds an unexpected Primary role:

oc --context cluster1 get dres -n <vm-namespace> -o wide
oc --context cluster2 get dres -n <vm-namespace> -o wide

Do not manually demote DRBD resources. Contact Trilio support with a support bundle.

Zone credential expiry warning

Symptom: The ProtectionZone's CredentialsValid condition shows reason CredentialExpiring.

Cause: The zone credential token is approaching expiry (within the configured warning window, default 30 days).

Effect: This is advisory only; it does not change the zone phase.

Fix: Rotate the credential before it expires:

tsrctl zone-credential rotate

Test failover stuck in CreatingSnapshots or CreatingVolumes

Symptom: A TestFailover stays in CreatingSnapshots or CreatingVolumes for an extended time.

Likely cause: The standby cluster's VolumeSnapshotClass is missing or does not match the StorageClass provisioner of the protected volumes.

Fix: On the standby workload cluster, verify that a VolumeSnapshotClass exists whose driver matches the StorageClass used by the protected VMs' PVCs:

oc --context cluster2 get volumesnapshotclass

If missing, create a VolumeSnapshotClass with the appropriate driver.

Node agent holds siterecovery.trilio.io/not-ready taint longer than expected

Symptom: A workload cluster node carries the siterecovery.trilio.io/not-ready:NoSchedule taint beyond the expected startup window. The DRBDResource for a VM shows NodeStartupGate=False.

Cause: DRBD resources on that node have not converged within startupDeadlineSeconds. This can happen on heavily loaded nodes or when the peer site is unreachable.

Fix: The condition clears automatically once DRBD converges; no manual action is required. If the condition persists and the out-of-sync count is not decreasing, check network connectivity between worker nodes (TCP 7000–8000) and check the node agent logs:

# Get pods on the specific node
oc --context cluster1 get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/component=drbd-node-agent \
  --field-selector spec.nodeName=<node-name> -o name

# Stream logs
oc --context cluster1 logs <pod-name> -n trilio-site-recovery-system -c agent -f

If you need more time for DRBD to converge, increase startupDeadlineSeconds on the ProtectionZone:

oc --context tsrmanager patch pz prod -n dr-prod \
  --type merge \
  -p '{"spec":{"nodeAgent":{"startupDeadlineSeconds":300}}}'

oc adm drain does not complete on a node hosting a protected VM

Symptom: oc adm drain <node> hangs indefinitely.

Cause: Draining a node that hosts a running protected VM is blocked by design. The protected VM is pinned to that node for the lifetime of the protection; live migration and cold migration are denied by the workload admission webhooks.

Fix: You have two options:

1. Fail over the ProtectionGroup to move the VM to the DR cluster, then drain.
2. Unprotect the VM (ProtectionRequest deletion), drain the node, then re-protect (which requires a full initial resync).

Do not attempt to bypass the admission webhooks.