Installation

This page walks you through installing Trilio Site Recovery on OpenShift. Installation proceeds in two stages: first, you install the operator on your TSR-Manager cluster using OLM, which deploys the ProtectionZone controller, the TSR web console, and all required CRDs; second, you create zone credentials for your two workload clusters and declare a ProtectionZone CR, which causes the ProtectionZone controller to automatically deploy the TSR-Manager and workload control plane stacks across all three clusters. You do not run Helm commands directly — the Helm charts are embedded in the operator binary and applied by the controller on your behalf.

Before you begin, ensure the following are in place across all three clusters:

Clusters

• One dedicated TSR-Manager cluster running OpenShift 4.17 or later
• Two workload clusters running OpenShift 4.17 or later
• Each workload cluster must serve the kubevirt.io/v1 API (VirtualMachine, VirtualMachineInstance, VirtualMachineInstanceMigration)
• Single-Node OpenShift (SNO) is supported for all three cluster roles

Node requirements (workload clusters)

• Worker nodes must run RHEL CoreOS 9 (el9 kernels). RHEL CoreOS 10 nodes are not validated in Tech Preview.
• The DRBD 9 kernel module will be loaded automatically by the node agent's init container; no manual module installation is required, but nodes must be able to pull images from quay.io/triliodata
• Each worker node must publish an InternalIP address

Network

• TCP ports 7000–8000 inclusive must be open between all primary and DR worker nodes for DRBD replication traffic (one port per replicated VM volume). Port 8000 is mandatory; a firewall rule that stops at 7999 will silently prevent replication on one volume.
• Round-trip latency under 10 ms between workload cluster worker nodes is required for Protocol C synchronous replication (the only supported mode)

Tooling

• OpenShift CLI (oc) — required for OLM installation and support bundle collection
• kubectl — required for creating and managing Site Recovery custom resources
• tsrctl — supplied by Trilio with the release; contact Trilio support to obtain it. Used exclusively to create and manage zone credentials.
• OLM (Operator Lifecycle Manager) — present by default on OpenShift 4.17+
• Cluster-admin rights on the TSR-Manager cluster for OLM installation
• Cluster-admin rights on each workload cluster for credential creation with tsrctl

Storage backend

• ProtectionZone.spec.storageBackendMode must be set to DRBD. The Linstor value is accepted by the CRD schema but rejected by the admission webhook with: Linstor backend is not supported yet; use DRBD

Installation consists of four ordered stages:

1. Install the operator on the TSR-Manager cluster via OLM
2. Obtain tsrctl and create zone credentials for both workload clusters
3. Create the kubeconfig Secrets and ProtectionZone CR on the TSR-Manager cluster
4. Verify that the zone reaches Available and that workload stacks are deployed

Stage 1 — Install the operator via OLM

All OLM resources are created in the trilio-site-recovery-system namespace on your TSR-Manager cluster.

1.1 Create the namespace

oc create namespace trilio-site-recovery-system

1.2 Create a CatalogSource

This registers the Trilio Site Recovery catalog with OLM.

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: tsr
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/triliodata/site-recovery-catalog:<release-tag>
  displayName: Trilio Site Recovery
  publisher: Trilio Data, Inc.

oc apply -f catalogsource.yaml

Replace <release-tag> with the tag provided by Trilio for your release. Do not use a floating tag.

1.3 Create an OperatorGroup

apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: trilio-site-recovery-og
  namespace: trilio-site-recovery-system
spec: {}

The empty spec: {} is required. The operator supports only the AllNamespaces install mode. Setting spec.targetNamespaces selects OwnNamespace or SingleNamespace mode, which the CSV does not support, and the install stalls permanently in Failed with reason UnsupportedOperatorGroup and the message OwnNamespace InstallModeType not supported. If you hit that, remove the field and the install proceeds:

Copy

oc patch operatorgroup trilio-site-recovery-og -n trilio-site-recovery-system \
  --type=json -p '[{"op":"remove","path":"/spec/targetNamespaces"}]'

oc apply -f operatorgroup.yaml

1.4 Create a Subscription

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: site-recovery
  namespace: trilio-site-recovery-system
spec:
  channel: stable
  name: site-recovery
  source: tsr
  sourceNamespace: openshift-marketplace

oc apply -f subscription.yaml

1.5 Verify the operator is running

Wait for OLM to install the CSV and start the operator pods:

# Check the CSV reaches Succeeded
oc get csv -n trilio-site-recovery-system

# Verify the ProtectionZone controller pod is running
oc get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/component=protectionzone-controller

# Verify the TSR web console pods are running
oc get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/component=backend
oc get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/component=frontend

At this point the ProtectionZone CRD is registered on the TSR-Manager cluster. No workload-cluster components have been installed yet.

Stage 2 — Create zone credentials with tsrctl

Each workload cluster must have a zone credential — a kubeconfig Secret on the TSR-Manager cluster — before you can create a ProtectionZone. tsrctl creates the required ServiceAccount, ClusterRole, and time-limited token on each workload cluster, then writes the resulting kubeconfig to standard output for you to store as a Secret on the TSR-Manager cluster.

tsrctl is supplied by Trilio with the release. Contact Trilio support to obtain it.

2.1 Create the credential for cluster1 (primary workload cluster)

Run tsrctl with cluster-admin credentials against the primary workload cluster:

tsrctl zone-credential create \
  --kubeconfig /path/to/cluster1-admin.kubeconfig \
  --zone my-zone

This creates a ServiceAccount in the trilio-site-recovery-access namespace on the workload cluster and emits a kubeconfig. Save the output:

tsrctl zone-credential create \
  --kubeconfig /path/to/cluster1-admin.kubeconfig \
  --zone my-zone > cluster1-zone-kubeconfig.yaml

2.2 Create the credential for cluster2 (DR workload cluster)

tsrctl zone-credential create \
  --kubeconfig /path/to/cluster2-admin.kubeconfig \
  --zone my-zone > cluster2-zone-kubeconfig.yaml

2.3 Inspect a credential

Verify the credential is valid before proceeding:

tsrctl zone-credential inspect \
  --kubeconfig /path/to/cluster1-admin.kubeconfig \
  --zone my-zone

Stage 3 — Create the zone namespace, kubeconfig Secrets, and ProtectionZone

All zone-scoped resources live in a dedicated namespace on the TSR-Manager cluster named dr-<zone-name>.

3.1 Create the zone namespace

oc create namespace dr-my-zone

3.2 Store the zone credentials as Secrets

oc create secret generic cluster1-kubeconfig \
  --from-file=kubeconfig=cluster1-zone-kubeconfig.yaml \
  -n dr-my-zone

oc create secret generic cluster2-kubeconfig \
  --from-file=kubeconfig=cluster2-zone-kubeconfig.yaml \
  -n dr-my-zone

3.3 Create the ProtectionZone CR

The following manifest registers both workload clusters with the TSR-Manager. Creating this CR triggers the ProtectionZone controller to validate cluster reachability, check RBAC and SCC prerequisites, install the TSR-Manager control plane in dr-my-zone, and install the workload control plane (including the DRBD node agent DaemonSet) on both workload clusters.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: my-zone
  namespace: dr-my-zone
spec:
  displayName: "Primary DR Zone"
  description: "Two-cluster DRBD protection zone pairing the primary and DR sites."
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

oc apply -f protectionzone.yaml

Stage 4 — Verify installation

4.1 Watch the ProtectionZone reach Available

oc get pz my-zone -n dr-my-zone -w

The zone progresses through Provisioning → Available. This takes a few minutes as the controller deploys workload stacks on both clusters.

4.2 Verify the ClustersValidated condition

The ClustersValidated condition must be True before you can create ProtectionGroup or ProtectionRequest resources. Check it:

oc get pz my-zone -n dr-my-zone -o jsonpath='{.status.conditions}' | \
  python3 -m json.tool

Look for "type": "ClustersValidated", "status": "True".

4.3 Verify the TSR-Manager control plane on the TSR-Manager cluster

oc get pods -n dr-my-zone \
  -l app.kubernetes.io/component=manager-control-plane

Expect one pod with status 1/1 Running.

4.4 Verify the workload control plane on each workload cluster

Run the following against each workload cluster:

# Workload control plane manager
kubectl get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/component=workload-control-plane \
  --kubeconfig /path/to/cluster1-admin.kubeconfig

# DRBD node agent DaemonSet — one pod per worker node
kubectl get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/component=drbd-node-agent \
  --kubeconfig /path/to/cluster1-admin.kubeconfig

All node agent pods must reach Running. Each pod's init container (drbd-module-loader) loads the DRBD kernel module before the agent starts.

4.5 Verify the default DRBDReplicationPolicy was created

The ProtectionZone controller automatically creates a default replication policy in the zone namespace. You do not create this object yourself:

oc get drp -n dr-my-zone

You should see one policy named my-zone-default with Default: true.

4.6 Access the TSR web console (OpenShift only)

The ProtectionZone controller creates an OpenShift Route for the TSR web console. Retrieve the URL:

oc get route site-recovery -n trilio-site-recovery-system

The console is accessible at https://trilio-site-recovery.<cluster-apps-domain>. Authentication is handled through OpenShift OAuth.

This build does not set a fixed hostname on the Route — the router assigns it, and it is reported under status.ingress, not spec.host. To print the URL directly:

echo "https://$(oc get route site-recovery -n trilio-site-recovery-system \
  -o jsonpath='{.status.ingress[0].host}')"

The Route terminates with reencrypt and redirects, so use https:// directly to avoid the extra hop.

Upgrading to a newer build

Moving to a newer build on the same compatibility line is a single change — patch the CatalogSource image and OLM does the rest:

oc patch catalogsource tsr -n openshift-marketplace --type=merge \
  -p '{"spec":{"image":"quay.io/triliodata/site-recovery-catalog:<new-tag>"}}'

OLM detects the new catalog, generates a fresh InstallPlan (auto-approved by the Subscription above) and rolls the CSV forward in place. You do not need to touch the Subscription, the OperatorGroup, or any ProtectionZone.

In-place upgrade applies from 0.2.0 onward; check the release notes before assuming an older build upgrades in place.

The ProtectionZone CR is the primary configuration object. All workload-cluster and TSR-Manager-side components are configured through its spec, not through chart values files.

ProtectionZone spec fields

Optional tuning fields

Node agent startup behavior

The DRBD node agent applies a siterecovery.trilio.io/not-ready:NoSchedule taint to its node at startup while DRBD resources are converging. The taint is removed when all resources are up or when the deadline passes.

spec:
  nodeAgent:
    startupDeadlineSeconds: 120  # default; range 10–3600

Increasing this value gives DRBD more time to converge on nodes with many replicated volumes before workloads are allowed to schedule. Changing it rolls the node agent DaemonSet once on each workload cluster.

Runtime log levels

You can adjust log verbosity for all zone components without restarting pods. The change propagates via the site-recovery-logging ConfigMap within approximately 15 seconds.

spec:
  logging:
    level: info              # required; one of: error | info | debug | trace
    controlPlaneLevel: debug # optional override for TSR-Manager and workload control planes only
    nodeAgentLevel: info     # optional override for DRBD node agents only

The warn level does not exist in the accepted value set. Use info for normal operations and debug or trace when diagnosing issues.

Hook job retention

Configure how long hook Jobs remain on workload clusters after finishing, so their logs remain accessible via kubectl logs:

spec:
  hooks:
    jobTTLSeconds: 3600          # default: 3600 — how long a succeeded hook Job is kept
    failedJobRetentionSeconds: 604800  # default: 604800 (seven days) — how long a failed hook Job is kept

What the ProtectionZone controller deploys automatically

You do not configure the following directly; the ProtectionZone controller manages them:

• TSR-Manager control plane (site-recovery-manager-control-plane chart) — installed into dr-<zone-name> on the TSR-Manager cluster. Hosts the ProtectionRequest, FailoverRequest, TestFailover, replication monitor, and DRBDReplicationPolicy reconcilers.
• Workload control plane (site-recovery-workload-control-plane chart) — installed into trilio-site-recovery-system on each workload cluster. Hosts the DRBDResource placement reconciler and admission webhooks.
• DRBD node agent DaemonSet — one pod per worker node on each workload cluster, named <release>-site-recovery-workload-control-plane-agent. Manages DRBD kernel state and metadata PVCs.
• Default DRBDReplicationPolicy — named <zone-name>-default with isDefault: true and Protocol C. You do not author or modify replication policies.
• Webhook serving certificates — provisioned and renewed automatically by the ProtectionZone controller. You do not create or rotate these.

RBAC persona ClusterRoles

Three ClusterRoles ship with the OLM bundle. They are inert until bound to users or groups by a cluster administrator.

# Example: grant cluster-wide admin access to a user
oc create clusterrolebinding site-recovery-admin-alice \
  --clusterrole=trilio-site-recovery-admin \
  --user=alice

# Example: grant operator access scoped to one zone namespace
oc create rolebinding site-recovery-operator-bob \
  --clusterrole=trilio-site-recovery-operator \
  --user=bob \
  -n dr-my-zone

Once the ProtectionZone reaches Available and ClustersValidated is True, you are ready to protect virtual machines and manage DR operations. All Day 2 operations are performed by creating and managing custom resources on the TSR-Manager cluster.

Understanding cluster identity in custom resources

The clusterID values cluster1 and cluster2 are the stable identifiers the controllers use. They correspond to whichever clusters you placed in those slots in the ProtectionZone spec. The human-readable name you chose (for example, primary and dr) is for display only.

When creating ProtectionRequest or ProtectionGroup resources, use cluster1 or cluster2 — not the human-readable name.

Checking zone and cluster health

# View all protection zones
oc get pz -n dr-my-zone

# View zone conditions in detail
oc describe pz my-zone -n dr-my-zone

# Check workload cluster reachability status
oc get pz my-zone -n dr-my-zone \
  -o jsonpath='{.status.clusters}' | python3 -m json.tool

The status.clusters[].status field reports Reachable, Unreachable, or Degraded for each workload cluster. The status.conditions array carries detailed reasons when the zone is not Available.

Checking the node agent on workload clusters

The node agent applies and removes the siterecovery.trilio.io/not-ready:NoSchedule taint during startup. To check agent health on a workload cluster:

# List all node agent pods
kubectl get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/component=drbd-node-agent \
  --kubeconfig /path/to/cluster1-admin.kubeconfig

# Stream logs from a specific node agent pod
kubectl logs <pod-name> -n trilio-site-recovery-system \
  -c agent \
  --kubeconfig /path/to/cluster1-admin.kubeconfig -f

To find the agent pod on a specific node:

# Step 1: get the pod name for the node
kubectl get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/component=drbd-node-agent \
  --field-selector spec.nodeName=<node-name> \
  -o jsonpath='{.items[0].metadata.name}' \
  --kubeconfig /path/to/cluster1-admin.kubeconfig

# Step 2: tail its logs
kubectl logs <pod-name-from-step-1> -n trilio-site-recovery-system \
  -c agent \
  --kubeconfig /path/to/cluster1-admin.kubeconfig

Adjusting runtime log levels

You can change the log level of any zone component without restarting pods by patching the site-recovery-logging ConfigMap. The preferred method is to patch the ProtectionZone spec, which propagates the change automatically:

oc patch pz my-zone -n dr-my-zone --type merge \
  -p '{"spec":{"logging":{"level":"debug"}}}'

The change takes effect within approximately 15 seconds on all zone components.

Rotating zone credentials

Zone credentials expire. The ProtectionZone status sets a CredentialExpiring advisory condition approximately 30 days before expiry. To rotate a credential:

# Issue a new token (keeping the same ServiceAccount)
tsrctl zone-credential rotate \
  --kubeconfig /path/to/cluster1-admin.kubeconfig \
  --zone my-zone > cluster1-zone-kubeconfig-new.yaml

# Update the Secret on the TSR-Manager cluster
oc create secret generic cluster1-kubeconfig \
  --from-file=kubeconfig=cluster1-zone-kubeconfig-new.yaml \
  -n dr-my-zone \
  --dry-run=client -o yaml | oc apply -f -

If RBAC on the workload cluster needs to be refreshed (for example after a new release adds permissions), add --refresh-rbac:

tsrctl zone-credential rotate \
  --kubeconfig /path/to/cluster1-admin.kubeconfig \
  --zone my-zone \
  --refresh-rbac

Verifying that the zone is ready to accept ProtectionGroups

Before creating ProtectionGroup or ProtectionRequest resources, confirm the ClustersValidated condition is True. While it is not True, the TSR-Manager webhooks will reject those creates:

oc get pz my-zone -n dr-my-zone \
  -o jsonpath='{.status.conditions[?(@.type=="ClustersValidated")]}'

The zone reports seven conditions. Five of them are stages that feed the overall Ready roll-up; one is informational only:

List them all at once:

oc get pz my-zone -n dr-my-zone \
  -o jsonpath='{range .status.conditions[*]}{.type}{"\t"}{.status}{"\t"}{.reason}{"\n"}{end}'

Keeping the zone credential valid

tsrctl issues bound tokens with a lifetime of about one year (--token-lifetime sets it). When a credential expires, TSR operations against that cluster stop — the zone will not protect VMs or fail over.

The zone raises CredentialsValid with reason CredentialExpiring roughly 30 days before expiry. Treat that as the trigger to rotate, rather than waiting for the credential to lapse.

A credential can also be disturbed without expiring — for example if routine cleanup on a shared cluster removes the ServiceAccount or its RBAC. The zone then reports AuthenticationFailed or InsufficientClusterPermissions after having been Available. Rotate as shown above; if rotate errors naming a missing ServiceAccount, it was deleted outright, so run zone-credential create instead.

Order matters when removing a zone. If a ProtectionZone is stuck Deleting, recreate it before any still-attached ProtectionGroup or ProtectionRequest finishes deleting. Their cleanup finalizer needs a valid, existing ProtectionZone to complete the cluster-side mirror cleanup, and will hang indefinitely if the zone is gone first.

Example 1 — Minimal ProtectionZone manifest

A minimal zone registering two clusters with RHEL CoreOS 9 nodes:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: prod-zone
  namespace: dr-prod-zone
spec:
  displayName: "Production DR Zone"
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

Example 2 — ProtectionZone with logging and extended node agent startup timeout

Useful when worker nodes have many existing DRBD resources that take longer to converge at startup:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: prod-zone
  namespace: dr-prod-zone
spec:
  displayName: "Production DR Zone"
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig
  nodeAgent:
    startupDeadlineSeconds: 300
  logging:
    level: info

Example 3 — Creating zone credentials with tsrctl

# Create credential for the primary workload cluster
tsrctl zone-credential create \
  --kubeconfig /path/to/cluster1-admin.kubeconfig \
  --zone prod-zone > cluster1-zone-kubeconfig.yaml

# Create credential for the DR workload cluster
tsrctl zone-credential create \
  --kubeconfig /path/to/cluster2-admin.kubeconfig \
  --zone prod-zone > cluster2-zone-kubeconfig.yaml

# Inspect the cluster1 credential
tsrctl zone-credential inspect \
  --kubeconfig /path/to/cluster1-admin.kubeconfig \
  --zone prod-zone

Example 4 — Storing zone credentials as Secrets on the TSR-Manager cluster

# Create the zone namespace
oc create namespace dr-prod-zone

# Store credentials as Secrets
oc create secret generic cluster1-kubeconfig \
  --from-file=kubeconfig=cluster1-zone-kubeconfig.yaml \
  -n dr-prod-zone

oc create secret generic cluster2-kubeconfig \
  --from-file=kubeconfig=cluster2-zone-kubeconfig.yaml \
  -n dr-prod-zone

Example 5 — Watching zone provisioning progress

# Apply the ProtectionZone
oc apply -f protectionzone.yaml

# Watch the zone phase (Provisioning -> Available)
oc get pz prod-zone -n dr-prod-zone -w

# Verify ClustersValidated condition
oc get pz prod-zone -n dr-prod-zone \
  -o jsonpath='{.status.conditions[?(@.type=="ClustersValidated")]}{"\n"}'

# List the auto-created default DRBDReplicationPolicy
oc get drp -n dr-prod-zone

# Verify workload control plane on cluster1
kubectl get pods -n trilio-site-recovery-system \
  -l app.kubernetes.io/part-of=site-recovery \
  --kubeconfig /path/to/cluster1-admin.kubeconfig

Example 6 — Binding a persona ClusterRole

# Grant trilio-site-recovery-admin to a specific user (cluster-wide)
oc create clusterrolebinding tsr-admin-alice \
  --clusterrole=trilio-site-recovery-admin \
  --user=alice

# Grant trilio-site-recovery-operator scoped to one zone namespace
oc create rolebinding tsr-operator-bob \
  --clusterrole=trilio-site-recovery-operator \
  --user=bob \
  -n dr-prod-zone

# Grant trilio-site-recovery-viewer cluster-wide (read-only)
oc create clusterrolebinding tsr-viewer-carol \
  --clusterrole=trilio-site-recovery-viewer \
  --user=carol

Example 7 — Rotating a zone credential with RBAC refresh

tsrctl zone-credential rotate \
  --kubeconfig /path/to/cluster1-admin.kubeconfig \
  --zone prod-zone \
  --refresh-rbac > cluster1-zone-kubeconfig-new.yaml

# Update the Secret in-place
oc create secret generic cluster1-kubeconfig \
  --from-file=kubeconfig=cluster1-zone-kubeconfig-new.yaml \
  -n dr-prod-zone \
  --dry-run=client -o yaml | oc apply -f -

Zone stuck in Provisioning

Symptom: oc get pz shows Status: Provisioning for more than five minutes.

Likely causes and fixes:

• ClustersValidated is False. Check oc describe pz <name> -n dr-<name> for the condition reason:

  • InsufficientClusterPermissions — the zone credential is missing required RBAC verbs. Rotate the credential with tsrctl zone-credential rotate --refresh-rbac and update the Secret.
  • KubeVirtNotInstalled — the workload cluster does not serve kubevirt.io/v1. Install OpenShift Virtualization or KubeVirt on the cluster.
  • ClusterAlreadyClaimed — the workload cluster is already registered in another zone. Each workload cluster can belong to only one zone at a time.
  • ClusterIdentityChanged — the credential now reaches a different cluster than the one originally registered (the kube-system namespace UID changed). There is no override; restore the original cluster or re-create the zone.
  • SCCPrerequisiteMissing — the privileged SCC or its ClusterRole does not exist on the workload cluster.

• Kubeconfig Secret missing or mis-keyed. Verify the Secret exists in the zone namespace and that kubeconfigSecretRef.key matches the key inside the Secret.

oc get secret cluster1-kubeconfig -n dr-my-zone -o jsonpath='{.data}' | python3 -m json.tool

Node agent pods stuck in Init

Symptom: Node agent pods on a workload cluster stay in Init:0/1 or Init:CrashLoopBackOff.

Likely cause: The DRBD kernel-module loader init container (drbd-module-loader) cannot find a precompiled kmod matching the node's kernel ABI, or the node cannot pull the loader image.

Fix:

• Confirm the node's kernel is RHEL CoreOS 9 (el9): oc get node <node> -o jsonpath='{.status.nodeInfo.kernelVersion}'
• Confirm the node can reach quay.io/triliodata to pull images
• Check init container logs: kubectl logs <agent-pod> -n trilio-site-recovery-system -c drbd-module-loader --kubeconfig /path/to/cluster-admin.kubeconfig

Zone condition SCCAdmissionDenied

Symptom: The zone condition WorkloadStacksReady=False with reason SCCAdmissionDenied.

Likely cause: The node agent DaemonSet pods were rejected by OpenShift SCC admission after installation. This typically means the agent's dedicated ServiceAccount (<release>-site-recovery-workload-control-plane-agent) was not successfully bound to the privileged SCC.

Fix: Check the DaemonSet events on the workload cluster:

kubectl describe daemonset \
  -l app.kubernetes.io/component=drbd-node-agent \
  -n trilio-site-recovery-system \
  --kubeconfig /path/to/cluster-admin.kubeconfig

Verify the ClusterRoleBinding to system:openshift:scc:privileged exists for the agent ServiceAccount.

ProtectionGroup or ProtectionRequest creation rejected by webhook

Symptom: kubectl apply returns a webhook admission error when creating a ProtectionGroup or ProtectionRequest.

Likely cause: The zone's ClustersValidated condition is not True. The TSR-Manager webhooks block ProtectionGroup and ProtectionRequest creates and spec changes until this condition is satisfied.

Fix: Resolve any zone validation issues first (see "Zone stuck in Provisioning" above), then retry.

oc get pz my-zone -n dr-my-zone \
  -o jsonpath='{.status.conditions[?(@.type=="ClustersValidated")]}{"\n"}'

kubectl get commands return empty on the wrong cluster

Symptom: kubectl get pz or kubectl get pr returns No resources found or server doesn't have a resource type.

Likely cause: You are running the command against the wrong cluster or namespace.

• All TSR-Manager-side resources (ProtectionZone, ProtectionGroup, ProtectionRequest, FailoverRequest, TestFailover, ReplicationGroupStatus, RPOEvent, Hook, DRBDReplicationPolicy) exist on the TSR-Manager cluster in the zone namespace dr-<name>.
• DRBDResource CRs exist on each workload cluster in the protected VM's namespace. Running kubectl get drbdresource against the TSR-Manager cluster returns server doesn't have a resource type because that CRD is not registered there.
• The workload namespace on each workload cluster is trilio-site-recovery-system. Running commands against trilio-site-recovery-system on the TSR-Manager cluster returns empty results.

CredentialExpiring condition on zone

Symptom: oc describe pz shows a CredentialsValid condition with reason CredentialExpiring.

Cause: The zone credential token for one or more workload clusters is within the expiry warning window (default 30 days before expiry). This is advisory and does not change the zone phase.

Fix: Rotate the credential before it expires:

tsrctl zone-credential rotate \
  --kubeconfig /path/to/cluster-admin.kubeconfig \
  --zone my-zone > new-kubeconfig.yaml

oc create secret generic cluster1-kubeconfig \
  --from-file=kubeconfig=new-kubeconfig.yaml \
  -n dr-my-zone \
  --dry-run=client -o yaml | oc apply -f -

Node carries siterecovery.trilio.io/not-ready taint after agent start

Symptom: New workloads are not scheduling on a node that has a running node agent pod. kubectl describe node shows the taint siterecovery.trilio.io/not-ready:NoSchedule.

Cause: The DRBD node agent applies this taint at startup while DRBD resources are converging. The taint is removed as soon as all resources on the node are up, or when spec.nodeAgent.startupDeadlineSeconds (default 120 s) expires.

Fix: Wait for the taint to lift. If it persists beyond the deadline, check the DRBDResource CRs on the workload cluster for conditions indicating why a resource has not converged:

kubectl get dres -A --kubeconfig /path/to/cluster-admin.kubeconfig

If the deadline is consistently too short for your environment, increase spec.nodeAgent.startupDeadlineSeconds in the ProtectionZone spec.