Getting Started

This page walks you through installing Trilio Site Recovery on OpenShift and creating your first protected virtual machine. By the end you will have the operator running on your TSR-Manager cluster, a ProtectionZone pairing your primary and DR clusters, and a KubeVirt VirtualMachine actively replicating via DRBD with RPO=0. Site Recovery is currently in Tech Preview; all components share that status.

Before you begin, ensure you have the following in place:

Clusters

• Three OpenShift clusters, each at version 4.17 or later:
  • One TSR-Manager cluster — hosts all orchestration controllers and holds zone credentials; runs no VM workloads
  • One primary (cluster1) workload cluster — runs your active KubeVirt VMs
  • One DR (cluster2) workload cluster — standby target for failover
• Single-Node OpenShift is supported for all three cluster roles

OpenShift version note: The OLM bundle annotation (com.redhat.openshift.versions) in deploy/olm-catalog/site-recovery/metadata/annotations.yaml reads v4.14-v4.19. However, the validated and supported floor for all clusters (TSR-Manager and both workload clusters) is OpenShift 4.17. Versions 4.14–4.16 appear in the installable range but are not supported.

Software on each workload cluster node

• KubeVirt installed and operational
• DRBD kernel module (drbd9) — the drbd-node-agent init container loads it automatically on RHEL CoreOS 9 nodes; RHEL CoreOS 10 nodes are not validated in Tech Preview
• RHEL CoreOS 9 is the validated node OS; RHEL CoreOS 10 is not validated in Tech Preview

Networking

• TCP ports 7000–8000 inclusive open between every primary and DR worker node (one port per replicated VM volume; port 8000 is required — a firewall rule stopping at 7999 will silently prevent replication on one volume)
• Round-trip latency under 10 ms between workload cluster worker nodes (required for Protocol C synchronous replication, which is the only supported mode)

Tools on your workstation

• kubectl — for creating and inspecting custom resources on all three clusters
• oc — required for OpenShift-specific operations (oc adm must-gather, Routes)
• OLM (Operator Lifecycle Manager) — must be running on the TSR-Manager cluster
• tsrctl — supplied by Trilio with the release; contact Trilio support to obtain it

Access

• cluster-admin rights on all three clusters during installation
• Network access from your workstation to all three API servers
• Pull access to quay.io/triliodata from all three clusters

The minimal path from zero to a protected VM:

1. Install the operator on the TSR-Manager cluster via OLM — create a CatalogSource, OperatorGroup, and Subscription targeting the site-recovery package on the stable channel.

2. Create zone credentials — run tsrctl zone-credential create on each workload cluster to mint kubeconfig Secrets the TSR-Manager uses to reach those clusters.

3. Create the zone namespace and kubeconfig Secrets on the TSR-Manager cluster — create the dr-<name> namespace and store the kubeconfig Secrets there.

4. Create a ProtectionZone CR — declare the two workload clusters; the operator automatically installs control-plane stacks on both workload clusters and starts the DRBD node agents.

5. Wait for the zone to reach Available — verify with kubectl get pz -n dr-<name>.

6. Create a ProtectionGroup — declare which VMs to protect together.

7. Create a ProtectionRequest — declare a single VM for protection; it transitions through Syncing to Protected once initial block-level replication completes.

The sections below expand each step in detail.

Step 1 — Install the operator on the TSR-Manager cluster

Site Recovery is delivered as an OLM bundle. Install it by creating three objects on the TSR-Manager cluster. All three live in the trilio-site-recovery-system namespace, which OLM uses as the operator's suggested namespace.

Create the namespace if it does not already exist:

kubectl --context <tsr-manager-context> create namespace trilio-site-recovery-system

Create the CatalogSource. Replace <tag> with the release tag Trilio provides:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: site-recovery-catalog
  namespace: trilio-site-recovery-system
spec:
  sourceType: grpc
  image: quay.io/triliodata/site-recovery-catalog:<tag>
  displayName: Trilio Site Recovery
  publisher: Trilio Data, Inc.

kubectl --context <tsr-manager-context> apply -f catalogsource.yaml

Create the OperatorGroup:

apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: site-recovery-og
  namespace: trilio-site-recovery-system
spec: {}

kubectl --context <tsr-manager-context> apply -f operatorgroup.yaml

Create the Subscription:

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: site-recovery
  namespace: trilio-site-recovery-system
spec:
  channel: stable
  name: site-recovery
  source: site-recovery-catalog
  sourceNamespace: trilio-site-recovery-system

kubectl --context <tsr-manager-context> apply -f subscription.yaml

OLM installs the ProtectionZone controller, the TSR web console frontend, and the Flask backend as CSV-embedded Deployments. Confirm the CSV reaches Succeeded:

kubectl --context <tsr-manager-context> get csv -n trilio-site-recovery-system

The CSV name follows the pattern site-recovery.<version>. When the PHASE column shows Succeeded, the operator is ready.

Step 2 — Mint zone credentials with tsrctl

Each workload cluster is reached through a kubeconfig Secret stored on the TSR-Manager. You create these credentials using tsrctl zone-credential create, which you must run with cluster-admin rights on each workload cluster. tsrctl is supplied by Trilio with the release — contact Trilio support to obtain the binary.

Run tsrctl zone-credential create for the primary workload cluster (cluster1). The --zone flag names the zone this credential will belong to; the --access-namespace flag controls where the ServiceAccount is created on the workload cluster (defaults to trilio-site-recovery-access):

tsrctl zone-credential create \
  --zone <zone-name> \
  --namespace <zone-namespace-on-tsr-manager> \
  --kubeconfig <path-to-cluster1-kubeconfig> \
  --context <cluster1-context>

Repeat for the DR workload cluster (cluster2):

tsrctl zone-credential create \
  --zone <zone-name> \
  --namespace <zone-namespace-on-tsr-manager> \
  --kubeconfig <path-to-cluster2-kubeconfig> \
  --context <cluster2-context>

On each cluster tsrctl creates a ServiceAccount in the trilio-site-recovery-access namespace, a ClusterRole with exactly the permissions the TSR-Manager needs for day-to-day operations, and a time-limited token with the cluster CA embedded. The resulting kubeconfig Secrets are stored on the TSR-Manager cluster in the zone namespace you specify.

Verify the credentials were created:

tsrctl zone-credential inspect \
  --zone <zone-name> \
  --namespace <zone-namespace-on-tsr-manager>

Step 3 — Create the zone namespace and kubeconfig Secrets on the TSR-Manager cluster

Every ProtectionZone lives in its own namespace named dr-<zone-name>. All zone-scoped resources — the ProtectionZone CR, ProtectionGroup and ProtectionRequest CRs, and kubeconfig Secrets — live here.

Create the namespace:

kubectl --context <tsr-manager-context> create namespace dr-<zone-name>

If tsrctl zone-credential create did not automatically place the kubeconfig Secrets in this namespace, create them manually. Each Secret must have a key that holds the kubeconfig content:

kubectl --context <tsr-manager-context> create secret generic cluster1-kubeconfig \
  --from-file=kubeconfig=<path-to-cluster1-kubeconfig> \
  -n dr-<zone-name>

kubectl --context <tsr-manager-context> create secret generic cluster2-kubeconfig \
  --from-file=kubeconfig=<path-to-cluster2-kubeconfig> \
  -n dr-<zone-name>

Step 4 — Create a ProtectionZone CR

The ProtectionZone CR is the top-level configuration object. Creating it triggers cluster validation, RBAC checks, and automated installation of the TSR-Manager control plane stack (in dr-<zone-name>) and the workload control plane stacks (in trilio-site-recovery-system on each workload cluster), including the drbd-node-agent DaemonSet. You do not run any Helm commands — the ProtectionZone controller handles all of that from its embedded charts.

Create the ProtectionZone:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: <zone-name>
  namespace: dr-<zone-name>
spec:
  displayName: "Primary DR Zone"
  description: "DRBD protection zone pairing primary and DR sites."
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

kubectl --context <tsr-manager-context> apply -f protectionzone.yaml

The storageBackendMode must be DRBD. The admission webhook rejects any other value. Each cluster entry requires a unique clusterID (cluster1 or cluster2), a name, and a kubeconfigSecretRef pointing to the Secret you created in Step 3.

The ProtectionZone controller also automatically creates a default DRBDReplicationPolicy named <zone-name>-default with isDefault: true and Protocol C (synchronous, RPO=0) in the zone namespace. You do not create or manage this object.

Step 5 — Wait for the zone to reach Available

Watch the ProtectionZone status:

kubectl --context <tsr-manager-context> get pz -n dr-<zone-name> -w

The STATUS column cycles through Provisioning as the controller installs workload stacks, then reaches Available when:

• Both clusters are reachable and their identities confirmed
• RBAC checks pass for both zone credentials
• The ClustersValidated condition is True
• Workload control plane Deployments and drbd-node-agent DaemonSets are running on both workload clusters

The ClustersValidated condition must be True before the API server accepts any ProtectionGroup or ProtectionRequest creates. Check conditions in detail:

kubectl --context <tsr-manager-context> describe pz <zone-name> -n dr-<zone-name>

Verify the workload stacks on each cluster:

# Check workload control plane on cluster1
kubectl --context <cluster1-context> get deploy,daemonset \
  -n trilio-site-recovery-system \
  -l app.kubernetes.io/part-of=site-recovery

# Check workload control plane on cluster2
kubectl --context <cluster2-context> get deploy,daemonset \
  -n trilio-site-recovery-system \
  -l app.kubernetes.io/part-of=site-recovery

The drbd-node-agent DaemonSet is named <release>-site-recovery-workload-control-plane-agent. Each node carries a siterecovery.trilio.io/not-ready:NoSchedule taint while the agent converges; the taint lifts automatically once DRBD is ready on that node (within startupDeadlineSeconds, default 120 s).

Step 6 — Create a ProtectionGroup

A ProtectionGroup groups one or more KubeVirt VirtualMachines for coordinated failover. All CRs in this step are created on the TSR-Manager cluster in the zone namespace.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: my-first-pg
  namespace: dr-<zone-name>
spec:
  sourceCluster: cluster1
  virtualMachines:
    - name: my-vm
      namespace: default
  sla:
    rpoTargetSeconds: 0
    rtoTargetSeconds: 300

kubectl --context <tsr-manager-context> apply -f protectiongroup.yaml

The sourceCluster field identifies which cluster the VMs are currently running on. virtualMachines lists the KubeVirt VM names and their namespaces on the workload cluster. The sla.rpoTargetSeconds feeds the replication monitor's objective; with Protocol C (the only supported mode) the effective RPO is always 0.

Check the ProtectionGroup:

kubectl --context <tsr-manager-context> get pg -n dr-<zone-name>

Step 7 — Create a ProtectionRequest

A ProtectionRequest declares a single VM for block-level DRBD protection. The controller provisions a DRBDResource pair on both workload clusters, rewires the VM's PVCs to DRBD-backed frontend PVCs, and drives the VM to the Protected phase.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-my-vm
  namespace: dr-<zone-name>
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: my-vm
    namespace: default

kubectl --context <tsr-manager-context> apply -f protectionrequest.yaml

Watch the phase progress:

kubectl --context <tsr-manager-context> get pr protect-my-vm -n dr-<zone-name> -w

The PHASE column transitions through:

Once the phase reaches Protected, the VM's KubeVirt VirtualMachine on the workload cluster is labelled siterecovery.trilio.io/protected-vm=true. The VM is now pinned to its current node for the lifetime of this protection; live migration and node drains are blocked while the VM is protected.

Verify protection on the workload cluster:

kubectl --context <cluster1-context> get vm -A \
  -l siterecovery.trilio.io/protected-vm=true

Inspect the DRBDResource on each workload cluster (the resource is named drbd-<namespace>-<vm>-<hash8>):

kubectl --context <cluster1-context> get dres -n default
kubectl --context <cluster2-context> get dres -n default

The ROLE column shows Primary on the active cluster and Secondary on the standby. The STATE column shows Connected when replication is healthy.

Example 1 — Minimal ProtectionZone manifest

This is the smallest valid ProtectionZone. Replace <zone-name>, Secret names, and cluster names with your values.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: prod-dr
  namespace: dr-prod-dr
spec:
  displayName: "Production DR"
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

Example 2 — ProtectionGroup with boot ordering

When failing over multiple VMs, bootOrder controls which VMs start first on the target cluster. Lower values start earlier; VMs with the same value start in parallel.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: app-stack
  namespace: dr-prod-dr
spec:
  sourceCluster: cluster1
  virtualMachines:
    - name: vm-database
      namespace: production
      bootOrder: 1
    - name: vm-app-server
      namespace: production
      bootOrder: 2
    - name: vm-cache
      namespace: production
      bootOrder: 2
  sla:
    rpoTargetSeconds: 0
    rtoTargetSeconds: 300

Example 3 — ProtectionRequest with custom metadata storage

By default DRBD external-metadata PVCs are 128 Mi and use the cluster default StorageClass. Override both if your cluster requires it:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-db-vm
  namespace: dr-prod-dr
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: vm-database
    namespace: production
  replicationConfig:
    drbdMetadataConfig:
      size: 256Mi
      storageClassName: fast-ssd

The storageClassName for metadata PVCs must bind immediately; WaitForFirstConsumer classes are rejected by the controller during validation because DRBD-managed PVCs have no pod consumer to trigger binding.

Example 4 — Checking replication health

The replication monitor auto-creates a ReplicationGroupStatus named <pg-name>-replication for each ProtectionGroup:

kubectl --context <tsr-manager-context> get rgs -n dr-prod-dr

Filter RPO audit events for a specific group using the label stamped by the replication monitor:

kubectl --context <tsr-manager-context> get rpoe -n dr-prod-dr \
  -l siterecovery.trilio.io/protection-group=app-stack

Example 5 — Rotating a zone credential

Zone credentials carry a time-limited token. When the CredentialExpiring advisory appears on the zone's CredentialsValid condition, rotate the credential. The --refresh-rbac flag also updates the ClusterRole on the workload cluster to match the current permission requirements:

tsrctl zone-credential rotate \
  --zone prod-dr \
  --namespace dr-prod-dr \
  --refresh-rbac

To immediately revoke the previous token after issuing a new one:

tsrctl zone-credential rotate \
  --zone prod-dr \
  --namespace dr-prod-dr \
  --refresh-rbac \
  --revoke-previous

Example 6 — Accessing the TSR web console (OpenShift only)

On OpenShift the ProtectionZone controller creates an oauth-proxy Route named site-recovery in the operator namespace. The console is reachable at:

https://trilio-site-recovery.<cluster-ingress-domain>

Browser access requires OpenShift; no ingress or authentication path is provided on upstream Kubernetes.

ProtectionZone stays in Provisioning or Unavailable

Symptom: kubectl get pz -n dr-<zone-name> shows STATUS as Provisioning or Unavailable for more than a few minutes.

Likely causes and fixes:

1. ClustersValidated condition is False — InsufficientClusterPermissions The zone credential authenticates but lacks required API permissions. The condition message names the missing verbs. Rotate the credential with --refresh-rbac to update the ClusterRole:

   Copy

   tsrctl zone-credential rotate \
     --zone <zone-name> \
     --namespace dr-<zone-name> \
     --refresh-rbac
   

2. ClustersValidated condition is False — ClusterIdentityChanged The credential now reaches a cluster whose kube-system Namespace UID differs from the UID recorded when the zone first claimed that cluster. There is no override. You must restore the original cluster or delete and re-create the zone.

3. ClustersValidated condition is False — ClusterAlreadyClaimed The workload cluster is already leased to another zone (the siterecovery.trilio.io/owner-zone annotation on its trilio-site-recovery-system Namespace points to a different zone). A cluster can belong to only one zone at a time.

4. ClustersValidated condition is False — KubeVirtNotInstalled KubeVirt is not detected on the workload cluster. Install KubeVirt before creating the zone.

5. WorkloadStacksReady condition is False — SCCAdmissionDenied The drbd-node-agent DaemonSet was rejected by OpenShift SCC admission after install. Check the DaemonSet events on the workload cluster:

   Copy

   kubectl --context <cluster-context> get events \
     -n trilio-site-recovery-system \
     --field-selector reason=FailedCreate
   

   Ensure the zone credential's ClusterRole includes use on system:openshift:scc:privileged and that the dedicated agent ServiceAccount is the sole subject of that binding.

6. ClustersValidated condition is False — SCCPrerequisiteMissing The privileged SCC or its associated ClusterRole does not exist on the workload cluster. This SCC ships with OpenShift; verify the cluster is a supported OpenShift version (4.17+).

ProtectionRequest stuck in Syncing

Symptom: The PHASE column stays at Syncing for an extended period; SYNC percentage is not advancing.

Likely causes and fixes:

1. DRBD replication port blocked TCP ports 7000–8000 must be open between all worker nodes on the primary and DR clusters. Port 8000 is required — a firewall rule stopping at 7999 silently prevents replication on one volume. Verify that all 1001 ports in the range are open in both directions.

2. drbd-node-agent not running on the VM's node The agent pod may be in Init (module loader failed) or CrashLoopBackOff. Check:

   Copy

   # Find the agent pod on the specific node
   kubectl --context <cluster-context> get pods \
     -n trilio-site-recovery-system \
     --field-selector spec.nodeName=<node-name> \
     -l app.kubernetes.io/component=drbd-node-agent
   
   # Then inspect logs
   kubectl --context <cluster-context> logs <agent-pod-name> \
     -n trilio-site-recovery-system -c agent
   

   If the init container drbd-module-loader fails, the DRBD kernel module could not be loaded. Ensure the node is running a supported RHEL CoreOS 9 kernel. RHEL CoreOS 10 is not validated in Tech Preview.

3. Node still tainted The agent applies siterecovery.trilio.io/not-ready:NoSchedule to its node until DRBD converges. Check node taints:

   Copy

   kubectl --context <cluster-context> describe node <node-name> | grep Taints
   

   If the taint persists beyond startupDeadlineSeconds (default 120 s), inspect the DRBDResource conditions on that cluster.

4. DRBDResource reports RoleConflict The ProtectionRequest shows Ready=False with reason RoleConflict. Both sides have promoted to DRBD Primary simultaneously. Check the DRBDResource on both clusters and contact Trilio support.

ProtectionRequest reaches Failed

Symptom: kubectl get pr -n dr-<zone-name> shows PHASE as Failed.

Fix: Describe the ProtectionRequest for condition details:

kubectl --context <tsr-manager-context> describe pr <name> -n dr-<zone-name>

The conditions array explains the failure. Common causes include: the VM namespace not found on the workload cluster, the VM referencing a StorageClass not available on both clusters, or a WaitForFirstConsumer StorageClass specified for DRBD metadata PVCs (which must bind immediately). Re-protecting always requires a full unprotect followed by a new ProtectionRequest and a full initial resync.

TSR web console not accessible

Symptom: Navigating to https://trilio-site-recovery.<cluster-ingress-domain> returns an error.

Likely causes and fixes:

1. The Route is only created on OpenShift. If you are on upstream Kubernetes, browser access is not supported.
2. The ProtectionZone must have reached Available before the controller creates the Route and ConsoleLink. Check the zone phase first.
3. Verify the Route exists:
   Copy

   oc --context <tsr-manager-context> get route site-recovery \
     -n trilio-site-recovery-system
   

4. The oauth-proxy requires that you are authenticated with your OpenShift credentials. Ensure your OpenShift user has been bound to one of the three persona ClusterRoles (trilio-site-recovery-admin, trilio-site-recovery-operator, or trilio-site-recovery-viewer).

kubectl get dres returns "server doesn't have a resource type"

Symptom: Running kubectl get dres against the TSR-Manager cluster returns a "server doesn't have a resource type" error.

Cause: DRBDResource CRs and their CRD are registered only on each workload cluster, not on the TSR-Manager cluster. Always query DRBDResources using a workload cluster context:

kubectl --context <cluster1-context> get dres -n <vm-namespace>
kubectl --context <cluster2-context> get dres -n <vm-namespace>

Collecting a support bundle

If you need to open a support case, collect a full diagnostic bundle from the TSR-Manager cluster. The oc adm must-gather command uses the site-recovery-must-gather image and reaches through to all zone workload clusters automatically. Replace <tag> with your release tag:

oc adm must-gather \
  --image=quay.io/triliodata/site-recovery-must-gather:<tag> \
  -- /usr/bin/gather

To collect only the last 2 hours of logs:

oc adm must-gather \
  --image=quay.io/triliodata/site-recovery-must-gather:<tag> \
  -- /usr/bin/gather --since 2h