Concept

Key Concepts

Kubernetes concepts and site-recovery-specific terminology such as recovery plans, failover, and failback

Overview

This page defines every term you will encounter when reading Site Recovery documentation, creating custom resources, or diagnosing a failing operation. Understanding these concepts — particularly how the TSR-Manager cluster relates to workload clusters, how DRBD replication is configured, and how failover operations are declared — is prerequisite reading before you follow any procedural guide. Each term maps to a concrete Kubernetes object, controller behavior, or infrastructure component.

Content

Cluster topology

TSR-Manager cluster

The TSR-Manager cluster is a dedicated Kubernetes (OpenShift) cluster that hosts all Site Recovery orchestration state. It runs the ProtectionZone controller, the TSR-Manager control plane, and the TSR web console. It holds every TSR-Manager-side custom resource — ProtectionZone, ProtectionGroup, ProtectionRequest, FailoverRequest, TestFailover, Hook, DRBDReplicationPolicy, ReplicationGroupStatus, and RPOEvent — as well as the kubeconfig Secrets used to reach workload clusters.

The TSR-Manager cluster runs no VM workloads and carries no DRBD replication traffic. It is required in every deployment; there is no supported configuration that omits it.

All TSR-Manager-side resources, including the controller pods themselves, live in a per-zone namespace named dr-<deployment-name>.

Workload clusters

A Site Recovery zone pairs exactly two workload clusters: one primary (active) cluster and one DR (standby) cluster. Each cluster occupies a fixed deployment slot identified by a ClusterID of either cluster1 or cluster2. These identifiers are stable for the lifetime of the zone — they appear in ProtectionRequest and FailoverRequest specs, and the failover mechanism uses them as the lever for role flips.

Each workload cluster runs the workload control plane and the drbd-node-agent DaemonSet in the trilio-site-recovery-system namespace. KubeVirt VirtualMachines and their persistent volumes live on workload clusters; the TSR-Manager cluster never holds VM workloads.

Protection zone namespace

Every ProtectionZone occupies its own namespace on the TSR-Manager cluster, named dr-<deployment-name>. This namespace contains the ProtectionZone CR, all ProtectionGroup and ProtectionRequest CRs for that zone, the kubeconfig Secrets for both workload clusters, and the TSR-Manager controller pods. A workload cluster can belong to only one zone at a time; the zone-exclusivity lease is an annotation (siterecovery.trilio.io/owner-zone-namespace-uid) on the trilio-site-recovery-system namespace of the workload cluster.

Storage and replication

Storage backend mode

Every ProtectionZone declares a storageBackendMode in its spec. The only supported value in Tech Preview is DRBD. The schema also admits Linstor, but the admission webhook rejects it: Linstor backend is not supported yet; use DRBD. Every supported deployment therefore uses the DRBD Operator backend, which manages DRBD replication via the drbd-node-agent DaemonSet and the DRBD kernel module on each workload node.

Protocol C / synchronous replication

Site Recovery replicates VM persistent volumes at the block level using DRBD. The only supported wire protocol in Tech Preview is Protocol C (synchronous replication). With Protocol C, a write is acknowledged only after both the primary and secondary sides have confirmed it, delivering RPO = 0. The admission webhook rejects replicationProtocol: A with ReplicationProtocolA is not supported yet.

Because replication is synchronous only, the RPO is always 0 and is not tunable. The rpo field on a ProtectionRequest must be 0; a non-zero value with Protocol C is rejected at admission.

Protocol C requires round-trip latency under 10 ms between workload cluster worker nodes.

DRBDReplicationPolicy

A DRBDReplicationPolicy CR defines DRBD replication parameters — wire protocol, network and disk tuning, storage class mappings — applied to VMs in a zone. You do not create this object. When a ProtectionZone reaches the Available phase, its controller creates one default policy named <zoneName>-default with isDefault: true in the zone's namespace. Only the default policy in a ProtectionRequest's namespace is ever consulted.

The policy's spec.clusters is a two-entry list. Each entry identifies a cluster by its ClusterID (cluster1 or cluster2) and may specify storage class mappings that control how source PVC storage classes are translated to target storage classes when provisioning replica PVCs on the DR cluster.

The spec.networkConfig.portRange field is accepted by the API but is not read by the port allocator. Ports are always allocated from the fixed range TCP 7000–8000 inclusive.

DRBDResource

A DRBDResource is a workload-cluster-side CR that represents all of a single VM's replicated disks as one DRBD resource. The ProtectionRequest controller creates a pair — one DRBDResource per cluster — when a VM is first protected. Both copies share the same name, port, volume numbers, and replication configuration, and differ in their side (Source or Target), role (Primary or Secondary), and local PVC names.

DRBDResource CRs are named drbd-<namespace>-<vm>-<hash8> and live in the VM's namespace on each workload cluster. They are not registered on the TSR-Manager cluster; kubectl get drbdresource against the TSR-Manager cluster returns "server doesn't have a resource type".

The spec.side field records the protection origin of the copy (Source = the cluster the data originated on) and is immutable — it does not change on failover. The spec.role field records the desired DRBD role and is updated by the ProtectionRequest controller on a role flip.

Metadata PVCs backing each DRBD volume's external metadata are named drbd-md-<resourceName>-v<volumeNumber>.

DRBD replication port range

DRBD replication uses TCP ports 7000–8000 inclusive, with one port allocated per protected VM. Port 8000 is allocatable; a firewall rule that stops at 7999 will silently prevent replication for one VM. These ports must be open between primary and DR worker nodes.

Protection objects

ProtectionZone

The ProtectionZone CR is the top-level configuration object that registers a pair of workload clusters with the TSR-Manager. Creating a ProtectionZone triggers cluster reachability validation, RBAC verification, and automated installation of the TSR-Manager and workload control plane stacks on all clusters. The zone's spec.storageBackendMode must be DRBD. The spec.clusters list must contain exactly two entries, each with a unique clusterID (cluster1 or cluster2) and a reference to a kubeconfig Secret.

The ProtectionZone carries a ClustersValidated condition. While this condition is not True, the TSR-Manager admission webhooks reject new ProtectionGroup and ProtectionRequest creates and spec changes. This condition is the gate for the entire DR configuration workflow.

Zone-level logging, hook retention windows, and node agent startup timeout are all configurable on the ProtectionZone spec and propagate to all components in the zone without pod restarts.

ProtectionGroup

A ProtectionGroup CR groups one or more KubeVirt VirtualMachines for coordinated failover as an atomic unit. It lives on the TSR-Manager cluster in the zone namespace. The spec.virtualMachines list references VMs by name and namespace. Boot ordering within a failover is controlled by the optional bootOrder field on each VM reference — lower values boot first, VMs sharing a value boot in parallel.

The ProtectionGroup carries an SLA block (spec.sla) where rpoTargetSeconds feeds the replication monitor's objective and is reflected in ReplicationGroupStatus.spec.rpoObjectiveSeconds. rtoTargetSeconds is recorded for compliance reporting.

The group's observed state is reported in status.state (Protected, Syncing, Degraded, or Failed) and status.activeClusterID, which records which workload cluster slot currently holds the running VMs.

ProtectionRequest

A ProtectionRequest CR declares that a single KubeVirt VM should be protected. It lives on the TSR-Manager cluster and drives the full VM protection lifecycle: the controller provisions a DRBDResource pair on both workload clusters, wires the VM to DRBD-backed frontend PVCs, and advances through the phases Pending → Validating → CreatingDRBD → Syncing → ReadyToActivate → Activating → Protected.

The spec.sourceCluster field is the failover lever. Patching it to the DR cluster's ID triggers a DRBD role flip — the controller promotes the DR copy to Primary and demotes the former primary — constituting a failover for that VM. The FailoverRequest controller uses this mechanism and is the only component that should patch spec.sourceCluster.

A protected VM is pinned to one node per cluster for the lifetime of the protection. Node name is immutable once set. Live migration, cold migration, and oc adm drain of a node hosting a running protected VM are all denied.

Re-protecting a VM always requires a full unprotect followed by a full initial resync.

The spec.standbyPlacement field allows node-level scheduling rules on the DR copy of the VM (nodeSelector overrides, node affinity changes, toleration additions/removals) to be declared at protection time. This field is immutable once set.

FailoverRequest

A FailoverRequest CR triggers a planned or unplanned failover of all VMs in a ProtectionGroup to a specified target cluster. It lives on the TSR-Manager cluster in the zone namespace.

Planned failover (failoverType: planned): graceful, waits for DRBD sync to complete, stops source VMs cleanly, then promotes the DR copy. Targets zero data loss.
Unplanned failover (failoverType: unplanned): force-promotes the DR copy without waiting for sync, prioritizing RTO. Because the supported replication mode is DRBD Protocol C (synchronous, RPO=0), every write the source acknowledged is already on the target, so an unplanned failover does not lose acknowledged data. What can be lost is writes still in flight at the moment of failure and anything the guest OS had not yet flushed to disk — the DR copy is a crash-consistent recovery point, exactly as if the VM had lost power.

The FailoverRequest controller patches each member ProtectionRequest's spec.sourceCluster and monitors the DRBD Primary role on the target side to confirm completion. Every failover is initiated by creating a FailoverRequest; failover never happens automatically.

Per-VM progress is tracked in status.vmStatuses. The overall operation state is status.state (Pending, InProgress, Completed, Failed, or Cancelled) and status.phase provides granular tracking through Initializing, Validating, RunningPreHooks, StoppingSource, WaitingForLocks, RemovingTaints, StartingTarget, Monitoring, RunningPostHooks, Completed, and Failed.

The spec.drainTimeoutSeconds field is accepted and range-checked by the admission webhook but has no effect on drain or failover behavior in Tech Preview.

TestFailover

A TestFailover CR triggers a non-disruptive DR validation drill on the non-active workload cluster. The controller creates DRBD snapshots of the protected VMs' volumes, provisions isolated copies of the VMs in a controller-generated namespace on the standby cluster, runs verification checks, and reports results — without touching production VMs.

The test namespace is generated by the controller as site-recovery-<protection-group>-<random> and reported in status.testNamespace. You cannot specify the test namespace in the spec.

Test VMs are fenced by a default-deny NetworkPolicy applied to the generated namespace. On OpenShift with OVN-Kubernetes, this isolation is enforced. On non-OVN CNI configurations, isolation silently fails.

TestFailover is supported in DRBD backend mode only.

The lifecycle phases are: Queued → PreparingEnvironment → CreatingSnapshots → CreatingVolumes → CreatingVMs → StartingVMs → Verifying → Succeeded (or Failed) → CleaningUp → Cleaned.

Cleanup is controlled by spec.cleanupPolicy (Automatic or Manual) and spec.retentionTime. Setting spec.cleanupRequested: true triggers cleanup while preserving the TestFailover CR as a record.

Hook

A Hook CR is a reusable, namespaced definition of a Kubernetes Job to run at specific lifecycle points of a failover, failback, or test-failover operation. Hook types include pre-failover, post-failover, pre-failback, post-failback, pre-test, post-test, and test-verification.

Hooks are bound to a ProtectionGroup via spec.hooks[]. The admission webhook prevents spec edits and deletion of a Hook while any operation has a Pending or Running plan entry naming it. Hooks have no status and no controller.

Hook execution is tracked in the FailoverRequest or TestFailover status.hookStatuses[], which is snapshotted from the ProtectionGroup at validation time so mid-flight edits cannot reshape a running plan.

Observability objects

ReplicationGroupStatus

A ReplicationGroupStatus CR is auto-created by the replication monitor controller for each ProtectionGroup. It aggregates DRBD replication metrics polled from the active workload cluster: out-of-sync bytes, connection state, current RPO, sync counts, and promotion eligibility. It is named <protection-group-name>-replication and lives on the TSR-Manager cluster.

The spec.rpoObjectiveSeconds field is set from the ProtectionGroup's spec.sla.rpoTargetSeconds and drives status.rpoStatus (Met, Degraded, Violated, or Unknown).

Note that status.volumeReplicationStatuses contains one entry per VM (one per DRBD resource), not one per disk. A VM with three disks contributes one entry because all its disks are volumes of a single DRBD resource.

Filter ReplicationGroupStatus objects using the label siterecovery.trilio.io/protection-group — field selectors on spec.* or status.* are not supported on custom resources.

RPOEvent

An RPOEvent CR is an immutable audit record created by the replication monitor when a replication status transition occurs — for example, sync completed, RPO degraded or violated, connection lost or restored, or resync started or completed. RPOEvent objects are used for SLA compliance reporting and post-incident analysis.

Because RPOEvent is immutable, any update attempt is rejected by the admission webhook. Deletion is restricted to the replication monitor service account.

RPOEvent objects carry the label siterecovery.trilio.io/protection-group and can be filtered by it.

Credentials and access

Zone credential

A zone credential is a kubeconfig Secret on the TSR-Manager cluster that grants the orchestration controllers access to one workload cluster's API server. Zone credentials are created, rotated, inspected, and deleted exclusively by the tsrctl zone-credential subcommand. The credential is bound to a specific cluster by the kube-system Namespace UID recorded at first claim and cannot be redirected to a different cluster.

Credentials are stored in the trilio-site-recovery-access namespace on each workload cluster and referenced as Secrets in the zone's dr-<name> namespace on the TSR-Manager cluster.

The tsrctl CLI is the only tool that manages zone credentials. It is supplied by Trilio with the release; contact Trilio support to obtain it.

ClusterIdentityChanged

ClusterIdentityChanged is a condition reason on the ProtectionZone CR indicating that the zone credential for a workload cluster now reaches a cluster whose kube-system Namespace UID differs from the UID recorded when the zone first claimed that cluster. There is no override. The only resolution is to restore the original cluster or re-create the zone.

ClustersValidated condition

The ClustersValidated condition on the ProtectionZone CR must be True before the TSR-Manager admission webhooks accept new ProtectionGroup or ProtectionRequest creates or spec changes. It gates the entire DR configuration workflow for a zone. Related condition reasons include ClusterAlreadyClaimed (the workload cluster is leased to another zone), InsufficientClusterPermissions (the credential lacks required RBAC), ClusterIdentityChanged, SCCPrerequisiteMissing, and KubeVirtNotInstalled.

Runtime components

ProtectionZone controller

The ProtectionZone controller runs on the TSR-Manager cluster in the zone namespace. It validates cluster reachability, identity, and RBAC for each zone; Helm-installs the TSR-Manager and workload control plane stacks from charts embedded in the operator binary; and server-side-applies CRDs on every reconcile. Customers do not run Helm commands directly — chart installation is fully automated by this controller once the zone reaches Available.

TSR-Manager control plane

The TSR-Manager control plane (component label: manager-control-plane) runs on the TSR-Manager cluster. It hosts the reconcilers for ProtectionRequest, FailoverRequest, TestFailover, pg-sync, replication monitor, and DRBDReplicationPolicy — the controllers that drive all DR lifecycle operations.

Workload control plane

The workload control plane (component label: workload-control-plane) runs on each workload cluster in the trilio-site-recovery-system namespace. It hosts the DRBDResource placement reconciler and admission webhooks for VM and DRBDResource guards, and exposes metrics.

drbd-node-agent DaemonSet

The drbd-node-agent DaemonSet runs one pod per worker node on each workload cluster (identified by label app.kubernetes.io/component=drbd-node-agent). It owns drbdadm configuration and .res files, provisions frontend PV-PVCs and metadata PVCs on its node, manages the DRBD kernel state, and applies a siterecovery.trilio.io/not-ready:NoSchedule taint to its own node during startup. The taint is removed once all DRBD resources on the node have converged or when the startup deadline (spec.nodeAgent.startupDeadlineSeconds, default 120 seconds) expires, whichever comes first.

The DaemonSet object name follows the pattern <release>-site-recovery-workload-control-plane-agent.

TSR web console

The TSR web console (Next.js frontend + Flask backend) is deployed via OLM on the TSR-Manager cluster. It provides dashboards for zone health, VM protection status, and one-click failover and test-failover operations. On OpenShift, it is accessible via an OAuth-proxy Route at https://trilio-site-recovery.<cluster ingress domain>. The console is not accessible on upstream Kubernetes — no ingress or authentication path is provided.

The three UI personas are delivered as OLM-shipped ClusterRoles: trilio-site-recovery-admin, trilio-site-recovery-operator, and trilio-site-recovery-viewer. What a user can do is decided per capability by SelfSubjectAccessReview with the caller's token.

tsrctl

tsrctl is the only Site Recovery CLI tool. It is used exclusively to create, rotate, inspect, and delete zone kubeconfig credentials via the zone-credential subcommand. It does not drive DR operations — failover, protection, and all other lifecycle operations are performed through CRDs. tsrctl is supplied by Trilio with the release; contact Trilio support to obtain it.

tsr-gather (support bundle collector)

The site-recovery-must-gather container image is the diagnostic bundle collector for Site Recovery. Run via oc adm must-gather on the TSR-Manager cluster, it collects logs, CRs, DRBD state, and configuration from the TSR-Manager and all zone workload clusters into a single read-only archive. The entrypoint inside the image is /usr/bin/gather.

Runtime configuration

site-recovery-logging ConfigMap

The site-recovery-logging ConfigMap controls the runtime log level of Site Recovery components without pod restarts. It is a per-namespace ConfigMap. The global key is level; per-component overrides use level.control-plane, level.drbd-node-agent, and level.protectionzone. Accepted values are error, info, debug, and trace. Changes take effect within approximately 15 seconds.

The ProtectionZone spec's spec.logging block propagates these settings to all components in the zone.

Scale limits

Up to approximately 1,000 protected VMs per deployment (one DRBD port per VM from the range 7000–8000).
Up to 15 replicated disks per VM.
Exactly two workload clusters per zone.
One ProtectionZone per namespace.
A 5-minute failover wall-clock budget.

Examples

Minimal ProtectionZone manifest

The following manifest creates a ProtectionZone pairing two OpenShift clusters in the DRBD backend mode. The kubeconfig Secrets (cluster1-kubeconfig and cluster2-kubeconfig) must already exist in the dr-prod namespace on the TSR-Manager cluster before or immediately after the zone is created.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: prod
  namespace: dr-prod
spec:
  displayName: Production DR Zone
  storageBackendMode: DRBD
  clusters:
    - name: cluster1
      clusterID: cluster1
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
      distributionName: RHEL-CoreOS-9
    - name: cluster2
      clusterID: cluster2
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig
      distributionName: RHEL-CoreOS-9

Protecting a VirtualMachine

A ProtectionRequest declares that a single KubeVirt VM running in the vm-apps namespace on cluster1 should be protected. The sourceCluster field identifies which deployment slot currently runs the VM.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-web-server
  namespace: dr-prod
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: web-server
    namespace: vm-apps
  replicationConfig:
    replicationProtocol: C
    rpo: 0

Grouping VMs for coordinated failover

A ProtectionGroup groups multiple VMs for failover as a unit, with boot ordering defined per VM.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: web-tier
  namespace: dr-prod
spec:
  desiredState: running
  virtualMachines:
    - name: web-server
      namespace: vm-apps
      bootOrder: 2
    - name: database
      namespace: vm-apps
      bootOrder: 1
  sla:
    rpoTargetSeconds: 0
    rtoTargetSeconds: 300

Triggering a planned failover

A FailoverRequest moves all VMs in a ProtectionGroup from cluster1 to cluster2 using a graceful, planned failover.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: web-tier-failover
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: web-tier
    namespace: dr-prod
  targetCluster: cluster2
  failoverType: planned
  skipHooks: false

Running a test failover

A TestFailover validates DR readiness without disrupting production VMs. The test namespace and target cluster are chosen automatically by the controller.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: web-tier-drill
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: web-tier
    namespace: dr-prod
  cleanupPolicy: Automatic
  retentionTime: 2h
  schedulingCheck: Fail

Checking replication health

Query the auto-created ReplicationGroupStatus for a ProtectionGroup. Use the short name rgs.

kubectl get rgs web-tier-replication -n dr-prod -o yaml

Listing RPO events for a ProtectionGroup

kubectl get rpoe -n dr-prod -l siterecovery.trilio.io/protection-group=web-tier

Inspecting a DRBDResource on a workload cluster

DRBDResource CRs live on workload clusters, not the TSR-Manager cluster. Use the short name dres.

kubectl get dres -n vm-apps --context workload-cluster1

Checking protected VMs with the label selector

The label siterecovery.trilio.io/protected-vm=true is stamped on KubeVirt VirtualMachine objects on workload clusters.

kubectl get vm -A -l siterecovery.trilio.io/protected-vm=true --context workload-cluster1

Adjusting runtime log levels without a restart

Patch the site-recovery-logging ConfigMap in the zone namespace to enable debug logging for the control plane:

kubectl patch configmap site-recovery-logging -n dr-prod \
  --type merge \
  -p '{"data":{"level":"info","level.control-plane":"debug"}}'

Related concepts

Install and configure the control plane — how the ProtectionZone controller and TSR-Manager control plane are deployed via OLM and how the ProtectionZone CR drives automated stack installation
Provision zone credentials — how tsrctl zone-credential mints and rotates the kubeconfig Secrets that allow the TSR-Manager to reach workload clusters
Create a ProtectionZone — step-by-step guide to creating the ProtectionZone CR and verifying the ClustersValidated condition
Protect a virtual machine — how to create a ProtectionRequest and monitor the VM through to the Protected phase
Monitor replication health — how to read ReplicationGroupStatus and RPOEvent objects for SLA reporting
Run a test failover — how to create a TestFailover CR and interpret the scheduling check and verification results
Execute a planned failover — how to create a FailoverRequest and confirm the role flip on both clusters
Execute an unplanned failover — how unplanned failover differs from planned and when to use it
Fail back to the original cluster — how to return VMs to the original cluster after a failover by reversing the FailoverRequest direction
Rotate zone credentials — how to use tsrctl zone-credential rotate to renew expiring kubeconfig Secrets
Collect a support bundle — how to run oc adm must-gather with the site-recovery-must-gather image to collect a diagnostic archive
Diagnose a degraded node or stalled protection — how to interpret DRBDResource conditions, node agent taints, and ProtectionRequest phases during troubleshooting