--- title: Custom Resource Schemas product: trilio-site-recovery-for-kubernetes doc_type: guide version: 0.2.0 source: git2docs (code-derived, validation-filtered) canonical: https://docs-tsr.trilio.io/trilio-site-recovery-for-kubernetes/site-recovery/crd-schema --- # Custom Resource Schemas _CRD structure and validation rules for site recovery resources_ ## Overview This page is the authoritative reference for every Custom Resource Definition (CRD) in the `siterecovery.trilio.io/v1alpha1` API group. It documents each resource's purpose, its full spec and status schema, the validation rules enforced at admission, and the enum values accepted by each field. All DR operations in Trilio Site Recovery are declared through these custom resources — understanding their schemas is essential for configuring protection, triggering failovers, monitoring replication health, and interpreting operator status. Nine CRDs are registered on the TSR-Manager cluster and one (`DRBDResource`) is registered on each workload cluster. ## Prerequisites Before working with these custom resources, ensure the following are in place: - **OpenShift 4.17 or later** on all three clusters (TSR-Manager and both workload clusters). OpenShift 4.14–4.16 are within the supported floor but are not validated for Tech Preview. - **KubeVirt** installed on both workload clusters (the `kubevirt.io/v1` API group must be served with `VirtualMachine`, `VirtualMachineInstance`, and `VirtualMachineInstanceMigration` kinds). - **DRBD 9 kernel module** (`drbd`, `drbd_transport_tcp`, `drbd_transport_lb-tcp`) available on every workload node that will host a protected VM. - **The Trilio Site Recovery operator** installed on the TSR-Manager cluster via OLM (CatalogSource + OperatorGroup + Subscription for package `site-recovery`, channel `tech-preview`). - **A `ProtectionZone` CR** that has reached `Available` phase before any `ProtectionGroup`, `ProtectionRequest`, `FailoverRequest`, `TestFailover`, or `Hook` can be created or modified. The `ClustersValidated` condition on the ProtectionZone must be `True`; while it is not, the TSR-Manager admission webhooks reject those CR creates and spec changes. - **`kubectl`** and, for OpenShift-specific operations, **`oc`**. - **`tsrctl`** (supplied by Trilio with the release; contact Trilio support to obtain it) for provisioning zone kubeconfig credentials before creating a ProtectionZone. - **TCP ports 7000–8000 inclusive** open between primary and DR worker nodes for DRBD replication (one port per replicated VM; port 8000 is mandatory — a firewall rule stopping at 7999 causes silent replication failure on one volume). - **RHEL CoreOS 9 nodes** on workload clusters (RHEL CoreOS 10 is not validated in Tech Preview). ## Configuration All configuration for Site Recovery is expressed through the custom resources described in this page. There are no separate configuration files for end users. The sections below describe the key configuration choices within each CRD. --- ### API group and version All Site Recovery CRDs share the API group and version: ``` siterecovery.trilio.io/v1alpha1 ``` --- ### Cluster placement | CRD | Where it lives | Where to query it | |---|---|---| | `ProtectionZone` | TSR-Manager cluster, `dr-` namespace | TSR-Manager | | `ProtectionGroup` | TSR-Manager cluster, `dr-` namespace | TSR-Manager | | `ProtectionRequest` | TSR-Manager cluster, `dr-` namespace | TSR-Manager | | `DRBDReplicationPolicy` | TSR-Manager cluster, `dr-` namespace | TSR-Manager | | `FailoverRequest` | TSR-Manager cluster, `dr-` namespace | TSR-Manager | | `TestFailover` | TSR-Manager cluster, `dr-` namespace | TSR-Manager | | `Hook` | TSR-Manager cluster, `dr-` namespace | TSR-Manager | | `ReplicationGroupStatus` | TSR-Manager cluster, `dr-` namespace | TSR-Manager | | `RPOEvent` | TSR-Manager cluster, `dr-` namespace | TSR-Manager | | `DRBDResource` | Each workload cluster, protected VM's namespace | Workload cluster only | Running `kubectl get drbdresource` against the TSR-Manager cluster returns `server doesn't have a resource type` because the CRD is not registered there. --- ### ClusterID values Every field that references a cluster slot uses the `ClusterID` enum, which has exactly two values: | Value | Meaning | |---|---| | `cluster1` | The first workload cluster slot | | `cluster2` | The second workload cluster slot | These are stable identifiers used across ProtectionZone, ProtectionGroup, ProtectionRequest, FailoverRequest, and DRBDReplicationPolicy. The human-readable cluster name lives in `ProtectionZone.spec.clusters[].name`. --- ### Replication protocol Only Protocol C (synchronous, RPO = 0) is supported in Tech Preview. The admission webhook rejects `replicationProtocol: A` with the message `ReplicationProtocolA is not supported yet`. Do not attempt to configure asynchronous replication. --- ### Storage backend Only `storageBackendMode: DRBD` is supported. The webhook rejects `Linstor` with `Linstor backend is not supported yet; use DRBD`. --- ### Runtime log levels Log levels for all Site Recovery components can be changed at runtime without pod restarts by editing the `site-recovery-logging` ConfigMap in the relevant namespace. Accepted values are `error`, `info`, `debug`, and `trace`. There is no `warn` level. The global key is `level`; per-component overrides are `level.control-plane`, `level.drbd-node-agent`, and `level.protectionzone`. The ProtectionZone `spec.logging` field propagates these settings automatically. --- ### DRBDReplicationPolicy The ProtectionZone controller creates one default `DRBDReplicationPolicy` named `-default` with `isDefault: true` in the zone namespace when a ProtectionZone is created. You do not create or modify this object. Only the default policy in the ProtectionRequest's namespace is consulted by the controller. ## Examples ### ProtectionZone — minimal example This CR pairs two workload clusters in a zone using the DRBD storage backend. Both clusters run RHEL CoreOS 9 nodes. ```yaml apiVersion: siterecovery.trilio.io/v1alpha1 kind: ProtectionZone metadata: name: primary-dr-zone namespace: dr-primary-dr-zone spec: displayName: Primary DR Zone description: Two-cluster DRBD zone pairing the primary and DR sites. storageBackendMode: DRBD clusters: - name: primary clusterID: cluster1 distributionName: RHEL-CoreOS-9 kubeconfigSecretRef: name: cluster1-kubeconfig key: kubeconfig - name: dr clusterID: cluster2 distributionName: RHEL-CoreOS-9 kubeconfigSecretRef: name: cluster2-kubeconfig key: kubeconfig ``` --- ### ProtectionZone — with logging and node agent tuning ```yaml apiVersion: siterecovery.trilio.io/v1alpha1 kind: ProtectionZone metadata: name: primary-dr-zone namespace: dr-primary-dr-zone spec: displayName: Primary DR Zone storageBackendMode: DRBD clusters: - name: primary clusterID: cluster1 distributionName: RHEL-CoreOS-9 kubeconfigSecretRef: name: cluster1-kubeconfig key: kubeconfig - name: dr clusterID: cluster2 distributionName: RHEL-CoreOS-9 kubeconfigSecretRef: name: cluster2-kubeconfig key: kubeconfig logging: level: info controlPlaneLevel: debug nodeAgentLevel: info nodeAgent: startupDeadlineSeconds: 180 hooks: jobTTLSeconds: 7200 failedJobRetentionSeconds: 604800 ``` --- ### ProtectionGroup — two VMs with boot ordering Both VMs fail over as one atomic unit. `web-server` boots first (lower `bootOrder`); `app-server` boots after. ```yaml apiVersion: siterecovery.trilio.io/v1alpha1 kind: ProtectionGroup metadata: name: web-tier namespace: dr-primary-dr-zone spec: virtualMachines: - name: web-server namespace: production bootOrder: 1 - name: app-server namespace: production bootOrder: 2 sourceCluster: cluster1 desiredState: running sla: rtoTargetSeconds: 300 rpoTargetSeconds: 0 ``` --- ### ProtectionRequest — protect a single VM The `sourceCluster` field is also the failover lever: patching it to the DR cluster triggers a role flip. ```yaml apiVersion: siterecovery.trilio.io/v1alpha1 kind: ProtectionRequest metadata: name: protect-web-server namespace: dr-primary-dr-zone spec: sourceCluster: cluster1 virtualMachine: name: web-server namespace: production replicationConfig: replicationProtocol: C drbdMetadataConfig: storageClassName: fast-ssd size: 128Mi ``` --- ### ProtectionRequest — with DR-side placement override When the DR cluster uses different node labels, use `standbyPlacement` to adjust the scheduling rules applied only to the DR copy. `standbyPlacement` is immutable once set. ```yaml apiVersion: siterecovery.trilio.io/v1alpha1 kind: ProtectionRequest metadata: name: protect-web-server namespace: dr-primary-dr-zone spec: sourceCluster: cluster1 virtualMachine: name: web-server namespace: production replicationConfig: replicationProtocol: C standbyPlacement: nodeSelector: add: topology.kubernetes.io/zone: dr-zone-a remove: - topology.kubernetes.io/zone ``` --- ### FailoverRequest — planned failover to cluster2 `failoverType: planned` stops VMs gracefully on the source cluster, waits for DRBD sync, then promotes the DR side. ```yaml apiVersion: siterecovery.trilio.io/v1alpha1 kind: FailoverRequest metadata: name: failover-web-tier-to-dr namespace: dr-primary-dr-zone spec: protectionGroupRef: name: web-tier targetCluster: cluster2 failoverType: planned batchBootTimeoutSeconds: 300 skipHooks: false ``` --- ### FailoverRequest — unplanned failover `failoverType: unplanned` force-promotes the DR side without waiting for the source to stop cleanly. Use during a site outage when the primary cluster is unreachable. ```yaml apiVersion: siterecovery.trilio.io/v1alpha1 kind: FailoverRequest metadata: name: emergency-failover-web-tier namespace: dr-primary-dr-zone spec: protectionGroupRef: name: web-tier targetCluster: cluster2 failoverType: unplanned ``` --- ### TestFailover — DR drill with automatic cleanup Spins up isolated VM copies from DRBD snapshots in a network-fenced namespace on the DR cluster. Resources are removed automatically after 2 hours. ```yaml apiVersion: siterecovery.trilio.io/v1alpha1 kind: TestFailover metadata: name: drill-web-tier-01 namespace: dr-primary-dr-zone spec: protectionGroupRef: name: web-tier cleanupPolicy: Automatic retentionTime: 2h schedulingCheck: Fail verification: reachability: - type: tcp vmName: web-server port: 443 timeout: 30s - type: http vmName: web-server port: 8080 endpoint: /healthz expectedStatus: 200 timeout: 30s ``` After creating this CR, read the controller-generated test namespace from status: ```bash kubectl get tfo drill-web-tier-01 -n dr-primary-dr-zone \ -o jsonpath='{.status.testNamespace}' \ --context ``` --- ### TestFailover — manual cleanup to preserve logs ```yaml apiVersion: siterecovery.trilio.io/v1alpha1 kind: TestFailover metadata: name: drill-web-tier-02 namespace: dr-primary-dr-zone spec: protectionGroupRef: name: web-tier cleanupPolicy: Manual schedulingCheck: Warn ``` To trigger cleanup while keeping the TestFailover record: ```bash kubectl patch tfo drill-web-tier-02 -n dr-primary-dr-zone \ --type merge \ -p '{"spec":{"cleanupRequested":true}}' \ --context ``` --- ### Hook — reusable lifecycle job definition ```yaml apiVersion: siterecovery.trilio.io/v1alpha1 kind: Hook metadata: name: notify-on-failover namespace: dr-primary-dr-zone spec: description: Posts a Slack notification after failover completes. jobTemplate: image: quay.io/myorg/notifier:v1.2.3 command: ["/usr/bin/notify"] args: ["--channel", "#dr-alerts"] env: - name: SLACK_WEBHOOK_URL valueFrom: secretKeyRef: name: slack-credentials key: webhook-url timeoutSeconds: 60 retries: 2 retryDelaySeconds: 10 timeoutSeconds: 60 ``` Bind this Hook to a ProtectionGroup by adding it to `spec.hooks`: ```yaml hooks: - name: notify-post-failover hookName: notify-on-failover type: post-failover runOn: target onFailure: Continue order: 0 ``` --- ### Inspect ReplicationGroupStatus for a ProtectionGroup The `ReplicationGroupStatus` is auto-created by the replication monitor. Its name is always `-replication`. ```bash kubectl get rgs web-tier-replication -n dr-primary-dr-zone \ -o yaml --context ``` --- ### Inspect RPO audit events filtered by ProtectionGroup ```bash kubectl get rpoe -n dr-primary-dr-zone \ -l siterecovery.trilio.io/protection-group=web-tier \ --context ``` --- ### Inspect DRBDResource status on a workload cluster ```bash kubectl get dres -n production \ --context ``` To get per-node agent pod logs, first find the pod on the relevant node: ```bash kubectl get pods -n trilio-site-recovery-system \ --field-selector spec.nodeName= \ -l app.kubernetes.io/component=drbd-node-agent \ -o jsonpath='{.items[0].metadata.name}' \ --context ``` Then retrieve its logs: ```bash kubectl logs -n trilio-site-recovery-system \ --context ```