Trilio Site Recovery for OpenStackOpenStack native DR service
Concept

Intended Audience

Operators deploying/running the service and developers calling its API


Overview

This page describes who Trilio Site Recovery for OpenStack is built for and what each audience needs to know to work with it effectively. Whether you are an OpenStack operator deploying and running the service, or a developer integrating with its REST API and Python client, understanding your role shapes which parts of this documentation are most relevant to you.


Content

Who This Documentation Serves

Trilio Site Recovery for OpenStack has two primary audiences: operators who deploy and run the service, and developers who build integrations against its API. Many readers will occupy both roles at different times — an operator writing an automation script is acting as a developer, and a developer who needs to stand up a test environment is acting as an operator.


Operators

Operators are responsible for deploying the service, keeping it running, and onboarding tenants into DR workflows. You are in this group if you:

  • Deploy and configure the protector-api and protector-engine services using Juju, Kolla-Ansible, or OpenShift manifests
  • Manage the service's relationships with Keystone, MariaDB, and RabbitMQ
  • Register DR sites, create tenant mappings, and assign the dr_site_admin and dr_operator Keystone roles
  • Run database migrations via protector-manage db sync or the Juju db-sync charm action
  • Diagnose degraded nodes, failed operations, and inconsistent protection groups
  • Execute site-level failover using openstack dr site failover or openstack dr failover to this site

What operators need to understand

Deployment model. The service is deployed as a standard OpenStack service registered in the Keystone catalogue under the disaster-recovery service type. Depending on your environment, you deploy it with:

  • The trilio-site-recovery-for-openstack Juju charm on Ubuntu Jammy or Noble machines in a Charmed OpenStack model
  • The Kolla install script (deploy/kolla/install.sh) alongside a Kolla-Ansible deployment
  • The bash deploy/openshift/scripts/install.sh script on RHOSO18, which applies eleven of the twelve manifests in the correct order (the twelfth, job-purge.yaml, is uninstall-only) and runs the init Jobs that create the database, RabbitMQ user, and Keystone service registration

Configuration. All service configuration is managed through /etc/protector/protector.conf using oslo.config INI-style sections. The single most important option is local_site in the [DEFAULT] section: the service cannot participate in any DR operation until this is set to the name of the site it belongs to. Under Juju, the charm writes this file and rewrites it on every hook, so configuration changes must go through juju config or the config-flags option — not by editing the file directly.

RBAC. Access control uses oslo.policy. Two custom Keystone roles must exist on both sites before the service starts:

  • dr_operator — allows tenants to manage protection groups and execute DR operations within their own project
  • dr_site_admin — a cloud-wide role for operators who register sites, create tenant mappings, and execute site-level failover

The dr_site_admin role carries no project-owner clause. A holder can register sites and map tenants for every project regardless of which project the role was granted on. Grant it to cloud operators on the admin project, not to individual tenant DR users.

Primary workflows for operators:

  • Reconcile an inconsistent protection group left behind by an interrupted failover or failback: openstack dr protection group reconcile <pg>
  • Roll out a new service version: run protector-manage db preflight before protector-manage db sync when crossing migration 022, then upgrade packages
  • Restore from a database backup: re-run protector-manage db sync after the restore to verify schema consistency
  • Diagnose a degraded node: check the operation record with openstack dr operation show — the source_site, target_site, progress, and error_message fields carry the diagnostic information; there is no estimated completion time for any operation
  • Execute a site-level failover: openstack dr site failover <site> or openstack dr failover to this site

Observability. The service does not emit oslo.messaging notifications. The supported observability surface is polled: use openstack dr replication health, openstack dr replication audit, openstack dr recovery point list, and the operation records from openstack dr operation list / openstack dr operation show.


Developers

Developers write code that calls the Trilio Site Recovery API — either directly over HTTP or through the Python client library. You are in this group if you:

  • Automate DR workflows in scripts or orchestration tools
  • Build monitoring or reporting integrations that query protection group and operation state
  • Write tooling that provisions or tears down protection groups as part of infrastructure lifecycle management
  • Extend or test the service itself

What developers need to understand

Three interaction surfaces. Users interact with the service in three ways:

  1. The openstack dr CLI subcommand family, provided by the python3-triliositerecovery package
  2. The versioned REST API served by protector-api on port 8788, with all endpoints under /v1/{tenant_id}/
  3. The Python client library imported as from triliositerecovery.v1.client import Client

All three are covered in this documentation set. The CLI and API are equivalent surfaces; the Python client is a thin wrapper over the same HTTP calls.

Authentication. The API validates Keystone tokens on every request. Clients must obtain a token from Keystone and present it in the X-Auth-Token header. The service is registered in the service catalogue under the disaster-recovery type; use standard OpenStack service discovery to locate the endpoint.

Python client conventions. When using the client library directly:

  • Import from triliositerecovery.v1.client: from triliositerecovery.v1.client import Client
  • Instantiate with a keystoneauth1 Adapter as session and an explicit endpoint — omitting endpoint causes every request URL to begin with None/ and fail at the first call
  • All managers return plain dicts and lists parsed from JSON, not resource objects: write site['id'], not site.id
  • The resource ID is always a positional argument: call client.operations.failover(pg_id, ...), not client.operations.failover(protection_group_id=...)
  • Network and flavor mappings are dicts at the library level: network_mapping={'src-net-name': 'dst-net-id'}. The src=dst string syntax is argparse-only and belongs to the CLI layer
  • DR operation methods — failover, failback, test_failover, cleanup_test_failover, reconcile, cancel — are on client.operations (the OperationManager), not on client.protection_groups

Key API behaviours developers encounter:

  • POST /v1/{tenant_id}/protection-groups/{id}/action accepts the keys failover, failback, failback_plan, test_failover, test_cleanup, and reconcile. The failback_plan key is read-only and returns HTTP 200 with a plan object rather than starting an operation.
  • InvalidState always returns HTTP 409, on every endpoint including /action
  • ResourceConflict (HTTP 409) is synthesised for any database unique-constraint violation
  • ServiceUnavailable (HTTP 503) is returned for database connectivity errors and is retryable
  • The resource-mappings create endpoint accepts resource_type values network, flavor, floating_ip_pool, and floating_ip; subnet and security_group return HTTP 400
  • Without a tenant mapping, the engine assumes the same project UUID on the peer site. This works for federated Keystone but fails silently for two independent clusters — which is the typical deployment this service exists for. Always create explicit tenant mappings for independent clusters.
  • Recovered instances are renamed: failover creates <instance name>-failover; failback creates <instance name>-restored. The original name is not restored after a round trip.

Network and flavor mappings use = as the separator in CLI arguments (--network-mapping src-name=dst-id). The left side of a network mapping should be the source network name, not its UUID, because captured Nova address metadata is keyed by name. At the library level, pass a dict.

The --probe flag on openstack dr replication health asks the recovery site whether the protection group's networks, security groups, and flavors resolve there as the project it recovers into. It does not test the storage backend or replication lag. Because it queries the peer site, it is slower than the plain command but is the most valuable check to run before a planned failover window.


Shared concerns

Both operators and developers need to understand the protection group lifecycle:

CREATING → ACTIVE → FAILING_OVER → FAILED_OVER → FAILING_BACK → ACTIVE
                \→ ERROR (on failure)

And the constraints on what can be protected:

  • VMs must boot from a Cinder volume (no ephemeral-disk boot)
  • LUKS-encrypted volumes cannot be added to a protection group; use array-side encryption (Pure data-at-rest) instead
  • For async replication, --volume-type is required at protection group creation time; for active_dr replication it is not required
  • Both sites must have Cinder policy relaxed to allow volume_extension:volume_manage, volume_extension:volume_unmanage, volume_extension:services:index, volume_extension:type_get, and volume_extension:type_get_all for the member role, because the service operates as a tenant through a Keystone trust

Examples

Identify your role before sourcing credentials

The test suite illustrates a principle that applies equally to operators and developers: DR operations must run as a tenant (a user with dr_operator), not as admin. Running as admin masks the Cinder policy errors that tenant users encounter in production.

# Source tenant DR credentials — not the admin RC
source drtest-cluster1-openrc.sh   # druser credentials

# Confirm the active user
openstack token issue -f value -c user_id

Verify the service is registered (operator)

openstack catalog show disaster-recovery

Check which site this instance belongs to (operator / developer)

openstack dr site list
# The local_site option in /etc/protector/protector.conf names this instance's site

List all protection groups as a tenant (developer)

openstack dr protection group list

List all protection groups across all projects (operator)

openstack dr protection group list --all-projects

Instantiate the Python client (developer)

from keystoneauth1 import session as ks_session
from keystoneauth1.identity import v3
from keystoneauth1 import adapter
from triliositerecovery.v1.client import Client

auth = v3.Password(
    auth_url='https://keystone.site-a.example.com:5000/v3',
    username='druser',
    password='...',
    project_name='my-dr-project',
    user_domain_name='Default',
    project_domain_name='Default',
)
sess = ks_session.Session(auth=auth)
adp = adapter.Adapter(
    session=sess,
    service_type='disaster-recovery',
    interface='public',
    region_name='RegionOne',
)
endpoint = adp.get_endpoint()
client = Client(session=adp, endpoint=endpoint, region_name='RegionOne')

# Managers return plain dicts — use dict access, not attribute access
for pg in client.protection_groups.list():
    print(pg['id'], pg['name'], pg['status'])

Execute a failover via the Python client (developer)

# DR operation methods are on client.operations, not client.protection_groups
# pg_id is positional; network_mapping is a dict, not a 'src=dst' string
operation = client.operations.failover(
    pg_id,
    failover_type='unplanned',
    network_mapping={'prod-network': 'dr-network-uuid'},
)
print(operation['id'], operation['status'])

Reconcile an inconsistent protection group (operator)

# Repairs a PG left inconsistent by an interrupted failover or failback.
# Works even when the primary site is unreachable.
openstack dr protection group reconcile <pg-id>

Check replication health before a failover window (operator / developer)

# Plain check: storage metrics and status
openstack dr replication health <pg-id>

# With --probe: also asks the recovery site whether networks,
# security groups, and flavors resolve in the target project.
# Slower, but catches permission and mapping gaps before they
# become failover failures.
openstack dr replication health <pg-id> --probe

Inspect a completed or failed operation (operator / developer)

openstack dr operation show <operation-id>
# Fields include: id, protection_group_id, operation_type, status,
# progress (0-100%), source_site (name), target_site (name),
# started_at, completed_at, error_message, and conditionally:
# warnings, floating_ips, restore_point, diverged_copy.
# There is no estimated completion time.

Related concepts
  • Architecture overview — how protector-api and protector-engine relate to each other and to OpenStack services
  • Deployment guide — step-by-step instructions for Juju, Kolla-Ansible, and OpenShift deployments
  • Configuration reference — all protector.conf options by section, with defaults and descriptions
  • RBAC and policy — the dr_operator and dr_site_admin roles, default policy rules, and how to override them with policy.yaml
  • Protection groups — lifecycle, member constraints, replication types, and the --volume-type requirement
  • Failover and failback — planned vs. unplanned failover, recovery point selection, network and flavor mapping
  • Reconciliation — repairing a protection group left inconsistent by an interrupted operation
  • REST API reference — all endpoints under /v1/{tenant_id}/, request and response shapes, and error codes
  • Python client reference — Client instantiation, manager methods, return types, and common pitfalls
  • CLI reference — all openstack dr subcommands with flags and usage notes
  • Site-level failover — using openstack dr site failover and openstack dr failover to this site for full-site DR events