Disaster Recovery: RTO, RPO and Continuity

Disaster Recovery (DR) is a set of policies, procedures and tools for the rapid recovery of IT infrastructure after a catastrophic event that causes an interruption of operations.

What Disaster Recovery Is

Disaster Recovery encompasses the planning and execution of processes to restore critical systems, data and operations after disasters. It differs from backup by focusing on the complete recovery of the operating environment, not just data.

DR is a crucial component of Business Continuity Planning (BCP), focusing specifically on the recovery of the technological infrastructure required for business operations.

Types of Disasters

Natural Disasters: Earthquakes, floods, hurricanes, fires.

Technical Failures: Hardware failure, data corruption, software bugs.

Cyberattacks: Ransomware, DDoS, data wiper, sabotage.

Human Errors: Accidental deletions, misconfigurations, operational error.

Infrastructure Failures: Power outages, network failure, cooling problems.

RTO and RPO

Recovery Time Objective (RTO): Maximum acceptable time to restore services after an incident. It determines the required speed of recovery.

Example: an RTO of 4 hours means systems must be operational within at most 4 hours after a disaster.

Recovery Point Objective (RPO): Maximum acceptable data loss measured in time. It defines the required backup frequency.

Example: an RPO of 1 hour means the maximum tolerable loss is the data from the last hour.

DR Strategies

Cold Site: Basic facility with minimal infrastructure. Requires complete configuration before use. RTO: days/weeks. Lowest cost.

Warm Site: Partially configured infrastructure with hardware and connectivity. Requires installation of data and applications. RTO: hours/days. Medium cost.

Hot Site: Complete replica of the production environment, always active and synchronized. Near-instant failover. RTO: minutes. High cost.

Cloud DR: Uses the cloud for replication and recovery. Flexibility and scalability. RTO varies according to configuration.

Disaster Recovery as a Service (DRaaS): Managed DR service in the cloud.

Components of the DR Plan

Business Impact Analysis (BIA): Identify critical systems and the impact of unavailability.

Risk Assessment: Assess the probability and impact of different types of disasters.

Recovery Procedures: Detailed step-by-step procedures for recovery.

Roles and Responsibilities: Define the DR team and clear responsibilities.

Communication Plan: How to communicate during and after a disaster.

Testing Schedule: A regular plan of tests and simulations.

DR Technologies

Data Replication: Synchronous or asynchronous between primary and DR sites.

Snapshots: Point-in-time captures of systems and data.

Failover Automation: Systems automate the switch to the DR environment.

Load Balancers: Distribute traffic and facilitate failover.

Virtual Machine Replication: Replication of VMs between datacenters.

Database Replication: Continuous replication of databases.

Recovery Process

1. Disaster Declaration: Assess the situation and declare activation of the DR plan.

2. Team Activation: Mobilize the DR team according to the plan.

3. Assessment: Evaluate the extent of the damage and affected systems.

4. Failover: Redirect operations to the DR environment.

5. Restoration: Restore data and applications according to prioritization.

6. Validation: Test the functionality of recovered systems.

7. Operation: Maintain operations in the DR environment while the primary is recovered.

8. Failback: Return operations to the primary environment once restored.

DR Testing

Regular testing is essential to validate the DR plan:

Tabletop Exercise: Simulation in a meeting room, without activating systems.

Walkthrough: Detailed review of the procedures with the team.

Simulation Test: Complete simulation without impacting production.

Parallel Test: Activate the DR environment in parallel with production.

Full Interruption Test: Shut down production and operate entirely on DR.

Recommended frequency: at least annually, or after significant changes.

Commercial Solutions

Veeam Backup & Replication: Backup and DR for virtual environments.

Zerto: Continuous replication and DR for VMs and cloud.

AWS Disaster Recovery: DR solutions on AWS.

Azure Site Recovery: DR as a Service from Microsoft.

VMware Site Recovery Manager: DR orchestration for VMware.

Best Practices

• Define realistic RTO and RPO based on BIA
• Document procedures in detail
• Keep documentation accessible offline
• Test regularly and after changes
• Train the team on DR procedures
• Automate as much as possible
• Keep an up-to-date inventory of assets
• Review and update the plan annually
• Consider DR for critical third-party data