How to Build a Disaster Recovery Plan That Actually Works Under Pressure

The worst recovery I ever worked on started with a voicemail at 2:14 in the morning. A storage controller at a regional professional services firm had failed, and the failover array refused to take over. The good news was that the company had a disaster recovery plan. The bad news was where that plan lived: on the very file server that had just gone dark.

We got them back online, but it took most of two days instead of the six hours their executives believed they had bought. Nobody on that call was careless. The plan simply had never met a real crisis before that night.

I have spent most of my career as a disaster recovery specialist and later as a DR manager, and that story is the rule rather than the exception. Most organizations do not lack a disaster recovery plan. They lack one that holds up when people are tired, systems are down, and the phone keeps ringing. This article is about closing that gap.

What a Disaster Recovery Plan Actually Is

A disaster recovery plan is the documented, tested set of procedures your organization follows to restore technology, applications, and data after a serious disruption. It answers three questions in order: what do we recover first, how do we recover it, and who does the work.

People often confuse it with a business continuity plan. They are related but not the same. Continuity covers how the business keeps operating, including people, facilities, suppliers, and manual workarounds. Disaster recovery is the technology engine underneath it. Ready.gov makes the point clearly: technology recovery strategies should restore hardware, applications and data in time to meet business recovery needs, and teams should build an IT disaster recovery plan together with the business continuity plan.

That pairing matters more than most teams realize. If your continuity plan says the billing department can work manually for 48 hours, but your DR plan restores billing in 12, you have overspent. When the numbers run the other way, you have a crisis waiting to happen. The two documents must agree with each other.

Why Most Disaster Recovery Plans Fail When the Pressure Hits

In my experience, plans fail for boring reasons, not dramatic ones. Someone wrote them to pass an audit. They describe an environment that changed two years ago. They assume the one engineer who knows the database will be available, rested, and reachable.

The data backs this up. Uptime Institute’s outage research has tracked these failures for years, and its 2025 analysis found that 54% of respondents said their most recent significant outage cost more than $100,000, with 20% reporting costs above $1 million. Even more telling for anyone who writes procedures: the share of human error outages caused by failure to follow procedures rose by ten percentage points compared with 2024.

Read that last statistic again. Procedures existed. People did not follow them. Sometimes that is a discipline problem, but far more often the procedure was unclear, outdated, or impossible to execute in the real situation. When I review a failed recovery, the question I ask is never “why didn’t they follow the plan?” It is “what about the plan made it hard to follow?”

There is also a quieter cause of failure: the plan protects the wrong things. I once audited a firm that had flawless recovery for its email platform and nothing documented for the licensing server that every engineering workstation depended on. Email came back in an hour. Nobody could do billable work for three days.

Start Your Disaster Recovery Plan With the Business Impact Analysis

Every strong disaster recovery plan I have built started in a conference room with department heads, not in a server room. The business impact analysis, or BIA, is where you find out what actually matters and how fast it matters.

The Questions Every BIA Should Ask

A good BIA asks each business unit a simple set of questions:

• Which processes generate revenue, serve clients, or satisfy legal obligations?
• How long can each process be down before the damage becomes serious?
• How much data can each process afford to lose?
• What systems, vendors, and people does each process depend on?

From those answers come the three numbers that drive everything else. NIST’s contingency planning guidance treats them as outputs of the BIA itself, and RTO, RPO, and Maximum Tolerable Downtime emerge from the BIA and drive recovery strategy selection.

The Three Numbers That Drive Your Disaster Recovery Plan

Here is how I explain them to executives:

Recovery Time Objective (RTO): How long until the system is working again. If your RTO for client billing is four hours, the clock starts when you declare the disaster and stops when users can bill again.

Recovery Point Objective (RPO): How much data you can afford to lose, measured in time. An RPO of 15 minutes means you can lose, at most, the last 15 minutes of transactions.

Maximum Tolerable Downtime (MTD): The point where the outage stops being painful and starts being fatal, whether through lost clients, regulatory penalties, or contract breaches. Your RTO must always sit comfortably inside your MTD.

A word of advice from the field: department heads will tell you everything is critical. They are not lying; they just only see their own corner. Your job is to push back gently with cost. When a manager learns that a 15 minute RPO costs five times more than a four hour RPO, the real priority list appears quickly.

Set Disaster Recovery Plan Objectives You Can Actually Defend

Once the BIA is done, I group systems into recovery tiers. This keeps the plan readable and prevents the classic mistake of trying to restore 200 systems at once.

Notice what sits at the top of Tier 1: identity. This is the lesson teams miss most often. If your directory service or single sign on platform is down, or internal DNS stops resolving, nothing else matters, because nobody can log in to recover anything. I have watched skilled teams restore a database perfectly and then sit idle for hours because no one could authenticate to the application that used it.

The other thing to defend is the gap between your stated objectives and your proven ones. An RTO written in a policy is a wish. An RTO measured in a test is a fact. I keep two columns in every plan I manage: target and last tested result. When those numbers drift apart, that is my work queue for the next quarter.

Build Your Backup Strategy Around the Attacker, Not the Accident

Ten years ago, most disasters I responded to were accidents: failed hardware, a flooded server room, an administrator deleting the wrong volume. Today, the scenario that keeps DR managers awake is ransomware, and it changes how you must design backups.

Why the Classic Backup Rule Needs an Upgrade

The classic guidance is the three, two, one backup rule: three copies of your data, on two different types of media, with one copy offsite. It is still the foundation, and CISA continues to recommend it as a backup strategy. The problem is that attackers now know the rule too. Many modern ransomware strains hunt down and encrypt backup targets before they attack primary data.

That is why the stronger version, often written as three, two, one, one, zero, has become my baseline. It adds one offline or air gapped copy that can survive ransomware aimed at connected backup systems, and zero errors, meaning your team verifies and tests every backup before it is needed.

Backup Controls Every Disaster Recovery Plan Needs

In practice, that means a few non negotiable controls:

• Immutable storage. Immutable backups stay locked for a set retention period, so no one can modify, encrypt, or delete them, including ransomware and administrators holding full credentials.
• Separate credentials. Backup administration accounts should never share passwords, identity providers, or management networks with production. At minimum, isolate backup infrastructure on its own VLAN. If one stolen login reaches both, you do not have a backup; you have a second target.
• Longer retention than you think. Attackers often sit quietly inside a network for days or weeks. If your retention window is shorter than their dwell time, every restore point you have may already be infected.
• Restore testing, not backup reporting. A green checkmark on a backup job only proves the software copied data. It does not prove you can bring a working system back from it.

I tell every client the same thing: you do not have backups until you have restored from them.

Write Your Disaster Recovery Plan for the Person Who Will Use It at 3 A.M.

This is where most disaster recovery plan documents go wrong. Their authors write for auditors, filling pages with policy language and architecture diagrams, when the real reader is a stressed engineer with a dead system, a manager on the phone, and two hours of sleep.

Here is how I structure a plan that people can actually use under pressure.

Define Exactly When the Plan Is Activated

Ambiguity at the start burns hours. Teams spend precious time debating whether a problem is “a disaster” while the clock runs. The UK Ministry of Justice guidance gets this right by requiring each plan to state the point at which teams should use the recovery plan, along with a list of people who can take the role of recovery manager.

I write declaration criteria as plain triggers, backed by alerts from your network monitoring tools so the clock starts on data, not on someone noticing. For example: “If any Tier 1 system is unavailable for more than 30 minutes with no confirmed fix in progress, the on call lead contacts the DR manager, who decides within 15 minutes whether to declare.” No interpretation required.

Assign Disaster Recovery Plan Roles, Not Names Alone

People go on vacation, change jobs, and get sick. Every role in the plan should list a primary, a backup, and a third option. The roles I always include are the recovery manager who owns decisions, technical leads for each tier, a communications lead, and a scribe who logs every action and timestamp. That last role sounds minor. It is not. The log becomes your evidence for insurers, regulators, and your own after action review.

Write Runbooks as Checklists

Each system gets its own runbook, written as numbered steps a competent engineer could follow without having built the system. Include the exact commands, the order of operations, the dependencies that must be running first, and how to confirm success. If a step says “restore the database,” it is not finished. A complete step names the backup set, the server, the command, and what a successful result looks like.

Keep Copies Where the Disaster Cannot Reach

Remember my 2 a.m. story. Store the plan in at least three places: a secure cloud location outside your main identity provider, encrypted copies on the laptops of key responders, and a printed binder at a secondary location. The MoJ guidance adds that everyone named in the plan should know where it lives and be able to access it. That sounds obvious until you test it and discover half your team cannot.

The Communication Layer Most Disaster Recovery Plans Forget

Technical recovery is only half the incident. The other half is people asking questions, and if you do not answer them, they will interrupt the engineers who are trying to fix things.

Ready.gov notes that when an emergency happens, customers want to know how it affects them, the business must notify regulators and local officials, and employees and their families will be worried. Plan for all three audiences before you need to.

What Every Disaster Recovery Plan Communication Section Needs

These are the elements I build into every plan:

• Out of band channels. If your email and chat platform are down or compromised, how does the team talk? Keep a pre agreed fallback, such as a separate messaging workspace or a phone bridge that does not rely on corporate systems.
• Prepared message templates. Draft holding statements for clients, staff, and partners in advance. Under pressure, nobody writes clear prose. Fill in the blanks instead.
• A single voice. One communications lead approves all outbound updates. Mixed messages from different managers damage trust faster than the outage itself.
• A regular update rhythm. Even “no change, next update in 60 minutes” calms people down. Silence breeds rumor.
• Regulatory timelines. Depending on your industry and region, some incidents trigger notification deadlines. Your legal counsel should map these before an event, not during one.

Why Clients Care More Than You Think

For professional services firms in particular, client communication is the asset you are really protecting. Clients forgive outages. They rarely forgive being kept in the dark.

Test Your Disaster Recovery Plan Until It Breaks

A disaster recovery plan that has never faced a test is a hypothesis. TechTarget puts it bluntly: if an organization does not invest time and resources in testing, there is a real chance the plan will not work as expected when it is needed.

There are three basic types of disaster recovery testing, and I use all of them on a rotating schedule.

Tabletop Exercises

Stakeholders gather and walk step by step through every component of the plan. No one touches live systems. I run these quarterly, usually for about two hours, with a realistic scenario and a few surprises injected along the way. Halfway through, I will announce that the recovery manager has lost cell signal, or that the backup vendor’s support line has a four hour wait. The goal is not to embarrass anyone; it is to find the assumptions nobody questioned.

Partial or Functional Tests

Here you actually restore specific systems into an isolated environment and time it. This is where your RTO and RPO meet reality. I have seen a “two hour” restore take nine hours because nobody accounted for the time needed to rehydrate data from cold storage. Better to learn that on a Tuesday afternoon than during a real event.

Full Disaster Recovery Plan Tests

The most demanding test is a complete failover, where production runs from your recovery environment. These carry real risk and cost, so most organizations run them once or twice a year for Tier 1 systems. They are worth it. Nothing else reveals hidden dependencies the way a full test does.

After every test, I hold an honest review. We look at what worked, what broke, what took longer than planned, and who owns each fix by what date. A test that produces no action items was probably too easy.

Keep Your Disaster Recovery Plan Alive

The most common failure I find in audits is not a bad plan. It is a good plan that went stale. The infrastructure team migrated servers, a SaaS platform replaced an internal app, and the network team reorganized, but the DR plan still describes the world from three years ago.

The MoJ guidance offers a sensible rule: teams should review plans regularly and update them whenever systems, services, personnel, or communication chains change. I go one step further and wire DR into change management, which in most organizations runs through the service desk. Any change request that affects a Tier 1 or Tier 2 system must include a short answer to one question: “Does this change the recovery procedure?” If the answer is yes, the runbook update becomes part of the change, not an afterthought.

I also schedule a formal annual review with business owners to revisit the BIA. Priorities shift. The system that was Tier 3 last year might now support your biggest client.

A Practical Disaster Recovery Plan Build Sequence

If you are starting from scratch or rebuilding a plan that no longer holds up, this is the order I recommend:

1. Secure executive sponsorship. Recovery needs budget and authority. Without a named executive owner, the plan will stall.
2. Run the business impact analysis. Interview every department and document processes, dependencies, and tolerances.
3. Set recovery tiers and objectives. Assign RTO, RPO, and MTD to every critical system, and get business owners to sign off.
4. Inventory everything. Map systems, data stores, vendors, licenses, and the dependencies between them. A mature IT asset management practice makes this far easier. As the IONOS guide on creating an IT disaster recovery plan notes, documenting all IT resources forms the basis of every step that follows.
5. Design recovery strategies. Choose backup architecture, secondary sites or cloud recovery, and vendor arrangements that meet your objectives at a cost the business accepts. Do not forget the WAN links your offices will need to reach the recovery site.
6. Write the plan and runbooks. Cover declaration criteria, roles, communication procedures, and step by step recovery checklists.
7. Distribute and train. Make sure every responder can find the plan without relying on production systems.
8. Test on a schedule. Run tabletop exercises quarterly, functional tests for critical systems at least twice a year, and full tests as risk and budget allow.
9. Review after every test and every incident. Turn findings into owned, dated action items.
10. Maintain through change management. Treat the plan as a living system, not a document.

Final Thoughts From the Recovery Room

I have been on enough recovery calls to know that disasters rarely look like the scenario in the plan. Floodwater reaches the building you thought was safe. Ransomware arrives through a vendor you trusted. Meanwhile, the person who knows the critical system is on a flight with no WiFi.

A disaster recovery plan does not work under pressure because it predicted the exact disaster. It works because its owners built it on honest business priorities, protected it against modern threats, wrote it for real people, and tested it until the weak spots surfaced. Pressure exposes every shortcut. The goal is to find those shortcuts yourself, on your own schedule, before an outage finds them for you.

Start with the BIA. Protect your backups like an attacker is looking for them, because one probably is. And test, test, test.

Frequently Asked Questions

What is the difference between a disaster recovery plan and a business continuity plan?

A business continuity plan covers how the whole organization keeps operating during a disruption, including people, facilities, and manual workarounds. A disaster recovery plan focuses on restoring technology, applications, and data. You should build them together, as the Ready.gov business preparedness resources recommend.

How often should a disaster recovery plan be tested?

At a minimum, run a tabletop exercise quarterly and functional restore tests for critical systems at least twice a year. Full failover tests are often annual. The TechTarget guide to disaster recovery testing explains the three main test types in detail.

What are RTO and RPO in disaster recovery?

RTO, or Recovery Time Objective, is how quickly a system must be working again after a disaster. RPO, or Recovery Point Objective, is how much data loss, measured in time, the business can tolerate. Both come from your business impact analysis, as N-able’s overview of disaster recovery testing describes.

How does ransomware change disaster recovery planning?

Ransomware actively hunts backups, so at least one copy must be offline or immutable, and backup credentials must stay separate from production. The AvePoint guide to the three two one backup rule covers the expanded rule built for this threat.

Who should own the disaster recovery plan?

An executive sponsor should own the budget and authority, while a DR manager or recovery lead owns the plan day to day. Every role should have a named primary and backup. If you rely on an outside partner for IT support, write their role and response times into the plan too. The UK Ministry of Justice IT disaster recovery guide recommends listing everyone who can step in as recovery manager.

Is there a standard framework for building a disaster recovery plan?

Yes. The most widely used reference is NIST’s contingency planning guidance, which walks through business impact analysis, strategy development, testing, and maintenance. You can read it through the NIST Contingency Planning Guide for Federal Information Systems, which is useful well beyond government agencies.

References

• NIST Contingency Planning Guide for Federal Information Systems
• Ready.gov: Ready Business Planning Resources
• Uptime Institute: Annual Outage Analysis Report 2025
• Data Center Knowledge: Data Center Outages Decline for Fourth Straight Year, But Issues Persist
• TechTarget: How to Conduct a Disaster Recovery Test
• N-able: Disaster Recovery Test Methods, Steps, and Cadence
• UK Ministry of Justice: IT Disaster Recovery Plan and Process Guide
• IONOS Digital Guide: Creating an IT Disaster Recovery Plan
• AvePoint: The Three Two One Backup Rule Complete Guide
• NinjaOne: The Three Two One Backup Rule Explained

About the Author

Sarah Whitmore

Administrator

Technology writer for Foretec.com, covering IT infrastructure, networking, cybersecurity, and practical technology solutions for modern businesses.

View All Posts