IT runbook documentation for the day something fails — OC Security Audit
Runbooks and recovery

IT runbook documentation for the day something fails

Step-by-step procedures for the work your IT does every week and the recovery you hope never to need, written so someone other than the author can follow them.

Know what comes back first, and how — OC Security Audit
Restore order

Know what comes back first, and how

A restore order agreed with the business before an outage, so recovery follows priorities instead of whoever calls first.

Procedures that still open when the network does not — OC Security Audit
Offline copy

Procedures that still open when the network does not

An encrypted offline copy of runbooks, contacts and the restore order, for the outage that takes your documentation platform down too.

25+ yearsIT & cybersecurity experience
Certified expertsCISSP · C|CISO · Microsoft · Cisco
Southern CaliforniaLocal and on-site, based in Irvine
NationwideRemote documentation & audits
From memory to procedure

IT runbook documentation turns know-how into procedure

IT runbook documentation writes down how your IT work is actually done: the steps, the order, the checks and the people to call. Runbooks cover specific technical tasks, such as restoring a server or renewing a certificate. Standard operating procedures (SOPs) cover recurring business processes, such as onboarding a new hire. Together they let someone other than the original author do the work correctly.

In many small firms this knowledge lives in one or two people’s heads. That works until one of them is on vacation during an outage, or leaves. A runbook makes the procedure part of the company instead of part of a person.

OC Security Audit writes and reviews runbooks for businesses in Irvine, across Orange County and Southern California, and remotely nationwide.

IT team planning disaster-recovery steps in a server room
Template

What every runbook contains

The same fields on every runbook make them faster to write, review and follow.

FieldWhat it holds
Title, ID and ownerA clear name, a reference number, the person responsible and the last review date
Purpose and triggerWhat the procedure achieves and the situation in which to use it
Systems affectedLinks to the servers, applications and network items in the asset inventory
PrerequisitesAccess needed (as vault references, never passwords), tools, and approvals required
StepsNumbered, one action per step, with the expected result after each
VerificationHow to confirm the procedure worked, and who confirms it
If a step failsRollback or the point at which to stop and escalate
ContactsWho to call, in order, with after-hours numbers
Test historyWhen the procedure was last tested, by whom, and how long it took

Prerequisites link to the credential index and systems link to the IT asset inventory, so a runbook never goes out of date in two places at once.

Where to start

The runbooks a small firm needs first

Start with the tasks that hurt most when the usual person is not available.

01

Onboarding

Accounts, groups and licenses, device setup, MFA enrollment, access to line-of-business applications, and the security training a new hire completes.

02

Offboarding

Disable sign-in, revoke sessions, transfer the mailbox and files, recover devices, and change every shared credential the person could reach.

03

Backup check and restore

How to confirm last night’s backup, and how to restore a single file, a mailbox and a whole server.

04

Certificate and domain renewals

What expires, when, who renews it, and what breaks if nobody does.

05

Internet failover

How to move to the backup circuit and back, using the details in your WAN circuit documentation.

06

Security incident first steps

Isolate the affected device, preserve logs, call the contact tree, and record who notifies the insurer and legal counsel.

Sample

Sample runbook excerpt

SAMPLE · Harborview Dental Group (fictional company) · illustrative data
Runbook
RB-07 · Restore practice-management server SRV-PM-01 from backup
Owner
J. Ortega · last reviewed 2026-08-20
Trigger
SRV-PM-01 will not start, or its data is damaged or encrypted
Prerequisites
Backup console access (vault reference VLT-0012) · NAS-BK-01 reachable · approval from the practice manager
Steps
1. Confirm the failure and open an emergency change.
2. Notify the practice manager and front desk using the contact tree.
3. In the backup console, select the last good restore point for SRV-PM-01 on NAS-BK-01.
4. Restore to the standby host. Leave the original powered off for investigation.
5. Start the practice-management service and open a test patient record.
6. Tell staff the system is back; record start and end times.
Verification
Front desk confirms today’s schedule loads; the vendor confirms the database version
If a step fails
Stop at step 4 and call the practice-management software vendor’s support line
Last test
2026-08-20 restore test · 2 h 10 min end to end
Sample

Sample restore order

SAMPLE · Harborview Dental Group (fictional company) · illustrative data
TierSystemDepends onAgreed target
1Internet circuit and firewall FW-HQ-01ISP circuitFirst hour
1Core switch SW-CORE-01, DHCP and DNSPower, FW-HQ-01First hour
2Practice-management server SRV-PM-01Tier 1, NAS-BK-01Same business day
2Microsoft 365 mail and TeamsInternetConfirm access; service is cloud-hosted
3Imaging workstations and Imaging-SyncSRV-PM-01Next business day
4File shares and printersTier 1Within two business days

The restore order is agreed with the business owner before an outage, not during one. It is the practical result of what the HIPAA Security Rule calls an applications and data criticality analysis, an addressable part of the contingency plan standard at 45 CFR 164.308(a)(7).

DR and BCP

How runbooks connect to disaster recovery and business continuity

A disaster-recovery plan says what to recover and in what order. Runbooks say how. A business continuity plan says how the office keeps working while IT recovers.

The HIPAA contingency plan standard at 45 CFR 164.308(a)(7) lists five parts, and each one maps to documentation:

  • Data backup plan (required): what is backed up, where, how often, and the backup check runbook
  • Disaster recovery plan (required): the restore order and the restore runbooks
  • Emergency mode operation plan (required): how critical work continues during the outage, such as paper schedules or call forwarding
  • Testing and revision procedures (addressable): the test history on each runbook
  • Applications and data criticality analysis (addressable): the tiers in the restore order

For help designing the backup and recovery capability itself, see our business continuity and disaster recovery services.

When systems are down

Contact tree, offline copy and testing

Recovery documentation has to work in exactly the conditions that break everything else.

01

Incident contact tree

Who declares an incident, then who calls whom: IT lead, MSP, ISP support line, key vendors, the cyber insurance carrier’s claims line and legal counsel. Each entry has a backup person and an after-hours number.

02

Encrypted offline copy

Runbooks, the contact tree, the restore order and the network diagram exported to encrypted media kept off-site. Refreshed after significant changes. See securing your IT documentation.

03

Tabletop exercises

Walk through a scenario, such as ransomware on the server or a dead internet circuit, with the people who would be involved. Gaps in the runbooks show up quickly.

04

Restore tests

Restore a file, a mailbox and a server on a schedule, and record the time each took. We suggest at least one restore test each quarter.

Frameworks and packages

Procedures auditors can review, and your team can use

ISO/IEC 27001:2022 Annex A 5.37 calls for documented operating procedures. The HIPAA Security Rule at 45 CFR 164.316 requires policies, procedures and required records in written form (electronic is acceptable), retained for six years from creation or the date last in effect, whichever is later, and reviewed and updated as needed. Runbooks with owners, review dates and test history are evidence for both. See IT documentation for compliance.

The IT Documentation Build includes runbooks and a restore order for your environment. The IT Documentation Audit reviews the procedures you already have against all 13 domains. Both are $3,500 fixed, one-time. See the IT documentation audit or IT documentation pricing.

Help desk lead guiding a technician through a support runbook
Related

Where to go next

IT change management

Every change updates the runbooks it affects, so procedures match the systems.

Securing your IT documentation

Protect your runbooks and keep an encrypted offline copy for the day the platform is down.

Ali Hassani, CISO, OC Security Audit
CISO-led

Guided by Ali Hassani, CISO

Certified CISO (C|CISO) and CISSP with 25+ years in IT, cybersecurity, compliance and infrastructure, and dozens of Microsoft and Cisco certifications, including MCSE, MCITP, MCSA Security, CCNP and CCNA. Ali writes runbooks so that someone who has never touched the system can follow them at two in the morning.

FAQ

Runbook and recovery questions

What is the difference between a runbook, an SOP and a DR plan?

A runbook is the step-by-step procedure for a technical task. An SOP covers a recurring business process. A DR plan sets what is recovered and in what order, and points to the runbooks that do it.

Who should write our runbooks?

The person who does the task, reviewed by someone who does not. During an IT Documentation Build we write the first versions with your team, so they match how your systems really work.

How often should runbooks be tested?

We suggest a quarterly restore test and an annual tabletop exercise, plus a review after any real incident or significant change.

Where should runbooks be kept?

In your documentation platform, linked to the systems they cover, with an encrypted offline copy for outages that take the platform down.

Will OC Security Audit run our recovery?

We document the procedures and the restore order. Running a recovery is hands-on work for your IT team, your MSP or our implementation partner IT Perfection.

Implementation partner

OC Security Audit documents and audits; it stays independent of the administration of your systems. If you want hands-on work, IT Perfection can fix what the documentation uncovers and keep the records current as part of managed IT.

IT Perfection managed IT
Engineer typing on a laptop in front of a network rack

Write it down before you need it

We will look at which procedures live only in someone’s head today and which to document first. Schedule a consultation.

Irvine, Orange County and Southern California on-site; remote nationwide. Fixed one-time price: $3,500.