The Recovery Test Should Start at the Teller Line

A practical weekly article for community-bank and credit-union boards and senior leaders on treating recovery tests as business-service evidence, not just technical restoration...

A recovery test can pass on paper while the branch is still dead in the water.

The backup restored. The server came online. The vendor ticket closed. The dashboard turned green. Someone updated the risk committee packet with the phrase “successful test.”

Successful for whom?

For community banks and credit unions, business continuity is not a data-center hobby. It is a promise about member access, cash movement, loan operations, wires, cards, online banking, branches, and the human beings stuck explaining the mess at the counter.

That makes the recovery test a governance artifact.

The board does not need to know every replication setting, storage tier, endpoint image, or network route. Please do not make directors memorize acronyms as punishment for asking good questions.

But the board and senior leaders should know whether the institution can run the business in a degraded state long enough to protect customers, members, staff, liquidity, reputation, and regulatory obligations.

Green infrastructure does not mean green operations

Technology teams naturally test the things they can measure cleanly. Recovery time. Backup integrity. Ticket closure. Server availability. Endpoint restoration. Vendor response. Those are important.

They are also incomplete.

A bank or credit union does not exist to restore servers. It exists to serve people and move money safely. If the teller line cannot verify balances, wire staff cannot confirm release authority, and branch managers do not know what to tell customers, the recovery test has reached the equipment, not the business.

That is why the post-test conversation matters. After a tabletop, failover exercise, ransomware drill, core outage rehearsal, branch connectivity test, or endpoint recovery event, leadership should ask for more than a technical success report.

Ask for the operating story.

Where did work stop? Which roles had authority to switch to manual procedures? Which forms were current? Which customer promises had to change? Which vendors were needed? Which communications were ready?

This is where continuity stops being a binder and starts being management.

Very glamorous stuff. Like plumbing. You only notice it when it ruins your day.

Example one: regulators expect continuity to follow the business

The FFIEC Business Continuity Management booklet frames continuity as an enterprise process, not just an IT recovery exercise. It emphasizes business impact analysis, risk assessment, testing, training, exercises, and board and senior management oversight.

Source: FFIEC IT Examination Handbook, “Business Continuity Management,” November 2019: https://ithandbook.ffiec.gov/it-booklets/business-continuity-management.aspx

That matters because a business impact analysis should connect systems to business processes.

Not “the core is critical,” although yes, thank you, that was not exactly a plot twist.

The useful version is more specific:

1. Which member or customer services stop when this system fails? 2. Which teams need read-only access, manual logs, phone trees, printed procedures, or alternate approvals? 3. Which processes can continue in degraded mode? 4. Which processes must stop because the risk is too high? 5. Who decides when the institution moves from normal operations to contingency operations?

A recovery test should prove those answers are real.

If the disaster recovery report says the core was restored in four hours, the risk committee should also know what happened during those four hours. Could branches handle cash withdrawals? Could wires be held safely? Could debit card issues be triaged? Could the call center explain the situation without inventing policy at high speed?

Example two: CrowdStrike showed why endpoint recovery is an operating issue

The July 2024 CrowdStrike content update incident was not a banking-specific event, but it is a useful reminder for every financial institution that depends on endpoints, cloud services, vendors, and tightly coupled operations. CrowdStrike’s own Preliminary Post Incident Review described how a content update for Windows hosts triggered system crashes and required remediation.

Source: CrowdStrike, “Preliminary Post Incident Review,” July 2024: https://www.crowdstrike.com/wp-content/uploads/2024/07/CrowdStrike-PIR-Executive-Summary.pdf

The governance lesson is not “never use endpoint security.” That would be silly. Endpoint security exists because the alternative is also bad, usually in more expensive ways.

The lesson is that recovery plans need to account for the messy middle.

If a large number of workstations fail at once, the question is not only how IT repairs them. The question is which business functions depend on those workstations, which staff can work from alternate devices, which users have recovery priority, and which decisions require executive authority.

A teller workstation, wire operations workstation, contact center desktop, lending processor laptop, and CEO laptop may all be Windows machines. They do not carry the same business consequence.

Boards should not micromanage the endpoint plan. They should expect management to show how endpoint recovery priorities map to business services.

If every device is priority one, nothing is priority one.

Example three: TSB showed that migration recovery is customer-impact recovery

In 2022, the UK Financial Conduct Authority and Prudential Regulation Authority fined TSB Bank plc over operational resilience failings related to its 2018 IT migration. The FCA final notice described serious disruption, including customers being unable to access accounts and services, and noted weaknesses in planning, governance, testing, and risk management.

Source: FCA Final Notice, “TSB Bank plc,” Dec. 20, 2022: https://www.fca.org.uk/publication/final-notices/tsb-bank-plc-2022.pdf

That was a large UK bank and a major migration. It is not the same fact pattern as a community bank core upgrade, credit union digital banking conversion, or branch network outage.

Still, the lesson travels well.

A technology change is not fully tested until the customer-impact path is tested. Can people access their money? Can staff explain what is happening? Can complaints be captured and routed? Can exceptions be handled? Can leadership see the operational damage while there is still time to act?

Community institutions may have smaller teams and simpler environments, but they also have less spare capacity when something goes sideways. One broken integration can turn into a call-center spike. One missing procedure can become branch-by-branch improvisation. One unclear owner can delay the customer message while everyone waits for someone else to blink first.

That is why recovery tests should include the teller line, the contact center, the operations queue, the digital support team, and the executive communication path.

Not as theater.

As evidence.

What should be in the recovery test packet

A useful packet does not need to bury the board in screenshots.

Start with five exhibits.

Show the critical services that matter to customers and members: cash access, card support, wires, ACH, online banking, mobile banking, loan closings, fraud response, statements, and contact center support. Tie each service to the systems, vendors, roles, and manual procedures it needs.

Define what the institution can still do when the normal system is unavailable. Name what continues, pauses, requires approval, or stops because the risk is too high.

3. Role and authority matrix

Name who can declare contingency mode, approve manual workarounds, prioritize restoration, pause risky activity, release customer messaging, and escalate to regulators or key vendors.

4. Customer and member communication evidence

Show draft messages, branch talking points, contact center scripts, website banners, digital banking notices, and escalation paths. The first customer message should not be written while everyone is already irritated.

5. Test results by business impact

Do not stop at “restored successfully.” Report what failed, which workarounds were used, which queues grew, which procedures were outdated, which staff were confused, and which calls took too long.

That last part is the part people like to soften.

Do not soften it.

The point of a test is to find the ugly bits while they are still cheap.

The board should ask what survived before everything recovered

Recovery governance is not about pretending the institution can avoid every outage. It cannot. Vendors fail. Patches break. Networks drop. Integrations misbehave. Humans click things. Sometimes technology just wakes up and chooses violence.

The board’s job is to make sure management has decided how the institution will protect customers and critical obligations while technology is impaired.

That means the recovery test should start where the business feels pain.

At the teller line. In the call center. In wire operations. In digital support. In the lending queue. In the fraud queue. In the executive update cycle.

If the test cannot explain what happened there, the packet is not finished.

A restored server is good.

A working institution is better.

Board and executive discussion questions

1. During our last recovery test, which customer or member services could continue in degraded mode, which had to stop, and who had authority to make that call? 2. If a workstation, core, digital banking, or vendor outage hit during branch hours, which roles would receive restoration priority and what evidence supports that priority? 3. What did the last test reveal about staff confusion, stale procedures, customer messaging, vendor escalation, or manual workarounds, and who owns the cleanup before the next test?