Designing, conducting, and evaluating tabletop exercises: A primer on optimizing this important planning tool

(CIDRAP Source Weekly Briefing) – The US Centers for Disease Control and Prevention (CDC) devoted 2 days last week to testing its response to a possible outbreak of pandemic influenza on American soil. During the drill, participants were told that 12 people in four states had become ill, infected with a new strain of the deadly H5N1 virus, and 25% of them had died. The CDC had to consider shutting US borders and closing schools.

Reports of such scenario-based exercises seem to surface with increasing frequency, often from governmental agencies, highlighting the importance of exercising pandemic plans. A company ready to test its plans can choose from a variety of options—orientations, drills, functional or full-scale exercises, or tabletops—depending on what aspect of its plan it wants to test and its stage of preparedness.

Tabletop exercises, one of the most talked-about ways to challenge and examine pandemic plans, can sharpen group problem-solving under pressure and elevate your company's preparedness—provided you properly design, carefully conduct, fully evaluate, and actually use the results of your test. Although no tabletop exercise can convey a realistic picture of a pandemic, it helps executives and planners find the plan's gaps.

Because a virtual cottage industry has emerged of consultants ready to help companies examine and test plans, you need to pick carefully. A comprehensive, vetted list does not yet exist, says Kristine Moore, MD, MPH, medical director of the Center for Infectious Disease Research and Policy (CIDRAP) at the University of Minnesota (publisher of Weekly Briefing). Moore and colleague Jill DeBoer, MPH, associate director of CIDRAP, experienced firsthand the growing demand for credible information at CIDRAP's 2007 Business Preparedness for Pandemic Influenza Second National Summit in February. Their tabletop exercise workshops filled to capacity with continuity planners, many from Fortune 500 companies and full of questions.

In their training sessions, Moore and DeBoer caution that just as a detailed pandemic preparedness plan is not an indicator that a company is prepared for a pandemic, a tabletop exercise is not necessarily the best way to test the plan. The Mar 15, 2007, issue of Weekly Briefing addressed how to decide what kind of exercise best suits your company and its stage of readiness.

If you choose a tabletop, Moore and DeBoer recommend this 10-step process.

1. Review your pandemic business continuity plan

First, make sure your company's plan is comprehensive and ready to be tested. "People come to me and say they want to test their plans, but when I start working with them we realize the plan is too general, does not deal with specific details, and is not ready to be tested," Moore says. In such a case, you may opt to conduct an educational tabletop to help flesh out the details of the plan, DeBoer adds.

Keep in mind that the reason to conduct a tabletop is to make sure your plans are feasible, comprehensive, and workable during an actual pandemic. As you conduct your review, pay special attention to the portion of the plan that details the area you're most interested in testing—telecommunications, for example.

2. Define a goal for the exercise

Goals can range from ensuring that essential job functions remain operational during a pandemic to exploring human resources policy changes. The goal should be matched to your company's past, current, and future planning activities, DeBoer says. If procedures are not yet defined, for example, your goal may be to educate the participants about the specific challenges posed by a pandemic to identify any needed procedures. Goals can range from specific to comprehensive.

Sun Microsystems Inc, based in Santa Clara, Calif., has conducted two exercises to test communications and decision-making processes at the local, regional, and corporate levels. Karen Dye, global crisis planning manager at Sun and aWeekly Briefing editorial board member, says her company picked this goal because decision making is extremely decentralized at Sun; communications between different countries and executives at the corporate headquarters are rare. Sun, which provides servers, software, and computer services, keeps a general checklist with possible goals for its tabletops that includes:

  • Identifying and understanding the gaps in current plans
  • Setting a baseline for performance prior to the exercise
  • Having floors inspected and reported on in X minutes
  • Having communication links operational at all assembly locations

3. Form an exercise design team

Assign the task of designing the exercise to a small group who are a natural fit for the job, DeBoer says. To test executive communications, for example, pick employees who are familiar with how executives communicate. Participants in the exercise should not be on the design team. To develop an effective exercise, the typical design team members will need to meet for at least 1 hour about four to six times.

DeBoer also recommends choosing what stage of a pandemic the exercise will simulate. "Pandemic influenza is a long-term problem, but during a tabletop exercise you only have a small amount of time, so it is best to focus the energy on one phase of the pandemic," she says.

4. Develop exercise objectives

Specific objectives help you define a scenario, ensure that participants share an understanding of what to accomplish, provide a way to organize the modules, and form the basis for the after-action report.

Sun used the following objectives when it conducted its tabletop: (1) identify gaps in the plan and inconsistencies with corporate crisis management and crisis communications, (2) walk through the plan and identify what needs to be changed, (3) identify what training is needed, (4) test policy and determine if it is feasible, (5) test reporting of absentees, (6) determine practicality of country-specific policies, and (7) evaluate capability in the event of phase 6 of the World Health Organization's (WHO's) stages of pandemic alert.

DeBoer and Moore recommend using the mnemonic SMART when identifying objectives. Each objective should be:

  • Simple: Phrase language simply and clearly.
  • Measurable: Set the level of performance so results are observable.
  • Achievable: Make sure the objective can be achieved using the resources your organization can commit.
  • Realistic: Present a realistic expectation of the exercise.
  • Task-oriented: Focus on a behavior or procedure, ideally an individual issue.

5. Develop the exercise scenario

The scenario should be based on the objectives, say Moore and DeBoer. During Sun's first tabletop exercise, for example, participants in Bangalore, India; Singapore; and Menlo Park, Calif., were presented with a scenario designed to test the communication objective. Participants each were asked to explain how they would react to the situations, Dye says.

During a second exercise, participants from the company's headquarters in California and the Toronto office met to make decisions during a scenario in which the group received reports of 11 cases of respiratory illness outside of Toronto. Two of the cases were identified as H5N1 influenza, and the press was reporting on the situation. In addition, the group was told the country manager was ill and unable to work.

6. Identify the players

Participants may be employees or other people critical to the company's business continuity. These might include the CEO, COO, and participants from the finance, human resources, and public relations departments, as well as contractors, vendors, and supply-chain partners. Select 25 to 40 participants.

7. Decide on a format

Base the format on your goal and objectives. Sample formats Moore and DeBoer suggest include the following seating arrangements:

  • Each department together. If, for example, the goal is to test communications between human resources, public relations, and technology departments, the exercise design team may want to seat each department together.
  • Different departments mixed. If you want departments to learn how each functions, mix the groups together.
  • According to incident command divisions, or the command chain members. Have the command leaders, operational leaders, planning team, finance team, and logistics team sit at their own tables.
  • By functionality. Place law enforcement, hospital staff, and emergency management personnel at individual stations.
  • By location. Position employees from the same country or office together.

8. Develop scripts and data injects

Once the design team determines the scenario, it can move into designing the script and data injects (new information that the participants must consider).

Sun Microsystems' first tabletop exercise lasted 4 hours and included four data injects. A group of 38 employees from Menlo Park, Bangalore, and Singapore met via telephone conference at 8 a.m. Data injects included:

  • Being told of an increase in cases of human-to-human transmission of H5N1 influenza in a village 100 miles outside Bangalore
  • Learning 2 days later that a male and female were infected with influenza near the center of Bangalore
  • Receiving reports 5 days later of multiple clusters of H5N1 influenza among humans
  • Discovering that a Sun employee died of H5N1 influenza in Bangalore

9. Address facilitation issues

Choose either a moderator or facilitator to conduct the exercise. A moderator presides over a meeting; a facilitator acts as a neutral third party who helps a group increase its effectiveness by improving its process. Most companies benefit from using a facilitator. When selecting one, says DeBoer, look for a professional who:

  • Plans and prepares
  • Guides, but does not participate
  • Calls people by name
  • Stays on track and on time
  • Allows group members to talk to each other
  • Gives clear instructions
  • Provides clarification and focus
  • Always remains neutral and fair
  • Isn't afraid of productive conflict
  • Isn't afraid to cut people off

To keep the groups focused, the facilitator should tell participants to write down any additional questions, suggestions, or concerns, says DeBoer. Time permitting, these notes can be considered at the end of the exercise.

Once the participants are in place, the facilitator can present the scenario and begin to inject updates. DeBoer doesn't offer a formula for the number of data injects and how long the exercise lasts; both need to be determined by the exercise goals and objectives.

10. Plan the after-action report

Assign someone who has good observation, note-taking, and writing skills to compile a report. Before the exercise begins, pick someone who will function only as an observer and be willing to take the time to compile a good report based on who will read it, how the information will be used, and what action will be taken.

Moore and DeBoer describe four types of reports, including:

  • An after-action report, which has general observations and recommendations
  • A corrective action report, which addresses areas of assessment, gaps, and corrective actions to remedy the gaps
  • An improvement plan matrix, which has goals, action steps, and benchmarks
  • A work plan, which covers specific actions, timelines, responsible departments or individuals, tracking method, and monitoring steps

DeBoer emphasizes the importance of follow-through at this point. Use the information gathered to make changes in the pandemic preparedness plans, she says, or your company might find itself with a list of dilemmas and no one to fix them once a pandemic starts.

Buy-in from high-level executives early on will give you the support you need to correct items addressed in the report. To make changes, DeBoer says, "You want someone with the resources, expertise, and managerial support to handle resolving the issues." Assign the task of correcting problems to people with managerial experience in the area in which the problem exists.

An ongoing strategy

Moore and DeBoer stress that scenario-based exercises are part of an ongoing strategy to advance your preparedness. Conducting tabletop exercises on a regular basis will help illuminate problems lurking on the pages of the plan.

At Sun, for example, Dye says tabletop exercises helped the company identify three opportunities for improvement: (1) a way of determining exact locations of all employees and customers, (2) the need for a bilingual emergency phone line, and (3) planning that did not include WHO pandemic alert phases as triggers for actions, because they weren't effective in that role. Had the company not conducted tabletop exercises, it could have risked discovering these problems during an actual pandemic, at a point too late to correct them.

The exercise was so helpful, Dye says, "We're going to conduct 11 tabletops in May with our offices in other countries."

—Natasha Rotstein

This week's top reads