How to Improve: Model for Improvement: Testing Changes

Image Block Hero fallback image.

Model for Improvement: Testing Changes

Model for Improvement: Plan-Do-Study-Act (PDSA) Cycles

Once a team has identified testable change ideas, the next step is to test them. The Plan-Do-Study-Act (PDSA) cycle is a method for learning how the change works in the local environment — by planning it, trying it, observing the results, and acting on what is learned, teams can build their knowledge about the potential of a change to result in improvement in their local context. This is the scientific method, used for action-oriented learning. Most changes will require many PDSA cycles, planned and executed in a sequence, to develop a change, test it under varying conditions, and eventually implementing it, if the change is shown to result in improvement.

Image
PDSA Ramp: How to Improve Testing Changes

Reasons to Test Changes

  • To increase your belief that the change will result in improvement.
  • To decide which of several proposed changes will lead to the desired improvement.
  • To evaluate how much improvement can be expected from the change.
  • To decide whether the proposed change will work in the actual environment of interest.
  • To decide which combinations of changes will have the desired effects on the important measures of quality.
  • To evaluate costs, social impact, and side effects from a proposed change.
  • To minimize resistance upon implementation.

Steps in the PDSA Cycle

Step 1: Plan

Plan the test or observation, including a plan for collecting data.

  • State the objective of the test.
  • State the questions the test will be designed to answer
  • Make predictions about what the results of the test will be (answers to the stated questions)
  • Develop a plan to test the change. (Who? What? When? Where? What data need to be collected?). Early PDSA cycles should be scoped as small as possible.

Step 2: Do

Execute the plan.

  • Carry out the test as planned.
  • Document problems and unexpected observations.
  • Begin analysis of the data.

Step 3: Study

Analyze the data and study the results.

  • Complete the analysis of the data.
  • Compare the data and result to your predictions.
  • Summarize and reflect on what was learned.

Step 4: Act

Refine the change, based on what was learned from the test.

  • Determine what modifications should be made.
  • Prepare a plan for the next test.

Example of a First Test of Change (PDSA Cycle)

Depending on their aim, teams choose promising changes and use Plan-Do-Study-Act (PDSA) cycles to test a change quickly often on a small scale at first, see how it works, and refine the change as necessary before testing it on a broader scale, and implementing if shown to lead to the desired improvement.

The following example shows how a team started with a small-scale test.

Example First PDSA: Diabetic patient planned visits for blood sugar management support

Objective: To learn about how patients respond to being offered an appointment for blood sugar management support

Questions:

  • How will patients respond to being asked?
  • What barriers might arise with the scheduling?

Prediction: Patients who are struggling with blood sugar management will be interested in scheduling a dedicated appointment. We will be able to find a time that works for the patient and the diabetes educator.

  • Plan: We will start the test with one doctor and one patient. On Tuesday, Dr. J will ask one patient if they would like more information on how to manage their blood sugar. The scheduler will confirm an appointment with the diabetes educator if they say yes.
  • Do: Dr. J asked their first patient with diabetes on Tuesday. The patient was excited to schedule the appointment.
  • Study: Patient was appreciative of the offer. We were able to schedule an appointment within one week.
  • Act: No change to the process yet, but we will have to keep an eye on the workload of the diabetes educator. For the next PDSA cycle, Dr. J will ask the next five patients and work with the scheduler to create planned visits for those who say yes.

Tips for Testing Changes

  • Plan multiple cycles for a test of a change and think a couple of cycles ahead. When designing a test, imagine at the start what the subsequent test or two might be, given various possible findings in the "Study" phase of the PDSA cycle. For example, teams that are redesigning same-day admission criteria should also be planning how those criteria will be applied.
  • Scale down the size of the test. For example, scale down the number of people involved in the test ("sample the next 10" instead of "get a sample of 200"), and the location or duration of the test ("test it in Operating Room #1 for one week"). Be innovative to make the test feasible.
  • Choose easy changes for early tests and don’t reinvent the wheel. Look for the concepts that seem most feasible and will have the greatest impact. Adopt or adapt successful changes made elsewhere, for example, instead of creating your own treatment protocol try modifying another hospital’s protocol.
  • Test with willing volunteers.
  • Do not try to get consensus when testing as this may delay your efforts and learning. When possible, choose changes that do not require a long process of approval, especially during the early testing phase. Consensus and "buy-in" are necessary for implementation, but not when testing.
  • Test over a wide range of conditions (day of the week, provider, etc.). Try a test quickly; ask, "What change can we test by next Tuesday?"
  • Collect useful quantitative and/or qualitative data during each test. Avoid technical slowdowns, like waiting for new software to be installed, and instead try recording test measurements and charting trends with paper and pencil.
  • Reflect on the results of every change. After making a change, a team should ask: What did we expect to happen? What did happen? Were there unintended consequences? What was the best thing about this change? The worst? What might we do next? Too often, people avoid reflecting on failure. Remember that teams often learn very important lessons from failed tests of change.
  • Be prepared to end the test of a change. If the test shows that a change is not leading to improvement, the test should be stopped. Note: "Failed" tests of change are a natural part of the improvement process. If a team experiences very few failed tests of change, it is probably not pushing the boundaries of innovation very far.  

Linking Tests of Change

​Testing changes is an iterative process: the completion of each Plan-Do-Study-Act (PDSA) cycle leads directly into the start of the next cycle.

A team learns from the test — What worked and what didn't work? What should be kept, changed, or abandoned? — and uses the new knowledge to plan the next test. The team continues linking tests in this way, refining the change until it is ready for broader implementation.

Note: People are far more willing to test a change when they know that changes can and will be modified as needed. Linking tests of change helps overcome an individual’s natural resistance to change by developing staff trust that ineffectual changes will not be mandated, and making the results observable.

Example Linked Tests of Change

Decrease length of stay (LOS) for emergency department (ED) patients with x-rays

  • PDSA 1: Test quick-look for extremity x-rays on one shift. Monitor LOS for patients with x-rays and error rate. Review results with Radiology.
  • PDSA 2: Revise documentation process and try quick-look for two days.
  • PDSA 3: Redesign viewing area and continue quick-look for two weeks.
  • PDSA 4: Make quick-look standard practice and monitor.

Testing Multiple Changes

​Typically, teams test more than one change at a time. All of the changes are aimed at achieving the same ultimate goal. Teams must develop linked tests of change, moving from testing to implementation, for each change, thinking through how the changes are likely to interact.

Image
Multiple PDSA Ramps: How to Improve Testing Multiple Changes

Example of Testing Multiple Changes

A team working on reducing the average extubation time for elective coronary artery bypass graft (CABG) patients worked on several changes at the same time. Each of the changes went through several linked Plan-Do-Study-Act (PDSA) cycles which are often referred to as a PDSA Ramp. Throughout the project, data on extubation time was collected in order to determine if the changes were resulting in improvement.

Change 1: Standardize pain management

In order to be extubated early, patients must not be too heavily sedated. The team began by revising the existing standards for postoperative pain management. Instead of using the traditional high dose of morphine, the team ran a series of PDSA cycles to develop, test, and eventually implement the use of smaller, more frequent doses. In this way, patients' pain was managed adequately, yet patients were awake enough to be extubated safely.

Change 2: Standardize anesthesia management

Patients cannot be extubated if they are heavily sedated. The team ran a series of PDSA cycles to develop, test, and eventually implement having anesthesiologists use lower doses of sedatives to prevent patients from remaining heavily sedated long after the surgery was completed.

Change 3: Establish a rapid weaning and extubation protocol run by nurses and respiratory therapists

The team ran a series of PDSA cycles to develop, test, and eventually implement a set of criteria that patients need to meet in order to be extubated safely, given the changes in anesthesia and pain management.

Change 4: Reduce delays in obtaining arterial blood gas (ABG) results

The team identified delays in obtaining ABG results and weaning parameters as barriers to early extubation. They ran a series of PDSA cycles to develop, test, and eventually implement assigning a dedicated respiratory therapist to obtain these results.