LEAD 6523 Module 5 Full Evaluation Plan Example

Six Steps, Two Years, One Decision: The Complete Evaluation Plan for a Rural Community Paramedicine Program, Organized by the Public Health Evaluation Framework

Student Name

American College of Education

LEAD6523: Planning, Evaluation, and Accountability

Module 5 Assignment

Instructor Name

November 9, 2026

Introduction

Over four modules I have carried one idea, home visits by paramedics for the county's heaviest 911 users, from a list of needs to a plan for accountability. The needs assessment found that 72 people account for about a fifth of calls; the logic model traced how home visits are supposed to reduce crises; the evaluation design proposed a matched comparison and a time series; and the accountability plan matched measures to audiences. This paper assembles that work into a single evaluation plan, organized by a published framework, and ties it to the decision the county must make at the end of the grant.

The Framework

The public health evaluation framework moves through six linked tasks, from bringing the people with a stake into the work, through describing the program, choosing a design, collecting believable evidence and reasoning to conclusions, to seeing the findings put to work, and it pairs those tasks with standards for usefulness, practicality, fairness and correctness (Centers for Disease Control and Prevention, 1999). Patton (2008) argued in his work on utilization-focused evaluation that evaluations should be designed from the start for specific intended users and intended uses, because evaluations that no one has asked for are rarely used. The plan follows the framework's steps and Patton's emphasis on use.

Step 1: Stakeholders and Intended Users

The primary intended users are the county commission, which will decide whether to fund the program after the grant, and the regional foundation, which will decide whether to extend its support. Secondary users are the hospital, the clinic and the ambulance service's crews, who will use findings to improve the program, and enrolled patients and families, whose experience the evaluation should reflect. An evaluation advisory group with one representative from each, including a family member of an enrolled patient, will review the plan, interim findings and the final report.

Steps 2 and 3: Program Description and Evaluation Focus

The program is described by the logic model from Module 2: five trained paramedics, a dedicated vehicle and clinic oversight producing post-discharge and scheduled home visits, medication reviews and referrals, with outcomes ranging from self-management in the short term to fewer calls and readmissions in the intermediate term. The evaluation focuses on the questions organized by RE-AIM in Module 3, with particular weight on effectiveness and maintenance, because those are what the commission must decide. The design compares enrolled frequent callers with a matched group of eligible callers not yet enrolled and examines county ambulance availability as an interrupted time series.

Step 4: Credible Evidence

Evidence comes from the dispatch system, hospital data under a data-sharing agreement, the program's visit records, a self-care questionnaire at enrollment and six months and two rounds of interviews with patients, crews and partners at months nine and twenty. The calendar is fixed in advance: baseline extraction before the first enrollment, monthly dispatch and visit data, quarterly hospital data and the questionnaire and interviews on schedule. The data coordinator reconciles sources monthly, and an outside reviewer from the regional university checks the analysis plan before data collection begins. Evidence counts as credible here when the commissioners who doubt the program would accept it, not only when the program's supporters do.

Step 5: Justifying Conclusions

Before results are known, the advisory group will agree on what would count as success: a reduction in transports among enrolled patients at least 25% greater than among matched patients, no increase in serious events after delayed calls and costs avoided that cover at least half of program costs. Results will be analyzed with comparisons of change between groups and with segmented regression for the time series, and they will be interpreted with the threats to validity identified in Module 3, particularly regression to the mean and changes in the clinic's capacity. Interviews will help explain the numbers, for example why some patients disenrolled.

Step 6: Use and Sharing

Findings will be used at three points. At month twelve, interim results will guide program changes and an early conversation with the commission. At month twenty-two, the advisory group will review draft findings before the final report. At month twenty-four, the report will go to the commission's open meeting with a recommendation on whether to keep, reshape or close the program, together with a two-page summary for the public and patients. Lessons will also be shared with the state emergency medical services office and at a regional rural health meeting, since other rural counties face the same pattern of calls.

Ethical Safeguards

The evaluation studies vulnerable people in their homes, and its protections are designed in from the first page. Enrollment in the program does not depend on joining the evaluation; patients may receive visits and decline to have their hospital data used. Consent forms use short words and short sentences, and staff read them aloud to anyone who prefers. Interview participants may stop at any time, and quotes in reports will be altered enough that no one in a small county can be recognized. The comparison group receives the program as soon as capacity allows, so the design never withholds care for the sake of the evaluation. The regional university's institutional review board has been asked to review the plan, since the outside reviewer's involvement may make the work research rather than routine program evaluation.

Meeting the Standards

The plan's design choices map onto the standards that the Joint Committee on Standards for Educational Evaluation set out for program evaluation (Yarbrough et al., 2011). Utility is served by designing for the commission's decision and involving intended users. Feasibility is served by relying mainly on existing data and a quarter-time coordinator. Propriety is served by consent, privacy protections and the commitment to report disappointing results as fully as good ones. Accuracy is served by matching, a time series, reconciled data and an outside reviewer.

Budget

The evaluation will cost about $46,000 over two years: the quarter-time data coordinator, $28,000; the outside reviewer's stipend, $8,000; interview transcription and participant gift cards, $4,000; and the cost analysis and reporting, $6,000. That is about 15% of the program grant, a share the foundation agreed to fund separately so that the evaluation does not compete with patient services.

If Circumstances Change

Two contingencies are planned for. If the grant ends early, the evaluation will report whatever twelve-month results exist, clearly labeled as preliminary, so the commission has evidence rather than none. If enrollment falls far short of 45 patients, the matched comparison will lack statistical power, and the plan shifts weight toward the time series, implementation measures and interviews, while stating plainly that effectiveness cannot be judged with confidence.

Conclusion

The plan gives the county a credible way to decide whether a new kind of care is worth keeping. It is built around the people who will make that decision, uses the strongest design the setting allows, agrees on success before results arrive and commits to sharing what is learned whatever the outcome. Planning, evaluation and accountability, the three words in this course's title, become in this plan a single process aimed at one honest answer.

References

Centers for Disease Control and Prevention. (1999). Framework for program evaluation in public health. MMWR Recommendations and Reports, 48(RR-11), 1-40. https://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm

Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Sage.

Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Sage.