RES 6023 Module 4 Power Analysis and Sample Size Plan Example

Is Four Clinics Enough? A Power Analysis and Sample Size Plan for a Stepped-Wedge Trial of Text-Message Immunization Reminders

Student Name

American College of Education

RES6023: Quantitative Research Designs

Module 4 Assignment

Instructor Name

November 2, 2026

Introduction

Modules 1 through 3 designed a stepped-wedge trial in which our county health department's four clinics begin sending text-message immunization reminders in random order, measured series completion through the state registry and planned to adjust for parent hesitancy. What remains is to ask whether the study could find an effect the department would care about. A study that is too small can miss a real benefit and leave the department believing reminders do not work. This paper sets the inputs for a power analysis, explains why the usual formulas do not apply, reports power for the planned design and weighs what to do about the result.

Inputs and Their Sources

Every power analysis rests on assumptions, and each needs a source. The baseline rate of series completion by 24 months at our clinics, from registry data for the past three years, is 62%. The smallest effect worth detecting is an 8-percentage-point gain, to 70%. That figure was chosen with the department's leadership as the smallest gain that would justify the system's cost, and it is larger than the roughly 4-point gain that Stockwell et al. (2012) found for influenza vaccine text reminders in New York, where a single seasonal vaccine was the outcome rather than a full series. Significance is set at a two-sided α of .05 and the target power at .80, the convention Cohen (1992) proposed. The intraclass correlation, the share of variation in completion that lies between clinics rather than within them, is set at .02, with .01 and .05 tested as alternatives because no estimate exists for our clinics.

Why the Simple Formula Does Not Apply

If children were randomized individually, comparing 62% with 70% at 80% power would require about 547 children per group, roughly 1,100 in all. That figure is misleading here for two reasons. First, children are grouped in clinics whose families resemble one another, so 1,100 children in four clinics carry less information than 1,100 independent children. In a conventional cluster trial with two clinics per arm and 300 children per clinic, an intraclass correlation of .02 would produce a design effect of about 7, reducing the effective sample to fewer than 90 children per arm and power to about 20%.

Second, a stepped wedge is not a conventional cluster trial. Because every clinic contributes periods both with and without reminders, part of the comparison is within clinics, which removes much of the between-clinic variation. The design must therefore be analyzed with a method built for it.

Power for the Planned Design

Hussey and Hughes (2007) derived the variance of the treatment effect in a stepped-wedge trial analyzed with a mixed model that includes fixed effects for time periods and a random effect for clusters, and their formula allows power to be computed directly. I applied it to the planned design: four clinics, five six-month periods, one clinic crossing over at the start of each period after the first and about 75 children per clinic per period, the number registry data suggest our clinics will enroll. The study would include about 1,500 children over two and a half years.

With these inputs, the planned study has only 47% power to detect an 8-point gain. The result changes little with the intraclass correlation, 48% at .01 and 46% at .05, which reflects the within-clinic comparisons at the heart of the design. The planned study could detect a 12-point gain with 82% power, but an effect that large is more than the published evidence supports. To reach 80% power for an 8-point gain with four clinics, each clinic would need about 174 children per period, more than twice what our clinics serve.

Checking the Calculation

Because the result is disappointing, I checked it two ways. First, the calculation was repeated by hand for the simplest case, with no between-clinic variation, to confirm that the formula reproduced the expected precision for a comparison of within-clinic periods. Second, the intuition was checked against the design. With four clinics and five periods, only 10 of the 20 clinic-periods are under reminders, and the earliest and latest periods contribute little to the comparison because nearly all clinics are in the same condition. The effective information is therefore much smaller than 1,500 children suggests, which is consistent with the low power.

Three Ways Forward

The first option is to accept the study's limits and treat it as a demonstration that can rule out very large effects but not moderate ones. That is honest but would leave the department's main question unanswered if the true effect is near 8 points.

The second option is to lengthen the study. Doubling the number of six-month periods to 10, with clinics crossing over every other period, would raise power to about 74%, but it would take five years, and the reminder system's vendor contract and the department's patience are both shorter than that.

The third option is to add clinics. The neighboring county's health department, which serves a similar population, has expressed interest in reminders. With eight clinics, two crossing over at each step, and 75 children per clinic per period, power rises to about 76%, and with 85 children per clinic per period it reaches 80%. Adding clinics strengthens the design in other ways as well, since four more clusters make chance imbalance less likely and broaden the setting for external validity.

Recommendation

I recommend the third option: a two-county stepped-wedge trial with eight clinics, five six-month periods and a target of 85 children per clinic per period, about 3,400 children in all. If the neighboring county's clinics enroll fewer children than expected, the study will still have about 76% power at 75 per period, a modest shortfall that will be reported. The added cost lies mainly in coordination between departments and a data-sharing agreement for the registry, both of which are simpler than extending the study by years.

Attrition and Missing Outcomes

The calculations assume complete outcome data. Module 3 noted that children who move out of state will be classified as lost, and registry records from past years suggest about 8% of children are lost before 24 months. Inflating the target by dividing by .92 raises it from 85 to about 93 children per clinic per period, which the two counties' combined enrollment should support. Losses that differ between conditions would bias the result as well as reduce power, so the analysis will report losses by condition and period.

What the Analysis Will Report

Whatever the final size, the study will report the estimated effect with its 95% confidence interval, not only whether it is significant. With the recommended design, the interval around an 8-point estimate would run roughly from 2 to 14 points, wide enough that a nonsignificant result would still carry information about which effects are plausible. The observed intraclass correlation will also be reported so that future studies of reminders in rural health departments can plan with a real estimate rather than an assumption.

Conclusion

The power analysis turned a promising design into a sobering finding: four clinics cannot reliably detect the 8-point gain the department cares about. Adding four clinics from a neighboring county is the most practical way to reach adequate power, and it strengthens the design's external validity. Module 5 assembles the full quantitative design proposal around that decision.

References

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. https://doi.org/10.1037/0033-2909.112.1.155

Hussey, M. A., & Hughes, J. P. (2007). Design and analysis of stepped wedge cluster randomized trials. Contemporary Clinical Trials, 28(2), 182-191. https://doi.org/10.1016/j.cct.2006.05.007

Stockwell, M. S., Kharbanda, E. O., Martinez, R. A., Vargas, C. Y., Vawdrey, D. K., & Camargo, S. (2012). Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: A randomized controlled trial. JAMA, 307(16), 1702-1708. https://doi.org/10.1001/jama.2012.502