RES 6023 Module 1 Quantitative Designs Comparison Example

Five Quantitative Designs for One Question: Comparing Ways to Test Whether Text-Message Reminders Raise Childhood Immunization Completion at a Nevada County Health Department

Student Name

American College of Education

RES6023: Quantitative Research Designs

Module 1 Assignment

Instructor Name

October 12, 2026

Introduction

I manage the childhood immunization program at a county health department in northern Nevada. Our clinics serve many families who miss the second and third doses in a vaccine series, and next year the department plans to send automated text-message reminders to parents before each due date. My supervisor has asked a simple question: will the reminders raise the share of children who complete the recommended series by 24 months of age? The question is causal, since it asks whether the reminders produce a change. This paper compares five quantitative designs that could address it and explains how much each could say about cause.

What the Evidence Already Shows

The question is not new. Pooling trials of reminders and recall notices sent to patients, a Cochrane team judged that such outreach probably raises vaccination coverage across many settings and vaccines (Jacobson Vann et al., 2018). Text messaging has been tested directly. Stockwell et al. (2012) randomized about 7,600 children and adolescents in a low-income, largely Latino urban population in New York and found that 43.6% of those whose parents received influenza vaccine text reminders were vaccinated by the end of March, compared with 39.9% under usual care. The evidence suggests reminders help, but the size of the effect varies by setting, and no study has examined our population, which includes many rural families and Spanish-speaking parents with limited data plans.

Design 1: Randomized Controlled Trial

In a randomized controlled trial, each eligible child's family would be randomly assigned to receive text reminders or the usual mailed postcard, and series completion at 24 months would be compared between groups. Random assignment is the design's great strength: on average, it makes the groups equivalent on every characteristic, measured or not, so a difference in completion can be attributed to the reminders. Its weaknesses here are practical. Withholding a low-cost reminder from half of families may be hard to justify to the county board, families in the same household or neighborhood may share reminders, blurring the comparison, and the trial would take more than two years to reach its outcome for the youngest enrollees.

Design 2: Quasi-Experimental Nonequivalent Groups

If the department starts reminders at two of its four clinics first, the families at those clinics could be compared with families at the other two. Harris et al. (2006) described this nonequivalent control group design as one of the most common quasi-experiments in health services research and noted that its credibility depends on how similar the groups are before the intervention. Clinics differ in the families they serve, so any difference in completion could reflect those differences rather than the reminders. Measuring completion rates at all four clinics in the year before reminders begin, and adjusting for child and family characteristics, would strengthen the design but cannot remove the problem of unmeasured differences.

Design 3: Interrupted Time Series

An interrupted time series would track the monthly percentage of children at our clinics who are up to date on their series for several years before and after reminders start, asking whether the level or trend changes at the point of introduction. Lopez Bernal et al. (2017) described the design as one of the strongest available for evaluating population-level public health interventions when randomization is impossible, because each period serves as a comparison for the next and long-term trends are modeled directly. The design's chief risk is that something else changes at the same moment, such as a state school-entry requirement or a vaccine shortage, and the analysis would credit the reminders for it. Adding a comparison series, such as a neighboring county without reminders, would reduce that risk.

Design 4: Causal-Comparative

A causal-comparative study would look back after reminders have run for a year, comparing completion among families who chose to enroll in text reminders with those who did not. The data are cheap and quick to gather from the immunization registry. The design is weak for causal claims, however, because families who sign up for reminders are probably more organized or more committed to vaccination to begin with. Any advantage for enrolled families could reflect who they are rather than what the reminders did, which is the selection problem in its plainest form.

Design 5: Cross-Sectional Survey

A cross-sectional survey of parents could ask whether they received reminders, how useful they found them and whether they remember missing appointments. A survey would answer questions the other designs cannot, such as whether Spanish-language messages are understood or whether parents with prepaid phone plans receive them at all. It cannot answer the causal question, since it measures reported experience at one time, and parents who are satisfied with reminders may differ in many ways from those who are not. It is best used alongside one of the other designs.

Measuring the Outcome

Whatever the design, the outcome must be defined the same way. Series completion will mean that a child has received every dose recommended for children by 24 months of age on the national schedule, as recorded in Nevada's statewide immunization registry. The registry is the right source because it captures doses given at private practices and pharmacies as well as at our clinics; relying on clinic records alone would count a child vaccinated elsewhere as incomplete. The registry has its own gaps, since families who move out of state disappear from it, and the analysis will need a rule for children lost in that way. Defining the outcome now prevents choosing a definition later that happens to favor the reminders.

Comparing the Designs

Ranked by the strength of causal inference they allow, the designs run from the randomized trial, through the interrupted time series with a comparison series and the nonequivalent groups design, to the causal-comparative study, with the survey unable to support causal claims. Ranked by feasibility for a county health department, the order nearly reverses: the registry already holds the data for a causal-comparative study, while a randomized trial needs the most planning, consent and time. The task is to find the strongest design the setting will allow.

My Choice

I recommend a randomized design adapted to the department's rollout. Because the reminder system will be phased in across the four clinics anyway, the order in which clinics begin can be randomized, a stepped-wedge approach in which every clinic eventually receives reminders but the timing is decided by chance. This keeps randomization's protection against selection, avoids permanently withholding reminders from anyone and fits the department's own schedule. With only four clinics, chance could still produce imbalance, so the analysis will adjust for clinic characteristics and the registry's monthly data will allow a time series view as a check. The design's limits, including the small number of clinics and the long wait for the 24-month outcome, will shape the threats to validity examined in Module 2.

Conclusion

Each design answers the supervisor's question with a different degree of confidence. A randomized trial offers the strongest causal claim but the most practical obstacles, while registry-based designs are easy but vulnerable to selection and history. Randomizing the order of clinic rollout captures much of the trial's strength within the department's real constraints. Module 2 examines what could still go wrong.

References

Harris, A. D., McGregor, J. C., Perencevich, E. N., Furuno, J. P., Zhu, J., Peterson, D. E., & Finkelstein, J. (2006). The use and interpretation of quasi-experimental studies in medical informatics. Journal of the American Medical Informatics Association, 13(1), 16-23. https://doi.org/10.1197/jamia.M1749

Jacobson Vann, J. C., Jacobson, R. M., Coyne-Beasley, T., Asafu-Adjei, J. K., & Szilagyi, P. G. (2018). Patient reminder and recall interventions to improve immunization rates. Cochrane Database of Systematic Reviews, 2018(1), Article CD003941. https://doi.org/10.1002/14651858.CD003941.pub3

Lopez Bernal, J., Cummins, S., & Gasparrini, A. (2017). Interrupted time series regression for the evaluation of public health interventions: A tutorial. International Journal of Epidemiology, 46(1), 348-355. https://doi.org/10.1093/ije/dyw098

Stockwell, M. S., Kharbanda, E. O., Martinez, R. A., Vargas, C. Y., Vawdrey, D. K., & Camargo, S. (2012). Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: A randomized controlled trial. JAMA, 307(16), 1702-1708. https://doi.org/10.1001/jama.2012.502