RES 6003 Module 5 Statistical Analysis Plan Example

A Statistical Analysis Plan for a Six-Year Study of Licensure Success in an Associate Degree Nursing Program: Research Questions, Matched Tests, A Priori Power and Error Control

Student Name

American College of Education

RES6003: Applied Statistics

Module 5 Assignment

Instructor Name

November 9, 2026

Introduction

Over four modules, I analyzed records for the 186 students who graduated from our Kentucky nursing program in its last three classes. That work found a strong link between nursing GPA and exit examination scores, lower grades among the heaviest student workers and several questions the data were too thin to answer, above all whether the remediation course helps. This final paper sets out an analysis plan for a prospective study that would answer those questions properly. It states the research questions, matches each to a statistical test, sets the sample sizes the tests need and explains how error rates and missing data will be handled.

Purpose and Design

The proposed study will follow every student who enters the program over six years, from admission to the first licensure attempt, rather than only those who graduate. Module 4 showed why: a model built on graduates alone cannot represent students who left, and the progression rule hides the low end of the GPA range. The design is a prospective cohort study using program records, with a short survey at entry and at the start of each semester to capture work hours, caregiving and finances. Because the remediation course cannot ethically be withheld from struggling students, the study will compare remediated students with a matched group of similar students from cohorts before the course existed.

Research Questions and Hypotheses

RQ1: Do students who complete the remediation course score higher on the exit examination than matched students who did not have access to it? H0: the population mean exit scores are equal; H1: they differ.

RQ2: Does nursing GPA differ among students grouped by weekly work hours, at up to 10, 11 to 20 or over 20 hours? H0: all three means are equal; H1: at least one differs.

RQ3: What is the relationship between second-semester GPA and exit examination score among all students who reach the final semester? H0: ρ = 0; H1: ρ ≠ 0.

RQ4: Which of exit examination score, GPA, work hours and remediation predict first-attempt licensure success when considered together? H0: none of the predictors is associated with the odds of passing; H1: at least one is.

Variables

The outcome variables are exit examination score, a continuous scale score; nursing GPA, continuous from 0 to 4.0; and first-attempt licensure result, a binary pass or fail. The predictors and grouping variables are remediation status, binary; work-hour group, categorical with three levels; and second-semester GPA, continuous. Covariates for matching and adjustment are entry GPA, admission test score, age and enrollment status, full- or part-time.

Matching Each Question to a Test

RQ1 compares two independent group means and will use Welch's t test, chosen because the remediated and comparison groups may differ in size and variance. Matching on entry GPA and admission test score will make the groups more comparable, and a sensitivity analysis using linear regression with those covariates will check whether the conclusion depends on the matching. Its effect size, Cohen's d, will carry a 95% confidence interval.

RQ2 compares three group means and will use a one-way ANOVA, followed by Tukey comparisons if the omnibus test is significant. If Levene's test shows unequal variances, Welch's ANOVA with Games-Howell comparisons will replace the standard tests. Effect size will be reported as eta squared.

RQ3 concerns two continuous variables and will use Pearson's correlation and simple linear regression, with residual plots to check linearity and constant variance and Spearman's correlation as a backup if the relationship proves monotonic but not linear.

RQ4 has a binary outcome and several predictors, so it requires binary logistic regression, which this course has not yet covered but which follows naturally from the regression in Module 4. Each predictor's result will appear as an odds ratio and its 95% interval.

Sample Size and Power

Sample sizes were set before data collection from published power tables (Cohen, 1992), targeting 80% power at a two-sided α of .05. For RQ1, detecting a medium effect, d = 0.50, requires 64 students per group. For RQ2, a medium effect with three groups requires 52 students per group, 156 in all. For RQ3, detecting a medium correlation, r = .30, requires 85 students. At the program's current enrollment of about 75 entering students a year, six years will yield roughly 450 entrants and, allowing for attrition, about 370 who reach the exit examination, enough for RQ2 and RQ3.

RQ1 is harder. Module 1 found 31 remediated graduates in three years, so six years will yield about 60 to 65, just enough if every one can be matched. RQ4 is harder still. For logistic regression, the binding constraint is the number of failures, not the number of students. In simulation work, Peduzzi et al. (1996) showed that coefficient estimates grew unstable once the ratio of outcome events to predictors dropped below roughly 10. With four predictors, and work hours counting as two because it has three levels, the model needs about 50 failures. At the observed failure rate of about 11%, six years of graduates would produce roughly 40, so the plan either pools a seventh year or reduces the model to three predictors.

Controlling Error Across Tests

With four research questions and their follow-up tests, the probability that some result is a false positive climbs well past 5%. The four primary tests will each be run at α = .05 because they address distinct questions set out in advance, but follow-up comparisons within RQ2 will use Tukey's procedure, and any exploratory tests beyond the four questions will be labeled as such and interpreted as hypotheses for future work. Sullivan and Feinn (2012) recommended pairing each test with an effect size, so every result will be reported with an effect size and confidence interval, so readers can judge practical importance and not only statistical significance.

Data Management and Missing Data

Program records are usually complete, but the semester surveys will have gaps. Missing data will be described by variable and by whether missingness relates to other variables, such as work hours or GPA. If less than 5% of a variable is missing, complete-case analysis will be used; if more, multiple imputation will be used, with complete-case results reported alongside for comparison. Records will be stripped of identifiers before analysis and stored on the college's secure server, and no data will be gathered until the college's review board has approved the study.

Limitations of the Plan

The matched historical comparison for RQ1 is the plan's weakest link. Students from earlier cohorts studied under a slightly different curriculum and, in some years, a different version of the exit examination, so differences between groups could reflect those changes as well as the remediation course. Matching on entry measures cannot remove that threat. The plan will report it plainly and, if the college later allows it, recommend a stronger design such as randomly assigning students near the cutoff to different forms of remediation. Self-reported work hours are a second weakness, since students may underreport or round their hours, and the analysis will treat the three groups as approximate.

Conclusion

The plan turns four modules of exploratory analysis into a confirmatory study. Each research question is matched to a test that fits its variables, sample sizes come from power analysis rather than convenience and the plan admits where the program's size limits what it can learn, most of all for the logistic model of licensure success.

References

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. https://doi.org/10.1037/0033-2909.112.1.155

Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49(12), 1373-1379. https://doi.org/10.1016/S0895-4356(96)00236-3

Sullivan, G. M., & Feinn, R. (2012). Using effect size: Or why the P value is not enough. Journal of Graduate Medical Education, 4(3), 279-282. https://doi.org/10.4300/JGME-D-12-00156.1