RES 6003 Module 2 Hypothesis Testing Report Example

Is 88.7% Different From 88%? Testing a Nursing Program's First-Attempt Licensure Pass Rate Against Its Benchmark, With a Confidence Interval and a Power Check

Student Name

American College of Education

RES6003: Applied Statistics

Module 2 Assignment

Instructor Name

October 19, 2026

Introduction

The Module 1 report summarized our program's last three graduating classes, 186 students, and found that 165 of them, 88.7%, were licensed after one attempt. The program's faculty set a target of 88% when they revised the curriculum three years ago. A dean reading the figure might conclude the program beat its target, while a skeptical faculty member might say the difference is too small to mean anything. This report uses probability and a formal hypothesis test to settle which reading the data support, then asks a question that is easy to skip: whether a sample of 186 could have detected a meaningful shortfall had one existed.

A Probability Model for Pass Rates

Treating each graduate's first attempt as a trial that ends in pass or fail gives a binomial model. If the program's true pass rate were exactly 88%, the expected number of passes among 186 graduates would be 163.7, with a standard deviation of about 4.4 passes. Under that model, results anywhere from about 155 to 172 passes would be unremarkable, and the probability of 165 or fewer passes would be about .65. The observed count of 165 sits barely more than one pass above the expected value, so even before any test, the model suggests the program's result is ordinary for a program whose true rate is 88%.

The binomial model rests on two assumptions worth stating. Each graduate's result is treated as independent of the others, which is reasonable for an individually taken examination but would weaken if, for example, one instructor's weak teaching depressed a whole cohort. And the three cohorts are treated as draws from one process with a single underlying rate, which Module 1's cohort figures of 87.7%, 92.1% and 86.4% do not contradict.

Hypotheses

The null hypothesis states that the program's true first-attempt pass rate equals the 88% target, H0: p = .88. Because the faculty want to know whether the program differs from the target in either direction, the alternative is two-sided, H1: p ≠ .88. A one-sided alternative would have been defensible only if the question had been fixed in advance as whether the program fell short; picking the direction only after looking at the results would make a spurious finding more likely. Before any calculation, α was fixed at .05.

The Test

With 186 graduates and an expected count of failures under the null of about 22, the normal approximation to the binomial is adequate, so a one-sample z test for a proportion was used. The standard error under the null is the square root of .88 × .12 divided by 186, or .0238. The observed proportion, .887, lies .0071 above the target, giving z = 0.30 and a two-sided p value of .77. The result does not reject the null hypothesis: a pass rate of 88.7% is entirely consistent with a true rate of 88%, and the data give no basis for saying the program beat its target.

The test also does not show that the program's true rate is exactly 88%. A nonsignificant result does not show that the target rate is the true rate; it shows only that these data cannot tell the two apart. Greenland et al. (2016) listed this error, along with reading a p value as the chance that the null hypothesis holds, in their guide to how tests are misread.

Confidence Intervals

A confidence interval says more than the test because it shows the range of true pass rates the data are compatible with. The familiar Wald interval, which takes .887 and moves 1.96 standard errors in each direction, runs from 84.2% to 93.3%. Agresti and Coull (1998) showed that the Wald interval often covers the true value less often than its nominal 95% when proportions are near 0 or 1, and recommended alternatives such as the Wilson score interval. The Wilson interval here runs from 83.4% to 92.5%. Both intervals contain 88%, consistent with the test, and both are wide: the data are compatible with a true pass rate as low as about 83% or as high as about 93%. For the faculty, that width is the most useful finding in the report.

Type I and Type II Errors

Two kinds of mistakes are possible. A Type I error would mean concluding that the program differs from its target when it does not; setting α at .05 holds that risk to 5% if the program truly sits at its target. A Type II error would mean missing a real difference. In this setting the second mistake is arguably the more costly, because a program whose true rate had slipped to, say, 83% would face a falling pass rate and possible scrutiny from the state board of nursing while its faculty believed all was well.

How Much Power Did the Test Have?

Power is the chance that a test will flag a difference of a stated size if that difference really exists. Cohen (1992) suggested .80 as a conventional minimum. With 186 graduates and a two-sided α of .05, the test had about 55% power to detect a true pass rate of 83%, five points below the target. Power rose to about 87% for a true rate of 80% and about 96% for 78%. Put plainly, if the program had quietly lost five points, this test would have caught it only a little more than half the time. To reach 80% power against a true rate of 83% would require about 364 graduates, roughly six years of cohorts at current enrollment.

What the Results Mean for the Program

Three conclusions follow. First, the program cannot claim to have exceeded its target; it can say only that its results are consistent with meeting it. Second, the confidence intervals show that three years of data leave real uncertainty about the program's true rate, so one cohort's dip or spike should not drive curriculum decisions. Third, the low power against a five-point drop means the program should not rely on an annual significance test as an early warning. Tracking a running multi-year rate with its confidence interval, and watching leading indicators such as exit examination scores, would serve that purpose better. Wasserstein and Lazar (2016) cautioned that scientific and policy conclusions should not hinge on a p value landing on one side of .05, and the program's case shows why.

Limitations

The analysis treats the 88% target as fixed and known, though it was itself a judgment by faculty. The independence assumption may be imperfect if cohort-level factors, such as a change in clinical sites, affected groups of students together. The data are a composite drawn from program records, and the conclusions apply to this program's graduates rather than to nursing programs in general. Finally, the test asks only about first attempts; a program's eventual pass rate, counting retakes, could tell a different story and would need its own analysis.

Conclusion

A one-sample z test found no evidence that the program's 88.7% first-attempt pass rate differs from its 88% target, z = 0.30, p = .77, 95% Wilson CI [83.4%, 92.5%]. The more important finding is the interval's width and the test's limited power, which together argue for multi-year monitoring rather than year-by-year verdicts. Module 3 turns from a single proportion to comparisons between groups.

References

Agresti, A., & Coull, B. A. (1998). Approximate is better than "exact" for interval estimation of binomial proportions. The American Statistician, 52(2), 119-126. https://doi.org/10.1080/00031305.1998.10480550

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. https://doi.org/10.1037/0033-2909.112.1.155

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337-350. https://doi.org/10.1007/s10654-016-0149-3

Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129-133. https://doi.org/10.1080/00031305.2016.1154108