RES 5303 Module 2 Sampling Analysis Paper Example

Who Answered and Who Did Not: What a 19 Percent Patient Experience Survey Sample Can Honestly Represent at an Outpatient Surgery Center

Student Name

American College of Education

RES5303: Research Methods and Applied Statistics in Healthcare

Module 2 Assignment

Instructor Name

September 13, 2027

The Claim and Its Sample

An outpatient surgery center, a composite created for this assignment, performs about 7,200 procedures a year in orthopedics, ophthalmology, gastroenterology and general surgery. Its marketing materials state that 94 percent of patients would recommend the center to friends and family. The figure comes from a patient experience survey mailed to every patient within a week of their procedure. In the most recent year, 1,368 of the 7,200 surveys were returned, a response rate of 19 percent, and 1,286 of the returned surveys answered the recommendation question positively.

The claim is about all patients: 94 percent of them would recommend the center. The evidence behind it comes from the 19 percent who chose to respond. Whether the claim is justified depends on how those respondents relate to everyone else, and that is a question about sampling. A sample does not represent a population simply because it came from it; it represents the population only to the extent that the process of getting into the sample did not depend on the thing being measured.

Population, Frame and Method

The target population, the group about which the center wants to make its claim, is all patients who had a procedure at the center during the year. The sampling frame, the list from which the survey was actually drawn, is the scheduling system's list of completed procedures with a valid mailing address. The frame is close to the population but not identical: about 4 percent of patients had no usable address, including patients with unstable housing and some who gave a relative's address, so they could not be surveyed at all. That is a small coverage gap, but not a random one.

Technically, the center did not draw a sample; it attempted a census, surveying every patient on the frame. The sample was produced by patients themselves, through the decision to return the survey. That makes it a self-selected sample, a form of nonprobability sampling in which each patient's chance of inclusion depends on their own choice and cannot be known in advance. Polit and Beck (2021) note that nonprobability samples can be informative, but that their results cannot be generalized to a population with the confidence that probability sampling allows, because the selection process may be related to the outcome.

Who Responded

Comparing respondents with the full frame on data the center already holds shows that the respondents were not a miniature of the patient population. Among respondents, 58 percent were aged 65 or older, compared with 39 percent of all patients. Medicaid and self-pay patients made up 21 percent of all patients but 9 percent of respondents. Ophthalmology patients, most of them older adults having cataract surgery, were 30 percent of patients and 44 percent of respondents, while general surgery patients were underrepresented.

The same pattern appears in published work. Tyser et al. (2016) examined responses to a widely used commercial patient satisfaction survey among orthopedic patients and found that only 16.5 percent responded, that older age strongly increased the odds of responding and that male sex, Medicaid or self-pay insurance and trauma subspecialty care decreased them. The center's respondents resemble that profile: older, more often privately insured or on Medicare and weighted toward elective, predictable procedures. If those patients also tend to report more positive experiences, the 94 percent figure would overstate the experience of the whole population.

Response Rate Is Not the Same as Response Bias

It is tempting to conclude that a 19 percent response rate makes the survey worthless. That conclusion is too quick. Groves (2006) reviewed studies that measured both response rates and nonresponse bias and found that the response rate alone was a poor predictor of bias. What matters is whether the likelihood of responding is related to the variable being measured. A survey with a low response rate can be nearly unbiased if people respond for reasons unrelated to the question, and a survey with a high response rate can be badly biased if the missing few are systematically different.

For this survey, the question is whether patients who had a worse experience were less likely to respond. Two findings suggest they may have been. Groups that responded less, younger patients, Medicaid and self-pay patients and general surgery patients, are also groups that in the center's own complaint log report more problems with wait times and billing. And survey mode matters: Elliott et al. (2009), in an experiment with a national hospital patient experience survey, found that patients randomized to telephone and interactive voice modes gave more positive evaluations than those randomized to mail, and that mode and patient mix adjustments were needed for fair comparisons. The low response rate is a warning; the pattern of who did not respond is the evidence.

What the Sample Can Honestly Represent

The sample can support a narrower claim than the one the center makes. It can honestly say that 94 percent of patients who returned the survey would recommend the center, and it can report that figure for subgroups where respondents are numerous, such as ophthalmology patients aged 65 and older. It cannot support a claim about all patients, and it cannot be used to compare service lines fairly, because the service lines differ so much in who responded.

The sample is still useful for improvement. Comments from 1,368 patients identify specific problems, and trends over time within the same survey method can show whether a change made a difference among the kinds of patients who respond. What the sample cannot do is stand in for the patients who did not answer, who are younger, less often privately insured and more often in the service lines with the most complaints.

A Design That Matches the Claim

If the center wants to make claims about all its patients, it needs a probability sample with active follow-up. The proposed design stratifies the patient population by service line and insurance type, creating eight strata, and draws a random sample of 80 patients from each stratum every quarter, or 640 patients a quarter. Each sampled patient receives the mailed survey, followed by a text message with a web link after five days and a telephone call after ten days for those who have not responded. Mixing modes this way tends to raise response among groups less likely to return mail, and the design records the mode of each response so that mode effects can be adjusted for in analysis, as Elliott et al. (2009) recommend.

Results would be weighted back to the population using the known size of each stratum, so that underrepresented groups count in proportion to their share of patients. With 640 patients sampled a quarter and a target response rate of 45 percent, the center would have about 290 responses a quarter, enough to estimate the overall recommendation rate within about 6 percentage points with 95 percent confidence. That is a smaller sample than the current census, and a far more honest one.

Conclusion

The center's 94 percent figure rests on a self-selected sample of 19 percent of patients who are older, more often privately insured and concentrated in one service line. The low response rate alone does not prove bias, but the pattern of nonresponse and the evidence on survey mode suggest the figure overstates the experience of patients as a whole. The existing survey can honestly describe its respondents and track trends among them. A stratified probability sample with mixed-mode follow-up and weighting would allow the claim the center wants to make, supported by a sample designed to make it.

References

Elliott, M. N., Zaslavsky, A. M., Goldstein, E., Lehrman, W., Hambarsoomians, K., Beckett, M. K., & Giordano, L. (2009). Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Services Research, 44(2, Pt. 1), 501-518. https://doi.org/10.1111/j.1475-6773.2008.00914.x

Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675. https://doi.org/10.1093/poq/nfl033

Polit, D. F., & Beck, C. T. (2021). Nursing research: Generating and assessing evidence for nursing practice (11th ed.). Wolters Kluwer.

Tyser, A. R., Abtahi, A. M., McFadden, M., & Presson, A. P. (2016). Evidence of non-response bias in the Press-Ganey patient satisfaction survey. BMC Health Services Research, 16, Article 350. https://doi.org/10.1186/s12913-016-1595-z