HLTH 5453 Module 2 Criteria-Based Policy Evaluation Example

Criteria First, Verdict Second: Judging the 340B Drug Pricing Program on Effectiveness, Equity, Efficiency and Accountability

Student Name

American College of Education

HLTH5453: Health Policy Evaluation and Development

Module 2 Assignment

Instructor Name

November 8, 2027

Choosing the Criteria

The previous module described the 340B Drug Pricing Program, which requires manufacturers to sell outpatient drugs at discounted prices to safety-net providers, and translated its aims into four evaluable objectives. This module evaluates the program against criteria chosen in advance. Bardach and Patashnik (2020) describe criteria as the evaluative standards used to judge policy outcomes, and they list among the most commonly used efficiency, equity, legality, political acceptability and robustness, while emphasizing that the analyst must decide and state which criteria matter for the problem at hand.

Four criteria were chosen. Effectiveness asks whether the program achieves its stated aim of increasing resources for covered entities to reach more eligible patients. Equity asks whether the benefits reach low-income and uninsured patients, the group most associated with the program's purpose. Efficiency asks whether the public cost of the program, borne through discounts required of manufacturers and passed through the drug market, produces benefits in proportion to that cost. Accountability asks whether the program's design allows policymakers and the public to know how the benefits are used. The criteria are fixed here, before a single study is read, so that no finding can quietly move the standard it is judged against.

What Each Criterion Requires

For each criterion, the standard of evidence was also set in advance. Effectiveness would be met if participating hospitals show measurable financial benefit and some expansion of services. Equity would be met if evidence showed that program benefits are concentrated among low-income patients or the communities where they live, or at least that participation increases care for them. Efficiency is the hardest to judge with available data and would be met only partially if benefits for the target population could be identified but not compared directly with program cost. Accountability would be met if covered entities were required to report savings and their use, or if federal oversight could verify compliance with program rules.

The evidence base chosen for the evaluation consisted of peer-reviewed studies using national data and designs capable of separating the program's effect from other hospital characteristics, together with a federal audit of program oversight. Advocacy reports from hospital and manufacturer associations were excluded, not because they are necessarily wrong but because each is produced by a party with a direct financial interest in the verdict.

Effectiveness

The evidence shows that hospitals respond strongly to the program's financial incentive. Desai and McWilliams (2018) compared hospitals just above and just below the eligibility threshold, a regression discontinuity design that approximates random assignment near the cutoff, and found that eligibility was associated with 2.3 more hospital-employed hematologist-oncologists per hospital and with substantially more hospital-billed claims for infused drugs in oncology and ophthalmology. That is clear evidence that the program shifts resources toward participating hospitals. Evidence that those resources translate into more comprehensive services is weaker. Nikpay et al. (2020) found that hospitals newly joining the program increased charity care spending by about 29 percent, roughly $880,000 per hospital, but that total community benefit spending did not change, suggesting the increase was offset by reductions elsewhere.

Verdict on effectiveness: partly met. The program reliably increases resources for participating hospitals, which satisfies the capacity reading of its aim, but the evidence that it expands the range of services those hospitals provide is mixed.

Equity

The equity evidence points the other way. Desai and McWilliams (2018) found that program eligibility was associated with lower proportions of low-income patients in hematology-oncology and ophthalmology and with no significant differences in hospitals' provision of safety-net or inpatient care for low-income groups or in mortality among low-income residents of their service areas. Conti and Bach (2014) found that hospital-affiliated clinics joining the program in 2004 or later served wealthier and better-insured communities than earlier participants. The modest increase in charity care found by Nikpay et al. (2020), and the greater likelihood that participating hospitals offered discounted care, is the strongest equity-positive finding.

Verdict on equity: not met. The best available evidence does not show that program benefits are concentrated among low-income patients, and some evidence suggests that expansion has moved toward more affluent communities. The program succeeds at moving money toward hospitals and has not been shown to move care toward the patients its purpose is usually taken to describe.

Efficiency and Accountability

Efficiency cannot be judged directly, because the program's total benefit to covered entities is not reported and its cost is spread across manufacturers, insurers and ultimately purchasers of drugs. What can be said is that some of the program's effect appears to be a shift of drug administration and physician employment from independent practices to hospitals, which Desai and McWilliams (2018) describe as consolidation, and that such shifts can raise prices paid by insurers without corresponding gains for the target population. Verdict on efficiency: cannot be established, with indirect evidence of costs that do not produce target benefits.

Accountability is the clearest failure. Covered entities are not required to report their 340B savings or how they use them, and federal auditors found weaknesses in the oversight of contract pharmacies through which many entities dispense 340B drugs, including limits on the agency's ability to ensure compliance with rules against duplicate discounts and diversion (U.S. Government Accountability Office [GAO], 2018). Verdict on accountability: not met.

The Verdicts Together

Assembled, the verdicts read as follows: effectiveness partly met, equity not met, efficiency not established and accountability not met. The pattern is consistent. The program does what its design rewards, generating revenue for participating hospitals, and the evidence does not show that it does what its purpose is commonly understood to require, directing that value toward low-income patients. The accountability failure explains why the equity question is so hard to settle: without reporting, even hospitals that use their savings well cannot demonstrate it. This verdict does not mean the program should end. It means that the program's design, not its existence, is the most promising target for policy change, which the following modules take up.

It is worth testing how sensitive the verdict is to the choice of criteria. An evaluator who adopted only the narrowest reading of the program's aim, increasing resources for eligible providers, and who weighed effectiveness alone would conclude that the program works, since the evidence of financial response is strong. An evaluator who weighed equity most heavily would conclude that it falls short of its purpose. The criteria used here give weight to both, and the resulting verdict, a program that succeeds at its mechanism but not demonstrably at its purpose, holds under either emphasis because the accountability failure is present in both. Showing that sensitivity openly allows readers with different priorities to see where their conclusions would diverge and why.

Conclusion

Setting four criteria and their standards before reading the evidence produced a clear and defensible evaluation of the 340B program. Strong quasi-experimental evidence shows that hospitals respond to the program's financial incentive, but not that benefits reach low-income patients, and the absence of reporting prevents a direct test. Because the criteria were fixed first, the verdicts reflect the evidence rather than a preferred conclusion, and a reader who disagrees can see exactly which standard they would set differently.

References

Bardach, E., & Patashnik, E. M. (2020). A practical guide for policy analysis: The eightfold path to more effective problem solving (6th ed.). CQ Press.

Conti, R. M., & Bach, P. B. (2014). The 340B drug discount program: Hospitals generate profits by expanding to reach more affluent communities. Health Affairs, 33(10), 1786-1792. https://doi.org/10.1377/hlthaff.2014.0540

Desai, S., & McWilliams, J. M. (2018). Consequences of the 340B drug pricing program. New England Journal of Medicine, 378(6), 539-548. https://doi.org/10.1056/NEJMsa1706475

Nikpay, S. S., Buntin, M. B., & Conti, R. M. (2020). Relationship between initiation of 340B participation and hospital safety-net engagement. Health Services Research, 55(2), 157-169. https://doi.org/10.1111/1475-6773.13278

U.S. Government Accountability Office. (2018). Drug discount program: Federal oversight of compliance at 340B contract pharmacies needs improvement (GAO-18-480).