BUS 6513 Module 4 Ethics of an Innovation Decision Example

Selling Faces to the Police: The Ethical and Societal Dimensions of the 2020 Facial Recognition Decisions at IBM, Amazon and Microsoft

Student Name

American College of Education

BUS6513: Innovation Theory and Organizational Practice

Module 4 Assignment

Instructor Name

February 26, 2029

Introduction

Innovation decisions are usually framed as questions of feasibility and return. Some also raise questions of rights, fairness and power that markets do not settle. In June 2020, amid national protests after the killing of George Floyd, three large technology firms changed course on selling facial recognition to law enforcement: IBM said it would no longer offer general-purpose facial recognition products, Amazon announced a one-year moratorium on police use of its Rekognition service and Microsoft said it would not sell the technology to police without federal regulation. This paper analyzes the ethical and societal dimensions of those decisions and what they teach leaders about governing innovation.

The Technology and Its Uses

Facial recognition systems compare an image of a face with stored images to verify identity or search for possible matches. Police agencies used such systems to generate investigative leads from surveillance footage and to search databases of booking or license photos. The technology promised faster identification of suspects and missing persons. It also raised concerns about mass surveillance, the chilling of lawful protest, misidentification and the absence of clear legal limits on its use, because most of the law governing police searches was written before such systems existed.

Evidence of Unequal Accuracy

Independent research shaped the debate. Raji and Buolamwini (2019) audited commercial facial analysis systems, including products from IBM, Microsoft and Amazon, and reported that error rates in classifying gender were markedly higher for darker-skinned women than for lighter-skinned men; after an earlier audit was published, several targeted vendors reduced those gaps. A large evaluation by the National Institute of Standards and Technology found demographic differences in false positive rates across many algorithms, with higher false match rates for some groups, including African American and Asian faces, in many systems, although the most accurate algorithms showed much smaller differences (Grother et al., 2019). In policing, a false match can lead to a wrongful stop or arrest, so unequal error rates translate into unequal risk.

A Framework for Responsible Innovation

Stilgoe et al. (2013) proposed that responsible innovation rests on four dimensions: anticipation of possible impacts, including unintended ones; reflexivity, meaning that innovators examine their own assumptions and values; inclusion of affected publics and stakeholders in deliberation; and responsiveness, the capacity to change direction as knowledge and values change. Owen et al. (2012) argued that innovation should move from science conducted for society toward science conducted with society, especially where technologies raise questions that experts alone cannot answer. The framework is useful because it evaluates not only the final decision but the process by which firms reached it.

Anticipation and Reflexivity

On anticipation, the firms' record before 2020 was weak. Concerns about bias and surveillance had been raised publicly by researchers and civil liberties groups for several years, yet the products were marketed to police without binding limits on use. Reflexivity was uneven. Some firms initially disputed audit findings, while others acknowledged them and worked to reduce error gaps. The pattern suggests that commercial incentives to grow a promising market dampened attention to foreseeable harms until external pressure made those harms impossible to ignore.

Inclusion and Responsiveness

Inclusion was limited before 2020; the communities most likely to be misidentified had little voice in decisions about deployment. The June 2020 decisions were, above all, acts of responsiveness, changing course in light of evidence and public values. Responsiveness after harm, however, is weaker than responsiveness built into the innovation process. The firms' statements also differed in scope. IBM described an exit from general-purpose products, while Amazon's pause applied only to police use and was time-limited at first, and Microsoft's commitment depended on future legislation. Each left questions about sales to other agencies and about the many smaller vendors that continued to supply police.

Stakeholders, Rights and Consequences

Three lines of ethical reasoning point in the same direction. A stakeholder view asks whose interests are affected; here, people subject to searches, especially those in groups with higher error rates, bore risks without consent or recourse. A rights view emphasizes due process and freedom of assembly, which unregulated surveillance can erode regardless of accuracy. A consequences view weighs benefits such as solved crimes against harms such as wrongful arrests and chilled protest; because the distribution of harms was unequal and the legal safeguards absent, the expected harms were difficult to justify. Each view supports restricting sales until rules and accuracy improved.

Was It the Right Decision?

The decisions can be defended as ethically sound but late and partial. They were sound because the evidence of unequal accuracy, the lack of legal limits and the high stakes of policing justified restraint. They were late because the risks had been documented years earlier, and partial because a pause by three large firms did not end police use when other vendors filled the gap. A skeptic might add that the decisions cost the firms little, since facial recognition sales to police were a small part of their business. That point does not make the decisions wrong, but it cautions against treating them as models of corporate courage.

What Came Afterward

Events after 2020 show how partial voluntary restraint was. Amazon extended its moratorium on police use of Rekognition indefinitely in 2021, while other vendors continued to supply police agencies, and several cities and states adopted their own limits on government use of the technology. Reports of wrongful arrests based on facial recognition matches continued to appear in the press. The pattern supports the view that firm-level decisions mattered as signals but could not substitute for public rules on how police may use the technology and what safeguards must accompany a match.

Lessons for Innovation Leaders

Four lessons follow. First, anticipation should be part of innovation governance from the start, through structured reviews of who could be harmed and how, especially for technologies used by powerful institutions on less powerful people. Second, independent audits should be invited rather than resisted, because they reveal problems internal testing may miss. Third, affected communities should have a voice before deployment, not only after controversy. Fourth, firms should support public rules for high-stakes uses, because voluntary restraint by some firms shifts business to others without protecting the public.

Implications for Strategy

Ethics and strategy are connected in this case. Reputation with employees, customers and regulators is an asset that a controversial product can damage. Firms that build responsible innovation practices early can move with more confidence into sensitive markets, because they can show how risks are managed. And firms that help shape sensible regulation can compete on a level field rather than racing toward the least careful practices. These considerations will inform the evidence-based innovation strategy in the final module.

Conclusion

The 2020 decisions by IBM, Amazon and Microsoft to stop or pause selling facial recognition to police responded to evidence of unequal accuracy, absent legal limits and high stakes for civil rights. Judged by the responsible innovation framework, the firms showed responsiveness but weak anticipation, uneven reflexivity and little inclusion before the controversy. Stakeholder, rights and consequences reasoning all support restraint. The decisions were sound but late and partial, and they show that ethical governance of innovation must begin before deployment and extend to support for public rules.

References

Grother, P., Ngan, M., & Hanaoka, K. (2019). Face recognition vendor test (FRVT) part 3: Demographic effects (NIST Interagency Report 8280). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.IR.8280

Owen, R., Macnaghten, P., & Stilgoe, J. (2012). Responsible research and innovation: From science in society to science for society, with society. Science and Public Policy, 39(6), 751-760. https://doi.org/10.1093/scipol/scs093

Raji, I. D., & Buolamwini, J. (2019). Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial AI products. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (pp. 429-435). Association for Computing Machinery. https://doi.org/10.1145/3306618.3314244

Stilgoe, J., Owen, R., & Macnaghten, P. (2013). Developing a framework for responsible innovation. Research Policy, 42(9), 1568-1580. https://doi.org/10.1016/j.respol.2013.05.008