Policing by Algorithm: Bias, Accountability, and the Crisis of Equal Justice
Author: Priyam Pratik
Student, Faculty of Law, University of Allahabad
—————————————————————————————
💡 3 Quick Takeaways
- Predictive policing systems often reproduce historical patterns of discriminatory enforcement because they are trained on data shaped by decades of unequal policing practices.
- Existing constitutional and statutory safeguards struggle to address algorithmic decision-making due to issues of opacity, proprietary protections, and doctrinal limitations.
- Meaningful accountability requires transparency, independent audits, enforceable oversight mechanisms, and accessible legal remedies for those adversely affected by algorithmic systems.
Introduction
Predictive policing has become an increasingly significant feature of modern law enforcement. Across numerous jurisdictions, machine-learning tools are now used to determine patrol deployment, assess pretrial detention risks, and inform sentencing and parole decisions. These systems are frequently promoted as objective and data-driven mechanisms capable of enhancing efficiency and reducing human error.
Yet the appearance of neutrality conceals a deeper concern. The data used to train predictive policing systems is not generated in a vacuum. Rather, it reflects decades of law enforcement practices, including patterns of surveillance, arrest, and prosecution that have often disproportionately affected racial and ethnic minorities.
The deployment of PredPol by the Los Angeles Police Department in 2013 illustrates this tension. Officials credited the technology with reducing arrests in targeted areas. However, questions remained regarding whether the observed outcomes reflected genuine reductions in crime or merely changes in enforcement practices. Officers were repeatedly directed toward locations identified as high-risk, generating more stops, more arrests, and more data that reinforced the system’s future predictions.
This cycle raises a fundamental question: are predictive policing systems genuinely identifying crime, or are they simply reproducing historical policing patterns under the guise of technological objectivity?
The answer carries significant implications for constitutional governance and the rule of law. Public authority must operate through transparent standards, equal treatment, and meaningful opportunities for review. This article argues that contemporary predictive policing practices frequently undermine these principles and require substantial legal reform.
Understanding Predictive Policing and Algorithmic Bias
Predictive policing systems generally operate through two primary models.
Place-Based Predictive Systems
Place-based systems, such as PredPol, identify geographic locations where criminal activity is predicted to occur within a specified period. Law enforcement agencies use these predictions to allocate patrol resources and direct officer presence.
Person-Based Predictive Systems
Person-based systems generate risk assessments relating to individuals. These assessments may influence decisions concerning:
- Bail;
- Pretrial detention;
- Sentencing;
- Probation; and
- Parole.
One of the most widely studied examples is the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) system.
Although these systems differ in application, both rely heavily on historical crime and enforcement data. Consequently, they inherit the biases embedded within that data.
The central problem is straightforward. Historical crime databases do not merely record criminal activity; they record law enforcement activity. Arrest statistics often reflect where officers were deployed, whom they stopped, and whom they chose to arrest. Where policing practices have disproportionately targeted particular communities, those disparities become embedded in the data used to train predictive systems.
As a result, algorithms frequently learn patterns of enforcement rather than patterns of criminal conduct.
Drug enforcement provides a particularly revealing example. Studies consistently indicate broadly similar rates of drug use across racial groups. Yet arrest rates often vary significantly. An algorithm trained on arrest data therefore learns who was arrested rather than who engaged in drug-related activity.
This distinction lies at the heart of contemporary concerns regarding algorithmic bias.
Empirical Evidence of Algorithmic Discrimination
One of the most influential examinations of algorithmic bias emerged from ProPublica’s 2016 investigation into COMPAS.
The study analysed thousands of criminal defendants in Broward County, Florida, and found substantial disparities in risk assessments. Black defendants were more frequently classified as high-risk despite not reoffending, while white defendants were more frequently classified as low-risk despite subsequent recidivism.
The findings suggested that racial disparities existed not merely in outcomes but also in the distribution of prediction errors.
The company responsible for COMPAS disputed the methodology employed by the researchers. However, it declined to disclose the algorithm’s internal workings, citing proprietary and trade secret protections.
This refusal to permit meaningful scrutiny highlighted a broader challenge. If the mechanisms generating predictive scores remain inaccessible, evaluating their fairness becomes exceptionally difficult.
Similar concerns have arisen in relation to place-based predictive policing systems.
Research conducted by the Stop LAPD Spying Coalition and the Human Rights Data Analysis Group examined predictive policing practices in Los Angeles and found that patrol resources were disproportionately directed toward predominantly Black and Latino neighbourhoods. Increased police presence generated additional arrests, which were subsequently incorporated into future predictions, thereby reinforcing existing enforcement patterns.
The resulting feedback loop illustrates how predictive systems may perpetuate structural inequalities even without explicitly incorporating race as a variable.
Constitutional Challenges
The increasing use of predictive policing technologies raises significant constitutional concerns relating to the Fourth Amendment, Equal Protection, and Due Process.
The Fourth Amendment and Reasonable Suspicion
The Fourth Amendment protects individuals against unreasonable searches and seizures.
In Terry v. Ohio, the United States Supreme Court held that investigatory stops require reasonable suspicion based upon specific and articulable facts.
Predictive policing complicates this requirement. When officers rely upon algorithmic predictions rather than direct observations, questions arise regarding whether predictive outputs satisfy constitutional standards designed for human judgment and individualized suspicion.
Federal courts have yet to establish a clear and consistent position regarding the extent to which algorithmic predictions may contribute to reasonable suspicion.
This uncertainty creates significant legal ambiguity concerning the constitutional limits of predictive policing.
Equal Protection and Structural Discrimination
The Equal Protection Clause presents a different challenge.
Under Washington v. Davis and Village of Arlington Heights v. Metropolitan Housing Development Corp., proving unconstitutional discrimination generally requires evidence of discriminatory intent rather than merely discriminatory effects.
Predictive systems are typically designed to appear facially neutral. They rarely include race as an explicit variable. Instead, they rely upon factors such as geographic location, prior contacts with law enforcement, and social associations.
These variables often function as proxies that correlate strongly with race without expressly identifying it.
Consequently, algorithmic systems may generate discriminatory outcomes while remaining insulated from traditional Equal Protection challenges because intentional discrimination is difficult to establish.
The result is a significant doctrinal gap between constitutional protections and contemporary forms of algorithmic decision-making.
Due Process and the Problem of Opacity
Due process concerns become particularly acute when predictive systems influence decisions affecting individual liberty.
Meaningful procedural fairness requires that individuals understand the basis of decisions affecting them and possess a genuine opportunity to challenge those decisions.
This principle becomes difficult to implement where proprietary algorithms remain undisclosed.
The controversy surrounding Loomis v. Wisconsin illustrates this problem. In that case, the sentencing court considered a COMPAS risk score when determining an appropriate sentence. The defendant argued that reliance upon a proprietary system denied him a meaningful opportunity to challenge the basis of the assessment.
Although the Wisconsin Supreme Court rejected the challenge, the broader constitutional concern remains unresolved.
If individuals cannot examine the factors underlying an algorithmic assessment, meaningful participation in legal proceedings becomes significantly constrained.
Limitations of Existing Legal Frameworks
Current statutory frameworks provide only partial responses to the challenges posed by predictive policing.
The First Step Act
The First Step Act of 2018 requires federal risk assessment tools to be evaluated for racial and gender bias.
While this represents an important development, its practical scope remains limited. The legislation primarily governs federal sentencing systems and does not extend to the vast majority of predictive policing practices occurring at the state and local levels.
Title VI of the Civil Rights Act
Title VI prohibits discrimination by recipients of federal financial assistance, including many law enforcement agencies.
In principle, Title VI could address algorithmic practices that generate racially disparate outcomes.
However, the Supreme Court’s decision in Alexander v. Sandoval significantly limited private enforcement of disparate-impact regulations. Individuals generally cannot bring private actions based solely on disparate impact and must instead rely upon administrative enforcement mechanisms.
This limitation reduces the effectiveness of Title VI as a tool for challenging algorithmic discrimination.
State and Local Responses
Several jurisdictions have attempted to regulate algorithmic decision-making independently.
California, Illinois, New York City, and various municipalities have adopted measures addressing algorithmic governance, while cities such as San Francisco, Boston, and Portland have imposed restrictions on certain surveillance technologies.
Although these initiatives represent important developments, they remain fragmented and inconsistent. The absence of a comprehensive national framework creates significant variation in accountability standards across jurisdictions.
The Rule of Law and the Need for Reform
The concerns surrounding predictive policing extend beyond technical questions of algorithmic design. They implicate fundamental principles of the rule of law.
A legal system derives legitimacy from transparency, consistency, accountability, and public accessibility. Individuals must be capable of understanding the rules that govern them and challenging decisions that affect their rights.
Predictive policing systems frequently struggle to satisfy these requirements.
Transparency
Any algorithmic system that influences liberty-related decisions should be subject to meaningful disclosure.
Courts, legal representatives, and affected individuals must possess access to sufficient information to understand how decisions are generated and to challenge inaccuracies where necessary.
Trade secret protections should not prevent constitutional scrutiny when government action affects individual liberty.
Independent Auditing
Predictive systems should undergo mandatory independent evaluations before deployment.
These audits should assess:
- Predictive accuracy;
- Error rates;
- Demographic disparities;
- False-positive outcomes; and
- Overall reliability.
Independent review would provide greater confidence in system performance and reduce the risk of hidden biases.
Accountability Mechanisms
Accountability should extend to both government agencies and private vendors.
Developers who conceal known deficiencies or misrepresent algorithmic performance should face legal consequences. Agencies deploying predictive systems should also be required to publish periodic reports detailing performance metrics and demographic impacts.
A national registry documenting government use of algorithmic decision-making tools could further strengthen oversight.
Access to Legal Remedies
Individuals adversely affected by discriminatory algorithmic outcomes should possess meaningful avenues for legal redress.
Without accessible remedies, transparency and auditing requirements alone may prove insufficient.
Effective accountability requires mechanisms through which affected individuals can challenge unlawful or discriminatory practices.
Conclusion
Predictive policing is not inherently incompatible with constitutional governance. Data-driven decision-making has the potential to improve consistency, reduce certain forms of human bias, and enhance administrative efficiency.
The difficulty lies not in the use of data itself but in the quality of the data and the institutional structures surrounding its deployment.
Contemporary predictive policing systems frequently reproduce historical patterns of discrimination because they rely upon datasets shaped by unequal enforcement practices. Through the language of scientific objectivity, longstanding disparities risk being transformed into seemingly neutral algorithmic outcomes.
This transformation presents serious challenges for existing constitutional doctrines. Equal Protection jurisprudence focuses on discriminatory intent, while algorithmic systems often generate discriminatory effects without identifiable intent. Fourth Amendment standards were developed for individualized human judgment rather than machine-generated predictions. Due process protections are weakened when proprietary systems prevent meaningful scrutiny.
The reforms required are neither radical nor unprecedented. Transparency, independent auditing, accountability, and effective legal remedies represent longstanding principles of responsible governance.
As algorithmic systems increasingly influence decisions relating to liberty, public safety, and criminal justice, the rule of law must evolve to ensure that technological innovation remains subject to constitutional values. Without such safeguards, predictive policing risks becoming a mechanism through which structural inequality is reproduced under the appearance of objectivity.
Disclaimer: The views expressed in this article are those of the author and do not necessarily reflect the views of The Lawscape.
The Lawscape — clear, practical legal insight for students and future lawyers.