ABSTRACT
The application of AI in the criminal justice system has been gaining prominence in recent times in matters of facial recognition, crime prediction, identifying suspects and assessing risks. Though the use of such technology would make the process more efficient, it would also make the rationale behind such decision-making very complex. This phenomenon is referred to as algorithmic opacity. The use of an AI system in the investigative process, arrest and the sentencing process makes it very difficult for a person to comprehend the decision-making process. This paper analyses the issues related to the opacity of AI decisions from a legal perspective. In addition to the above, it focuses on issues arising out of AI-related decisions as per Articles 14 and 21 of the Indian Constitution.
INTRODUCTION
Central to any evidentiary decision-making is the burden of proof, which entails the legal duty of proving the matters alleged by a party. It is a general principle in litigation that is captured in the maxim Ei Incumbit Probatio qui Dicit, Non-Qui Negat which means, the burden of proof lies upon the person who asserts, not upon the one who denies.
Technologies have affected the manner in which crimes are being investigated and how criminal justice authorities make their decisions. The algorithms may process large amounts of data, detect facial matches, estimate possible places of crime commission and evaluate the probability of committing the crime in the future. However, algorithmic opacity becomes a significant challenge when it is impossible to comprehend the logic of an algorithm’s result.
Central to any evidentiary decision-making is the burden of proof, which entails the legal duty of proving the matters alleged by a party. It is a general principle in litigation that is captured in the maxim Ei Incumbit Probatio qui Dicit, Non-Qui Negat which means, the burden of proof lies upon the person who asserts, not upon the one who denies.
For instance, an individual may become a suspected person or a dangerous offender according to an algorithm’s decision, but the reasons why this person became a suspect will not be understandable. In criminal justice, it is especially alarming when a decision may affect human rights, including freedom, reputation and the right to a fair trial.
The fundamental issue here is whether judicial decision-makers should adopt AI-based decisions when their rationale cannot be adequately analysed. The purpose of this essay is to prove that AI can help criminal justice, but only in the case of its transparent, lawful and human-controlled use.
UNDERSTANDING ALGORITHMIC OPACITY IN CRIMINAL JUSTICE
Algorithmic opacity refers to the complexity of the process used by an AI algorithm to produce an outcome. It may be due to the use of complicated mathematics, the availability of huge amounts of data, or claims by the business entity about the confidentiality of the algorithm.
In one example, facial recognition technology identifies an individual as a potential match on a police watchlist. In another case, a prediction of crime software identifies an area as a place prone to committing criminal acts using records from the past. But there is a risk that the output is influenced by faulty data or bias in the records.
The challenge posed by AI is not just the possibility of making errors. Human beings also err in making decisions. But the issue here is that the output of an algorithm appears to be scientific and objective despite flawed reasoning. When police officers, prosecutors, and judges use the results without considering the limitations of the output, they become a source of influence without proper scrutiny.
CONSTITUTIONAL CONCERNS UNDER INDIAN LAW
The Indian Constitution offers significant checks on the discretion of the state. While Article 14 is concerned with equality before the law and equal protection of the law, Article 21 is concerned with the right to life and personal liberty. Such provisions are important where the application of AI systems impacts criminal investigation, custody or sentencing of offenders.
In the case of Justice K.S. Puttaswamy v. Union of India, (2017) 10 SCC 1 (India), the Supreme Court held that the right to privacy was an essential aspect of Article 21 and stressed the significance of dignity and autonomy. Such a decision is important in relation to AI systems that deal with the processing of personal data and biometric data.
Moreover, the problem is further aggravated if the algorithms are constructed based on historically discriminatory data. The algorithm can continuously recognise an area as high risk because it has been subjected to excessive police activity in the past. The above constitutional provisions do not outrightly prevent the use of AI but impose on the State the obligation to ensure that the technology is applied in a lawful, fair, and proportional manner. When the AI affects a person adversely, the use of the technology cannot be used as an arbitrary exercise of power.
JUDICIAL APPROACH: PRECEDENTS ESTABLISHED
In State v. Loomis, 881 N.W.2d 749 (Wis. 2016), the Supreme Court of Wisconsin reviewed the use of the COMPAS risk-assessment tool while sentencing. There were apprehensions regarding the use of the algorithm because the methodology of the same was not available for independent analysis. Use was allowed with certain limitations and precautions. One of such limitations is the restriction that the assessment cannot determine the sentence on its own.
This case shows how hard it is to apply the use of an algorithm in the criminal justice process when the methodology of the algorithm cannot be reviewed.
Another important concept is that of facts especially within a person’s knowledge. In the case of Shambhu Nath Mehra v. State of Ajmer, AIR 1956 SC 404, the Supreme Court of India has made it clear that facts especially within a person’s knowledge cannot be taken advantage of by the prosecution for relieving itself of the burden of proof. This is an interpretation of Section 106 of the Indian Evidence Act, 1872 and its counterpart, Section 109 of the Bharatiya Sakshya Adhiniyam, 2023.
The problem assumes critical significance in cases where AI-generated images, videos, audio, or text are used as evidence. Such data may seem real even if it is fabricated. When the technical details that can be used to verify their genuineness are not available, the party contesting the evidence will find itself in great trouble.
In Maneka Gandhi vs Union of India, (1978) 1 SCC 248, the Supreme Court has ruled that any process impacting personal liberty must be fair and reasonable. Though the ruling has not involved any discussion regarding AI, its relevance cannot be ignored while dealing with an automated system impacting liberty.
THE NEED FOR TRANSPARENCY AND ACCOUNTABILITY
Transparency in criminal justice need not mean the release of every piece of code used in an AI system. What is needed is enough information to enable the court, the accused, and their counsel to understand and contest any decision taken with the help of AI systems.
Firstly, any AI-assisted evidence must come with information on how it has been produced, what information was used, and what its limitations are.
Secondly, courts must make an accurate assessment of the accuracy of AI-assisted evidence. There must not be blind faith in the conclusion drawn from such a process.
Third, there should be accountability for mistakes, discrimination, and any abuse that results. Systems applied in the criminal justice process need to be tested for accuracy and discrimination, with their applications recorded so that the systems can be reviewed at a later date.
Finally, the accused needs the opportunity to challenge the evidence that is being used to prosecute them. The confidentiality of commercial information does not give the courts an excuse not to investigate the evidence presented.
WAY FORWARD
There should be well-established mechanisms regulating the use of AI technology in criminal investigations and the trial process in India. The requirements can be formulated in terms of testability, disclosure, audit, and human supervision of AI technologies. It is necessary to elaborate on the criteria for evaluating the evidence obtained via AI.
The provisions of Section 109 of the BSA might be applicable in situations where a person is responsible for proving the fact of knowledge. Still, it should be used prudently, and its application cannot provide a complete answer to the problem of algorithmic opacity. The need for prosecution to prove beyond reasonable doubt the accused person’s criminal responsibility must be maintained.
CONCLUSION
The use of AI can lead to more efficient criminal investigations and forensic analysis. But there is also a risk in the process because of the unexplainability and uncontrollability of the use of this technology. This might lead to a violation of the rights of people related to unfairness, privacy, and fair trial, especially due to the impact of AI products on decision-making regarding the liberty of people.
Several constitutional and evidentiary provisions in India can serve as a basis for ensuring the appropriate use of this technology. It should help the process of administration of justice, but not take control over it.
“PRIME LEGAL is a National Award-winning law firm with over two decades of experience across diverse legal sectors. We are dedicated to setting the standard for legal excellence in civil, criminal, and family law.”
WRITTEN BY: DEBARTAI CHATTERJEE
- Samyukta Ramaswamy, The Evidence Machine: Rethinking Admissibility and Privacy in India’s AI Surveillance State, 20(2) Indian J.L. & Tech. (2024), https://repository.nls.ac.in/ijlt/vol20/iss2/4/
- Setty Hrudai Tej, Regulating AI-Generated Deepfake Evidence Under the Bharatiya Sakshya Adhiniyam, 2023, 16(3) IUP J.L. Rev. 5 (2026), https://iupindia.in/ViewArticleDetails.asp?ArticleID=9415
- Brandon L. Garrett & Cynthia Rudin, The Right to a Glass Box: Rethinking the Use of Artificial Intelligence in Criminal Justice, 109 Cornell L. Rev. 561 (2024), https://publications.lawschool.cornell.edu/lawreview/wp-content/uploads/sites/2/2024/04/Garrett-Rudin-final.pdf
- Vincent Chiao, Fairness, Accountability and Transparency: Notes on Algorithmic Decision-Making in Criminal Justice, 15 Int’l J.L. Context 126 (2019), https://doi.org/10.1017/S1744552319000077
- Tatiana Dancy & Monika Zalnieriute, AI and Transparency in Judicial Decision Making, 46 Oxford J. Legal Stud. 1 (2026), https://doi.org/10.1093/ojls/gqaf030
- Sreehari V.S., Gautham Vishnu N., Kiran S. Sekhar, Surya Das & Charuta Nair, Artificial Intelligence and Predictive Policing in the Indian Criminal Justice System: Methods, Applications and Constitutional Concerns, 3 J. Intelligent Decision Making & Information Sci. (2026), https://doi.org/10.59543/jidmis.v3.1948
