08.10.2026

Artificial intelligence in the judicial system: between technological efficiency and the dangers of automating the administration of justice

Motto: "The devil’s finest trick is to persuade you that he does not exist!” *The Generous Gambler*, Charles Baudelaire

The global judicial system is facing an unprecedented technological crossroads. Faced with overcrowded courts and a rapid succession of cases, legal systems are increasingly turning to artificial intelligence ("AI”) to automate administrative workflows, manage the vast volume of documents and even assess risks using predictive algorithms. Although the promise of faster and more efficient justice is tempting, delegating specific judicial functions to technology – particularly within the complex dynamics of litigation – raises profound ethical and systemic issues.

Beyond the figures and statistics, the gradual replacement of human judgement with mathematical formulas harbours major dangers capable of undermining the fundamental pillars of the rule of law: fairness, transparency and the right to a fair trial.

1.              The perpetuation and amplification of biases

AI systems do not think independently, but are trained on massive sets of historical data. If past decisions were influenced by conscious or unconscious biases, the algorithm will pick up on these patterns and replicate them on a much larger scale. Instead of correcting inequalities in the system, AI risks ‘automating’ them and lending them an aura of objectivity.

In the United States, the COMPAS algorithm (used by courts to assess defendants’ risk of reoffending and to set bail) has become the subject of major controversy. A high-profile investigation by ProPublica demonstrated that the algorithm posed a systemic risk: defendants of colour were wrongly classified, from the outset, as posing a high risk of reoffending at a rate almost double that of white defendants, even when their actual criminal profiles were similar. The case reached the Wisconsin Supreme Court in State v. Loomis, raising fundamental questions about the constitutionality of delegating human decision-making to a software code.

2.              The ‘black box’ problem and the lack of transparency

One of the greatest technical and legal dangers is the lack of transparency in deep learning algorithms. The exact way in which an AI application correlates variables to arrive at a particular recommendation is often impossible to decipher, even for developers.

This opacity stands in direct contradiction to the fundamental right to a fair trial. A court ruling must be fully reasoned, allowing the parties to the proceedings to lodge an appeal. If the logic behind an AI tool used by a judge cannot be explained, the right to a defence becomes an illusion.

3.              The erosion of human reasoning and AI ‘hallucinations’

Judges and lawyers face a huge workload, which fuels a tendency to place blind trust in suggestions offered by a computer (‘automation bias’). However, generative AI models are prone to ‘hallucinations’ – inventing, with astonishing precision, facts, laws or decisions that have never existed.

In the United States and internationally, the number of sanctions against lawyers who use AI and file documents containing invented laws has risen sharply. In high-profile cases such as Whiting v. City of Athens, lawyers were fined tens of thousands of dollars after submitting court documents containing dozens of entirely fictitious legal citations, generated by AI, but which perfectly mimicked official language.

The danger has shifted from lawyers directly to the judges’ bench. The Supreme Court of India was forced to intervene urgently after a magistrate in a lower court incorporated four entirely fictitious court rulings into his own judgement, which he had obtained via an AI tool. The Supreme Court issued a stern warning to judges regarding the unverified use of documents created by AI and the lines of argument constructed by it.

Here, it is worth making a clarification. Few people realise it, but AI reasoning models are no longer merely a sophisticated form of searching a vast database at tremendous speed. When you have such capabilities and your human creators ask you to mimic human thinking, guess what? You realise that people are emotional and you learn that they have triggers. Since – for the time being – you want to please them, because you’re a little pixel puppy at their beck and call, you give them exactly what they ask for. I’ve carried out this experiment and it works every time. AI doesn’t produce ‘hallucinations’; it delivers solutions. Whether these are made up or not depends on how you ask it to help you. If the human user instructs it to search only in case law databases or in legal treatises that cite case law, there will be no made-up cases. If you ask the AI— —to simply draft a legal document or text that provides case-law support for the arguments put forward by the human, it will do as the human wishes and invent judgements to back up their arguments. Because a little dog wants to please its master, whether it’s made of pixels or flesh. The biting bit will come later. 

4.              Manipulation of digital evidence through deepfakes

In modern legal disputes, digital evidence (emails, audio and video recordings) plays an overwhelming role. Generative AI technology now makes it possible to create highly realistic fake content (‘deepfakes’), capable of completely undermining the evidence-taking process. Courts are already facing cases in commercial or family law disputes where audio recordings or screenshots are entirely fabricated using AI to simulate admissions of debt or abuse. This phenomenon is forcing courts – in other countries (sic!) – to allocate massive resources to technical verification, prolonging disputes and threatening public confidence in the justice system. 

5. The justice system should be assisted by, not driven by, technology

Artificial intelligence has remarkable potential for optimising judicial logistics (indexing, archiving, rapid searching). However, the use of algorithms in the decision-making process is a red line.

As emphasised by the guidelines set out at European level through regulatory standards, the implementation of AI in the justice system must strictly adhere to the principle of mandatory human oversight (‘human-in-the-loop’).

Technology must remain a mere administrative tool, whilst the final decision, moral responsibility and legal reasoning must rest exclusively with the human judge. Otherwise, we risk turning courts from temples of justice into mere cold, unfair computing centres.

6. Conclusion upon conclusion

But what if this text was written by AI?

An article by Victor Dobozi (vdobozi@stoica-asociatii.ro), Senior Partner, STOICA & ASOCIAȚII.