There is a tendency in discussions about legal AI to focus on the wrong question. Most commentary centres on whether artificial intelligence can write convincing legal prose. Can it summarise judgments? Can it answer legal questions? Can it draft legal advice? Can it adjudicate cases?
Recent court guidance has focused on the risks of generative AI: hallucinations, fabricated authorities and unverifiable reasoning. The Supreme Court of Victoria’s Practise Note SC Gen 25 does not prohibit generative AI. It reinforces that professional responsibility for verification and accuracy stays with the practitioner.
These questions are understandable, but they are not the most important. The central problem in legal AI has never been language generation. It has always been legal reasoning and authority chaining.
Modern AI systems are exceptionally good at producing fluent text. A large language model can generate persuasive sounding legal analysis within seconds. It can summarise lengthy judgments and synthesise authorities into seemingly coherent legal reports. To many observers, this appears to be legal reasoning.
But appellate advocacy and judicial reasoning do not ultimately turn on whether something sounds plausible. They turn on whether propositions can be justified and whether reasoning holds together under scrutiny.
The present generation of legal AI tools largely operates through semantic retrieval and probabilistic generation. These systems retrieve relevant authorities and use generative models to produce summaries or answers.
Some systems work surprisingly well because legal writing possesses highly structured statistical properties. Judicial reasoning is repetitive. Similar doctrines cluster semantically. Courts repeatedly employ familiar forms of reasoning and doctrinal language.
This can produce commercially useful legal products. However, there is a deeper jurisprudential problem underlying these approaches.
Senior barristers and appellate judges frequently disagree about the ratio decidendi of a case. Different levels of abstraction produce different ratios. Courts routinely reinterpret or narrow earlier authorities. Concurring judgments complicate matters further. The distinction between ratio and obiter dicta is not always clear.
When a legal AI system claims to “extract the ratio” from a judgment, it is rarely identifying some universally agreed legal truth. More commonly, it is generating a professionally acceptable characterisation of the case by relying upon subsequent judicial reframing of the ratio.
That distinction matters.
Many current legal AI systems function as sophisticated semantic compression engines. They retrieve authorities, identify doctrinally similar passages and generate coherent summaries.
But common law reasoning is not reducible to elegant summarisation. Legal propositions do not acquire force merely because they are expressed persuasively. They acquire force from their relationship to the authority from which they are drawn.
A proposition stated in a judgment must be understood by reference to the issue before the court, the material facts treated, the reasoning necessary to the result, and whether the proposition formed part of the rule that led to the final outcome or was merely incidental observation.
Once legal propositions are separated from the precise reasoning supporting them, distortions begin to emerge. Holdings become overgeneralised. Conditional reasoning becomes universalised. Minority reasoning is merged into majority reasoning. Fact-specific determinations become abstract doctrinal rules.
These problems are often difficult for non-specialists to detect because the generated prose remains highly convincing. However, experienced appellate lawyers immediately recognise the issue.
The challenge therefore is not merely generating legal language. The challenge is preserving authority-constrained reasoning.
This requires a fundamentally different approach to legal AI. If legal reasoning is to be reliable under scrutiny, every proposition must remain traceable to authoritative source material. The inferential chain connecting proposition and authority must remain recoverable. This is where machine-checkable legal reasoning becomes significant. Such systems attempt to preserve the structural integrity of legal inference itself.
Under this approach, legal propositions are linked to precise authoritative support, including pinpoint references. Inferential dependencies can be examined and tested. Unsupported propositions become identifiable. Contradictions can potentially be detected. The reasoning process itself becomes auditable.
Once legal AI systems are required to maintain this level of attributional discipline, the engineering difficulty increases dramatically. Even minor errors become consequential. If a proposition is framed too broadly, if dicta is treated as holding or if factual conditions activating a rule are omitted, the reasoning chain may become invalid.
These are not superficial drafting issues. They are structural failures in legal reasoning. The danger is therefore not merely hallucination in the ordinary sense. The deeper danger is synthetic doctrinal coherence. An AI system optimised primarily for fluent synthesis may inadvertently generate a cleaner and more unified version of the law than actually exists.
This creates serious risks in legal practice. The issue is therefore not whether AI can assist legal work. It clearly can. AI systems are already transforming legal research, document review, disclosure analysis, chronology construction and drafting workflows.
The more important question is what type of legal reasoning architecture should govern these systems. If legal AI remains focused primarily on generation and summarisation, it risks producing systems that are persuasive but insufficiently constrained by authority.
In legal AI, the most significant development may ultimately not be generation at all. It may be the emergence of computational systems capable of preserving the authority structures upon which the common law itself depends.
The implications of this shift are no longer theoretical. Authority-grounded and machine-checkable legal reasoning systems already exist. These systems are capable of tracing inferential chains through judgments, linking propositions to precise authoritative support, testing reasoning consistency and supporting appeal-checking functions by identifying unsupported propositions, doctrinal inconsistencies and potential defects in authority treatment.
The significance of this development has not yet been fully appreciated within the broader legal technology market because the underlying advance is structural rather than cosmetic.
First published in the following three publications.
https://itbrief.com.au/story/the-real-problem-in-legal-ai-is-not-generation
https://channellife.com.au/story/the-real-problem-in-legal-ai-is-not-generation
https://cfotech.com.au/story/the-real-problem-in-legal-ai-is-not-generation
