Using AI for Legal Document Review: Benefits and Limitations

In 2023, two New York attorneys submitted a legal brief citing six court cases to support their argument. All six were fabricated — invented wholesale by the AI chatbot they had used to draft the filing, complete with plausible-sounding docket numbers and judicial reasoning that never existed. The court sanctioned both lawyers. The case, Mata v. Avianca, is now the cautionary tale cited in nearly every CLE session on generative AI, and for good reason: it captures both halves of the story in one incident. AI genuinely can produce fluent, structured, seemingly authoritative legal work in seconds — and it can be confidently, catastrophically wrong while doing it.

Legal document review is one of the oldest and most labor-intensive parts of practicing law, and it is also one of the areas where AI has produced the clearest, most measurable gains anywhere in knowledge work. JPMorgan's COIN platform famously cut the time spent reviewing commercial loan agreements from roughly 360,000 attorney hours a year to a task measured in seconds. Predictive coding tools used in e-discovery have been judicially endorsed since 2012 for their ability to review millions of documents at a fraction of the cost of manual review. At the same time, the same underlying technology — large language models generating fluent text — is the reason bar associations across the U.S. have issued formal ethics opinions warning lawyers that they remain personally responsible for anything an AI tool puts in front of a judge.

This article walks through where AI fits into the legal document review workflow, what it's demonstrably good at, where it fails and why, and how firms and in-house legal teams are structuring their processes to capture the upside without inheriting the risk.

What "Legal Document Review" Actually Covers

The phrase covers several distinct workflows, and AI's suitability differs sharply across them:

The distinction matters enormously for how much you should trust the output. Tasks 1 through 4 are fundamentally classification and extraction problems — the AI is finding and labeling things that already exist in a document. Task 5 is a generation problem — the AI is producing new text, including citations, that must be independently true. Generation tasks carry categorically higher hallucination risk, which is why nearly every reported sanctions case involving AI in the legal profession falls into that fifth category.

How AI Fits Into the Review Pipeline

Regardless of which task above you're solving, a production-grade legal AI review workflow follows a broadly similar shape: documents come in, get prepared, get processed by the model, and then pass through some form of human checkpoint before anything becomes a decision that affects a case or a deal.

Two engines do most of the heavy lifting inside that pipeline, and they are not the same technology, even though marketing copy often blurs them together:

• Technology-assisted review (TAR / predictive coding) — a classifier trained on a small set of documents a human has already coded as responsive or not, then used to rank or bucket the remaining millions. This is the workhorse of e-discovery and has been recognized by U.S. courts as a legitimate discovery method since Da Silva Moore v. Publicis Groupe in 2012.
• Large language models (LLMs) — used for clause extraction, summarization, contract Q&A, and drafting. These are the tools behind most modern contract-review platforms (Kira Systems, Luminance, Evisort, Harvey, and similar), and they are also the source of the generation risk described above.

The Benefits: Where the Numbers Are Real

Speed and throughput at a scale humans cannot match

A trained associate might review 50–80 contracts a day for a specific clause. A well-tuned extraction model can process the same volume in minutes, and it doesn't get slower or less consistent on document 400 than it was on document 1. This is the single biggest driver of adoption: due diligence timelines that used to take weeks can compress to days.

Lower marginal cost per document

Legal review has historically been billed by attorney hour, which makes it one of the most expensive line items in litigation and transactional work. Shifting first-pass review to AI, with human validation layered on top, changes the cost structure from "linear in document count" to "roughly flat plus a smaller review layer" — which is why in-house legal departments and litigation funders have pushed hardest for adoption.

Consistency across a large document set

Human reviewers get tired, get distracted, and apply standards slightly differently at 4pm than at 9am — a well-documented phenomenon in large-scale manual review projects. A model applies the same criteria to document 1 and document 100,000, which is valuable specifically because consistency, not just speed, is what courts and regulators are increasingly asking parties to demonstrate in e-discovery.

Surfacing needles in enormous haystacks

In due diligence, the risk usually isn't in the 30 contracts everyone reads carefully — it's in the one non-standard indemnification clause buried in contract #287 of 400, in a portfolio nobody would have had time to read line-by-line under deal timelines. AI extraction tools are well-suited to exactly this kind of needle-in-haystack detection, because unlike a fatigued human, they apply full attention to every document.

Freeing attorney time for judgment work

The realistic framing isn't "AI replaces the associate," it's "AI absorbs the repetitive first pass and the associate spends their time on the ambiguous 10% that actually requires legal judgment." Firms that have restructured workflows this way report that junior lawyers spend more time on negotiation strategy and client counseling and less time on mechanical extraction — a shift that, done well, makes the work more valuable, not just faster.

The Limitations: Where It Breaks, and Why

Hallucination in generative tasks

When an LLM is asked to summarize a contract that's in front of it, it's grounded in real text and errors tend to be omissions or misreadings. When it's asked to recall a case, a statute, or a citation from memory, it can generate something fluent and specific that simply doesn't exist — as Mata v. Avianca and the string of similar sanctions cases since 2023 have shown. The failure mode is dangerous precisely because it doesn't look like a failure; a fabricated citation reads exactly like a real one.

Loss of context, negation, and cross-reference

Contracts are full of clauses that only make sense in relation to other clauses — a definition three pages earlier, an exception buried in a schedule, a "notwithstanding Section 4.2" that flips the meaning of the sentence that follows it. Extraction models can miss these dependencies, flagging a clause as standard when a cross-reference elsewhere in the document quietly changes its effect.

Confidentiality and privilege exposure

Feeding client documents into a third-party AI tool raises real questions about attorney-client privilege and data confidentiality, especially with consumer-grade tools that may retain or train on submitted content. Multiple state bar ethics opinions now specifically address this, generally concluding that lawyers must understand a vendor's data handling before using it on privileged material — not assume it's safe by default.

Training data reflects historical drafting, not current law

Models trained on large corpora of historical contracts and case law can be confidently out of date, especially in fast-moving areas like data protection or AI regulation itself. A model's "training cutoff" is invisible to the person reading its output, unlike a human associate who knows when they last checked a statute.

Explainability gaps

When a TAR system deprioritizes a document as unlikely to be responsive, or an extraction model flags a clause as low-risk, it's not always straightforward to reconstruct exactly why — which becomes a real problem if opposing counsel challenges the methodology, or if a regulator asks how a compliance determination was made. This is the same explainability problem seen across other high-stakes AI applications, and it's one reason courts have required detailed disclosure of TAR methodology in discovery disputes.

Over-reliance and skill atrophy

If junior associates never do first-pass contract review manually, they may reach senior roles without having built the pattern-recognition instincts that let a human reviewer sense "something about clause 14 feels off" — a form of tacit expertise that's hard to acquire by only checking an AI's homework.

A Decision Framework: How Much Human Review Does This Task Need?

Not every document review task carries the same stakes, and treating them identically is itself a mistake — either by over-trusting AI on high-stakes work, or by insisting on full manual review of low-stakes, high-volume work where AI adds clear value with little downside.

Getting the Benefit Without the Exposure: A Practical Checklist

• Match the tool to the task: classification/extraction tools for contract review and e-discovery; treat generative drafting and research tools as first drafts only
• Independently verify every citation, statute, and case reference an AI produces before it appears in anything filed or sent externally
• Confirm the vendor's data handling and confidentiality terms before uploading privileged or client-confidential material
• Disaggregate AI accuracy by document type and clause type — an aggregate 95% accuracy figure can hide much weaker performance on the specific clause you care about most
• Sample and manually re-check a defined percentage of AI-classified "non-responsive" or "low-risk" documents, not just the flagged ones
• Document the review methodology, including AI tool version and validation steps, in case it needs to be defended in discovery disputes or before a regulator
• Keep a qualified human reviewer accountable for every output that reaches a court, regulator, or counterparty — never let AI output go out the door unsupervised
• Retrain or recalibrate extraction models when contract templates, jurisdictions, or regulatory requirements change materially
• Preserve manual-review skill-building for junior staff, rather than letting AI fully replace the training function of first-pass review
• Check current bar association and court guidance on AI disclosure obligations in your jurisdiction — these are changing quickly

Frequently Asked Questions