In their own wordsWhat lawyers say goes wrong with AI.
29 posts from public forums where lawyers talk about their work, sorted into the four problems that come up most. Find the one that sounds like your week. Each links to what to do about it.
Doing the same work twice
Never Start From Scratch
Re-uploading the same files, re-explaining the same case, rebuilding the same document. What starting over actually costs. 6 posts.
“AI has the document. It doesn't have the history.”
“It [a legal AI tool] can't leave comments throughout for associates to address or suggest alternative docs to cite as it has no knowledge of the case.”
“GenAI is now good enough to do a very usable first mark-up of an SPA against a term sheet [...] loan agreements are usually 80-400 pages, exhausting the context window [...]”
“I use mock names for the attorneys and clients (like DOE and KOE), throw it into ChatGPT and ask for a rewrite for clarity, organization and professionalism, throw it in Word, [...] and throw it on letterhead.”
“Have tried endlessly to build a reliable Claude skill but Claude is flummoxed by anything that needs careful formatting fidelity.”
“whether the tool remembers the file across sessions (most chat tools do not; that is where a human still compounds).”
Checking what the AI wrote
The Verification Tax
Checking an AI-drafted citation takes minutes each. Lawyers on what that adds up to, and what happens when it gets skipped. 9 posts.
“I wouldn't use AI for anything that requires citing case law. You're going to have to check every citation individually anyway, so why take the risk you miss it changing the 2d to a 3d in a federal citation and get in serious trouble with the judge/bar? [...]”
“lawyers are getting sanctioned much more often than every week. On average it's more than one a day. [AI Hallucination Cases Database]”
“I have found it much more helpful for drafting than for research. [...] But citations to cases and statutes as well as any statements of law........you better believe I'm checking it independently. It's unbelievable to me at this point that some attorneys haven't seen the stories and don't verify everything it does! Hell I look over everything my paralegal does before it gets filed. Same reason!”
“It's very helpful if you feed it your notes and research and ask it to organize your materials into a motion. That way, you can also easily spot the hallucinations because you did the research yourself. [...] But I carefully research whatever it comes up with.”
“What happened here is inexcusable, and Butler Snow sincerely apologize[s].”
“I do like deep research, but I caught it [an AI tool's deep-research feature] hallucinating a quote from a case. It mashed together part of the actual opinion with the key header note and attributed it a quote. It was *correct*, but definitely not a quote. [...] I reported it.”
“the made-up case law is the worst part. we've checked AI-drafted docs and the fabrication rate is surprisingly high. perfectly formatted, plausible sounding. courts catching it is mostly luck - someone has to actually verify each one.”
“I wish I could bill for fixing my supervising attorney's AI briefs. Instead I'm stuck working late to clean up other attorneys messes. Fake citations with fake quotes, real citations with fake quotes, fake string citations... Every citation is a new surprise!”
“Ahh, sure - the solution is check the work. Fairly basic concept and surely one that's applied to the work produced by juniors (human ones). The lawyers who mature and evolve their craft to leverage the efficiency and time savings without sacrificing accuracy are going to dramatically outpace those who don't. Enjoy!”
Getting the firm to say yes
The Managing Partner Packet
The partner says no, the policy goes unread, and the confidential file ends up in a chatbot anyway. 8 posts.
“He's convinced that if we upload sensitive case files they're going to be used to train public models or somehow leak to other users. We've tried explaining that enterprise versions are different but he wants ironclad proof that we aren't violating attorney-client privilege.”
“I don't want to be made an example out of for being an early adopter.”
“It's more, to me, what data am I feeding into the research tool, does my client know I'm using it, and is my client aware that the training set might be used by others?”
“The managing partner is mixing three different risks into one sentence: (1) the vendor trains on your uploads, (2) the vendor retains your uploads, and (3) your matter data leaks to other customers. Those are separate contract terms. [...] For the proof pack your partner wants from vendors: (a) signed DPA language on training and retention, (b) whether human reviewers can see customer content, (c) data residency options if you need them, (d) an admin audit log of who uploaded what to which matter. If they cannot produce those four, walk.”
“One of the attorneys at my firm uses chatGPT for everything. [...] He has it prepare his briefs and uploads client information to it. He even has his paralegals run everything through chatGPT before showing it to him. [...] I am also concerned about the uploading of confidential information to AI platforms.”
“My problem is that people at our firm are still using all sorts of AI. Still, most of them have no idea of the limitations, and because we don't have a secure AI are uploading it into an LLM that will learn from that material and potentially breach confidentiality.”
“Your mistake might be trying to win this with arguments. Your managing partner is asking the right question, he's just asking it of the wrong person. You shouldn't be the one proving the data is safe. The vendors should. [...] Framed this way, your managing partner stops being the blocker and becomes the standard-setter.”
“being pressured to use AI tools without adequate human oversight, then held personally accountable for the consequences.”
Cleaning up someone else's AI
The AI Mess
Clients, opposing counsel and pro se filers now arrive with AI-drafted work. Cleaning it up is billable time nobody planned for. 6 posts.
“So I quoted a reasonable flat rate fee to resolve a real estate matter for a former divorce client. She then uses AI to draft up some bare-bones documents and instructions and emails them to me. The documents will not fully resolve her issues, and may cause other problems. She then asked me to requote my fee.”
“Except for the fact that your pro se opposing counsel is also using AI, and now their motions seem just credible enough to require way more effort to disentangle”
“One of my clients took a pretty complicated and novel royalty agreement and asked ChatGPT to review it. The "review" it spit out was insane. It referenced sections and paragraphs which didn't exist. It couldn't even accurately summarize how indemnity risk was allocated.”
“I just tell clients that I'm a lawyer and I'm not allowed to just submit somebody else's work without painstakingly reviewing it. And thus it would be more work for me to review their entire document and make all the necessary changes than it would be for me to draft it from scratch.”
“I have been dealing with similar situations for the past few months. I advise my clients that, in my experience, working from AI generated documents involves additional work, sometimes a lot of extra work, and that I would charge at my hourly rate based on the time spent.”
“I just caught OC with 49 hallucinations in 50 citations (!!!)”
How this page works
Where these come from
These are public posts from online forums and discussions where legal work and legal technology get talked about: what is breaking, what is being tried, and what it costs. We cannot verify who anyone is, so nothing here is presented as a credential.
Each line is copied exactly as written, typos included. Cuts are marked. No username, no firm, no link to the individual.
Each quote sits under the problem it describes, and links to the page where we answer it. This page is refreshed monthly as new lines are found.
These are individual accounts from public online discussion, not a representative sample or a guaranteed outcome.
See Irys on your own matter.
Tell us about your practice and we will set up a walkthrough on the work you actually do.
A focused 30 minutes built around your matters.
Your documents, not a generic demo dataset.
Answers on security and pricing in the same call.
No credit card. Transparent pricing. 7-day free trial.