Seoul Gazette - Courts cracking down on error-strewn AI-assisted legal briefs

That got him a fine and a mandatory course on how to use AI properly.

But this case was just one in a rising tide of AI "hallucinations" muddying the legal waters. Now some courts are trying to stem the tide by cracking down on the lawyers concerned.

It was judge Jerry Edwards Jr., of Louisiana district court, who sanctioned the lawyer in that recent US case last month. He found the AI-generated hallucinations while reviewing a motion to block evidence from being admitted in a personal injury case.

He caught seven quotes attributed to previous court rulings that either did not appear in the original decisions or had been misrepresented.

The hapless lawyer said a law clerk had caught other fabricated quotes in a first draft. Instead of learning from his mistake however, the attorney simply asked Claude to correct the errors -- then submitted the brief without reviewing it.

The attorney had believed the AI's output was accurate, said the court's May 18 memorandum order announcing the sanctions.

"Ignorance of the risks of AI usage is no longer an excuse," the judge wrote, fining the lawyer $1,000 and sending him on a three-hour course on AI-assisted legal practice.

But he at least granted part of the motion.

- Hundreds of cases -

AI-generated blunders do not necessarily destroy a lawyer's case, French lawyer Damien Charlotin told AFP.

"The lawyer's credibility is in tatters -- but there are cases where lawyers still win, despite the hallucinations, because they were right on the merits," he explained.

But there are other issues.

Judge Linda Kevins of New York State's Supreme Court warned in a January ruling that such blunders wasted the time and money of both the opposing party and the court.

In addition, she said, they were "potentially harming the reputation of judges and courts whose names are falsely invoked as authors of the bogus opinions".

Charlotin has been compiling a database of court documents containing AI hallucinations since April 2023.

So far, he has 1,600 examples from 35 countries: from fabricated quotes and cases, to plagiarised arguments or references and out-of-date advice.

Since it is a crowd-sourced database it cannot count as the definitive list, but by far the most cases -- 1,116 of them -- come from the United States.

Next comes Canada, with 173 cases, Australia with 74, then Britain with 59.

- 'Do your job!' -

"In the legal field, AI is used for research, finding precedents, and sometimes generating text," said Charlotin.

Lawyers and litigants representing themselves who submitted documents containing AI hallucinations were most often using mainstream chatbots, particularly OpenAI's ChatGPT.

The data suggests that specialised legal AI tools, used by many law firms, are more reliable -- but they are still not immune from hallucinations.

"AI thrives on recognising patterns. And legal citations and arguments are always formatted in the same way, so it's easy for an AI to follow a template and generate fake ones," said Charlotin.

"Because no lawyer knows all past cases, when a chatbot outputs something that looks like a law citation, the only way to know it actually exists is to check it," he added.

But some lawyers are still not doing that, said judge Scott Schlegel, who serves on the American Bar Association's Task Force on Law and Artificial Intelligence.

Too many attorneys still do not understand the technology's limits, he told AFP. Time pressures are compounding the problem, Charlotin's database shows.

But that's no excuse, Schlegel told AFP in February.

"At this point, I just can't understand how we still have the issue. Just do your job and read the cases. Come on!"

- Sanctions -

According to Charlotin's data, hallucinations in court filings have increased eightfold in the past year compared with the previous 12 months.

He is testing his own AI-powered tool to help lawyers spot hallucinations before they are submitted to courts. But he cannot guarantee it will not itself make mistakes.

"It's still up to the lawyers to adapt their verification process to the issues at hand," he said.

Databases such as Court Listener have a growing list of rulings on AI-generated calamities in the US courts. And in several such cases, judges are handing out more than just reprimands.

Last week, federal judge Sharion Aycock sanctioned lawyers on both sides of a civil case in Mississippi after they submitted AI-assisted legal briefs that cited non-existent cases.

Four lawyers were fined $8,000 in total, and two of them were barred from appearing in the Mississippi Northern District Court for two years.

"In an era of rampant unverified AI usage within the legal field, this case presents a prime example of the risk associated with serving as a rubberstamp when acting as local counsel", she wrote in the sanctions order.

L.Jeong--SG