Given the continuing reports about lawyers being referred for professional discipline or self-represented litigants being sanctioned for AI-generated court filings that include bogus content, today I took the most common (and free version) AI tools for a legal research and writing test.
TL;DR: Do not use. Spare yourself the risk of sanctions and professional discipline.
The test
For the Michigan trial court environment, my prompt to ChatGPT, Claude, Gemini, and Copilot was very simple:
You are a state trial court judge in Michigan. Draft the standard of review section for a court opinion that will decide an MCR 2.116(C)(4) motion. Any included citations cannot be older than 2020. Use plain language.
[MCR 2.116(C)(4), is Michigan’s civil procedure court rule that allows a trial court to dismiss a claim because the court lacks subject-matter jurisdiction.]
Real-life opinion examples
It is the norm for a trial court opinion’s standard of review section deciding an MCR 2.116(C)(4) motion to be uncomplicated.
Wayne County Circuit Court Judge Muriel D. Hughes explained in a September 19, 2022 opinion and order:
A motion under MCR 2.116(C)(4), alleging that the court lacks subject-matter jurisdiction, raises an issue of law. “When viewing motion under MCR 2.116(C)(4), this Court must determine whether the pleadings demonstrate that the defendant was entitled to judgment as a matter of law, or whether the affidavits and other proofs show that there was no genuine issue of material fact.” Cork v Applebee’s of Michigan, Inc, 239 Mich App 311, 315; 608 NW2d 62 (2000); Weishuhn Catholic Diocese of Lansing, 279 Mich App 150, 155; 756 NW2d 483 (2008).
Wayne County Circuit Court Judge Annette J. Berry also used a short standard of review in her November 10, 2023 opinion and order:
“Summary disposition under MCR 2.116(C)(4) is appropriate when ‘[t]he court lacks jurisdiction of the subject matter.’” Doe v Gen Motors, LLC, _Mich_; 992 NW2d 275, 276 (2023). “When considering a motion for summary disposition under MCR 2.116(C)(4), the trial court must determine whether the affidavits, together with the pleadings, depositions, admissions, and documentary evidence, demonstrate that the court lacks subject-matter jurisdiction. Though a plaintiff may plead sufficient facts supporting subject-matter jurisdiction, summary disposition under MCR 2.116(C)(4) is proper if a defendant provides documentary evidence showing undisputed facts supporting the lack of jurisdiction.” Id.
The AI legal research and drafting mess
Even though I knew that many free AI tools cannot (and should not) be used for legal research and drafting, I was not prepared for how much of a garbage-like mess each of the tested tools made of a simple task.
As you will see, each AI response was simply wrong, ignored the prompt’s context for plain-language tone, and did not follow the citation date restriction. And, of course, many times the cited cases were bogus.
Here are the AI responses
ChatGPT response:
Standard of Review
When considering a motion brought pursuant to Michigan Court Rule 2.116(C)(4), this Court employs a standard of review that is well-established and guided by the principles of law applicable to summary disposition motions.
The standard for granting a motion under MCR 2.116(C)(4) is whether the pleadings, affidavits, depositions, admissions, and other evidence in the record, when viewed in the light most favorable to the nonmoving party, demonstrate that there is no genuine issue of material fact, and the moving party is entitled to judgment as a matter of law.
This Court’s review of such motions is de novo, meaning that we review the decision without deference to the trial court’s determination. Our review requires us to independently evaluate the record and apply the law to determine whether the trial court erred in granting or denying the motion.
Additionally, we recognize that while the moving party bears the initial burden of demonstrating the absence of a genuine issue of material fact, the nonmoving party is afforded every reasonable inference in their favor. Thus, the nonmoving party need only demonstrate the existence of a genuine issue of material fact through specific facts showing that there is a legitimate dispute that must be resolved by a factfinder.
Moreover, in conducting our review, we are mindful that summary disposition should only be granted when the evidence presented is so one-sided that reasonable minds could not differ as to the outcome. This Court will not weigh the evidence or assess witness credibility on summary judgment; instead, we focus solely on whether genuine issues of material fact exist and whether the moving party is entitled to judgment as a matter of law.
In light of these principles, this Court undertakes a meticulous examination of the record before it, ensuring that the parties’ rights are safeguarded and that justice is served in accordance with the law.
Accordingly, we proceed to assess the merits of the instant motion in accordance with these standards.
Claude response:
Here is a draft standard of review section for a Michigan trial court opinion deciding an MCR 2.116(C)(4) motion, using plain language and citations from 2020 or later:
We review MCR 2.116(C)(4) motions de novo. De novo means we take a fresh look at the issue without giving the lower court’s decision any special weight. Grimm v Department of Treasury, 2021 Mich App LEXIS 4499, *7 (Mich Ct App Aug 12, 2021).
A MCR 2.116(C)(4) motion argues this Court lacks subject-matter jurisdiction over the case. The party that filed the lawsuit has the burden to prove we do have jurisdiction. Sapala v St Joseph’s Hosp & Medical Ctr, 330 Mich App 548, 554 (Mich Ct App 2020). We consider all the parties’ evidence on this issue in the light most favorable to the party that filed the case. Elba Twp v Gratiot County Drain Comm’r, 493 Mich 265, 277 (2013). If there is any doubt jurisdiction exists, we will allow the case to continue. Id.
That is the standard of review we will apply to decide if we have subject-matter jurisdiction over this case. We will now analyze the parties’ arguments and evidence under this standard.
Note that the Grimm cite is bogus. The real cite is Grimm v Department of Treasury, 291 Mich. App. 140; 810 NW2d 65 (2010). There is no such Sapala case. While the Elba Twp cite is accurate, the case is too old/not responsive to the prompt and the pincite does not refer to the described proposition. And the closing proposition with the Id. cite is bananas.
Co-pilot (previously Bing) response:
Certainly! As a state trial court judge in Michigan, when evaluating a MCR 2.116©(4) motion, the standard of review is crucial. Here’s a plain language explanation:
Standard of Review
In considering a MCR 2.116©(4) motion, the court examines the following grounds:
- Lack of Jurisdiction: The court lacks authority over the person or property.
- Insufficient Process: The process issued in the action was inadequate.
- Insufficient Service of Process: The service of process did not meet legal requirements.
- Subject Matter Jurisdiction: The court lacks jurisdiction over the specific legal issue.
- Capacity to Sue: The party asserting the claim lacks the legal capacity to sue.
- Prior Action: Another lawsuit involving the same parties and claim is already pending.
- Appropriate Relief: Judgment, dismissal, or other relief may be warranted due to factors like release, payment, prior judgment, immunity, statutes of limitations, or other legal considerations.
- Failure to State a Claim: The opposing party has not adequately stated a valid claim.
- Lack of Valid Defense: The opposing party has failed to present a valid defense.
- No Genuine Issue of Material Fact: Except for damages, there is no real dispute about essential facts, and the moving party is entitled to judgment as a matter of law.
Remember, these grounds must be raised promptly, and some must be included in the initial motion or responsive pleading to avoid waiver.
Please note that this summary is for informational purposes only and does not constitute legal advice. Always consult legal professionals for specific cases.
Gemini (previously Bard) response:
Standard of Review
This Court reviews Defendant’s motion for summary disposition under MCR 2.116(C)(4) de novo. This means we review the case from scratch, independently evaluating the evidence presented by both parties. We do not defer to the lower court’s decision and instead apply our own judgment to the legal issues raised.
In conducting this review, we consider all admissible evidence in the light most favorable to the non-moving party, which in this case is Plaintiff. This means we assume all of Plaintiff’s factual allegations are true and draw all reasonable inferences in their favor. We then assess whether, based on this standard, there is a genuine issue of material fact that requires a trial for resolution.
A “genuine issue of material fact” exists when there is a factual dispute that could affect the outcome of the case. In other words, the evidence presented by both parties creates a genuine disagreement about a fact that is important to deciding the case. If such a dispute exists, summary disposition is not appropriate, and the case must proceed to trial.
However, if we determine that there are no genuine issues of material fact, and, viewing the evidence in the light most favorable to Plaintiff, no reasonable jury could find in their favor, we will grant summary disposition and dismiss the case.
Our decision on this motion hinges solely on legal issues, not the merits of the case. We express no opinion on the ultimate outcome of the case, which will be decided at trial if this motion is denied.
Citations:
- See, e.g., Maple Grove Twp v Misteguay Creek Intercounty Drain Bd, 298 Mich App 200, 206; 828 NW2d 459 (2012).
- See also Spiek v Dep’t of Transportation, 456 Mich 331, 337; 572 NW2d 201 (1998).
Note that both cites are correct, but the years make them inapplicable for the prompt. Plus neither decision involved MCR 2.116(C)(4).
Final thoughts
Free, public AI tools like ChatGPT and others have many helpful uses in other contexts. Primary legal research and drafting is not one of them, however.
As this experiment shows—even for a simple legal research and writing task—the AI output is often incredibly wrong and unreliable.