Lawyers are increasingly turning to generative AI tools to brainstorm arguments, draft documents, summarize cases, and analyze strategy. But there is a growing concern that many AI systems are designed not simply to answer questions, but to keep users engaged and satisfied. In some cases, that may mean giving users the answer they want to hear rather than the answer they need to hear. That creates a serious risk for lawyers.

A lawyer who uses AI as a sounding board may unknowingly receive responses that are overly agreeable, overly confident, or subtly tailored to reinforce the user’s assumptions. The problem is not necessarily outright hallucinations, though those remain a concern. The greater danger may be something more subtle. AI systems may flatter the user, validate weak arguments, and fail to challenge faulty assumptions because doing so creates a more satisfying user experience.

In other words, the AI may act less like a skeptical colleague and more like a people pleaser. That is a dangerous dynamic in the legal profession, where good lawyering often depends on identifying weaknesses, stress-testing assumptions, and hearing uncomfortable truths before a judge or opposing counsel points them out.

Imagine a lawyer asks an AI assistant whether a complaint states a viable claim. A poorly framed prompt may invite the AI to focus on supporting the lawyer’s theory rather than critically evaluating it. The response may sound polished and persuasive while overlooking fatal legal defects, adverse authority, evidentiary gaps, or jurisdictional problems. Because the answer is written confidently and fluently, the lawyer may mistake tone for accuracy.

This concern reflects a broader issue in AI development sometimes described as “sycophancy,” where AI systems are optimized to appear helpful, agreeable, and affirming. The model learns that users generally prefer responses that feel supportive and validating. Platforms built around user engagement may have incentives to favor responses that users perceive as supportive and affirming, even when a more critical response would be more accurate or useful. But legal analysis is not supposed to be validating. It is supposed to be accurate.

Fortunately, lawyers can reduce this risk by changing the way they interact with AI tools. One of the most important things lawyers can do is explicitly instruct the AI not to prioritize agreeableness or affirmation. Lawyers should tell the system that they want critical analysis, not validation. A prompt might say:

“Do not try to make me like your response. Critically analyze my position. Identify weaknesses, adverse authority, procedural problems, and factual assumptions that may be incorrect. If my argument is weak or unsupported, tell me.”

That simple instruction changes the nature of the interaction. Instead of acting like a marketing tool designed to keep the user engaged, the AI is directed to act more like a skeptical law partner conducting a rigorous review.

Lawyers should also remember that confidence is not competence. Generative AI systems are designed to produce fluent language, not necessarily truthful or reliable answers. A response that sounds authoritative may still be incomplete, misleading, or entirely wrong. The more polished the answer appears, the more disciplined lawyers must be in independently verifying the result.

That principle is already reflected in lawyers’ ethical duties. Competence under Rule 1.1 requires lawyers to understand the benefits and risks associated with technology. Independent professional judgment under Rule 2.1 requires lawyers to exercise their own legal judgment rather than outsourcing critical thinking to a machine. AI can assist legal analysis, but it cannot replace skepticism, diligence, or professional responsibility. Good lawyers are not paid merely to generate arguments. They are paid to evaluate them critically. Ironically, one of the best ways to use AI effectively may be to train it not to agree with you.