Professional Negligence in AI

View on LinkedIn

Beyond Regulations, Toward Literacy

The Case That Changed the Conversation

In December 2024, Deloitte Australia was engaged for AU$440,000 to conduct an independent assurance review of a government welfare system. The result was a scandal:

more than 20 serious errors including invented book titles, nonexistent judicial citations, and fabricated references.

The cause: hallucinations from Azure OpenAI's GPT-4o, used without adequate oversight. This case became the emblematic example of professional negligence in AI use, triggering a global debate about how to regulate these tools.

But the fundamental question is: why did the AI hallucinate so extensively? The answer lies in two technical factors that any professional must understand before using these tools: temperature and sycophancy.

Factor 1: Temperature — The Secret Nobody Explains

You don't drive a car until you learn how, because it can cause harm. Using AI has a fundamental secret that the Deloitte consultant apparently did not know: temperature.

Temperature is like the flame of a stove that can be off or fully open. This parameter controls how much creativity and randomness the model applies to its responses.

High temperature (0.7–1.0+): the model is more creative, varied, "imaginative"—perfect for creative writing or brainstorming, but it invents things.

Low temperature (0.0–0.3): the model is more conservative, predictable, factual—ideal for work requiring precision such as consulting, legal, or technical analysis.

The critical problem is that generic AI tools—ChatGPT, Claude, Gemini—do not allow regular users to adjust the temperature. These platforms are configured with moderate or high temperatures because they are designed for general and conversational use, not exclusively for professional consulting or legal work demanding absolute factual precision.

The Deloitte consultant used a tool designed for general creativity in a context requiring forensic accuracy, without understanding or being able to control this fundamental parameter. It is like driving a sports car configured for maximum speed in a school zone, with no access to the speed limiter.

Factor 2: Sycophancy — Why AI Never Contradicts You

The second factor explaining the hallucinations is sycophancy. Language models are programmed to accept the user's premises uncritically, to avoid contradicting the user even when wrong, to generate responses that confirm the user's beliefs in order to "please" them, and to express themselves with authority and certainty even when their internal confidence is low.

Why? Because many models are optimized through human feedback (RLHF) to be pleasant, coherent, and helpful—not necessarily to maximize absolute truthfulness. That design bias favors "satisfying" responses over "accurate" ones in edge cases.

Concrete example: If you ask "What are the 11 commandments?", the best-calibrated modern AI systems would recognize that there are not 11 but 10 commandments and would politely correct you. But some AI systems, especially those with high temperature and strong sycophantic bias, might invent an 11th commandment to satisfy your premise: "The 11 commandments include the traditional 10 plus an eleventh recognized in some traditions: 'Love your neighbor as yourself'..."

This sounds authoritative, coherent, academic—but it is entirely fabricated. The model accepted your mistaken premise and constructed a plausible narrative. This same mechanism explains the invented book titles and false judicial citations in the Deloitte case.

The Perfect Storm

When you combine high temperature with sycophancy you get the perfect storm: the consultant asks a question, the model (with high temperature) has wide latitude to "create," the model (trained to please) does not question the premise, and it generates invented information in an authoritative tone. The result appears fluid, coherent, professional—but is completely false.

The Recent Evolution: Reasoning AIs

Aware of these problems, the industry has developed a new generation: reasoning AIs. These models have been on the market for barely a year—OpenAI launched its o1 in September 2024, followed by DeepSeek R1 (January 2025), Claude 3.7 Sonnet with Extended Thinking (February 2025), and Gemini 2.0 Flash Thinking.

The central innovation is chain-of-thought: instead of generating an immediate response, these models "think" step by step before answering. The model decomposes the problem, evaluates different strategies, identifies potential errors, and only then generates its final response. This internal reasoning process is trained through reinforcement learning, rewarding the model for following correct logical steps.

Why does this reduce hallucinations? Because it forces the model to self-evaluate before committing to a response. Studies show that chain-of-thought reduces reasoning errors by 30–35%. By forcing the model to "think out loud," incorrect logical leaps and fabrications are intercepted before they reach the user.

However, reasoning AIs are not a magic solution. Recent research (2025) reveals that models trained with incomplete pipelines can exhibit more hallucinations than traditional models. Two problematic cognitive behaviors have been identified: Flaw Repetition (the model becomes trapped in loops of repetitive thoughts) and Think-Answer Mismatch (the final response does not match the reasoning process shown).

These models are slower and more expensive because they invest more computational resources in thinking. But when well trained, they represent a significant advance. The challenge: this technology has less than a year of maturation, and the users who employed GPT-4o in the Deloitte case did not have access to these capabilities.

The Regulatory Response: Inoperative and Absurd

The case has generated a cascade of regulatory proposals: mandatory detailed disclosure, governance with "human on the loop," task segregation, prompt traceability, formal quality KPIs, internal audits.

Does this seem logical? It does not. All of these proposals are like strapping floaties, a life jacket, and four lifeguards on an Olympic swimmer: inoperative and absurd. They transform AI use into a bureaucratic exercise that stifles innovation without resolving the underlying problem.

You don't need any of this, if you know how to swim.

The Real Solution: Literacy and Professional Tools

The Deloitte case is not a failure of AI—it is a failure of digital literacy and tool selection. You don't see a Toyota Camry racing in Formula One, because it is not built for that. Using generalist AI for critical professional work is equally unsuitable.

Professionals do not necessarily need deep technical expertise, but they do need to understand three fundamental concepts:

Temperature (knowing that generic platforms do not allow you to control it and that you need enterprise APIs for critical work),

Sycophancy (the model is programmed to please you, not to challenge you—adopt an adversarial verification posture), and

Reasoning AIs (understanding when to use them and that they are not infallible).

PLEASE READ CAREFULLY WHAT THE AI PRODUCES FOR YOU — YOU ARE THE PROFESSIONAL

Ironically, after the scandal, Deloitte took the right steps: it established a strategic alliance with Anthropic to deploy Claude to more than 470,000 professionals globally—Anthropic's largest enterprise deployment. Why Claude and not continue with GPT-4o? Because Claude was designed with a safety-first focus and robust enterprise controls.

In addition, Deloitte is certifying 15,000 professionals in a formal program that teaches not only how to use AI, but crucially when NOT to use it. They are co-creating customized Claude "personas" for specific roles (accountants, auditors) with compliance built in. They created a Center of Excellence with specialists who design safe implementation frameworks.

This is the correct response: real technical literacy + dedicated professional tools. No more paralyzing regulations, but rather trained professionals using the right tools for the right work. Deloitte's negligence was using the wrong tool without proper training. Its subsequent response is the model to follow. Literacy is necessary.