The manager of a small logistics company starts using AI to prepare his weekly reports. At first, he reads every line carefully, checks the numbers and rewrites the conclusions in his own words. After a few months, he skims. Then he simply copies the summary into the report and sends it.
One week, the AI’s summary says delivery delays have improved. They haven’t; a change in how the data was exported has distorted the figures. The manager, who once would have spotted it instantly, doesn’t. His director does.
This gradual drift, from thinking alongside AI to letting it think for us, is something researchers have begun to measure. Their findings suggest the biggest risk lies in how much we trust AI, and how confident we feel in our own judgement.
One Like insight: The more you trust AI, the more deliberately you need to keep thinking.
What the research found
In 2025, researchers from Microsoft Research and Carnegie Mellon University, led by Hao-Ping (Hank) Lee, presented a study at CHI, the leading international conference on human–computer interaction. They surveyed 319 knowledge workers who used generative AI tools at work, collecting 936 real examples of tasks they had used AI for, and asked how much critical thinking they applied.
The pattern was clear. When people had higher confidence in the AI’s ability to do a task, they tended to report applying less critical thinking. When they had higher confidence in their own ability to do the task, and to evaluate the AI’s output, they tended to apply more.
The researchers also found that AI shifted the nature of critical thinking rather than removing it. Instead of gathering information and producing work from scratch, people increasingly spent their effort verifying information, integrating AI responses into their work and overseeing the task. That oversight role is valuable, but only if people actually perform it.
The researchers warned that over-reliance could, over time, reduce people’s independent problem-solving and leave them less practised at the very skills needed to catch AI’s mistakes.
The study relied on self-reported survey responses rather than direct measurement of thinking, and it captured a single point in time. But it gives a useful warning sign: confidence in the tool, not the tool itself, is what tends to switch people’s thinking off.
Keeping your thinking switched on
The research suggests a few habits that help people stay engaged when working with AI.
Decide what you think first. Before asking AI for an answer, jot down your own view or rough answer. Then compare. This keeps your judgement active and makes differences easier to spot.
Ask “how would I know if this is wrong?” For any important output, identify the one or two facts that would make the conclusion fall apart, and check those.
Keep doing some tasks yourself. Skills fade without practice. Regularly doing key analyses, writing or calculations without AI keeps your ability to judge its work sharp.
Ask AI to challenge you. Use it to argue against your plan, list weaknesses in a draft or suggest what you might be missing, rather than only to produce answers.
Rotate review responsibilities. In a team, have different people review AI-assisted work, so no one becomes complacent and fresh eyes catch drift.
These habits echo earlier articles in this series. As explored in AI Makes Good Work Better, and Some Work Worse, people who accepted AI’s reasoning without question did worse on tasks outside its strengths. And as Better Decisions Come From Better Conversations showed, good process protects against confident mistakes.
[Infographic placement] Who keeps thinking? A simple seesaw. One side labelled Confidence in the AI (higher → less critical thinking). The other labelled Confidence in your own judgement (higher → more critical thinking). Beneath, five habits: Decide first · Check the key facts · Practise without AI · Ask AI to challenge you · Rotate reviewers. Caption: Lee et al. (2025), CHI conference, survey of 319 knowledge workers.
A thinking check
Pick one recurring task where you or your team now rely heavily on AI, such as reports, proposals, analysis or customer replies. For the next month, apply one simple rule: before using the AI’s output, write down in a sentence or two what you expected it to say, and note anything that surprised you. Surprises are often where errors hide, and noticing them keeps your judgement in practice.
A few questions can help:
- Which tasks have we stopped checking carefully since we started using AI?
- Do we form our own view before reading the AI’s answer?
- Are we keeping our core skills in practice?
- Who reviews AI-assisted work, and do they bring fresh eyes?
Which brings us back to the logistics manager and his distorted delivery figures. AI hadn’t made him less capable; it had made him less engaged. A thirty-second habit, asking himself whether the numbers matched what he’d seen on the ground that week, would have caught the error long before his director did. The best use of AI frees time for thinking. It shouldn’t replace the thinking itself.
Sources
- Hao-Ping (Hank) Lee, Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks and Nicholas Wilson, “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers”, Proceedings of the CHI Conference on Human Factors in Computing Systems, 2025.
