A small marketing consultancy starts using AI for everything. It drafts proposals in minutes, generates campaign ideas, rewrites website copy and produces neat summaries of client research. The team is thrilled. Output doubles.
Then a client complains. A strategic recommendation in their report was confidently wrong: the AI had misread a subtle trend in the client’s sales data, and the consultant, trusting it, hadn’t questioned the conclusion. Ordinarily, she would have caught it.
This pattern, AI dramatically improving some tasks while quietly making others worse, isn’t just an anecdote. It was captured in one of the most careful studies yet of how AI affects real professional work, and it offers a practical lesson for any business using these tools.
One Like insight: AI’s abilities are uneven. Knowing where the edges are matters as much as using it at all.
The jagged frontier
In 2023, a team of researchers from Harvard Business School, Wharton, MIT and other institutions, led by Fabrizio Dell’Acqua, worked with Boston Consulting Group to run a controlled experiment with 758 of its consultants. Participants were randomly assigned to complete realistic consulting tasks either with or without access to GPT-4.
For tasks that fell within what AI could do well, such as generating ideas for a new product, writing, analysis and persuasive communication, the results were striking. Consultants using AI completed around 12% more tasks, worked about 25% faster and produced work rated more than 40% higher in quality than those without it. Those who had been weaker performers to begin with improved the most.
But the researchers also designed a task that looked similar yet fell outside what the AI could reliably do: a business problem requiring careful interpretation of subtle evidence in data and interviews. On this task, consultants using AI were noticeably less likely to reach the correct answer than those working without it, by around 19 percentage points. The AI produced plausible but wrong analysis, and many consultants accepted it.
The researchers described AI’s abilities as a “jagged technological frontier”: uneven, and not always where you’d expect. Tasks that seem equally difficult to a person can be easy or hard for AI. The study used one firm and one model in 2023, and AI has advanced since. But the central lesson still applies.
Finding where AI helps your business
The practical task for any small business is to map its own frontier: which tasks AI genuinely improves, and which it can quietly make worse.
Start with tasks that are easy to check. Drafting emails, summarising meeting notes, brainstorming names or ideas, reformatting documents and writing first drafts are often good candidates. Mistakes are visible and cheap to fix.
Be cautious with judgement-heavy work. Interpreting subtle data, making strategic recommendations, assessing risk and anything requiring deep context about your clients or industry are where AI can produce convincing but wrong answers.
Test before trusting. Try AI on a task where you already know the right answer. If it gets it wrong, that’s a sign the task is outside the frontier, at least for now.
Keep thinking, not just checking. The consultants who did worst outside the frontier often accepted the AI’s reasoning instead of applying their own. As we explored in When Can You Trust Your Gut?, expertise matters most where situations are complex and ambiguous.
Use AI to lift, not replace, weaker areas. The study found less experienced consultants gained most. AI can help newer staff produce better first drafts, provided experienced people still review the work.
[Infographic placement] The jagged frontier. A wavy line separating two zones. Inside the frontier (AI helps): ideas, drafting, writing, summarising, structured analysis — about 12% more tasks, 25% faster, 40%+ higher quality. Outside the frontier (AI can mislead): subtle data interpretation, judgement calls — about 19 percentage points less likely to be correct. Caption: Dell’Acqua et al. (2023), 758 BCG consultants.
Mapping your own frontier
List the ten tasks your team spends most time on. For each, ask two questions: could AI do a useful first draft or analysis, and how easy would it be to spot a mistake? Tasks where AI is likely to help and errors are easy to catch are your best starting points. Tasks where errors would be subtle and costly need much more care, or should stay with experienced people.
Revisit the list every few months. AI capabilities are changing quickly, and tasks outside the frontier today may move inside it, while new risks may emerge.
A few questions can help:
- Which of our tasks does AI genuinely improve, and how do we know?
- Where could AI produce a convincing but wrong answer that we might miss?
- Have we tested AI on tasks where we already know the right answer?
- Who reviews AI-assisted work in judgement-heavy areas?
Which brings us back to the marketing consultancy and its doubled output. AI was transforming the proposals, ideas and copy, all well within the frontier. The strategic recommendation was outside it. The answer wasn’t to stop using AI, but to recognise the boundary: let AI accelerate the writing and ideas, and keep experienced consultants firmly in charge of interpreting clients’ data and making strategic calls.
Sources
- Fabrizio Dell’Acqua, Edward McFowland III, Ethan Mollick, Hila Lifshitz-Assaf, Katherine Kellogg, Saran Rajendran, Lisa Krayer, François Candelon and Karim R. Lakhani, “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality”, Harvard Business School Working Paper 24-013, 2023.
