A small gift company launched a subscription box two years ago. Sales were slow, so the owner changed the packaging. Then the price. Then the contents, the name, the delivery schedule and the photography. Each version was a little better than the last, and each one sold a little, but never quite enough. The box is now on its seventh relaunch, and the owner still talks about it with real affection.
Nobody would call her lazy. She has done exactly what every article on iteration recommends: try, learn, adjust, try again. The problem is that nobody ever told her when to stop.
Most advice about iteration, including some we’ve published, focuses on starting: testing ideas cheaply, learning from customers, improving in small steps. Far less is said about the moment when the most useful thing an iteration can tell you is that the idea itself isn’t working. Recognising that moment is surprisingly hard, and there are good psychological reasons why.
One Like insight: Iteration helps you improve an idea. It can’t tell you when to let one go unless you decide in advance what “not working” looks like.
Why stopping is so hard
In 1985, the psychologists Hal Arkes and Catherine Blumer ran a neat field experiment at the Ohio University Theater. The first 60 people who bought season tickets were randomly sold them at one of three prices: full price, a small discount or a large discount. Everyone could see exactly the same plays. Yet during the first half of the season, those who had paid full price attended more plays than those who had paid less.
Arkes and Blumer called this the sunk cost effect: a greater tendency to continue something once money, effort or time has already gone into it. Their explanation was that people don’t like to appear wasteful. Having paid more, the full-price buyers felt they should get their money’s worth, even on evenings when they might rather have stayed at home. The money was already spent either way, but it still shaped their choices.
It’s worth being honest that the effect isn’t as tidy as it’s often presented. A 2007 study by economists led by Daniel Friedman found that the sunk cost effect in their experiments was surprisingly small and erratic. Even so, few business owners need a laboratory to recognise the feeling. Two years, seven relaunches and a garage full of stock make walking away feel like admitting all that effort was wasted.
Iteration can make this worse. Each round of improvement adds a little more investment, and each small sign of progress (a few more sales, a nicer comment) feels like a reason to keep going. The idea never fails badly enough to force a decision. It just never quite succeeds.
Deciding in advance
If the moment of decision is clouded by everything you’ve already invested, one solution is to make the decision earlier, before the investment piles up. Annie Duke, a former professional poker player who now writes about decision-making, calls these “kill criteria” in her 2022 book Quit. The idea is to agree, at the start, on the signals that would tell you to stop.
Duke suggests framing them as a combination of a state and a date: if by a certain date we haven’t reached a certain state, we stop or change course. For the subscription box, that might have been: “If we haven’t reached 150 regular subscribers by the end of month six, we’ll stop and put the time into our best-selling products instead.” It’s a simple sentence, but it does something powerful. It turns a painful judgement into a check against a decision you made when you were thinking clearly.
Kill criteria also make conversations easier. In a small business, the person most attached to an idea is often the owner, and colleagues may hesitate to say it isn’t working. A criterion agreed in advance gives everyone permission to raise it without making it personal.
[Infographic placement] Setting kill criteria. A simple fill-in card. The idea: ______. We’ll keep going if, by [date] we reach [measurable state]. We’ll change course if [weaker result]. We’ll stop if [clear failure signal]. Who will make the call: ______. Caption: Write it down before you start, while you can still be objective. Based on Annie Duke’s “states and dates” approach.
Highlight: Decide what failure looks like before you’re too invested to see it.
Stop, change course or keep going
Knowing when to stop doesn’t mean giving up at the first difficulty. Plenty of good ideas take time, and persistence is a real virtue in business. The useful question is what kind of signal you’re getting.
A few patterns can help tell them apart. If each round of iteration produces clearly better results, even slowly, that’s usually a sign to keep going. If customers love one part of the idea but not the rest, that may be a signal to change course: keep what’s working and drop the rest. The subscription box might, for example, reveal that customers adore one particular product inside it and would happily buy it on its own. And if repeated changes produce only tiny, inconsistent improvements, with the core measure barely moving, that’s often the clearest sign that the idea itself, not the execution, is the problem.
It also helps to count what continuing really costs. The obvious costs are money and stock. The less obvious one is attention. Every hour spent on the seventh relaunch is an hour not spent on something that might work better. Economists call this opportunity cost, and it’s the cost most owners forget to include.
Highlight: The hidden cost of keeping a struggling idea alive is everything else you could have done with that time.
Letting go well
If you have an idea that’s been iterating for a long time, try a simple thought experiment. Imagine you hadn’t spent anything on it yet. Knowing what you know now, would you start it today? If the honest answer is no, the money and effort already spent shouldn’t be the reason to carry on.
For new ideas, write down your kill criteria before you begin, and share them with someone who will hold you to them. Then iterate freely, knowing there’s a sensible point at which you’ll step back and decide.
A few questions can help:
- If we were starting from scratch today, would we choose this idea?
- What did we expect to have achieved by now, and how close are we?
- Is each round of changes producing real improvement, or just activity?
- What else could we do with the time and money this idea is using?
Which brings us back to the subscription box on its seventh relaunch. Stopping wouldn’t mean the last two years were wasted. Those relaunches taught the owner a great deal about her customers, her suppliers and what people will pay for. The point of iteration was always to learn. Sometimes the most valuable thing it teaches you is where to put your energy next.
Sources
- Hal R. Arkes and Catherine Blumer, “The Psychology of Sunk Cost”, Organizational Behavior and Human Decision Processes 35(1), 1985.
- Daniel Friedman, Kai Pommerenke, Rajan Lukose, Garrett Milam and Bernardo Huberman, “Searching for the Sunk Cost Fallacy”, Experimental Economics 10(1), 2007. Source of the caveat that the effect was small and erratic in their experiments.
- Annie Duke, Quit: The Power of Knowing When to Walk Away (Portfolio, 2022). Source of “kill criteria” and the states-and-dates approach.
