Most AI pilots are designed to be un‑killable

Of the AI pilots I’ve watched up close over the last two years, the ones that failed took an average of eleven months to die. The technology usually disappointed inside the first six weeks. The other nine and a half months went to managing the disappointment.

The pattern is the same everywhere. A pilot gets a sponsor, a line in the budget, and a demo day. By the first checkpoint the demo is behind schedule, so success gets redefined: fewer use cases, softer metrics, “learnings” promoted from byproduct to deliverable. Nobody is lying. The incentives do all the work — the sponsor can’t afford a failure, the vendor can’t afford a cancellation, and the team can’t afford to be the ones who said so.

The fix costs one page. Before kickoff, write down what dead looks like: a number, a date, and the name of the person who pulls the plug. “If drafting time isn’t down 20% by March 31, Elena kills it” survives every steering committee. “We’ll evaluate progress quarterly” survives nothing, which is the point of writing it that way.¹

Notice what a kill criterion is not. It isn’t pessimism, and it isn’t a lack of commitment — it’s the price of being trusted with the next budget. The teams that write one get something the others never do: a clean ending. Killed-on-schedule reads as judgment. Faded-out-after-eleven-months reads as waste, and everyone in the room knows which one they watched.

“A pilot without a kill criterion isn’t an experiment. It’s a subscription.”

I hold myself to the same page. Wildfire — the civic tool I’m building for Chicago — has its shutdown test published in the open: if this fall’s launch can’t turn curiosity into members who stay, I’ll shut it down and write about why.² Writing it down doesn’t make stopping painless. It makes continuing honest.

So before your next pilot: one page, one number, one date, one name. If the room won’t sign it, you’ve learned what the pilot was for.