philip@lilius

Wrong about where

· 5 min read

A few years ago I’d have bet that prompt engineering would become a serious craft: context, use case, steps, edge cases, carefully worded intent, and a healthy market in teaching people to do it. I’d have lost. The models got uncannily good at working out what you meant, and the joke going round now is that the 2026 version of a carefully engineered prompt reads “pls fix teh user flow to increse converstions. maek no mistakes”.

What’s interesting is the direction of the mistake: I didn’t overestimate the models, I underestimated them, and that’s exactly why the skill I’d bet on evaporated, because the machine absorbed it. The skill that got scarce instead was the one nobody was selling courses in, which is knowing when the answer is wrong. Talking to the model got cheap and reading what it says got expensive.

I suspect that’s the more common failure, and the more useful one to spot. Most AI forecasts got the size of the change roughly right and the place badly wrong. They put the value where it’s visible (the demo, the capability, the new thing you can do) while it kept turning up in the half nobody puts on a slide: checking, cost, and someone willing to sign for the result. So the test for a forecast isn’t whether it’s too optimistic. It’s whether it points at the right half.

If you’re so smart

There’s an old question for anyone selling predictions: if you’re so smart, why aren’t you rich? If they’re so sure, Polymarket takes bets. A forecast gets paid when it’s told, not when it comes true, so it only has to sound right on the day (Nassim Taleb would say it has no skin in the game), and nobody comes back in two years to check.

A few people have. Dan Luu went back over some well-known AI pundits and futurists and found that almost nobody looks at what happened, so the vibes just get repeated. Philip Tetlock did it at scale. For Expert Political Judgment he scored 28,000 forecasts from 284 experts and found that the confident ones with one big idea and a great story did worse than the cautious ones who kept updating, which is the case he later made for keeping score in Superforecasting.

The test catches everyone selling a view of AI, and it’s fair to sort them by what they’re selling:

The pre-tragic sell the future. Everything changes next quarter, buy now. Get in on the ground floor!

The tragic sell fear. Everything ends, subscribe for the details. Feel smart by being sceptical!

The post-tragic sell calm. It’s happening anyway, here’s how the grown-ups cope. Be smart by knowing the traps!

The third camp is where I try to operate, and since I sell “AI knowledge” too, it gives me no immunity at all. Annie Duke’s Thinking in Bets has the cheapest check I know against that kind of bias: before defending a belief, ask yourself “wanna bet?” and notice how sure you suddenly aren’t. A failure log is hard to fake and a forecast is easy, so my only defence is that I build things with it, at work and at home, and write down what broke. The prompt engineering bet is in there.

The telling business

What I’ll call the telling business is a product that is the story, already paid for by the time anyone could check it. Two specimens from one week of my filter bubble stand out, and both point at the visible half.

The first was a short video arguing that the next trillion-dollar company will be a marketplace where experts deploy themselves, plus their entourage of AI, as software, because “there’s no Amazon for software”. The closest thing we already have is Fiverr and Upwork, and if you’ve bought from them you know where a marketplace for packaged expertise tends to end up: a race to the cheapest gig that looks finished. That follows from the format. A marketplace can show you the package and the price, which is the visible half, while what clients of services mostly pay for is someone who answers for the result, and that doesn’t fit in the package.

The second was a software consultancy’s blog post on the Gartner hype cycle, with projections out to 2029. Read the edit note and you find the projections were researched with ChatGPT, Grok, Kimi and Gemini. It’s a prophecy the machines wrote for the seer, with figures that come with no source, and it ends with a button to book a consultation.

What’s left in the trough

The funny part is that the consultancy’s advice was mostly right, and all of it sits in the invisible half: start narrow, invest in boring engineering, sort out your data, build governance in from the start. Gartner’s 2026 cycle for AI says the same in a headline about putting control ahead of capability. It expects over 40% of agentic AI projects to be cancelled by the end of 2027, and it calls observability, the tooling that catches “plausible but incorrect outputs”, one of the year’s most consequential items. Gartner sells forecasts too, but it publishes them with a date, which is more than most.

In the generative AI cycle, according to The Register’s report on it, one category Gartner puts on the slope of enlightenment is plain AI-enabled applications, like coding assistants, image and video tools and summarisation, already used by over half their market. Nothing is on the plateau yet.

So when the market slides, my guess is that the dull layer is what’s still standing, because somebody needs it for dull reasons.

Finance will want to know what it costs and lawyers will want to know what it did, so the measuring, the budgets and the audit trail survive, kept alive by auditors. So do the narrow tools where a mistake has a price you can count, like transcription, extraction and sorting (sorry, “strategy” doesn’t make the cut). And so do processes kept in files. My novel’s process survived ten model versions because it lived in a repository; pat on my shoulder.

The experts stuck in the trough will be the ones whose product was the telling, and the ones who climb out will be the ones whose product worked anyway, with or without a better model. That guess is a forecast too, and it points at the boring half on purpose. It’s dated November 2026, so you can check it.