Past the Pilot The Model Is the Cheap PartThree changes in two weeks, and why the part you own is the logic.
| ||||
 | ||||
01 The Pattern 02 Keep the Logic 03 What a Decision Costs 04 Same-Answer Audit 05 What Builders Are Saying
In two weeks, the price of an AI decision fell to almost nothing, Anthropic removed the setting teams used to get consistent answers, and its own team told companies to build fewer agents. It's one story.
Each of those reads like a model story. They're really about what you own. The model is turning into the cheap, swappable part. The questions you ask it, the answers you allow and the rules around it are the part that stays.
01
The Pattern
Models are getting cheaper, more interchangeable and less tunable, all at once. That moves the value out of the model and into the business logic around it: the definitions, rules and thresholds your team writes down. If that logic only lives inside prompts, it isn't really yours.
| OpenAI's new Decisions API and TypeSafe's Jev both answer a narrow question you define, like "does this invoice match the PO" or "which of these five categories fits," and return a probability instead of a paragraph. In one public test, Jev sorted support messages for about $110 per million. Claude Opus cost 22 times that, and about 170 times without prompt caching. |
Per million $110 to sort support messages with a decision model |
I keep comparing AI to Excel, and this is why. Excel is one of the most useful tools a company has. Used badly, it's also where information goes to die: the formula one analyst wrote, in a file nobody else opens, that the quarter quietly depends on. Prompts are heading the same way. When the definition of adjusted EBITDA lives in a prompt, it lives in one tool, and nobody can see it change.
The fix isn't to slow down on AI. It's to put the logic somewhere you own, and let the model do what it's good at.
02
Swap the Model, Keep the Logic
A context layer is where the business logic lives: the definitions, the rules and the calculations an agent calls instead of making up. It can be an MCP server or an API. Point a different model at it and you get the same number.
Here's a test I like: if you switched models tomorrow, what would you lose? If the honest answer is "most of it," you've rented your AI rather than built it.
On one portfolio reporting workflow we built, the model never decides adjusted EBITDA. A rules engine computes it, and the definition sits in a context layer the agent calls like any other tool. The model reads the source documents and gathers the figures. The code does the math.
|
Changes every quarter The model Its price, its settings and the vendor's API version. |
Shouldn't have to Your logic What counts as revenue, approval thresholds, match rules and metric definitions. |
A data warehouse is the closest comparison. The data model is an opinionated view of your business, and that opinion is the asset, not the database under it. The comparison breaks on cost. A context layer covers one use case and usually pays for itself in the time it saves. It isn't a two-year migration.
Anthropic's guidance on building fewer agents fits the same pattern. Their Applied AI team found most enterprise agents work better as one loop with tools and skills than as a stack of specialist agents passing summaries to each other. One customer's single loop ran about twice as fast. The scoping doesn't go away. It moves out of the org chart and into the tools, which is exactly where a rules engine sits.
| Read the full piece: what goes in a context layer → |
|
From inside the build The model will change. Your business logic shouldn't have to. |
03
What a Decision Costs Now
A cheap decision model gets close to a frontier model on narrow questions, and handing only its unsure cases to the bigger model closes most of the gap. How you phrase the question matters more than which model answers it.
Cheap doesn't mean equal. In one public test on 3,080 banking support messages, Jev scored 81% and Claude Opus scored 84%. Sending only the cases Jev was unsure about up to Opus also got 84%, at just over a quarter of Opus's cost. We've started putting Jev into our builds for exactly this kind of step.
|
Asked once 63% "Is this email phishing?" |
Five narrow checks 95% Same model, split into specific questions. |
The phishing result is the one to remember. Same model, same kind of emails. Asked once, it scored 63%. Split into five narrow checks, like whether the links point to shortened URLs and whether a free email address claims to be a company, it scored 95%. Nothing about the model changed. The question did.
The model call was never the hard part. Writing the question, the allowed answers and the threshold is.
| Read the full piece: the numbers and where it breaks → |
|
04 · Use this week The Same-Answer AuditFor years, teams made AI more consistent by turning down a setting called temperature, a kind of creativity dial. Anthropic's new API version removes it, and their migration guide says no other setting replaces it. That dial never made answers right. It made them repeatable, and a wrong number you get every time is harder to catch than one that moves. A paper last week found the same thing about reviewers. When AI monitors approving payments all read the same poisoned evidence, they all approved the fraud together. Three reviewers who read the same thing are one reviewer. The fix that held up was a rules engine, because you can't talk code into anything. Take one AI workflow and ask your team four questions.
Every spot that shows up under "same twice" or "the swap" should be ordinary code: a calculation, a match against a purchase order, a threshold. If you can't write the fixed list of answers, the step isn't ready to automate yet, and that's a good thing to learn before launch.
| ||||||||||
|
From Blue Orange · Blueprint Where should your company look first?
| |||
|
This issue keeps coming back to one question: which steps are worth handing to AI, and which belong in code. Blueprint is how we start that conversation with a company, written down. You answer about ten minutes of questions, it adds public research on the company and its market, and it scores your answers the way we score a company in an engagement. You get AI opportunities ranked by value against effort, build, buy or an agent for each, and the first moves for the next 90 days. It's free by invitation.
|
05 · What I'm watching
What Builders Are Saying
The best reactions to these changes came from people running agents in production. Four worth reading, with my take on each.
|
elvis · @omarsar0 · Oct 24 "For most agent workflows, folding subagents into the main loop via tool calls and skills makes a ton of sense (as Anthropic recommends). ... But you can't just get rid of subagents if you have been using them extensively. You need to run evals ... The evals should always inform your strategy." My take: right call. Fewer agents is the default, not a rule. Measure before you fold. |
|
Gauri Gupta · @gauri__gupta · Oct 28 "On a sample support agent, folding one subagent into another cut latency and also broke a process that had been passing ... it only showed up in one run out of two. an average score would have stayed flat." My take: the most useful line on the whole debate. Run every task more than once and look at which ones flipped, not at the average. |
|
Akshay · @akshay_pachaar · Oct 13 "I tested it on 40 requests, and it was as accurate as letting Opus make every match, at 150x lower decision cost. ... low-confidence matches go back to Opus for review, and in my test, that review caught 5 wrong matches that would otherwise have gone through." My take: the handoff is the design. The confidence score tells you which answers deserve a second look. |
|
DAIR.AI · @dair_ai · Oct 17 "With the fitted threshold, tool and argument cascades cut cost by 14.2% without changing the outcome of any of 500 tasks. A fixed threshold of 0.5 lost 21.5 points." My take: the threshold is business logic too. Fit it on your own work, write it down, and own it. |
Reply and tell me the first step you'd move out of the prompt and into code. I read every reply.
|
