How to use decision models like Jev in software factories

Key Takeaways

• Decision models answer closed questions, not open ones. They return a typed value (a label, a score, a yes or no) from a fixed set of possible answers, with a probability attached, instead of generating free text.
• Jev is the furthest along, but it’s not alone. There are other options like OpenAI's Decisions API, Microsoft’s decision-1, and an open-weight model called Laya amongst others
• The gains from decision models appear in points of judgment, not reasoning (where LLMs shine). Testing puts Jev at equal or better accuracy than an LLM judge at a fraction of the latency and cost, perfect for tasks like ticket classification and completion checks.
• It sits between the agents and the rules. A decision model does not replace the coding agent or the workflow engine.
• Confidence is the best part. Answers come with a confidence score so you can decide what happens next more easily

A software factory makes hundreds, if not thousands, of small, closed-set decisions every day. For example, which team owns this ticket, or which runbook fits this incident, or whether this skill call actually resolved anything. Most of those decisions still run through an LLM. But LLMs are built for reasoning, and ultimately writing and code, which is much slower and much more expensive than the task requires. Decision models like Jev from TypeSafe AI are built to answer exactly these questions in under a second. OpenAI's Decisions API and a smaller open model called Laya take the same approach. Let’s understand what decision models are and where they can make your software factory more efficient.

How Jev Differs From the Models Already in Your Factory

A classic LLM and a decision model answer different kinds of questions. LLMs are built to produce open-ended text like a PR description, an explanation, or a block of code. Decision models are built to pick one answer from a list you’ve already defined. They also send along a confidence score.

TypeSafe calls Jev a "System One" model (a nod to Daniel Kahneman's book “Thinking, Fast and Slow”). It’s built to do the deciding rather than the writing. Positioned that way, it fits between two things that already exist in a factory: Workflows and conditions carry the deterministic rules like what triggers exist and what happens after what. Within the workflow, agents (built on LLMs) do the actual work: writing code, drafting a PR description, or explaining an incident. Decision models are narrower than both: they take a state and a closed question, and returns a typed answer with a probability, in milliseconds rather than seconds.

In short, if the decision in your software factory already has a name for every possible answer, an LLM is the wrong tool for it and you should look to replace that decision with a decision model instead.

The Decisions a Software Factory Makes Over and Over

Software factories are meant to be highly repeatable systems. So they’re going to make the same decisions over and over. In our experience, we’ve seen the following decisions show up constantly in an agentic SDLC:

• Ticket priority. Two bugs with near-identical titles can carry different urgency once you check service criticality, SLA flags, and recurrence count, data an LLM reading ticket text has no reliable way to weigh.
• Resource ownership. An orphaned service with a stale owner field needs an answer built from deploy history and catalog tags, not a narrative guess from commit messages.
• Routing to the right agent. Picking the owning agent for a ticket is a fixed list of teams, not open text, and a generative model asked to "pick one" states a guess with the same confidence as everything else it writes.
• Choosing a runbook during an incident. Ranking candidate runbooks by how often each one resolved a similar incident historically is a scoring problem, not a writing problem.
• Skill selection. Deciding which skill an agent should invoke, restart a service, query logs, open a PR, often depends on structured state such as environment and freeze status, which a generative model would have to infer from text instead of checking directly.
• Eval and skill health. Did the fix actually resolve the ticket, and does this skill's resolution rate hold up across thousands of invocations? Checking this per call with a full LLM pass is usually too slow and expensive to run at volume.

Each of these decisions have the same shape: a finite set of outcomes, so they deserve a cheap (or even free), fast decision model.

The economics of using decision models in software factories

A ticket moving from creation to a merged fix typically passes through two classification-type decisions, priority and routing, and two completion-type decisions, a step check and an eval. Running all four through Jev costs about $0.00042 per ticket. Running the same four through a generative model costs about $0.0082 to $0.0503 per ticket, roughly 20x to 120x more, depending on how powerful the model is. 

At 10,000 tickets a month that difference is $82 to $503 for LLMs versus $4.20 for decision models.

How to add decision models like Jev without redesigning the factory

You probably shouldn’t swap out all the decisions in your software factory for a decision model right away, but it is actually quite easy to get started:

1. Pick one decision with a closed set of answers. A completion check or an intent classification step is the strongest starting point.
2. Test it offline before it touches production. Run labeled, real data through the decision model and check whether confidence actually tracks correctness on your traffic.
3. Set a confidence threshold and keep the LLM as the fallback. Act automatically above it, send the middle band for review, and route anything below it to the model you already trust.

As more and more companies build out their software factories, we’re going to see more and more of the decisions on those factory lines move from humans, to LLMs, and then to decision models. Let generative models do the work, and let a decision model answer the closed questions between steps.

FAQ

Does Jev replace the coding agents already in the pipeline?

No. Jev answers closed questions with a fixed set of outcomes, while coding agents write code, draft explanations, and reason through open-ended problems. The two are complementary: Jev handles the narrow judgment calls between agent steps, and the generative model still does the actual work. TypeSafe's own documentation is explicit that Jev cannot replace the model behind a coding agent.

Can Jev run on a company's own infrastructure?

Not today. Jev is a hosted API, and every call leaves your infrastructure, since TypeSafe has not published a self-hosted, VPC, or on-premises option. TypeSafe offers zero data retention for enterprise customers through a sales agreement and states it does not train on user data. If local deployment is a requirement, Laya, a smaller open-weight model built on ModernBERT, is the closer fit, though it runs with weaker out-of-the-box calibration.

How much does it cost to run a check on every pipeline step?

Jev charges $0.042 per million input tokens with no charge for output, since it has no long generated response. At volume, that is roughly $0.07 per 1,000 completion checks in one published test, against about $0.40 per 1,000 for the equivalent LLM call. Latency, not price, is usually the real constraint at catalog scale, since larger inputs push response time toward a second.

What happens when Jev returns the wrong decision?

The model attaches a probability to every answer, so the working defense is a confidence threshold, not trust in the raw output. Code should act automatically only above a threshold you set from labeled data, send mid-confidence answers for review, and fall through low-confidence or refused answers to the existing LLM step. In Harness's own tests, every error landed inside the confidence band the model itself had flagged as uncertain.

Does Jev need to be trained on a company's own codebase?

No. TypeSafe states there is no per-customer fine-tuning. You customize behavior through the state, instructions, and criteria you send with each call rather than training a dedicated model. That keeps setup fast, but it also means accuracy depends entirely on how well you describe the decision and the evidence behind it in each request.

‍