Learning how to prompt Jev well is mostly about question size, not clever wording. On an independent benchmark, TypeSafe’s Jev scored 62.6% on a single compound judgement and 95.0% on the same decision split into five atomic questions, the difference between a model that looks unreliable and one that matches a general-purpose LLM on accuracy while beating it on cost and speed.
If your first Jev integration came back disappointing, this is very likely why. Jev is a System One model built for fast, narrow, typed judgements, not open-ended reasoning, and it punishes a question that tries to do several jobs at once. We replicated this pattern on our own content, and it held up.
If you haven’t covered the basics, start with what Jev and TypeSafe’s System One model are and how to use Jev’s Choice, Score and Noul primitives.
Why do Jev’s answers get worse when the question gets bigger?
Jev reads instructions literally rather than inferring intent, so a compound question forces it to silently blend several criteria into one probability. Ambiguity in that blend shows up as lower accuracy and worse calibration. A narrow, single-signal question gives Jev exactly one thing to judge, which is what its architecture, fast pattern-matching against a fixed state rather than multi-step deliberation, is built for.
Asked as one question, Jev averages several signals into a single number, and averaging destroys information. Asked separately, each signal gets full attention. Decomposition is the single highest-leverage move in this guide.
What happened when one question became five?
An independent evaluation on 2,000 synthetic phishing emails found Jev scored 62.6% on a single “is this phishing” question, against 81.3% for Claude Haiku 4.5, a statistically significant gap. Split into five atomic signal questions and combined with a logistic regression, Jev reached 95.0%, ahead of Haiku’s 93.2%, and the gap was no longer significant (p = 0.063).
The benchmark, built on the PhishNChips v5.2 dataset and reported in an independent write-up and reproducible repository, split 1,000 phishing and 1,000 legitimate emails evenly. Jev caught only 43.2% of phishing with an 18% false positive rate, weak against Haiku 4.5 (not a frontier model). The five decomposed probabilities were fed into a logistic regression fitted on 1,000 held-out emails and scored on another 1,000. That combination, not Jev alone, produced 95.0%.
In our own testing on 22 September 2026, we ran a small version of the same test: 16 compliance-training modules, 8 written to meet a defined five-part content standard, 8 deliberately missing pieces of it. A single compound “does this meet our standard” question scored 50.0% accuracy, no better than a coin flip. Split into five atomic Noul questions, one per required element, combined by majority vote, it scored 100.0%.
That swing is larger than the phishing study’s, and it needs a caveat: our 16 labels were constructed to test the mechanism, not sourced from real, ambiguous non-compliant content, and 16 modules is a small sample. It shows Jev applies a clearly defined standard consistently once decomposed, but it is not equivalent-strength evidence to a 2,000-example study. What it confirms, in a domain entirely different from phishing, is the same mechanism: decomposition removes the blending problem.
| Study | Compound accuracy | Decomposed accuracy | Comparison model | Notes |
|---|---|---|---|---|
| Independent phishing benchmark | 62.6% | 95.0% (with logistic regression on top) | Claude Haiku 4.5: 81.3% compound, 93.2% decomposed | n=2,000 synthetic emails, PhishNChips v5.2 |
| Our own L&D replication | 50.0% | 100.0% (majority vote, no model on top) | Not tested | n=16 constructed modules; mechanism check, not a real-world claim |
How do you decompose a judgement into atomic questions?
Decomposition means listing every independent signal an evaluator would check, writing one question per signal, and combining the answers afterward in your own code rather than asking Jev to combine them.
Step 1: List every component of the standard separately
Write the standard as an evaluator would check it, not as a sentence. Ours had five parts: an objective, a graded assessment, a regulation citation, a worked scenario, and a visible content date.
Step 2: Turn each component into one atomic question, sent in one call
Each part became its own Noul question: has_objectives, has_assessment, cites_regulation, has_scenario, dated_content. Because Jev evaluates questions in parallel and isolation, five cost no latency penalty over one.
Step 3: Combine the answers in your own code
A simple majority vote reached 100% against our constructed labels. For higher-stakes decisions, a weighted or trained combination, as in the phishing study’s logistic regression, will outperform a plain vote.
This applies outside Jev, too. Badly written training needs assessment questions fail the same way: they ask about more than one thing at once, forcing whoever answers to silently weigh the parts.
How should you write criteria descriptions?
Write criteria as a single, literal, checkable condition, worded as you’d explain it to someone who cannot ask a follow-up question. Jev does not infer unstated context, so a criterion depending on background you haven’t written down gets answered inconsistently.
We tested this across 15 of our own published and drafted posts, tagged into a content-family taxonomy, on 22 September 2026. Agreement with our human-assigned tags was 73.3%, with mean confidence of 0.849. That combination, middling agreement alongside high confidence, is diagnostic: it usually means criteria are clear enough for Jev to commit, but not precise enough to match your intended distinctions. The fix is rewriting the criteria, not lowering expectations.
The simplest test for a criterion: would you need to explain it verbally if a colleague misread it? If yes, that explanation belongs in the instruction text, not in your head.
Write Criteria As Checklists
Draft every criterion as a single yes/no sentence with no “and,” “or,” or “considering” in it. A criterion that needs a conjunction to state is at least two criteria.
Why does Jev need a “none of these” option?
A Choice question forces Jev to pick from the options given, even when none fits, producing a confidently wrong answer instead of a useful signal. An explicit “none of these” option lets Jev flag a mismatch instead of guessing.
This matters most where categories were designed for the common case and real input has edge cases you didn’t plan for. Without an escape option, edge cases get force-fit into the closest category, quietly degrading accuracy because Jev still reports a distribution that looks reasonable. Watch for flatness: near-equal probabilities across options signal unclear criteria or thin state. The escape option won’t fix unclear criteria, but it stops the model masking the problem.
Should you ask all your questions in one call?
Yes, when the questions are genuinely independent. Bundling multiple questions into one call, a pattern TypeSafe calls speculative fan-out, is cheaper and faster than sequential calls, because Jev evaluates every question in parallel and in isolation against the same state.
TypeSafe reports that bundling 13 independent questions into one call was 12.2 times cheaper and 10 times faster than 13 sequential calls, in a documented test of the fan-out pattern, described more generally in TypeSafe’s own documentation. Most of the saving comes from sending shared state once instead of thirteen times; treat the multiplier as directional. Because questions are evaluated in isolation, cost scales with tokens, not time: fan out what you need and stop.
How do you turn confidence into a routing rule?
Confidence-gated routing sets a different threshold per action, scaled to the cost of being wrong, and routes low-confidence answers to a human instead of acting automatically. Read-only actions can run at low confidence; irreversible ones should not.
def route_decision(answer, confidence, action_cost):
thresholds = {
"read_only": 0.55,
"reversible": 0.75,
"irreversible": 0.92,
}
threshold = thresholds.get(action_cost, 0.85)
if confidence >= threshold:
return execute(answer)
return escalate_to_human(answer, confidence)
The threshold is a business decision, not a modelling one. A content tag feeding an internal filter can run at 0.55 and get corrected cheaply later; a question auto-approving a payment should sit far higher. Our deep dive on Jev’s limitations and confidence calibration covers where these numbers hold up and where they don’t. If the packaged metric doesn’t fit, Jev exposes the raw probability distribution for threshold logic of your own design.
How do you combine several scores into one decision?
Composite scoring means judging each independent dimension separately, then applying your own weights to combine the scores in code, rather than asking one question for a blended verdict. This keeps each dimension auditable and lets you re-weight without another call.
Grading e-learning content on tone, accuracy and completeness as three Score questions, then applying weights such as 50/30/20 in your own function, gives a number you can defend and adjust. One blended question instead loses visibility into which dimension is dragging the score down. We tested this directly in our companion piece on composite scoring tested against real assessment grading.
Log Every Component Score
Store each individual Score and Noul output alongside the final composite number, not just the composite. When a weighting turns out wrong later, you re-run the math, not the model.
Why does adding context make accuracy worse?
The general concern is context rot: padding a call with material the question doesn’t need can dilute the signal Jev is looking for and lower accuracy. In our own testing, we did not observe this effect at the scale we tested.
On 22 September 2026, we ran three state sizes against the same 15 tagged blog posts: a lean state (title and objectives only), a full state (every field), and a bloated state (every field plus irrelevant padding such as fake enrollment counts and a generic instructor bio). All three scored 73.3%. Accuracy held completely flat.
This is a genuine null result, reported honestly because it runs counter to the general hypothesis that irrelevant context degrades accuracy. We are not claiming context rot is a myth: 15 items is likely too small to detect a real effect, and the title and objectives fields probably already carried most of the signal. With larger or noisier state, treat this as open, not settled, and test it against your own content.
What are the mistakes that waste the most time?
Most wasted time on a Jev integration traces back to a handful of repeatable mistakes: compound questions, missing escape options, one confidence threshold applied everywhere, and treating a flat probability distribution as a normal answer instead of a warning. Each has a straightforward fix, listed below.
| Mistake | What it looks like | Fix |
|---|---|---|
| Compound questions | One question with “and” or “considering” joining criteria | Split into atomic questions, combine in code |
| No escape option | Choice questions with only expected categories | Add “none of these” as a real option |
| One threshold for everything | Same cutoff for low-stakes and irreversible actions | Scale threshold to the cost of being wrong |
| Ignoring flat distributions | Near-equal probabilities treated as normal | Treat flatness as a signal to rewrite criteria |
| Assuming more state helps | Stuffing every field in “to be safe” | Test lean versus full state first |
| One blended verdict | Single composite number, no component scores | Score dimensions independently, combine yourself |
See our overview of using Jev for learning and development use cases and our look at how AI is showing up inside modern LMS platforms.
Conclusion
The 62.6% to 95.0% swing is not a benchmark curiosity. It is the clearest evidence available that Jev’s usefulness depends on how you write the question, not on raw model capability. Every pattern here is downstream of one principle: give Jev one narrow, checkable thing to judge, and combine results yourself.
Our own testing supports the same mechanism on different content, at small scale, with an honest null result on context size we are not smoothing over. Before concluding Jev “isn’t accurate enough,” count how many judgements your prompt is asking it to make at once.
Pull your worst-performing question, list every independent signal buried inside it, and rewrite it as separate Noul or Score questions against the same state. Test both versions against a small labelled sample of your own content before deciding the model is the problem.
FAQ
Q1. Why is Jev inaccurate on my first attempt?
Almost always because the question is compound. Jev is a System One model that reads instructions literally, so asking it to weigh several criteria at once forces it to blend them into one probability. An independent benchmark saw accuracy jump from 62.6% to 95.0% on the same judgement once it was split into five narrow questions.
Q2. How do you prompt Jev for the best accuracy?
Write one atomic, single-signal question per judgement instead of one compound question, phrase criteria literally with no unstated context, add a “none of these” option to Choice questions, and combine multiple question outputs yourself in code rather than asking Jev to blend them internally.
Q3. What is Jev decomposition and why does it work?
Decomposition means splitting a compound judgement into several atomic questions, one per independent signal, then combining the answers in your own code. It works because Jev’s architecture is built for narrow pattern-matching against a single signal, not for silently averaging several criteria into one probability.
Q4. Does asking Jev more questions in one call slow it down?
No. Jev evaluates every question in parallel and in isolation against the same state, so bundling questions barely affects response time. TypeSafe reports a 13-question call ran 12.2 times cheaper and 10 times faster than 13 sequential calls, though cost still scales with the number of tokens sent.
Q5. What does a flat probability distribution from Jev mean?
A flat, near-equal spread across options usually signals that your criteria are unclear or that the state you gave Jev doesn’t contain enough information to distinguish between them. Treat it as a prompt to rewrite the criteria or enrich the input, not as a normal answer to accept.
Q6. Does adding more context always hurt Jev's accuracy?
Not necessarily. The general “context rot” concern is that irrelevant material dilutes signal, but our own test across 15 tagged blog posts found no accuracy difference between lean, full, and deliberately padded state. The sample was small, so treat this as an open question worth testing on your own content.
Q7. Should I trust Jev's confidence score on its own?
Treat confidence as one input to a routing decision, not a final answer. Scale your acceptance threshold to the cost of being wrong for that specific action, and use Jev’s raw probability distribution instead of the packaged confidence metric when your use case needs finer-grained control.
