About 30 minutes · four recorded call transcripts to read against
Goal: name the four parts of a question and the three parts of a request.
A request to Jev is one JSON object with three keys (docs: API):
{
"state": /* string | object | array of strings: the thing to judge */,
"model": "jev-latest", /* or a pinned version like "jev-1.13.0" */
"questions": {
"your_id": { /* 1. id: your key, never shown to the model */
"type": "choice", /* 2. type: choice | score | noul */
"instructions": "…", /* 3. what to judge, in plain words */
"criteria": { … } /* 4. the options, the levels, or the yes/no descriptions */
}
}
}q1 and caller_wants_quote behave identically. Name them for the person reading the code.null). Score: an ordered array of level descriptions. Noul: optional {true: …, false: …}.The running example for the rest of this lesson is a small solar installer's phone line. Four short transcripts, five questions, all recorded live on 2026-09-19. Level 5 puts them all in front of you; levels 2 to 4 use them one primitive at a time.
Goal: write a Choice with good options and read its probabilities.
"outcome": {
"type": "choice",
"instructions": "What the caller wants from this call",
"criteria": {
"sales_lead": "Caller wants to buy, get a quote or book a consultation",
"support": "Existing customer with a technical or billing problem",
"vendor_or_spam": "Caller is selling something to the company",
"unclear": "Too little information to tell what the caller wants"
}
}vendor_or_spam already carries meaning.null when the key is self-explanatory. Describe when two options could overlap, or when the label is company jargon.other or unclear option (docs: Choice). Without one, a cut-off call still gets forced into a real category.The response carries three things: the winning choice, a probability for every option (they sum to 1), and a confidence between 0 and 1. Run the two transcripts below and watch the outcome question only; the others come later.
Both calls are unambiguous and the numbers say so: sales_lead at 1.0 with confidence 1.0, vendor_or_spam at 1.0 with confidence 0.99. Compare with lesson 1's Stripe message (0.69 / 0.31, confidence 0.53). Same primitive, very different shape of answer. The probabilities are the answer; the top label is a summary of them.
For a messier example straight from the docs: a shoe-store ticket asked "which department" came back returns 0.60, billing 0.38, shipping 0.02 with confidence 0.39, because the customer wanted a refund and complained about delivery (docs: Choice, full example). That is a ticket for a human, not a bug in the model.
Goal: explain why a score of 1.12 is not an error, and why "levels 1 to 3" is a bad ladder.
"caller_mood": {
"type": "score",
"instructions": "How the caller comes across",
"criteria": ["Friendly or neutral", "Impatient or worried", "Angry or hostile"]
}Criteria is an ordered array of 2 to 10 level descriptions. Level 0 is the first entry. The model never sees the numbers; it judges each description independently and returns a probability per level. The score is the probability-weighted mean of the level indices: Σ level_index × probability (docs: Score). So it can, and often does, land between two levels.
The same call was asked for its mood twice on 2026-09-19. With described levels it scored 1.00 with confidence 0.99. With the levels written as ["1", "2", "3"] and the instruction "rate the caller mood from 1 to 3", it scored 1.46 with probabilities 0.02 / 0.50 / 0.48 and confidence 0.26. The model has no idea what a "2" is. The docs report the same effect on their bug-severity ladder: numbers-only levels gave 0.57 with confidence 0.35, and adding a matching example to each level lifted confidence from 0.54 to 0.90 (docs: writing good criteria).
Two more things from the docs worth keeping: use the extra levels only when each one is a genuinely distinct situation (3 to 5 is the sweet spot for most workflows), and remember that legend in the response maps indices back to your descriptions so logs stay readable.
Goal: phrase a Noul so that high means yes, and know what it does not promise.
"needs_attention_today": {
"type": "noul",
"instructions": "The caller describes a problem that needs attention today"
}
"callback_requested": {
"type": "noul",
"instructions": "The caller asks to be called back or agrees to an appointment",
"criteria": {
"true": "An explicit request for a call, or accepting a proposed time",
"false": "No follow-up contact is requested or agreed"
}
}noul from 0 to 1, the probability the statement holds. There is no confidence field on a Noul; the number already carries it (docs: Noul).{true: …, false: …} sharpens the boundary when "yes" needs a definition, as with the callback above.| Call | callback_requested | needs_attention_today | Reading |
|---|---|---|---|
| 1 · roof quote, agreed a visit | 0.95 | 0.03 | Book it, no rush |
| 2 · inverter fault, second day dark | 0.07 | 0.93 | Escalate today, nobody asked for a callback |
| 3 · cut off after ten words | 0.05 | 0.20 | Nothing to act on |
| 4 · cold caller | 0.04 | 0.07 | Ignore |
On call 2 the statement "the caller is angry" scored 0.82 and "the caller is not angry" scored 0.29. They add to 1.11. The docs document the same effect and say it plainly: structural invariants like noul + not_noul = 1 are not guaranteed, and a Noul and a two-option Choice asking the same thing can disagree (docs: Jev 1.13 jaggedness). Ask one Noul per fact, phrased positively, and threshold it in code.
Goal: predict every answer before revealing it, and notice where you were surprised.
The five questions: outcome (Choice), callback_requested (Noul), caller_mood (Score), language (Choice with null descriptions), needs_attention_today (Noul). Pick a call, read the state, write your five predictions down, then run the replay. The live button opens the same request in the Playground on your own account.
unclear at 0.99. That only worked because unclear existed. Delete it and the model must pick a real category for ten words of nothing.The response envelope, for completeness: model (the versioned ID that actually answered, even when you asked for jev-latest), answers keyed by your ids, and usage with input_tokens and output_tokens (docs: API response).
Goal: three primitives, read cold.
1. A Score over three levels returns 1.46 with probabilities 0.02 / 0.50 / 0.48 and confidence 0.26. Best reading?
That is the recorded numbers-only ladder. A near-even split between two levels with low confidence means the descriptions did not let the model tell them apart. Describe situations.
2. Which field does a Noul answer NOT contain?
Noul returns noul only. Confidence exists on Choice and Score, where it summarizes a whole distribution.
3. Two Nouls, "caller is angry" and "caller is not angry", return 0.82 and 0.29. What does this tell you?
Documented behaviour: no structural invariants across questions. Ask the positive statement once and threshold it in code.
4. Why add an unclear option to a Choice about call outcomes?
Probabilities must sum to 1 across the options you give. If the honest answer is "nothing to see", it needs a place to go.
Take one of the three decisions you wrote down in lesson 1. Write it as one Choice, one Score and one Noul that all make sense for the same input, open the Playground, paste a real (anonymised) input, and run. Keep the JSON; lesson 3 will improve it.
Primary source: the three primitive pages, Choice, Score, Noul. Each has a complete request and response with numbers.