Jev — Thinking Fast, in One Pass

A camera pipeline asks the same small questions thousands of times a day. What colour is this jacket? Is that a bus or a truck? Each answer is one word from a short list. Sending every one to a large model that writes paragraphs is slow and costs money; a fast model is cheap but sometimes wrong. The whole trick is knowing when it is wrong.

Any fast-and-slow decision system can be stated in four steps, and this read hangs on them:

• state: the thing the questions are about, a text or a picture;
• one fast pass: every question answered at once, against the same state;
• probabilities over your options: never free text, only the options you offered;
• a threshold: act on the sure answers, escalate the rest to something slower, a bigger model or a person.

Daniel Kahneman called the fast half System 1 and the slow half System 2. TypeSafe AI named a product after the first: Jev, a "System One model". Here's the plan:

1. Two Systems in Your Head
2. From Brains to Models
3. Jev, From the Outside
4. Jev, From the Inside
5. Visual Jev and Other Models That Read Pictures
6. Conclusion

One example runs through all of it: a navy garment, and whether it counts as "blue".

Brush up ;)

• OpenAI’s GPT — Part 1: Unveiling the GPT Model: writing an answer token by token costs one pass per token.
• Query, Key, Value — Dot-Product Attention from Scratch: what one forward pass over a sequence computes.
• Is ALBERT short for BERT?: an encoder with a small head on top, the older way to classify text.
• CLIP — Learning Pictures From the Words Next to Them: zero-shot by comparing an image with one sentence per label, the older way to ask a picture a closed question.
• Benchmark Datasets: Gas-station for CV: where labels come from, and why that matters.

NOTE: some words mean exactly one thing here.

• System 1 / System 2: Kahneman's.
• System One model: TypeSafe's name for its product.
• Jev: TypeSafe's hosted jev-1.13, nothing else. Visual Jev: Yu & Yao's paper.
• probability: one per option. top probability: the largest of them. confidence: TypeSafe's formula (§3.3), never "the chance of being right".

1. Two Systems in Your Head

A bat and a ball cost $1.10 together. The bat costs $1.00 more than the ball. How much is the ball?

Answer before you read on. A number turned up straight away, didn't it? Hold on to it ;)

That number came from what Kahneman calls System 1. Checking it is System 2's job, and System 2 often doesn't bother. This section walks through Thinking, Fast and Slow [KAHN11] up to the question the rest of the read asks about models: when can a fast answer be trusted?

1.1 The Two Characters

Kahneman borrows the labels from Stanovich and West [STAN00] ([KAHN11] ch. 1 The Characters of the Story). His definitions [KAHN11a]:

• System 1 "operates automatically and quickly, with little or no effort and no sense of voluntary control."

• System 2 "allocates attention to the effortful mental activities that demand it, including complex computations."

• System 1: 2 + 2, hostility in a voice. System 2: a tax form, 17 × 24 [KAHN11a].

• System 1 runs all the time and proposes. System 2 idles and mostly endorses; when System 1 "runs into difficulty, it calls on System 2" [KAHN11a].
• Evans and Stanovich call this default-interventionist: the fast answer is the default, the slow one may step in ([EVAN13] p. 227).
• Keep this shape in mind. Everything later is built on it: a fast answer for every question, a slow check for a few.

And System 1 cannot be switched off:

bottom line length (% of the top one)

100

fin angle (°)

35

ruleroffon

• With the ruler on, you know the lines are equal. They still look unequal.
• System 2 can learn to distrust the picture; it cannot stop System 1 drawing it [KAHN11a].

NOTE: Kahneman is blunt about what the two systems are: "fictitious characters" [KAHN11a], shorthand for two kinds of processes, not two places in the brain. Every time this read calls a model "System 1", it borrows the vocabulary, not the brain.

1.2 The Lazy Controller

Back to the ball. If you said 10¢, the bat is $1.10 and the pair costs $1.20. The check:

• b is the ball's price in dollars, so the bat is b + 1.00.

• The ball is 5¢, the bat $1.05. One addition kills 10¢.

• This is puzzle one of Shane Frederick's Cognitive Reflection Test [FRED05]: three questions, each with an answer that jumps out and is wrong.

• People can do the sum. They just don't: the check is cheap and still gets skipped. System 2 is lazy ([KAHN11] ch. 3 The Lazy Controller).

puzzle◀1 / 3 · bat and ball▶ Nextyour answer (¢, minutes or day)

0

1.3 A Machine for Jumping to Conclusions

Part I explains why the fast answers convince us. Three ideas matter for models:

• Norms (ch. 6): ask how many animals of each kind Moses took into the ark, and few people notice that it was Noah: Moses fits ([KAHN11] ch. 6). Remember Moses. In section 4 an option's name plays the same trick on a model.
• WYSIATI, what you see is all there is (ch. 7): System 1 builds the most coherent story from what is in front of it, and how sure you feel tracks the story's coherence, not the evidence behind it ([KAHN11] ch. 7). Every calibration point later grows from here.
• Substitution (ch. 9): given a hard question, System 1 answers an easier one ([KAHN11] ch. 9). A model that predicts a label from a cue that merely goes with it does the same (section 5).

1.4 Small Numbers, Anchors and Linda

Part II catalogues the heuristics and the biases they leave. Three come back later.

The law of small numbers (ch. 10): small samples swing to extremes ([KAHN11] ch. 10). Remember it whenever two models differ by two points on a small test set.

Anchors (ch. 11): an arbitrary number seen first pulls an estimate towards it ([KAHN11] ch. 11).

Linda (ch. 15). Linda is 31, single, outspoken, studied philosophy, and as a student cared deeply about discrimination and social justice. Which is more probable?

• Linda is a bank teller.
• Linda is a bank teller and is active in the feminist movement.

Asked to rank eight statements about Linda, these two among them, 89% of 88 statistically naive undergraduates put the second above the first, and so did 85% of 32 decision-science doctoral students ([TVER83] Table 1, p. 298). That is the conjunction fallacy: the story that fits Linda stands in for the one that is likely ([KAHN11] ch. 15). The rule it breaks:

• T: Linda is a bank teller. F: she is an active feminist.
• P(F | T) is at most 1, so the pair can never beat T alone. Let's count:

share who are bank tellers (%)

10

share of the tellers who are feminists (%)

60

1.5 Base Rates: the Cab Problem

Next: ignoring how common things are to begin with ([KAHN11] ch. 14, ch. 16 Causes Trump Statistics).

• 85% of a city's cabs are Green, 15% Blue. A cab was in a hit-and-run at night.
• A witness says it was Blue. Tested in the same conditions, the witness is right 80% of the time.
• What is the chance the cab was Blue?

The usual answer, Kahneman reports, is 80%, with the 15% thrown away ([KAHN11] ch. 16). Let's devise the equation step-by-step:

• P(B) = 0.15 is the base rate; P(says B | B) = 0.80 is the witness's accuracy.
• Two ways to hear "Blue": a Blue cab seen right (0.12) and a Green cab seen wrong (0.17). Green cabs are so many that the mistakes about them outnumber the correct calls.

Blue cabs in the city (%)

15

witness is right (%)

80

A rare label plus a decent detector still means many false alarms. Now swap the witness for a fast model and Blue for a rare class. Same maths, same false alarms.

1.6 When Can You Trust an Intuition?

Part III, Overconfidence, turns to experts. Chapter 22, the one this read leans on hardest, retells his joint work with Gary Klein [KAHN09]. An intuition deserves trust only when

1. the environment is regular enough to be predictable, and
2. the person has had the chance to learn it, through long practice with quick, clear feedback.

Firefighters and chess players pass both; stock pickers fail the first ([KAHN11] ch. 22). And: "Subjective experience is not a reliable indicator of judgment accuracy" ([KAHN09] abstract). Feeling sure is not evidence; a track record on the same kind of question is.

NOTE: parts of the book have not held up. Priming (ch. 4) failed worst, and Kahneman himself said he had "placed too much faith in underpowered studies" [SCHI17]; ego depletion (ch. 3) failed a 23-lab replication [HAGG16]. None of these is a demo here: the demos are arithmetic or an illusion you can check with a ruler. The two-systems split has critics too: Evans & Stanovich stress that the slow process is not always right ([EVAN13] p. 226), and Melnikoff & Bargh call the split a "convenient and seductive myth" [MELN18].

System 1 here: the 10¢ that arrives before you ask for it, sure of itself; right where the world is regular and feedback is quick.
System 2 here: the one addition that checks it, and that you skipped.

Can a model's intuition be trusted on the same terms?

2. From Brains to Models

2.1 Thinking Out Loud: Chain of Thought as System 2

• Bengio, NeurIPS 2019: deep learning so far is System 1 (fast perception learned from fixed datasets); System 2, slow step-by-step reasoning, is the next step [BENG19].

• A language model writes one token at a time. Ask it to write the steps first and it starts to look like System 2:

• Chain of thought [WEI22]: worked examples with the steps written out lift accuracy on arithmetic, commonsense and symbolic problems; "Let's think step by step" alone lifts GSM8K (grade-school maths word problems) from 10.4% to 40.7% on text-davinci-002 [KOJI22].

• Reasoning models such as DeepSeek-R1 [DEEP25] are trained to write long chains before answering.

And on easy questions? Chen et al. [CHEN24a] ask o1-like models "what is the answer of 2 plus 3?"; in their Fig. 2 one of them (QwQ-32B-Preview) writes 13 solutions in 901 tokens, and the first, already right, took 39. System 2 spending over twenty times the effort on 2 + 3 is a failure Kahneman's lazy controller would never make ;)

Let's count what each style costs. One pass is one forward pass of the network; a sequential step is a pass that has to wait for the previous one. For k questions about one input, with a answer tokens per question and T thinking tokens:

• The 1 is the pass over the input. A model that reads every answer off that pass is done; a writer needs one more step per token.
• With k = 8, a = 10 and T = 500 (my estimates, not measurements): 1, 81 and 581 steps.

questions k

8

thinking tokens T

500

answer tokens per question a

10

my estimates2 + 3Reset

NOTE: steps are not time. On a GPU steps can be batched and cached; section 5 shows where the time really goes.

2.2 Routers and Cascades

Two ways to decide who answers:

• a router decides before anyone answers which model answers [ONG24];
• a cascade lets the fast model answer everything and escalates the items it is unsure of to the slow one [CHEN23].

Think of a receptionist who answers the easy calls and puts the hard ones through. Neither idea is new:

• Chow's reject option (1970): don't answer when your top probability is below a bar [CHOW70].
• Viola & Jones's face detector is a cascade: cheap early stages throw away most windows [VIOL01].

Let's do the cascade sums. The fast model F costs $c_F$, the slow model S costs $c_S$. F answers every item; items whose top probability is below a threshold τ go on to S. Call κ the share F keeps:

• Made-up numbers: $c_F=1$, $c_S=50$, κ = 0.8 → cost 1 + 0.2 × 50 = 11, so about 4.5× cheaper than sending everything to S.
• ${\text{acc}}_S(p<\tau )$ is S's accuracy on the escalated items only, the hard ones.
• Say F is 95% right on what it keeps, and S is 90% overall but 70% on the leftovers: 0.8 × 0.95 + 0.2 × 0.70 = 0.90, not the 0.94 you'd promise from S's overall 90%.
• That gap is a common mistake in quick cascade sums.

2.3 Where the Analogy Holds, and Where It Breaks

This table is my judgement, not a finding. The "breaks" rows are the useful ones.

Table. 1 The two systems against a fast model plus a slow one. My reading of the book's chapters 1, 3, 7 and 9 and of [EVAN13].

2.4 When to Trust a Fast Model's Top Probability

Kahneman & Klein's conditions [KAHN09] translate almost word for word:

• A regular environment → a task that is predictable from the input.
• A chance to learn, with feedback → the model saw data like yours, and you collect the outcomes to check it.

• "Subjective experience is not a reliable indicator" → a top probability is evidence only after it has been shown to be calibrated on your data.

• Calibrated: of all answers given with probability about q, about q of them are right.

• It is a property of many answers, never of one.
• To measure it: sort answers into B bins by top probability and compare each bin's accuracy with its mean top probability. N answers, $n_b$ of them in bin b:

• This is the expected calibration error [NAEI15] [GUO17]. A bin of 200 answers, out of 2,000, at mean 0.95 that is right 80% of the time adds 200 / 2,000 × 0.15 = 0.015 on its own.
• Dividing the logits (the scores before the softmax) by one number, the temperature, fixes much of the overconfidence of modern networks [GUO17].

A simulated model to play with. Start calibrated, make it overconfident, then keep it calibrated but move it to shifted data:

temperature it reports through0.250.5124datathe data it was tuned onshifted datacalibratedoverconfidentcalibrated, then shiftedReset

• Calibrated: of the simulated answers at 0.9 or more, 94% are right. Overconfident: 80%. Calibrated but on shifted data: 76%.

• Nothing in the model's output tells you the data shifted (WYSIATI again); only your own labels catch it.

• So, on Kahneman & Klein's terms: a fast model is worth having only if its probabilities tell you when to call the slow one.

• Free text gives a messy signal: a sequence's probability is biased by its length [GUPT24].
• A closed set of options gives one number. TypeSafe built a product on exactly that.

3. Jev, From the Outside

Think of an exam sheet with multiple-choice answers. You write the questions and the options. The examiner reads the passage once, fills in one bubble per question, and also tells you how hard it pressed the pencil on every bubble. It never writes a sentence in the margin. That is Jev.

3.1 What Jev Is

• Jev is TypeSafe AI's hosted jev-1.13, the only model TypeSafe lists as current on 10 Oct 2026 [TYPE26g], announced by Diogo Almeida, TypeSafe's founder and a co-author of the InstructGPT paper [ALME26], [OUYA22].
• TypeSafe calls it a System One model, after Kahneman's System 1 [TYPE26i]. In the launch post's words, "a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out" [ALME26].
• One request carries a state (a string, a JSON object or an array) and a map of named questions [TYPE26h]. Every question is answered against the same state, and "one answer does not become context for another question" [TYPE26e]. No text or explanation comes back [TYPE26i].

3.2 Three Kinds of Question

The whole answer vocabulary is three primitives [TYPE26e], [TYPE26h]:

Table. 2 The three question types and their answer fields, from TypeSafe's API reference and primitives pages [TYPE26h], [TYPE26e].

The option names and their descriptions are both sent to the model [TYPE26e]. Section 4 shows how much a name alone can move the answer.

3.3 Confidence Is Not a Probability

Every Choice comes back with a probability per option and one confidence, a statistic of the shape of the probabilities [TYPE26f]. For a Choice with n options and top probability $p_{max}$:

• $\frac{1}{n}$ is what an even split gives each option, so an even split scores c = 0 and certainty c = 1.
• The formula looks at the top probability alone: (0.6, 0.3, 0.1) and (0.6, 0.2, 0.2) both give (0.6 − 1/3)/(2/3) = 0.40 [TYPE26f].
• n is the number of options offered, not the ones that got mass. The worked call below needs this.

A Score's confidence instead asks how far the mass sits from the most likely level, so it sees the order of the levels [TYPE26f]; a Noul returns only p, and for a number on the same 0-to-1 scale the docs suggest |2p − 1| [TYPE26f].

NOTE: none of these means "right with probability c".

• Whether answers at 0.9 are right 90% of the time is calibration (§2.4), a question about many answers on your data. TypeSafe says calibration "does not guarantee that an individual answer is correct" [TYPE26i].
• There is no threshold inside Jev and no "abstain". The bar lives in your code.
• The docs' example: below 0.5 goes to a person; a high-stakes action must clear 0.9 [TYPE26f].

question typeChoiceScoreNoulChoice options offered39, as in the saved callraw score, first bar

1.5

raw score, second bar

0.5

raw score, third bar

-1

act above

0.9

send to a person below

0.5

saved call: upper colourdocs: (0.6, 0.3, 0.1)docs: Score (0, 0.57, 0.43)docs: Score (0.5, 0, 0.5)saved call: backpackReset

3.4 One Real Call, Worked Through

Numbers are always satisfying, so here is one real call: three questions about a man in a dark navy jacket, made on 10 Oct 2026 through a third-party gateway that forwards requests to TypeSafe. The description was written by hand; Jev reads words only (§3.5). The request, shortened (both colour questions offer the same nine options):

{"state": {"description": "A man walking on a sidewalk wearing a dark navy hooded jacket, light blue jeans and white sneakers, carrying a black backpack."},
 "questions": {
   "upper_color": {"type": "choice", "instructions": "Colour of the upper-body clothing",
                   "criteria": {"black": null, "white": null, "grey": null, "red": null, "blue": null,
                                "green": null, "yellow": null, "brown": null, "other": null}},
   "lower_color": {"type": "choice", "instructions": "Colour of the lower-body clothing", "criteria": {"...": "same nine"}},
   "backpack":    {"type": "noul", "instructions": "Is the person carrying a backpack?"}}}

Table. 3 The test call of 10 Oct 2026 (served by TypeSafe as jev-1.13, through the gateway), response as returned.

Let's check the confidence against §3.3:

• It matches with n = 9, returned rounded as 0.9. Offer only blue, black and other and the same 0.91 gives (3 × 0.91 − 1)/2 = 0.865: six empty options raise the confidence without moving a single probability.
• Navy went to blue, with 0.04 left on black. Nothing in the request said "navy counts as blue"; section 4 comes back to that.
• With the docs' example bars, the jacket at 0.9 does not clear the > 0.9 bar and lands in the "Verify first" branch. A threshold copied from the docs is an anchor.

3.5 How to Call It, and What It Cannot Do

TypeSafe serves it at POST https://api.typesafe.ai/v1/systemone [TYPE26h]. The Python SDK, after its quickstart [TYPE26j]:

from typesafe_sdk import TypeSafeClient, Choice, Noul

with TypeSafeClient() as client:              # reads TYPESAFE_API_KEY
    r = client.system_one(
        state={"description": "A man ... carrying a black backpack."},
        questions={
            "upper_color": Choice(instructions="Colour of the upper-body clothing",
                                  criteria={"black": None, "blue": None, "other": None}),
            "backpack": Noul(instructions="Is the person carrying a backpack?"),
        })
print(r.choices["upper_color"].choice, r.nouls["backpack"].noul)

• The SDK sends jev-latest unless you pass model= to the client. If your thresholds were tuned on one version, pin it [TYPE26g].
• Input is text only, up to 64k tokens for the state and all questions together [TYPE26g].

What it cannot do, by TypeSafe's own docs:

• see: no images, audio or video [TYPE26i];
• write: no free text, numbers or lists. The only answers are Choice, Score and Noul [TYPE26h], and Jev "is not trained to generate text" [TYPE26d];
• chain or explain: questions never see each other's answers [TYPE26e];
• soft spots, from its "jaggedness" page: literal reading, counting, questions with several hops, long irrelevant state, a lean towards the first option [TYPE26d].

System 1 here: one call, three answers, and a confidence you can check by hand.
System 2 here: your threshold. Jev never checks itself; the jacket at 0.90 waits for a person or a slower model.

So much for the outside. Now the inside, as far as anyone can see it.

4. Jev, From the Inside

Short answer: TypeSafe has published no paper, no architecture, no model card with a size and no weights. So we go in four steps:

• what TypeSafe says (§4.1);
• what its behaviour gives away (§4.2);
• what nobody outside knows (§4.3);
• how an open model with the same interface does it (§4.4).

4.1 Disclosed: What TypeSafe Says

1. Jev is built on "a new model architecture, parallel sampler … and training method we call Reinforcement Learning for Calibrated Decisions (RLCD)" [ALME26].
2. Sampling is "Parallel. Generates all outputs in a single query", where an LLM is "Sequential. Generates one token at a time" [ALME26].
3. "Jev ingests the state once and evaluates every question against it in parallel" [TYPE26g].
4. "Jev is neither small nor an LLM" [ALME26]; it "understands language, but doesn't generate free-form text" [TYPE26a].
5. RLCD's contract: higher probability "should correspond to a greater chance that the answer is correct" [TYPE26l].
6. Choices with many options use "a two-stage system of scoring independently then making an explicit choice" [ALME26].

In short: three named ingredients, a sentence about reading the state once, and a contract for the output.

4.2 Inferred: What the Behaviour Gives Away

None of these is TypeSafe's statement. Each is my reading of documented behaviour.

• The state is read once and each question reads it on its own. The limit is 64k tokens for the state plus all questions, but 32k for the state plus the single longest question [TYPE26g]; a budget set by the longest question only makes sense if each question meets the state separately.
• The answer lives only on your options. Probabilities sum to 1 over the names you declared [TYPE26h]. Nothing else can come out.
• Time barely grows with the number of questions. TypeSafe's cookbook (on jev-1.12) times 13 questions in one call at about 1.3 times one single call ([TYPE26c], my arithmetic).
• Time that stays nearly flat as questions grow fits one shared read of the state. It doesn't prove it.

4.3 Unknown

Nobody outside TypeSafe knows:

• the parameter count;
• whether the backbone is an encoder, a decoder or a hybrid;
• which pretrained model it starts from;
• how a question and its options are laid out and scored;
• what the "parallel sampler" samples;
• the RLCD objective.

Press pieces guess at a base model. None of the guesses comes from anyone who would know, so I leave them out.

4.4 In an Open Look-Alike: How this-that-model Does It

Everything in this part comes from this-that-model-1.0 ([CHEN26] §3), an open model built by Cheng, Dai and Sun to answer the same kind of request, and from Visual Jev [YU26], which reads its answer the same way. It shows one way the contract can be met. It says nothing certain about Jev. Let's devise the equations step-by-step.

Step 1, one read. Write the state $S$ and the questions $q_1,\dots ,q_N$ one after another, and end each question at its own answer slot $s_k$. One forward pass of the network $f_{\theta }$ over the whole input $x$ gives a hidden vector for every position:

• L is my count, assuming one slot token per question.

Step 2, pick the slot. The answer to question $k$ is read from one row only, $h_k=H_{s_k}\in {ℝ}^d$.

Step 3, score only your options. Each option $j$ of question $k$ gets one label token $\ell (k,j)$, a letter for up to ten options ([CHEN26] §3.1). The network's output table has a row $w_t$ for every token $t$; $h_k$ is compared only with the $n_k$ rows of your options:

Step 4, turn scores into probabilities over those options only (Eq. 2 of [CHEN26]; Visual Jev's Eq. 1 is the same readout without $\tau$):

The sum in the bottom runs over your $n_k$ options and nothing else. An answer outside your list is not unlikely; it has no place to exist.

Numbers make it clear. Take "what colour is the lower garment?" with options blue (incl. navy) / black / other, and say (made-up numbers, $\tau =1$) the scores come out $z=(2.1,1.4,-0.5)$:

Blue wins at 0.64. Notice what the network actually scored: the tokens "A", "B", "C". What blue means reached it only through the text of the question. Hold that thought for §4.6.

Step 5, the questions don't talk. No answer is ever written back into the input, so given the input the answers are independent (Eq. 3 of [CHEN26]):

Step 6, train against a proper scoring rule. this-that-model trains on a convex mix of cross-entropy and the Brier score $\Vert p-e_y{\Vert }^2$ (its Eq. 7), weights unstated ([CHEN26] §3.3), with $e_y$ the one-hot vector of the right option. Both are proper: the expected loss is lowest when the reported probability equals the true chance, so the model is pushed towards calibration. RLCD's name suggests the same goal; its objective is unpublished (§4.5).

Step 7, count the passes. One support ticket of about 300 tokens with 5 questions of about 40 tokens each (my estimates) takes 1 forward pass read this way, 5 as one call per question, and 41 to 61 for a text generator writing a short JSON answer, before any thinking.

What does asking many questions in one call buy? The figure uses TypeSafe's cookbook numbers and one third-party paper [LI26b], each marked with its source:

showcountcosttimequestions per state

5

answer tokens per question (text generator)

10

thinking tokens (text generator)

0

state length (thousand tokens)

11.1

the five-question ticketthe cookbook's thirteenwhat was measuredReset

NOTE: every model equation in §4.4 is this-that-model's or Visual Jev's. None is known to be Jev's.

4.5 RLCD: Teaching a Model to Mean Its Probabilities

A model trained only to be right has no reason to make its 0.9 mean 90%: 0.51 and 0.99 put the same answer on top and earn the same score. TypeSafe says Jev is trained with Reinforcement Learning for Calibrated Decisions (RLCD) [ALME26]. Here is the idea underneath, and why the reward decides honesty.

Reinforcement learning in brief. Finding the best coffee near the office: nobody hands you a ranked list. You try a shop, rate the cup, and go back to shops that beat your usual cup.

• Reinforcement learning (RL): learning which actions earn the most of a number, the reward, through trial and error ([SUTT18] §1.1).
• The loop: an agent sees a situation $s$; its policy ${\pi }_{\theta }$ (the model, weights $\theta$) picks an action $a$; the world returns a reward $R$. Repeat.
• Unlike supervised learning, you only get a score for what you did, never the right answer: feedback that "evaluates" rather than "instructs" ([SUTT18] Ch. 2).
• The baseline $b$ is your "usual cup", e.g. the average reward so far ([SUTT18] §2.8).

The simplest update [WILL92] ([SUTT18] §13.4):

• $g$ points to where $a$ gets likelier; $\alpha$ is the step size.
• $R>b$ pushes $a$ up, $R<b$ pushes it down. $b$ leaves the average step unchanged and usually makes it less noisy.

pulls so far

0

▶run (which random draws)

0

size of each nudge0.050.10.4compare each score withthe average so farzeroadd 4 to every scorenoyesfirst pullfifth pullafter 500 pulls+4, compared with zero+4, compared with the averageReset

• Press Play: three options pay 1 with hidden chances 0.2, 0.5, 0.8. After 500 pulls option 3 has chance 0.966.
• Turn on add 4 to every score and compare with zero: every pick looks good. Over 200 runs the best option ends at 0.53, against 0.95 with the average.

Three routes from a pretrained model. TypeSafe's primer: pretrained language models "have been adapted in two major ways. TypeSafe adds a third" ([TYPE26l], "Three post-training approaches").

• RLHF (human feedback): the reward is a model of people's preferences between outputs [CHRI17], [OUYA22]. It tends to make models say they are surer than they are [LENG24].
• RLVR (verifiable rewards, named in [LAMB24]): a program checks the answer [DEEP25]. Right is 1, wrong is 0, which "rewards models equally whether they are confidently correct or merely guessing" ([DAMA25] §1).
• RLCD optimises for "answers with epistemically honest probabilities on System One tasks" ([ALME26], table row "Optimizes for").

Why the reward decides honesty. Take a Noul: the model reports $q$, its probability of yes; yes ($y=1$) happens with true chance $p$. Let's devise the equations step-by-step.

Step 1, score only the decision. Reward 1 if $q$'s side of ½ is right. At $p=0.7$, reporting 0.51 or 0.99 both pay 0.70 on average. Honesty earns nothing.

Step 2, score the squared error (the Brier score <a class="cite" data-x="ref:BRIE50" href="https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2">[BRIE50] for one yes/no question, up to a factor of 2, as a reward):

• The last term ignores $q$, so the only best report is $q=p$; lying costs exactly $(q-p{)}^2$. At $p=0.7$: honest 0.7 earns −0.210, overconfident 0.9 −0.250.
• A reward whose expected value peaks at the true chance is a proper scoring rule [GNEI07]. The log reward (cross-entropy turned round) is one too; both are the losses of Step 6 in §4.4.

Step 3, reward confidence when right: $R=q$ if yes, $1-q$ if no. [XU24]'s ablation tries a similar linear "naive" reward.

• A straight line: for $p>\frac{1}{2}$ it peaks at $q=1$. At $p=0.7$, 0.99 earns 0.696 against 0.580 for the honest 0.7. It pays for overconfidence.

true chance of “yes” p

0.7

▶your report q

0.9

squared erroron its ownplus 1 when the decision is righthonest reportoverconfident reporta rare “yes”with a bonus for being rightReset

• Slide the true chance of "yes": squared error and log peak on $q=p$, confidence-when-right at the edge, decision-only is flat above 0.5.

NOTE: a proper reward makes honesty pay on the training data. Networks trained with cross-entropy, also proper, are still overconfident on new data [GUO17]. Check on your own labels (§2.4).

One way to build it, not Jev's. The closest public method is RLCR [DAMA25]: a language model writes an answer and a confidence $q$; with $c=1$ if the answer is right, else 0, the reward is (their Eq. 8; Eq. 1 in v1):

• Their Theorem 1: the expected reward peaks when $q$ is the answer's true chance, and then for the likeliest answer. A log score would break the second part ([DAMA25] App. A).
• The baseline is the average reward over several samples of the same question [SHAO24].
• Numbers (Table 1, HotpotQA): ECE (§2.4) 0.37 → 0.03 against plain RLVR, accuracy 63.0% → 62.1%. On six unseen datasets: base 0.40, RLVR 0.46, RLCR 0.21.

rewardscore only the decisionreward confidence when rightsquared errorlogtraining steps (thousands)

40

▶squared error, after 40,000 stepsconfidence when right, after 5,000confidence when right, after 40,000decision only, after 40,000log, after 40,000Reset

• A toy, not a language model. Pick a reward, press Play, watch the dots against the diagonal.
• Average gap over 30 runs, from 0.250: squared error 0.031, log 0.036, decision only 0.144, confidence-when-right 0.191 (after first nearing the diagonal).

Disclosed: RLCD is RL on a pretrained language model, returning decisions and probabilities, not text; "higher probability should correspond to a greater chance that the answer is correct" [TYPE26l]; probabilities are "optimized against outcomes" [TYPE26i]; one set of weights serves every account [TYPE26g]; "We make all the data ourselves" ([ALME26], FAQ).
Unknown: the reward; where the outcomes come from; the optimiser; any supervised stage before or recalibration after; any calibration figure from TypeSafe. Unpublished, as [IBRA26] note.
Inferred (mine): "calibrated" plus "optimized against outcomes" is what a proper scoring rule is for. TypeSafe has never named one.

System 1 here: a reward that makes 0.9 mean 90% on TypeSafe's data.
System 2 here: your labels, checking it still does on yours.

4.6 What the Claims Mean

"193.6× faster, 444.6× cheaper." These are TypeSafe's home-page numbers from its own evals [TYPE26a], and the launch post itself adds "we expect that these are on the higher end of real world gains" [ALME26]. Fair play to them for writing that. Treat the numbers as the vendor's best case.

"Cannot hallucinate". TypeSafe's own FAQ says what it means: Jev "guarantees the shape of its answers, not that every decision is correct" [TYPE26a]. So "no hallucination" = the answer is always one of your options, Step 4's denominator, true by construction. It tells you nothing about accuracy: a model that always answers "blue" also never gives an answer outside your list.

Type-safe and wrong. Remember the thought we held in Step 4? Sun, Xu, Shi and Yang [SUN26] tested it: does the option's name sway the model, or only its definition? They took 1,200 binary questions and swapped which definition each option name was bound to. A model that reads the definitions should flip its word and keep its meaning. On Jev (their §4.5):

• with yes/no names, 32.5% of decisions changed meaning; with neutral 0/1 names, 2.08%;
• every answer was still one of the options. Type-safe, and wrong.

The toy below shows one way such a flip can happen: slide how much the name pulls against the definition.

showflipsaccuracyhow much the name pulls

0.4

names ignorednames winReset

Their advice is cheap to follow: neutral option names, with the meaning in the descriptions ([SUN26] §6). They report it as a mitigation but do not evaluate it (§7); the 0/1 row above is the closest evidence on Jev. For our running example, "blue (incl. navy)" should say what it means in the description too, not only in the name.

System 1 here: one read of the state, every question answered off it, probabilities only over your options.
System 2 here: whatever your threshold calls in. Jev has no check of its own, and a well-named option matters more than it should.

Jev reads words only. For pictures we need someone else's model.

5. Visual Jev and Other Models That Read Pictures

TypeSafe doesn't do pictures yet: Jev 1.13 takes no image input [TYPE26g]. The paper worth reading carefully is Visual Jev [YU26], because it measures where the time goes.

Picture a building inspector with a checklist of twelve items: you look at the room once, then tick the boxes against what you saw. That is the whole paper; the rest is how to do it inside a vision-language model (VLM) without the ticks leaking into each other.

NOTE: Visual Jev is by Guanxu Yu and Yuhang Yao. The paper never names TypeSafe; the "Jev" link is the repository's request shape (state, a map of typed questions) [YU26b]. Nothing is distilled from Jev.

5.1 Look Once: Shared Visual Context

Visual Jev splits every request in two:

• Prefix = the system line + the image + any public text about it. It goes through the vision encoder and the language model once, and its key–value cache (Query, Key, Value — Dot-Product Attention from Scratch) is kept.
• Suffix = one question + its options + a fixed Answer: at the end. The N suffixes run as rows of one batch on top of copies of that cache.

Each row sees the image and its own question, never another question or its answer. The paper makes this isolation a requirement (its §3), and separate batch rows give it for free. The backbone is Qwen3-VL-4B-Instruct [BAI25].

A token count (my sketch, not the paper's): N questions alone read N(P + S) tokens and the image N times; a shared prefix reads P + NS tokens and the image once. With a prefix P ≈ 230, a suffix S ≈ 45 and N = 32: 8,800 vs 1,670 tokens, about 5.3× fewer. It is a count, not a time; §5.3 shows the clock does not agree with it.

5.2 Reading the Letter at "Answer:"

The options are written into the suffix as A. black, B. white, … and the model is told to reply with one letter. Then it is not allowed to reply. The hidden state $h_i$ at the colon of Answer: goes through the usual output layer that turns a hidden state into a score per token (the LM head), and only the logits of the $K_i$ option letters are kept:

This is the paper's Eq. 1 and §4.4's Step 4 again. The sum runs over the options offered for this question only, so the probabilities add up to 1 even when every option is wrong. If "none of these" can be the truth, it has to be an option.

• What is trained is small: the vision tower is frozen, the language tower moves through LoRA [HU21], 33.0 M trainable parameters [YU26a] against the backbone's 4.44 B [BAI25], about 0.7%.
• There is no new head: ordinary next-token cross-entropy at the readout position, with the right letter as target.
• Training lifts macro accuracy (the plain average over the four test sets) from 0.706 to 0.761 (the paper's Table 2).
• In the model card's words, "the entire gain sits on the two task families represented in training" (GQA [HUDS19], SNLI-VE [XIE19]) [YU26a].
• On the two it never saw, TextVQA moves by 0.001 and TallyQA by 0.005, against a seed spread of 0.014.
• So the fine-tune teaches the format and the trained tasks. It doesn't teach better eyesight.

Now the sobering part. Replace the image with a flat grey field and ask the same questions with the same options (the paper's Table 1):

questions fromTextVQAGQASNLI-VETallyQATextVQA, first buildcountinga leaky buildReset

On GQA the blind model scores 58.5% against 36.5% by chance, almost half of the way to seeing (84.1%). That is section 1's substitution, live: asked "what is in the picture?", the model answers an easier question, "which option sounds right for this question?". Any test built from options needs this blind row.

5.3 Where the Speed Comes From

Table 13 of the paper times eight ways of running the same questions about one image, on one RTX 5090, as milliseconds per question (total time divided by the number of questions):

questions about one image◀6 / 6 · 32▶ Nextcompare withbatched pathsone at a timewriting the answertimeper questionwhole groupone question32 questions, whole groupdoes writing cost much?Reset

At N = 32 questions:

• Batching the questions, image still re-read for each: ~2.6× faster than asking each alone.
• Reusing the image's prefix on top of batching: another ~3.4×. Together ~9×.
• At N = 1 sharing is slower (82.9 vs 48.1 ms), because building and copying the cache has no other question to pay for it. It beats asking each alone from N = 2, and plain batching from N = 4.

Here's the surprise. Generating one letter per question costs about the same as reading the letter off the logits. In the paper's own heading: "Skipping token generation is not the main saving." Fewer passes doesn't mean faster: reusing the image and batching the questions is where the time goes.

5.4 What the Paper Does Not Claim

• Calibration. The returned number is a raw softmax, and the paper says so: it "is not a calibrated probability that the answer is correct". The top probability is a signal to test on your own labels.
• Scope: one model family, English only, ≤ 16 options, frozen vision tower, and no transfer to the unseen sets (Limitations).
• The released adapter was trained on 2 or 4 options per question; its card calls wider option sets "untested coverage" [YU26a].

5.5 Open Jev-Style Models for Images

The most downloaded "Jev with images" on Hugging Face (on 10 Oct 2026) is not from TypeSafe or from the paper.

Table. 4 Jev-style models that take images, from their model cards and papers as of 10 Oct 2026. AutoTrust's JEV-9B card: the models are "not affiliated with, endorsed by, or a product of TypeSafe AI" [AUTO26a].

• AutoTrust's students learnt Jev 1.13's text answers; the JEV-27B-VL card says calibration on images "has not been measured systematically" [AUTO26].
• PixelJev adds a useful warning: on one texture set its adapter improves accuracy while calibration gets worse (their §5.5). Accuracy is not calibration.

System 1 here: one look at the picture, then every question against it at once.
System 2 here: your labels. The probability is uncalibrated by the paper's own account, so only your own checks can set the bar.

6. Conclusion

Fast for the many easy calls, slow for the few hard ones, and your own labels decide which is which. This part is my advice, not a paper's.

Reach for a System One model when

• the answer is one of a closed set you can write down, and your code branches on it (§3.2);
• the same kind of question comes back again and again, so you can collect outcomes: Kahneman & Klein's regular world with feedback (§1.6, §2.4);
• many questions share one state. Ask them in one call (§4.2), and for pictures read the image once (§5.3).

Escalate when

• the answer must be worked out rather than read off the input: maths, code, logic, several hops (§3.5);
• the top probability falls below a threshold you checked on your own labels, not one copied from the docs (§3.4);
• the input is unlike what you checked on. Shifted data looks the same from the outside (§2.4);
• "none of these" could be the truth and is not among the options (§5.2).

Deploy a cascade, and pick the threshold honestly

1. Label a sample of your own data, per kind of question; plan for hundreds of items, not dozens (§1.4).
2. Check that the fast model's top probability drops on its errors. A model that sits at 1.00 on its mistakes gives the gate nothing to catch (§2.4).
3. Measure the slow model on the items you would escalate, not on everything: those are the hard ones (§2.2).
4. Choose τ on one half of the items and report the other half.
5. Pin the model version (§3.5) and use neutral option names with the meaning in the descriptions (§4.6).

The limits

• Nobody outside TypeSafe knows Jev's size, architecture or training objective (§4.3). "Cannot hallucinate" means "always one of your options", not "right" (§4.6).
• Jev reads no pixels. The vision models here are other people's, and Visual Jev's probabilities are uncalibrated by its own account (§5.4).

The bat and the ball had a fast answer, a cheap check, and a check nobody ran. A cascade is that check, written into code, with the threshold as the moment System 2 wakes up. Just don't let the threshold be an anchor ;)

7. References

1. [KAHN11] Kahneman. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011, https://openlibrary.org/books/OL26208390M/Thinking_fast_and_slow.
2. [STAN00] Stanovich, West. Individual differences in reasoning: Implications for the rationality debate?. Behavioral and Brain Sciences 23(5):645–665, 2000, https://doi.org/10.1017/S0140525X00003435.
3. [EVAN13] Evans, Stanovich. Dual-Process Theories of Higher Cognition: Advancing the Debate. Perspectives on Psychological Science 8(3):223–241, 2013, https://doi.org/10.1177/1745691612460685.
4. [KAHN11a] Kahneman. Of 2 Minds: How Fast and Slow Thinking Shape Perception and Choice. Scientific American, 15 June 2012 (excerpt of ch. 1 of Thinking, Fast and Slow), https://www.scientificamerican.com/article/kahneman-excerpt-thinking-fast-and-slow/.
5. [FRED05] Frederick. Cognitive Reflection and Decision Making. Journal of Economic Perspectives 19(4):25–42, 2005, https://doi.org/10.1257/089533005775196732.
6. [TVER83] Tversky, Kahneman. Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review 90(4):293–315, 1983, https://doi.org/10.1037/0033-295X.90.4.293.
7. [KAHN09] Kahneman, Klein. Conditions for intuitive expertise: A failure to disagree. American Psychologist 64(6):515–526, 2009, https://doi.org/10.1037/a0016755.
8. [SCHI17] Schimmack, Heene, Kesavan. Reconstruction of a Train Wreck: How Priming Research Went off the Rails. Replicability-Index blog, 2 February 2017 (Kahneman's reply in the comments, 14 February 2017), https://replicationindex.com/2017/02/02/reconstruction-of-a-train-wreck-how-priming-research-went-of-the-rails/.
9. [HAGG16] Hagger, Chatzisarantis, Alberts, Anggono et al. A Multilab Preregistered Replication of the Ego-Depletion Effect. Perspectives on Psychological Science 11(4):546–573, 2016, https://doi.org/10.1177/1745691616652873.
10. [MELN18] Melnikoff, Bargh. The Mythical Number Two. Trends in Cognitive Sciences 22(4):280–293, 2018, https://doi.org/10.1016/j.tics.2018.02.001.
11. [BENG19] Bengio. From System 1 Deep Learning to System 2 Deep Learning. NeurIPS 2019 Posner Lecture (video: https://slideslive.com/38922304), https://neurips.cc/virtual/2019/invited-talk/15488.
12. [WEI22] Wei et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv 2201.11903.
13. [KOJI22] Kojima et al. Large Language Models are Zero-Shot Reasoners. arXiv 2205.11916.
14. [DEEP25] DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv 2501.12948.
15. [CHEN24a] Chen et al. Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs. arXiv 2412.21187.
16. [YU26] Yu, Yao. Visual Jev: Accurate and Efficient Decisions from Shared Visual Context. arXiv 2609.25845.
17. [TURP23] Turpin, Michael, Perez, Bowman. Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. arXiv 2305.04388.
18. [ONG24] Ong et al. RouteLLM: Learning to Route LLMs with Preference Data. arXiv 2406.18665.
19. [CHEN23] Chen, Zaharia, Zou. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. arXiv 2305.05176.
20. [CHOW70] Chow. On Optimum Recognition Error and Reject Tradeoff. IEEE Transactions on Information Theory 16(1):41–46, 1970, https://doi.org/10.1109/TIT.1970.1054406.
21. [VIOL01] Viola, Jones. Rapid Object Detection using a Boosted Cascade of Simple Features. CVPR 2001, https://doi.org/10.1109/CVPR.2001.990517.
22. [NAEI15] Naeini, Cooper, Hauskrecht. Obtaining Well Calibrated Probabilities Using Bayesian Binning. AAAI 2015, https://doi.org/10.1609/aaai.v29i1.9602.
23. [GUO17] Guo, Pleiss, Sun, Weinberger. On Calibration of Modern Neural Networks. arXiv 1706.04599.
24. [GUPT24] Gupta et al. Language Model Cascades: Token-level uncertainty and beyond. arXiv 2404.10136.
25. [TYPE26g] TypeSafe AI. Models. TypeSafe documentation, read 10 Oct 2026, https://docs.typesafe.ai/models.
26. [ALME26] D. Almeida. Introducing System One Models & Jev. TypeSafe AI blog, launch post (the page now shows Oct 12, 2026, a re-publish date; Jev released 15 Sept 2026), read 10 Oct 2026, https://typesafe.ai/blog/introducing-system-one-models-and-jev.
27. [OUYA22] Ouyang et al. Training language models to follow instructions with human feedback. arXiv 2203.02155.
28. [TYPE26i] TypeSafe AI. System One models. TypeSafe documentation, read 10 Oct 2026, https://docs.typesafe.ai/concepts/system-one.
29. [TYPE26h] TypeSafe AI. API reference. TypeSafe documentation (OpenAPI spec at api.typesafe.ai/openapi.json), read 10 Oct 2026, https://docs.typesafe.ai/api.
30. [TYPE26e] TypeSafe AI. Primitives. TypeSafe documentation (with /primitives/choice, /score, /noul), read 10 Oct 2026, https://docs.typesafe.ai/primitives.
31. [TYPE26f] TypeSafe AI. Confidence. TypeSafe documentation, read 10 Oct 2026, https://docs.typesafe.ai/confidence.
32. [TYPE26j] TypeSafe AI. Python SDK. TypeSafe documentation, read 10 Oct 2026, https://docs.typesafe.ai/sdk/python.
33. [TYPE26d] TypeSafe AI. Jev 1.13 jaggedness. TypeSafe documentation, reviewed 2 Oct 2026, https://docs.typesafe.ai/model-jaggedness/jev-1.13.
34. [TYPE26a] TypeSafe AI. TypeSafe: System One models. TypeSafe AI home page and FAQ, read 10 Oct 2026, https://typesafe.ai/.
35. [TYPE26l] TypeSafe AI. AI primer. TypeSafe documentation, read 10 Oct 2026, https://docs.typesafe.ai/introduction/machine-learning-primer.
36. [TYPE26c] TypeSafe AI. Parallel questions. TypeSafe AI documentation, cookbook run on jev-1.12 (priced as of Sep 2026), read 10 Oct 2026, https://docs.typesafe.ai/cookbooks/parallel_questions.
37. [SUN26] Sun, Xu, Shi, Yang. Type-Safe Is Not Error-Free: Typed Decision Models Follow the Option Name, Not the Definition Bound to It. arXiv 2609.26758.
38. [LI26b] Li, Wang, Gong, Lang, Yu. Replacing Large Language Models with Jev Decision Models for Low-Latency Edge Service Orchestration. arXiv 2609.22753.
39. [CHEN26] Cheng, Dai, Sun. this-that-model-1.0: A typed decision model that decides in 30 ms, for a millionth of a cent. arXiv 2609.23886.
40. [YU26b] Yu, Yao. Visual Jev: code and scored outputs. GitHub repository (README, REPRODUCE.md, data/, code/ at a392eae), read 10 Oct 2026, https://github.com/guanxuyu-sv/Visual-Jev.
41. [BAI25] Bai et al. Qwen3-VL Technical Report. arXiv 2511.21631.
42. [HU21] Hu et al. LoRA: Low-Rank Adaptation of Large Language Models. arXiv 2106.09685.
43. [YU26a] Yu, Yao. Visual Jev 4B — answer-supervised adapter. Hugging Face model card, read 10 Oct 2026, https://huggingface.co/guanxuyu/visual-jev-4b-answer-sft.
44. [HUDS19] Hudson, Manning. GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering. arXiv 1902.09506.
45. [XIE19] Xie, Lai, Doran, Kadav. Visual Entailment: A Novel Task for Fine-Grained Image Understanding. arXiv 1901.06706.
46. [ZHOU26] Zhou, Yang, Zhao. From Text Decisions to Pixels: An Study of Jev-Style Visual Choice Model. arXiv 2609.29283.
47. [AUTO26] AutoTrust AI. JEV-27B-VL. Hugging Face model card, read 10 Oct 2026, https://huggingface.co/autotrust/JEV-27B-VL.
48. [AUTO26a] AutoTrust AI. JEV-9B. Hugging Face model card, read 10 Oct 2026, https://huggingface.co/autotrust/JEV-9B.
49. [GAO26] Gao. RSI-Jev v6.1-VL-4B. Hugging Face model card, read 10 Oct 2026, https://huggingface.co/shgao/rsi-jev-v6.1-vl-4b.
50. [CHRI17] Christiano, Leike, Brown, Martic, Legg, Amodei. Deep Reinforcement Learning from Human Preferences. arXiv 1706.03741.
51. [LAMB24] Lambert et al. Tülu 3: Pushing Frontiers in Open Language Model Post-Training. arXiv 2411.15124.
52. [SHAO24] Shao et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv 2402.03300.
53. [WILL92] Williams. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning 8(3–4):229–256, 1992, https://doi.org/10.1007/BF00992696.
54. [GNEI07] Gneiting, Raftery. Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association 102(477):359–378, 2007, https://doi.org/10.1198/016214506000001437.
55. [SUTT18] Sutton, Barto. Reinforcement Learning: An Introduction. MIT Press, 2nd edition, 2018 (free PDF from the authors), http://incompleteideas.net/book/the-book-2nd.html.
56. [DAMA25] Damani, Puri, Slocum, Shenfeld, Choshen, Kim, Andreas. Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty. arXiv 2507.16806.
57. [LENG24] Leng, Huang, Zhu, Huang. Taming Overconfidence in LLMs: Reward Calibration in RLHF. arXiv 2410.09724.
58. [XU24] Xu et al. SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales. arXiv 2405.20974.
59. <a class="cite" data-x="ref:BRIE50" href="https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2">[BRIE50] Brier. Verification of Forecasts Expressed in Terms of Probability. Monthly Weather Review 78(1):1–3, 1950, https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.
60. [IBRA26] Ibrahim, Zaki. Evaluating Decision Models for Text Annotation in Computational Social Science. arXiv 2609.24574.