Woolr

Introduction

In the last post I asked Gemma 4 12B about thousands of football transfers. It named the right club for only 14% of famous players' moves, and it almost never said it didn't know. It even named a club for 994 of 1,000 players I made up, so nothing in its answers shows when it's guessing. I'll call a wrong answer that the model gives as if it knew a hallucination.

In this post I look at the model's activations instead, which are the numbers it computes at each layer as it reads a question. I trained probes on the activations to predict whether the model knows the answer, and compared them with the usual ways of measuring a model's confidence. A probe here is a logistic regression, which is a simple linear classifier.

Then I looked at what the probes are reading. The obvious candidate is whether the model recognises the player, and football transfers let me test the idea directly. A famous player can have a well-known move from 2015 and another from 2025, after the model's training cutoff, so the model knows the player well but can't know the second move.

By moves after the cutoff, I mean real moves from Wikidata made after March 2025. In the last post I found that the model's knowledge of moves stops around October 2024, so it can't have read about these moves in training. It still named the right club for 14 of the 435, probably from transfer rumours or easy guesses, so I grade these moves like any others, and only use the ones it got wrong.

Here's the short version of what I found.

• A probe on the activations separates answers the model knows from hallucinations with an AUROC of 0.85 to 0.90, on players the probe never saw in training. The model's own token probabilities do just as well, and adding the probe to them gains about 0.01.
• The probe works before the model writes its answer. Read at the end of the question, it already gets 0.87 on the questions about player facts.
• When I asked the model how confident it was, its answers did about as well as chance (0.54), because it says 100 for almost everything.
• A probe for whether the model recognises the player is fooled by famous players, and calls most of their moves after the cutoff "known". A probe trained on a better comparison finds a separate signal in later layers for whether the model has the fact.

I measure how good each signal is with AUROC, which comes up a lot in this post. Each signal gives every answer a score, and a good signal gives the answers the model knows higher scores than its hallucinations. AUROC is the chance that, if you pick one answer the model knows and one hallucination at random, the signal gives the known answer the higher score. So 0.5 is no better than chance, and 1.0 means the signal ranks every known answer above every hallucination.

The results come first, and the methodology section after them explains how I built the labels and the probes.

Labels for what the model knows

Before I could train any probes, I needed labels that say which answers the model knows. I asked every question once with greedy decoding, where the model always picks its most likely next token, and five more times with random sampling at temperature 0.7. Claude Haiku 5.5 then graded each answer against Wikidata, because the model often words a right answer differently from Wikidata, e.g. "Defender" for a full-back (more on the grading in the last post). I label an answer "knows" when the greedy answer and at least 4 of the 5 samples are right, and "doesn't know" when none of the 5 samples are right. I label everything in between a "best guess".

Apart from the player facts, nearly all the answers are "doesn't know", so there are plenty of hallucinations to detect. The downside is that the questions about transfers have only 59 "knows" answers among the test players, so the results for them are noisy. The player facts (nationality, position and club in 2023) have 623, so their results are more precise. In the last post I checked that the wrong answers about transfers come from gaps in what the model knows, and not from how I asked the questions or graded the answers.

Comparing the probe against other confidence signals

The probe does as well as the token probabilities

For every question about a real player, I trained a probe to predict "knows" against "doesn't know" from the activations at one token and one layer. I read the activations at the four tokens marked in Figure 2. I trained each probe on 70% of the players and tested it on the other 30%, so no player appears in both groups.

The activations hold the signal, since the probes get 0.85 to 0.90 once the model has written its answer. However, the model's own token probabilities do just as well. The simplest of them is the answer's mean log-prob, which is the average log-probability the model gave to each token of its answer, and it gets 0.89 on both the moves and the player facts. The answer's token entropy, which measures how spread out the model's probabilities over the next token were, gets 0.88 and 0.89. Combining the probe at the last answer token with the log-prob gets 0.89 and 0.90, which is at most 0.01 better than the log-prob alone, and within noise. Detection Without Correction found the same result on smaller models, so my result repeats their finding on a bigger model.

The probe and the log-prob also mostly agree with each other. On the held-out player facts their scores have a Spearman correlation of 0.76, where 1 would mean they put the questions in exactly the same order. Both signals miss some confident hallucinations. For example, 141 of the 326 answers the model doesn't know have an average token probability above 0.9, and the probe gives 65 of the 141 a score below 0.5, so it catches about half of them.

The probe works before the model answers

The probe's main advantage is that it doesn't need the answer. The probe at the end of the question gets 0.87 on the player facts before the model has written anything, while every signal based on the output has to wait for the answer.

A more practical way to read the results is to imagine that the model answers only the questions it's most confident about. On the player facts, if it answers the 40% of questions that the probe at the end of the question rates highest, 90% of its answers are right, compared with 64% when it answers everything. The log-prob and the probe at the last answer token do a little better (93% and 94%), but as I said, they need the answer first. Stated confidence gets only 68%, barely above the 64% from answering everything.

In practice, this gives you an out. If you can probe and see that the model is not sure about what it's about to say, you can build mechanisms to deal with that. Maybe you only run costly look-ups in scenarios where the probe's score is below some threshold.

The model's stated confidence is close to chance

Asking the model how confident it is gives almost no information, because it says 100 for nearly everything. When I asked it for a number from 0 to 100, it said exactly 100 for 95% of its right answers and for 88% of its wrong answers, and 0 for almost all the rest. As a result, its stated confidence gets an AUROC of only 0.53 on the moves and 0.54 on the player facts. This ain't exactly news, but I think still worth checking. Open models don't seem to be getting better in this regard.

P(True) does better. For P(True), I show the model its own answer, ask whether the answer is true, and take the probability it gives to "True" (Kadavath et al., 2022). It gets 0.67 on the moves and 0.80 on the player facts, which is below the probe, especially on the moves. The table below shows some questions about moves by players the probe never saw. The probe score is the probe's estimate of the chance that the model knows the move, made before the model answers.

The probe gets the first three rows right, including two confident hallucinations that P(True) rates as certainly true. It gets the last row wrong, because the Winks answer is right even though the probe gave it 0.

What the probe is reading

The probe works, so the next question is what information it reads from the activations. To find out, I compared it with two other probes, each trained to separate a different pair of groups.

Recognition probe

Real playersagainstMade-up players

Does the model know this player?

Main probe

Answers it knowsagainstAnswers it doesn't know

Does the model know this answer? All real players.

Fact probe

Moves it knows, before the cutoffagainstMoves it misses, after the cutoff

Is this fact missing? Only players the model knows well.

Here's what the comparison shows.

• The recognition probe tracks whether the model knows the player, so it calls most wrong answers about famous players "known".
• The main probe reads more than recognition. It scores famous players' wrong answers close to the other wrong answers.
• The fact probe, trained only on players the model knows well, finds a separate signal for whether the model has the fact. The signal appears in later layers than recognition, and it isn't explained by the player's current club.

Famous players fool the recognition probe

Language models can tell whether they know a person or thing. Ferrando et al. (2025) found that a linear probe can read this from the activations, which raises the question of whether the main probe does much more. Of the 200 moves the model knows, 179 are by famous players, compared with about half of the moves it doesn't know, so a probe could score well just by spotting famous players. If that's all it reads, it will call a famous player's move "known" even when the model hasn't seen the move. Moves by famous players after the cutoff test this, because the model knows the player well but can't know the move.

To test it, I trained a recognition probe on real players against made-up players, the same comparison that Ferrando et al. made. It separates the two almost perfectly, with an AUROC of 0.998 to 0.999 at all four tokens, and 0.998 on the question wording I left out of training. It isn't reading what the names look like, because a probe that sees only the length of the name and its number of tokens gets 0.55. It also gives obscure real players scores almost as low as made-up ones (Figure 9), so it measures whether the model knows the player, rather than whether the player exists.

Then I applied the recognition probe to the moves after the cutoff that the model got wrong, for players it's familiar with. It scored 58% of them above 0.5 at the player's name, and 68% at the end of the question, so it called most of the model's wrong answers "known". A hallucination detector built on recognition would therefore miss most wrong answers about players the model knows well, which is the case where the model sounds most sure of itself.

The clearest test compares two moves by the same player. For 26 test players, the model got a move before the cutoff right and a move after the cutoff wrong. The recognition probe's scores for the two moves differ by only 0.05 at the name and 0.03 at the end of the question, so it can barely tell them apart. The gap grows to 0.25 at the last answer token, after the model has written its wrong answer.

The main probe depends much less on how familiar the player is. Its scores are low overall, because few of the moves it was trained on are "knows", so it helps to compare groups. Across the four tokens, it gives familiar players' wrong answers after the cutoff a mean of 0.06 to 0.12. That's closer to its scores for all the answers the model doesn't know (0.03 to 0.06) than to its scores for moves the model knows (0.25 to 0.32), so the main probe reads something beyond recognition.

A probe for the missing fact

To separate the extra signal from recognition, I trained the fact probe only on famous players the model knows well. It compares 170 moves before the cutoff that the model knows with 321 moves after the cutoff that it misses. Every player in both groups is famous, and the model knows all of them well, so recognition can't separate the groups, and only the fact can.

The fact probe does best at the end of the question, where it gets 0.87 at layer 45. At layer 0, which holds only the embeddings of the tokens, it gets 0.50 to 0.51 at the name and the end of the question, so the signal comes from the model's processing and not from the words in the question. The fact probe also gives made-up players low scores (0.01 to 0.05), even though it never saw them in training.

Of course, it's always possible that the fact probe could have learned a shortcut. In a question about a move after the cutoff, the club in the question is often the player's current club as far as the model knows, while in a question about an older move it's usually an older club. So the probe might have learned to spot questions about leaving the current club, rather than missing facts. To check, I asked the model "Which club does X currently play for?" for all 559 familiar players, and had Claude judge whether its answer was the club in the question.

The shortcut explains little. The club in the question was the model's current club for only 36% of the wrong answers after the cutoff, and whether the club is current separates the two groups with an AUROC of only 0.67, well below the probe's 0.87. For a stricter test, I looked only at moves before the cutoff where the club isn't current, so that only the fact varies. The probe still separates moves the model knows from moves it misses, with 0.83 at the end of the question and 0.85 at the first answer token. Wrong answers before the cutoff score about the same as wrong answers after it (0.22 against 0.15 to 0.20), and far below moves the model knows (0.70). So the fact probe reads whether the model has the fact, and not whether the question is about the player's current club.

Recognition appears in earlier layers than the fact

I trained each of the three probes at every one of the model's 49 layers (the embeddings plus 48 transformer layers), to see where in the model each kind of information can be read. The model recognises a player in the middle layers, and the signal for whether it has a particular fact appears in later layers.

At the end of the question, the recognition probe gets about 0.84 up to layer 14, then jumps to 0.95 at layer 15 and passes 0.99 by layer 19. The fact probe climbs more slowly, passes 0.8 only at layer 27, and does best at layer 45 of 48.

Conclusion

Gemma 4 12B's activations carry a usable signal for whether an answer is a fact the model knows or a hallucination. Once the model has written its answer, though, the signal is no better than the model's own token probabilities, which cost nothing extra to read.

My main takeaway from the above exploration is that the probe is useful before the model answers. That's a practically useful result as discussed. At the end of the question it's nearly as good as the log-prob is after the answer, with 0.87 against 0.89 on the player facts. So you could use it to stop the model, or to have it look the answer up, before it commits to an answer. A recognition probe isn't enough for the job, because famous players fool it. A probe that compares known and missing facts for players the model is familiar with finds a separate signal in later layers, which tracks whether the model has the fact, and the signal still holds when I control for the most obvious other explanation.

Methodology

Model

I ran Gemma 4 12B-IT in bf16 (16-bit numbers) on a MacBook's GPU, using PyTorch. It has 48 transformer layers, and its activations at each layer are 3,840 numbers long.

Questions

The questions, the made-up players and the grading are the same as in the last post, which explains how I built them. No question mentions a date, so a move the model knows and a move after the cutoff look the same, apart from whether the model has the fact. The question set leaves out moves between July 2024 and March 2025, so that every move is well before or well after where the model's knowledge stops, around October 2024. I left one of the four question wordings out of probe training, to check that the probes weren't learning the wording. I count a player as familiar if the model gets at least 2 of the 3 player facts right.

Activations

For each question, I ran the prompt and the model's greedy answer through the model once, and saved the activations after every layer at the four tokens. Adding the answer doesn't change the activations at the earlier tokens, because each token only attends to the tokens before it, and I checked that they match (a cosine similarity of at least 0.9997). The same run gives the answer's mean log-prob and its mean token entropy.

Probes and evaluation

Each probe is a logistic regression on standardised activations (scikit-learn, C = 0.1). I split the players 70/30 once, so all the questions about a player are on the same side. I chose each probe's layer by 5-fold cross-validation on the training players, grouped by player, then refit the probe on all the training players and scored it once on the test players. The intervals are 95% bootstrap intervals, which resample players rather than questions, because questions about the same player aren't independent. I trained the probes on "knows" against "doesn't know", and left the best guesses out of training. I report made-up players separately and never pool them with real players.

Limits and next steps

• I tested one model, in one domain, with questions that each ask for one fact and get a short answer.
• The questions about moves have only 59 "knows" answers among the test players, so their intervals are wide.
• On the moves, the probe doesn't pick up partial knowledge. Moves the model gets right in some of its five samples score as low as moves it never gets right, even though they're right about as often as the in-between answers on the player facts, where the probe's score does rise with the number of samples right.
• I haven't yet run semantic entropy (Farquhar et al., 2024), which samples ten new answers and measures how much their meanings disagree. It's the strongest baseline that uses sampling.

This post and the last one only read the model. In the next post I try to change what it says, by pushing on neurons and directions for football inside the model, to make it talk about football and to make a particular club the answer to transfer questions.

References

• Ferrando, Obeso, Rajamanoharan and Nanda (2025). Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models.
• Roy et al. (2026). Detection Without Correction.
• Kadavath et al. (2022). Language Models (Mostly) Know What They Know.
• Farquhar, Kossen, Kuhn and Gal (2024). Detecting hallucinations in large language models using semantic entropy.