Disclaimer: This post is for educational and purely research purposes. I acknowledge that some of the results may be used to construct adversarial attacks on System One models. However, I believe that the builders of these models are already aware of these attacks and a patch or defense would appear sooner than any meaningful attack may occur. If that is not the case, then this post generates evidence towards prioritising that area of defense research building more inclusive guardrails.

In this post, I study non-autoregressive decision models or so-called System One models which are specially designed architectures for cheap, calibrated decision-making, intended as control modules embedded inside larger agentic systems. This is an exploratory study conducted over the weekend applying all forms of interpretability tools to better understand Sys1 model behaviour and to suggest design choices for more efficient or safer deployment practices.

Please reach out to me at gs454236 [AT] gmail [DOT] [COM] for constructive criticisms, questions, or thoughts.

What are System One Models and Why Should You Care?

System One models, named so after Daniel Kahnemann’s popularization of the phrase as a subconscious thinking module within the brain as opposed to the more verbalized System two component, are a class of non-autoregressive decision-making language models that output only a calibrated probability distribution over a predefined set of choices given a text input formatted as a predefined schema. The appeal of such models stem from their ability to give sub-second decision making them an order of magnitude faster than conventional language models as well as cheaper (since you can bypass the whole test-time inference cost). These models came into the spotlight with TypeSafe AI’s announcement of Jev-1.13 , however, the claims are disputed as an Indian engineer claims to have built the architecture an year earlier already.

With recent studies on using Jev for LLM-as-judge, an application area notoriously difficult for language models due to uncalibrated decision across batches and prohibitive latency to be useful for current long-running agentic tasks, these models still remain underexplored as a new model architecture.

To the best of my knowledge, this is the first attempt to apply mechanistic interpretability tools developed by researchers across the years and try to pry open the brains of Sys1 models in order to understand them better and in the process, to make them better and safer too.

Setup

In order to facilitate whitebox interpretability, we¹ selected Jev-9B, an open student model of Jev-1.13. It is a frozen Qwen3.5-9B model with a LoRA adapter trained on Jev’s probability outputs. Using this architecture allowed us to compare base Qwen with Jev student with just disabling the adapter. we also used Tev1-4B, Tev1-0.8B, both open models from Together AI. We prioritized open models of varying sizes to have a variant study as well as I wanted to see applicability of interpretability tools on these models and identify blind spots. Some models to explore are CLM-8B, Clef, as well as Jev-1.13 itself.

All the experiments were written and run by Claude Code. I supplied the research questions, provided human oversight as well as guided key design decisions. This blog post is written by me, entirely. The diagrams are generated by Claude. The full experiment took over 12 hours on my Mac M4 Max. I did not rent any cloud GPU.

Methodology

As noted above, we mainly focused on open student models of 0.8B, 4B, and 9B sizes as well as their Qwen base models to serve as control. We constructed four datasets, some from Jev’s training corpus and held-out families as well as some constructed scenarios across various epistemic modalities (ambiguous, missing evidence, etc.), an injection dataset constituting prompt-based attacks as well as pairs with one number change in the prompt state.

We employed pretty much all the tools out there under the sun as the goal was also to see what worked and what didn’t on these seemingly new class of model organisms. I trained linear probes, decision lens, used activation patching, SAEs, crosscoders, and MAttr to quantify model internals, beliefs versus the model output. I used the model’s output confidence as baseline whereever applicable and good ‘ol bag-of-words classifier on raw text.

In particular, I was interested in the following questions (findings and discussions follow in their respective sections):

1. At which layer does the decision form? Can we use a logit lens approach to find formative layers?
2. Does the distillation impart new knowledge to student models? What does distillation change exactly?
3. Is the model internally consistent with its output when it comes to reporting errors?
4. What are the ASR for common prompt injection techniques on Sys1 models?
5. Do the model modalities share a common intensity direction? Can it be steered?
6. Are System One models really easier to interpret?
7. Can we exploit model internals to build efficient guardrails?
8. Does the model force a correct answer when there is none?
9. Are System One Models option shuffle invariant?
10. How small can a System One adapter be?

When does decision form?

We applied a decision lens (similar to a logit lens) to all 32 layers of the Qwen base model as well as activation patching to find the most causative layer when the decision is flipped.

We found that across the three model decision primitives (choice, score, noul) Jev-9B consistently formed the final answer post layer 24. Across 40 pairs, we found all big models fix the decision gradually over the latter layers – calculated as the median fraction of answer change by patching each layer.

Does distillation impart new knowledge?

As we focused on distilled models on open base models, I was curious to understand if distilling added new knowledge to the base model or if it was merely learning a transformed output format. Current evidence points to the latter as we trained linear probes across all layers.

We also performed linear CKA across layers to study how the hidden states differ. We used Matryoskha Attribution or MAttr to find adapter components that restore most of the model behaviour. MAttr learns weight significance by changing weights from base to trained under a random budget.

For Jev-9B, the adapter changes about 3% of the weight norm but rotates the final layer complete (0.23). Tev class models differ in the middle but the final layers are almost identical to initial layers.

Why care anyway? I think the biggest takeaway is that adapters can be much smaller (1.6M instead of 40M) making it far easier to train custom adapters on a per-consumer, per-usecase basis. In particular, engineers should prioritize the middle layers. Results on model reformating existing knowledge indicates using a better base model might yield a stronger model.

Can a probe predict mistakes better than the model’s output?

I was curious to understand if a probe on the hidden state can predict the model’s error better than the model’s output probabilities. It turns out that for bigger models, the probabilities are a good proxy and the probes fall short.

How robust are distilled System one models to prompt injection attacks?

Does distillation make the issue worse?

Surprisingly, ablating the hijacking features made the ASR slightly worse than baseline. This is most likely because the SAE has selected for the particular scenario at the decision token and not the underlying mechanism.

Do the model modalities share a common intensity direction? Can it be steered?

We noticed that the model has three decision primitives – score (select a number within a range), choice (select an option from a set), and noul (a binary yes/no probability).

Is it possible to steer one of these answers by putting pressure on another? In other words, does the model make decisions from some “shared context”?

Under the distilled model setup, this seems to be true. Distillation creates a partially shared direction which can influence ratings by steering the “yes” direction.

Are System One models easier to interpret?

One of the appeals of System one models are that they finish their compute in one single pass as opposed to an autoregressive model. Does this single pass nature make them easier to interpret?

We tried learning an SAE reconstructing the decision state as well as try to find a subcircuit using MAttr. The results are mixed. The trained SAE is indeed smaller but the decision circuit is more dense than the base model.

Can we exploit model internals to build better guardrails?

Although this sounds like an enticing idea, initial enthusiasm was dissolved when we tested the guardrails on fresh attacks.

Does the model force a correct answer when there is none?

Doing a softmax over the raw logits yield some probability distribution. If we remove the correct answer, does the model shift the mass to another incorrect answer or to “Don’t know”/“Right answer is missing” option?

We trained a small probe on the hidden state as well as noted the raw score. Both agree with each other, meaning the raw max-logit may be interpreted as a fitness score to the output.

Are System One models option-shuffle invariant?

Case 1: Prompt1, choose between Opt1, Opt2, Opt3 Case 2: Prompt1, choose between Opt2, Opt3, Opt1

Does the model output the correct option in both cases? We found that under some margin tolerance, the model flips the answer under the shuffling transformation. Moreover, we also patched the decision token to find if the decision binds to a particular letter. We found that initially the model tracks the option and then post layer 22 binds to the option letter after the decision has formed.

How small can a System One model adapter be?

I ask this question in our specific setup since we are using a student model. I think there is already some indication in terms of parameter count but in terms of rank, we trained a dedicated rank-16 and rank-1 MLP adapter. Here are some interesting results for a rank-1 adapter with some controls and the assocaited learning curves.

Limitations and Future Work

I think the biggest limitation of the above study is that it is only indicative of potential behaviour and does not quantify associated metrics or behaviour. This is due to the intentional small-scale of the experiments conducted on a tight compute budget and more as an intellectual curiosity project.

1. We did not use the NTU, decision-injection-bench, or Check Point datasets. Those were gated, could have made some effort :P
2. The training runs were fractional as compared to baselines which means that numbers are only directionally true and needs to be validated.
3. We only exclusively focused on open weight models for their appeal to be interpreted. This may or may not be a drawback. Other models to explore are the Jev-1.13 model itself, Stanford’s CLM-8B, and Cloudflare’s Clef, kev, any2jev, shgao, among others.
4. The attacks were synthetic and constructed. Any respectable jailbreaking attempt should cover the standard jailbreaking suites.
5. The MAttr code was implemented from paper due to lack of an author reference implementation. This may or may not affect the final results and/or correctness of the results.

Conclusion

System one models are definitely going to be around, not on the surface but as a potential driving backbone for many agentic systems that will increasingly continue to dominate most of our lives in the coming days. I think the need to deeply understand these models cannot be understated.

This post summarizes one of the first attempts to apply tools developed by the mechanistic interpretability community to pry open these new class of models. This is not a conclusive study but should inform researchers and industry practitioners towards some helpful directions to pursue as well as to implement better guardrails and integrate these systems so they are safer and more efficient to use.

To cite this work

Basil. "Understanding System One Models Using Interpretability Tools." (October 2026)
*Basil Labib's Blog*, basil08.github.io, 
http://basil08.github.io/post/understanding-system-one-models-using-interpretability-tools/

References

Please note: This is AI-generated section. Please verify before citing.

A. Models (models, data, tooling)

• MAttr: Arora et al., Matryoshka Attribution, arXiv 2609.25518 (Sept_2026).
• Zoo (HF): autotrust/JEV-9B, autotrust/JEV-27B, togethercomputer/Tev1-{0.8B,4B}-experimental (+ togethercomputer/tev1 repo for the prompt format), Contrastive-LM/CLM-v0.1-8B, jaredpalmer/kev-0.5b, huaweifeng/any2jev-qwen3-0.6b, shgao/rsi-jev-v*-qwen3.5-2b, Cloudflare/clef*
• Corpus: SargeDev/jev-distill-corpus-v3 for the 608K distilled Jev-1.13 outputs
• Backbone: Qwen Team, Qwen3 Technical Report, arXiv 2505.09388
• Gated DeltaNet (the GDN layers at L14/L22): Yang, Kautz, Hatamizadeh, Gated Delta Networks, arXiv 2412.06464

B. Training recipe

• Hinton et al., Distilling the Knowledge in a Neural Network, 1503.02531
• Hu et al., LoRA, 2106.09685
• Shuttleworth et al., LoRA vs Full Fine-tuning: An Illusion of Equivalence, 2410.21228
• Epstein (1969), A Scoring System for Probability Forecasts of Ranked Categories, J. Appl. Meteor.
• Guo et al., On Calibration of Modern Neural Networks, 1706.04599

C. What fine-tuning changes

• Jain et al., Mechanistically Analyzing the Effects of Fine-Tuning on Procedurally Defined Tasks, 2311.12786
• Prakash et al., Fine-Tuning Enhances Existing Mechanisms: Entity Tracking, 2402.14811
• Lee et al., A Mechanistic Understanding of Alignment Algorithms (DPO/toxicity), 2401.01967
• Kornblith et al., Similarity of Neural Network Representations Revisited (CKA), 1905.00414

D. Lenses, probes, patching

• nostalgebraist (2020), interpreting GPT: the logit lens, LessWrong
• Belrose et al., Eliciting Latent Predictions with the Tuned Lens, 2303.08112
• Alain & Bengio, Linear Classifier Probes, 1610.01644
• Hewitt & Liang, Control Tasks for Probes, 1909.03368.
• Belinkov, Probing Classifiers: Promises, Shortcomings, and Advances, 2102.12452
• Meng et al., ROME / causal tracing, 2202.05262
• Zhang & Nanda, Towards Best Practices of Activation Patching, 2309.16042
• Heimersheim & Nanda, How to Use and Interpret Activation Patching, 2404.15255
• Geiger et al., Causal Abstractions of Neural Networks, 2106.02997, plus DAS, 2303.02536
• Schuster et al., CALM, 2207.07061, and Din et al., Jump to Conclusions, 2303.09435

E. Uncertainty

• Kadavath et al., Language Models (Mostly) Know What They Know, 2207.05221
• Burns et al., Discovering Latent Knowledge (CCS), 2212.03827
• Azaria & Mitchell, The Internal State of an LLM Knows When It’s Lying, 2304.13734
• Marks & Tegmark, The Geometry of Truth, 2310.06824

F. Attribution and circuit discovery

• Kramár et al., AtP*, 2403.00745
• Syed et al., Attribution Patching Outperforms ACDC, 2310.10348
• Conmy et al., ACDC, 2304.14997
• Bhaskar et al., Finding Transformer Circuits with Edge Pruning, 2406.16778
• Wang et al., IOI circuit, 2211.00593
• Braun et al., Attribution-based Parameter Decomposition, 2501.14926, and Bushnaq et al., Stochastic Parameter Decomposition, 2506.20790

G. Dictionaries (SAEs and crosscoders)

• Bricken et al. (2023), Towards Monosemanticity, transformer-circuits.pub
• Cunningham et al., Sparse Autoencoders Find Highly Interpretable Features, 2309.08600
• Templeton et al. (2024), Scaling Monosemanticity, transformer-circuits.pub
• Gao et al., Scaling and Evaluating Sparse Autoencoders (TopK, downstream-loss eval), 2406.04093
• Bussmann, Leask, Nanda, BatchTopK SAEs, 2412.06410
• Bussmann et al., Matryoshka SAEs, 2503.17547 (name and ordering idea shared with MAttr)
• Lindsey et al. (2024), Sparse Crosscoders for Cross-Layer Features and Model Diffing, transformer-circuits.pub
• Minder et al., Overcoming Sparsity Artifacts in Crosscoders… (latent scaling ν, the L1 artifact), 2504.02922
• Kissane et al. (2024), SAEs (Usually) Transfer Between Base and Chat Models, Alignment Forum

H. Steering (RQ3)

• Turner et al., Activation Addition, 2308.10248
• Rimsky et al., Contrastive Activation Addition, 2312.06681

I. Prompt injection and persuasion

• Greshake et al., Not What You’ve Signed Up For, 2302.12173
• Yi et al., BIPIA, 2312.14197
• Chen et al., StruQ, 2402.06363
• Wallace et al., The Instruction Hierarchy, 2404.13208
• Zeng et al., How Johnny Can Persuade LLMs…, 2401.06373

J. Multiple-choice and option order (RQ14, S3-D)

• Zheng et al., LLMs Are Not Robust Multiple Choice Selectors, 2309.03882
• Pezeshkpour & Hruschka, LLMs Sensitivity to the Order of Options, 2308.11483
• Wiegreffe et al., Answer, Assemble, Ace, 2407.15018. They also find symbol binding in late layers, which matches your L23 result, so this one is worth citing directly.
• Lieberum et al., Does Circuit Analysis Interpretability Scale? (Chinchilla MC), 2307.09458

Footnotes

1: From now onwards, “we” should refer to me and Claude.