Skip to content
Novandri.

Mechanistic Interpretability Examples

Mechanistic interpretability studies the computation inside a model. Read concrete examples, a worked method, and why a circuit is not an explanation.

Rachmadwipa Novandri4 min read

Mechanistic interpretability tries to identify the internal computations that produce a behaviour, which is different from asking a model to explain itself. The interpretability entry describes the field, as the superposition paper treats features shared across neurons, as Model Explainability: SHAP, LIME, and Interpreting Black Boxes covers post-hoc methods, as the pyvene literature treats interventions, as the causal-inference article warns about correlation, as the publication standard forbids guaranteed outcomes.

Direct answer:

Study a specific behaviour with a hypothesis, an intervention, and a measured change — not a narrative.

Wikipedia's interpretability entry provides the context, and the superposition paper provides a mechanism. The boundary: finding a circuit is not the same as explaining a behaviour fully, which the causal-inference article warns, as the risk article requires residual risk.

A four-step flow: pick a behaviour, state a hypothesis, intervene, then measure the change.
Conceptual: behaviour, hypothesis, intervention, change. Study a specific behaviour with a hypothesis, an intervention and a measured change.

The examples

Feature visualisation. A pattern that activates a component can suggest what it represents, which the interpretability entry describes, as the superposition paper treats features, as the data-governance article requires provenance, as the benchmarks article warns about broad claims, as the rubric article requires concrete criteria, as the risk article requires residual risk.

Activation patching. Replacing an internal value and observing the output can test a hypothesis, which the pyvene literature treats, as the causal-inference article distinguishes intervention from correlation, as the evaluation article requires a test, as the rubric article requires criteria, as the decision-memo article requires for a record, as the monitoring article advises for review.

Circuit analysis. A path of components can be traced for a narrow behaviour, which the interpretability entry describes, as Mechanistic Interpretability: Reading Circuits Inside a Model treats internal pathways, as the model-explainability article contrasts post-hoc methods, as the causal-inference article warns about causes, as the risk article requires residual risk, as the publication standard forbids guaranteed explanations.

Faithfulness checks. An explanation should predict behaviour under intervention, which the evaluation-rubric article requires, as the causal-inference article advises, as the benchmarks article warns about easy cases, as the observability article treats signals as evidence, as the data-governance article requires dated figures, as the risk article requires residual risk.

Three consequences follow. The behaviour should be narrow, as the rubric article requires, as the interpretability entry advises, as Mechanistic Interpretability: Reading Circuits Inside a Model treats narrow paths, as the benchmarks article warns, as the data-governance article requires dated figures, as the evaluation article requires a test, as the causal-inference article distinguishes causes, as the decision-memo article requires a record, as the monitoring article advises for review, as the risk article requires residual risk, as the model-explainability article contrasts methods, as the observability article treats signals.

A worked method

The following is hypothetical and illustrative. A researcher suspects a component carries information about a token's position.

Three checks decide. First, is the behaviour narrow and measurable, as the rubric article requires, with a fixed test, as the evaluation article requires? Second, does intervening on the component change the output as predicted, as the pyvene literature and causal-inference article advise, with a faithfulness check, as the rubric article requires? Third, is the result recorded with residual uncertainty, as the decision-memo and risk articles require, with dated data, as data governance requires, as the monitoring article advises for review?

Three conclusions follow.

Intervention is stronger evidence, as the causal-inference and pyvene literature advise. And the result should be recorded with uncertainty, as the decision-memo and risk articles require.

What interpretability cannot do

It cannot provide a complete explanation, which the publication standard forbids claiming, as the causal-inference article warns, as the risk article requires residual risk. It does not guarantee faithfulness, which the rubric and evaluation articles require, as the benchmarks article warns about easy cases, as the observability article treats signals, as the monitoring article advises for review. And it cannot replace evaluation, which the evaluation-rubric article requires, as the model-explainability article contrasts methods, as the data-governance article requires dated figures, as the decision-memo article requires for a record.

The counterargument that a circuit fully explains a behaviour is fair for narrow cases and incomplete when the behaviour generalizes, which the worked method shows, as the causal-inference and benchmarks articles advise, as the risk article requires residual risk.

A decision test

  1. Define a narrow behaviour, as the rubric article requires.
  2. State a hypothesis, as the decision-memo article requires.
  3. Intervene, as the pyvene literature advises.
  4. Check faithfulness, as the rubric article requires.
  5. Use dated data, as data governance requires.
  6. Avoid broad claims, as the benchmarks article warns.
  7. Record the result and residual risk, as the decision-memo and risk articles require.

For the mechanism, see Interpretability Reading Circuits; for the contrast, see Model Explainability.

References & Further Reading