MC-Lens probes concept access in vision-language models

MC-Lens probes concept access in vision-language models

Vision-language models (VLMs) combine visual and textual information, but their outputs do not reveal the concept representations inside them. A new paper tackles that gap with the Multimodal Concept Lens (MC-Lens), a multimodal Jacobian lens. The authors frame it through global workspace theory (GWT), which proposes that information is selected from parallel brain processes, integrated in a shared workspace, and made available to specialised processes. MC-Lens is built to study those three functions in how VLMs handle concepts.

The tool has two lenses, one on the vision encoder and one on the language decoder. Both map intermediate states into a shared semantic space defined by the decoder's output vocabulary, which lets the authors compare how accessible a concept is across the two modalities. MC-Lens also supports internal concept interventions, used to examine what role the accessed concepts play in downstream computation.

The abstract reports four sets of findings.

(i) Accessibility and selective access. Multimodal concept information is readable in intermediate states and selectively accessible as tasks require. To capture concept information better and to support counterfactual experiments, the authors extend "readout carriers" from a single name token to a set of tokens.

(ii) Concepts in downstream computation. Counterfactual experiments show that accessed concept information participates in downstream computation and reasoning.

(iii) Development of cross-modal alignment and selective access. In controlled VLM training, the authors develop two J-space metrics that track the development of cross-modal concept alignment more clearly than hidden state cosine does. The same experiment shows that reliable selective access develops later than concept accessibility.

(iv) Shared functions across models. Across VLM families and sizes, the authors find shared patterns of concept accessibility, query-dependent selectivity and cross-task intervention effects, despite differences in internal organisation.

The authors conclude that these findings offer an experimental basis for studying GWT in multimodal artificial systems, and for studying how it can guide improvements to cross-modal alignment.

Key facts

  • MC-Lens is a multimodal Jacobian lens with a vision encoder lens and a language decoder lens, both mapping intermediate states to a shared semantic space defined by the decoder's output vocabulary.
  • Concept information is readable in intermediate states and selectively accessible as tasks require; the authors extend "readout carriers" from one name token to a token set.
  • Counterfactual experiments show that accessed concept information participates in downstream computation and reasoning.
  • In controlled VLM training, two J-space metrics track cross-modal concept alignment more clearly than hidden state cosine, and reliable selective access develops later than concept accessibility.
  • Shared patterns of concept accessibility, query-dependent selectivity and cross-task intervention effects appear across VLM families and sizes.

Why it matters

VLM outputs do not reveal internal concept representations, so it is hard to see what a model actually holds in its intermediate states. MC-Lens offers a way to read concepts out of both the vision encoder and the language decoder in one shared vocabulary space, and to intervene on them. Its timing result is a concrete one: during controlled training, reliable selective access comes later than plain accessibility. The paper also uses global workspace theory as a structured set of questions about selection, integration and availability of information inside a model.

Who it affects

Mainly researchers working on interpretability and cross-modal alignment in vision-language models, and those interested in applying GWT-style ideas to artificial systems. The authors say the findings can guide cross-modal alignment improvements, which would be relevant to people training VLMs.

How to use it

The abstract describes MC-Lens as a probing method rather than a product. Its lenses read intermediate states from the vision encoder and language decoder, and its intervention support lets a researcher alter accessed concepts and watch the effect on downstream computation. The two J-space metrics can be used to follow cross-modal concept alignment during training, where the authors say they are clearer than hidden state cosine.

How solid is it

This is a preprint listed on Hugging Face Papers, and the account here rests on its abstract. The authors report experiments in four areas: readout and selective access, counterfactual interventions, controlled training, and comparisons across VLM families and sizes. The abstract reports no quantitative results such as accuracy, effect sizes or percentages, so the size of the effects cannot be judged from it. The one explicit comparison is that the J-space metrics track alignment more clearly than hidden state cosine.

Risks and caveats

The abstract does not claim that VLMs are conscious or have a global workspace. It says only that the findings offer an experimental basis to study GWT in multimodal artificial systems, so the global workspace framing should not be read as a claim about machine consciousness. The shared patterns across models are reported alongside differences in internal organisation, which means the similarity is partial. Specific models, sizes and datasets are not named in the abstract, so the breadth of the cross-model result is hard to assess from it.

“Multimodal concept information is readable in intermediate states and selectively accessible as tasks require.”

— Paper abstract, Hugging Face Papers