Glonce

AutoHebb: Hebbian networks that gate their own learning for zero-shot inference

researchThe Glonce editorsarXiv🔥56Glonce rating56Impact40Novelty60Relevance95Surprise30Players10Credibility70

A new arXiv paper, titled "AutoHebb: Self-Modifying Neural Networks that Zero-Shot Infer and Continually Learn", starts from a point about brains. Humans and animals can generalize and learn in novel circumstances, including "0-shot" inference (ZI) and continual learning (CL). The authors say efforts to model these two abilities are seriously limited in several ways. Existing models typically lack components that are critical to ZI and CL in the brain: strong inductive biases for ZI and autonomous control of learning for CL. Models can also be hard to analyse when their components are overly complex, or too restricted to one task when their components are overly bespoke.

To address this, the paper introduces autonomous Hebbian networks, or "AutoHebb". These are a class of online learning feedforward neural networks that, according to the authors, realize strong inductive biases and autonomously controlled learning while staying both simple and expressive.

The mechanism differs from standard training. Standard models learn through externally controlled optimization such as gradient descent. AutoHebb networks instead combine external feedback with internally generated activity to produce a global learning signal, called a "neuromodulator", which is fed back to the network. The neuromodulator in turn gates a biologically grounded local learning rule, a 3-factor Hebbian rule, that updates the network's plastic weights. The fixed (non-plastic) weights are found with meta-learning.

With that setup, the authors report four findings: AutoHebb networks (1) succeed on ZI tasks, (2) show emergent, untrained ZI and CL abilities, (3) admit reverse-engineering into explicit algorithms, and (4) outperform modern learning models on ZI and CL tasks. Performance comparisons to related models also implied an essential role for three model components: meta-learning, Hebbian plasticity, and autonomous control of learning.

The authors conclude that AutoHebb may be useful in hypothesis generation, analysis, and discovery of biologically grounded forms of learning and generalization.

Key facts

  • AutoHebb is a class of online learning feedforward networks in which a global neuromodulator signal gates a local 3-factor Hebbian rule that updates the plastic weights.
  • The neuromodulator is produced by combining external feedback with internally generated activity, in contrast to externally controlled optimization such as gradient descent.
  • Fixed (non-plastic) weights are found with meta-learning.
  • The authors list four findings: success on zero-shot inference tasks, emergent untrained zero-shot and continual learning abilities, reverse-engineering into explicit algorithms, and better results than modern learning models on those tasks.
  • Comparisons to related models implied that meta-learning, Hebbian plasticity and autonomous control of learning all play an essential role.

Why it matters

The paper targets two abilities that brains have and many machine learning systems handle poorly: inferring in novel situations without examples (zero-shot inference) and learning continually. Its argument is that existing models lack strong inductive biases and autonomous control of learning, and that the fix is a network that decides for itself when and how much to learn, instead of relying on an external optimizer such as gradient descent.

Who it affects

Mainly researchers working on continual learning, meta-learning, Hebbian or plastic networks, and computational models of how brains generalize. The authors frame the approach as a tool for generating and analysing hypotheses about biologically grounded forms of learning.

How to use it

This is a research result, not a product. The authors suggest AutoHebb may be useful in hypothesis generation, analysis, and discovery of biologically grounded forms of learning and generalization. The abstract mentions no code release, model size or compute cost.

How solid is it

All claims come from the authors' own abstract, and it is a preprint. The abstract says AutoHebb outperforms "modern learning models" on zero-shot and continual learning tasks, but it names no benchmarks, datasets, comparison models or quantitative results, so the size of the advantage cannot be judged from it. The finding that three components are essential is described by the authors as implied by performance comparisons, not proved.

Risks and caveats

The headline performance claim is self-reported and unspecified. The abstract names no authors or institutions, and it does not claim the networks match human or animal performance, only that they model biologically grounded learning. The authors' own conclusion is hedged: AutoHebb "may be useful" for hypothesis generation and discovery, not that it is proven to be.