UT Austin engineer swaps AI multiplication for lookup tables

Most neural networks work by multiplying inputs against learned weights, an operation performed millions or billions of times per query and, according to Lizy K. John, professor of electrical and computer engineering at the University of Texas at Austin, more work than the job actually requires. In an IEEE Spectrum interview, John describes five years spent developing weightless neural networks, which route binary inputs through interconnected lookup tables instead of multiplying them by weights, closer to consulting stored answers than repeating the same arithmetic. Depending on the task, she says, the resulting networks can be less than a thousandth the size or 1,000 times as fast as conventional alternatives while keeping comparable accuracy, though no accuracy percentages are given.

John traces the idea to a lookup based pattern recognition product someone in the UK commercialized in the 1980s before it disappeared, kept alive afterward by a small research group at the Federal University of Rio de Janeiro. A colleague invited her to a weekly meeting on the technique, and she assigned a student with spiking neural network experience to build a prototype. Within six months the team had a lookup based network running on an FPGA at 1,000 times smaller than what other researchers were achieving at the time.

The team's results so far are concentrated in small, specific problems. On datasets such as human activity recognition and medical monitoring, including ECG, EEG and blood pressure, John says weightless networks cut energy use 1,000 times against conventional alternatives. For one sensor problem, the best comparable small AI model runs 17 megabytes; the team's weightless version is 14 kilobytes, more than 1,000 times smaller. For keyword spotting, the kind of listening that detects a wake word like "Alexa," the current best industry model uses more than 5,000 nanojoules per inference; John's team's version uses 42 to 79 nanojoules depending on the variation. The team also built an arrhythmia detector that fits on a bendable plastic substrate holding only about 10,000 logic gates, versus the billions of gates on a conventional 2 or 5 nanometer chip; John says the plastic substrate is also more sustainable to manufacture because it uses less water than standard chipmaking.

John frames the appeal as energy efficiency: she notes the human brain runs on about 20 watts, while a large language model performs millions or billions of multiplications to predict a single next word, an operation she calls the most energy expensive part of the hardware. Multiplication, not the surrounding architecture, is what the technique targets; the underlying neuron model behind most popular networks today traces to a 1943 paper, the McCulloch-Pitts model, which the field scaled up to billions of neurons without, in John's view, establishing that scale is necessary.

On transformers, the model behind today's chatbots, John says her team has already replaced the multilayer perceptron portion, about half of a transformer layer, with lookup tables, but has not yet replaced the attention layer; language models remain a longer term target rather than a demonstrated result. She says adopting the approach elsewhere would require no change to the rest of the AI stack: it can use the same training data current models use, and training still happens on GPUs today only because they are the most powerful hardware available. She hopes to eventually train on FPGAs instead, since FPGAs already contain small lookup tables suited to both training and inference. John attributes the field's small following to inertia: existing multiplication based methods work well enough that few researchers feel pressure to try something that looks too simple to scale, and she says showing the approach succeed on a larger language model is what she believes would draw more people in.

Key facts

Weightless neural networks replace weight multiplication with binary lookup tables, and per University of Texas at Austin professor Lizy K. John can run at less than a thousandth the size or 1,000 times the speed of conventional networks on some tasks while keeping comparable accuracy, though no accuracy percentage is cited.
On sensor tasks like ECG, EEG and blood pressure monitoring, John says weightless networks use 1,000 times less energy; for one problem the team's model is 14 kilobytes versus 17 megabytes for the best comparable alternative.
For wake word keyword spotting, John's team's model uses 42 to 79 nanojoules per inference versus more than 5,000 nanojoules for the current best industry model.
The team has replaced the multilayer perceptron half of a transformer layer with lookup tables but has not yet replaced the attention layer, so full transformer scale language models remain an unproven, longer term target.
The technique traces to a 1980s UK commercial pattern recognition product that disappeared and was kept alive by a small research group at the Federal University of Rio de Janeiro before John's team began work on it five years ago.

Why it matters

Multiplying inputs by learned weights is the operation that drives most of the energy cost in today's neural networks, from chatbots to photo organization to song recommendations. John argues that much of this multiplication is unnecessary: the human brain runs on about 20 watts while a language model performs millions or billions of multiplications to produce a single next word. Weightless neural networks replace that arithmetic with binary lookup tables, an old technique from 1980s pattern recognition hardware that a small research group kept alive quietly for decades before John's team picked it up five years ago and got it running on an FPGA within six months.

Who it affects

The most immediate audience is engineers building battery powered or space constrained devices: medical sensor makers, wearable and activity tracking developers, and anyone building keyword spotting or wake word hardware. John's own target use case is medical monitoring, such as a Band-Aid style sensor a patient wears for a week, and chemistry experiments where an answer is needed at the point of measurement rather than after offline processing. Chatbot and language model builders are a longer range audience only: the technique has replaced half of a transformer layer so far, not the whole model.

How to use it

There is no commercial product yet; this is university research, and John names no funding source, company or planned release. She says adopting the approach requires no change to the surrounding AI stack: weightless networks can train on the same data current models use, and training still runs on ordinary GPUs today simply because they are the most powerful hardware available. She hopes to eventually train directly on FPGAs, which already contain the small lookup tables the technique needs for both training and inference; until then, she says, nothing else in the pipeline needs to change.

How solid is it

The results are demonstrated, not theoretical, but confined to small problems. John's team has built working FPGA implementations for human activity recognition, ECG/EEG/blood pressure monitoring and keyword spotting, and cites specific size and energy figures for each: 14 kilobytes versus 17 megabytes on one sensor task, 42 to 79 nanojoules versus over 5,000 nanojoules per inference on keyword spotting. Separately, the team built an arrhythmia detector on a bendable plastic substrate holding about 10,000 logic gates. On transformers, the team has replaced only the multilayer perceptron portion, about half a layer, and has not touched the attention mechanism; no accuracy percentages are given anywhere, only John's claim of "comparable accuracy."

Risks and caveats

Every demonstrated result in the interview is on a small, specific problem; John states plainly that for bigger problems, people lack confidence the approach will scale, because it seems too simple to work at that size. The technique has failed to gain traction before: a 1980s UK commercial version disappeared, and only a small Rio de Janeiro research group kept working on it in the intervening decades. The interview names no timeline for reaching transformer scale language models, no funding source, and no accuracy figures to compare directly against conventional networks; John herself frames a working large language model demonstration as a hope, not an achieved result.

“If you ask me a question, I'm not doing multiplications to answer you. I'm thinking, yes, no, I should say this, I shouldn't say that. And the human brain only consumes about 20 watts of energy doing that.”

— Lizy K. John, professor of electrical and computer engineering, University of Texas at Austin