Nibnib image format decodes two pixels per byte on RP2350

A GitHub repository presents nibnib, an image format tuned for extreme decompression speed. According to its README, it began as an experiment to see what image quality and performance were possible for DVI output on an RP2350 microcontroller. On devices of that class, low bit depth images and palette lookup are common. The budget is approximately 6 cycles per pixel, which is enough for a pixel fetch, a palette lookup and a store into the line buffer, but leaves little room to spare.

Nibnib keeps a palette at its core but changes what the palette holds. Instead of an absolute colour, each entry stores a delta. The README says this lets it surpass ordinary palette images in quality, vastly improving soft regions. Speed comes from decoding two pixels at a time from a single input byte, which gives 4bpp compression.

Decoding works on pixel pairs. The last decoded pixel is doubled up, a byte is read from the input stream and looked up in the palette, and the resulting 32-bit value is added to the doubled previous pixel. The result is two 16bpp pixels in RGB 565 format; the low 16 bits are the left pixel. The README shows the inner loop, lightly adapted from the Rust source: for each byte it computes pix as the upper half of the previous value multiplied by 0x10001, with wrapping addition of the palette entry, then writes the low and high 16 bits as two output pixels. An RP2350 on-the-fly decoder is included in the image demo of the vic-2350 repo.

Encoding is the hard part. Like most compression formats, nibnib is defined by its deterministic decoder, and encoding is a search through possible encoded images for one that decodes to a good replica of the original. The README says this search is quite asymmetrical: the current encoder is orders of magnitude slower than decoding. It contrasts this with JPEG, where the DCT is invertible and quantization is easy, so at baseline encoding and decoding run at about the same speed.

The search has two main dimensions: the choice of palette and the selection of codewords. In the initial version the palette is a cube of small delta values plus a block of random values chosen by weighted Gaussians. The cube is needed so that values near 0 and full scale can be reached without overflow; without it, the images show bad artifacts. The Gaussian process was tuned ad hoc, using guesses about reasonable values. The delta between the two pixels trades sharpness against noise: keeping it small smooths the image but reduces the ability to encode sharp edges. The palette is not chosen adaptively from the input image, which the README calls most definitely an opportunity to improve quality.

The basic encoding loop tries all 256 palette entries, picks the one with the minimum error, and distributes that error using Floyd-Steinberg weights, so later pixels fit the source value plus weighted earlier errors. This resembles error diffusion with a quantized palette, except that error diffusion usually handles one pixel at a time and nibnib produces two. The entry is chosen to minimize the error of both pixels, with somewhat higher weight on their sum than on their difference, because human vision is more sensitive to lower spatial frequencies.

The README calls this a basic, working technique and lists improvements. Quality could rise significantly with an optimized palette and a deeper search; two choices for one pixel may have roughly equal error while one makes the next pixel's search much more favourable. Artifacts currently trail to the right of sharp edges, which the README says is a consequence of the greedy search. Encoding right to left, or boustrophedonically with alternating row direction, while decoding stays left to right looks feasible because the addition step is invertible in practice, with some subtlety around the carry and the pixel duplication. Both improvements would slow the encoder considerably but leave decoding speed unchanged. Another direction is a faster encoder: brute-force palette search can almost certainly be improved, perhaps with a more structured palette. All of these changes work with the existing decoder.

The file format is described as extremely unsophisticated: 2 bytes each for width and height, then 256 32-bit palette entries, then the rows in order. Each row begins with two RGB 565 pixels followed by (width - 2) / 2 bytes, each indexing the palette, and the row stride is rounded up to a multiple of 4 bytes. There is no other metadata. The format is not intended for archival and is much more likely to be used when integrated tightly with the decoder. The code is licensed under Apache 2.0 or MIT, at the user's option.

Key facts

• Nibnib is an experimental image format that began as a test of image quality and speed for DVI output on an RP2350 microcontroller, where the budget is approximately 6 cycles per pixel.
• Each input byte indexes a 256-entry table of 32-bit palette deltas; adding the delta to the doubled previous pixel yields two RGB 565 pixels, so the format reaches 4bpp.
• The encoder is a greedy search with Floyd-Steinberg error diffusion over all 256 palette entries, and the README says it is orders of magnitude slower than decoding.
• Planned improvements are an image-adaptive palette, a deeper search and right-to-left or alternating-direction encoding; all would work with the existing decoder.
• The file layout is 2-byte width and height, 256 32-bit palette entries, then rows; the code is dual-licensed Apache 2.0 or MIT.

Why it matters

Microcontrollers that drive video output have almost no time per pixel: about 6 cycles on RP2350-class devices, just enough for a fetch, a palette lookup and a store. Plain palette images fit that budget but look worse in soft regions. Nibnib keeps the same lookup cost and stores a delta instead of an absolute colour, which the README says improves soft regions a lot. It gets two pixels from each byte, and the whole decoder is one add and a table lookup per pair. The design pushes all the hard work into the encoder, a trade that suits images prepared once and shown many times.

Who it affects

The audience is narrow: embedded developers who push images to a display from an RP2350, such as DVI output, and anyone interested in cheap decoders for constrained hardware. The README says the format is much more likely to be used when integrated tightly with the decoder, and it is not meant for archival, so it is not a general-purpose replacement for common formats.

How to use it

The code is licensed under Apache 2.0 or MIT, at your option. The RP2350 on-the-fly decoder implementation is included in the image demo of the vic-2350 repo, and the decoder's inner loop is shown in the README, adapted from the Rust source. To read the format by hand: 2 bytes each for width and height, then 256 32-bit palette entries, then the rows. Each row starts with two RGB 565 pixels and then (width - 2) / 2 palette-index bytes, with the row stride rounded up to a multiple of 4 bytes. Encoding is the slow part, so images would be converted ahead of time.

How solid is it

This is the project's own README describing an experiment, and it calls the result a basic, working technique. It explains the algorithm in detail, but no benchmark figures are given: no decode throughput, no cycle count measured on the RP2350, and no encoder timing beyond 'orders of magnitude slower'. No image-quality metrics such as PSNR or SSIM appear, and no comparison numbers against JPEG or other palette formats. Quality claims, such as the improvement over palette images, rest on the author's description and visual examples rather than on stated measurements.

Risks and caveats

The encoder is a greedy search, and artifacts currently trail to the right of sharp edges. The palette is not adapted to the input image, and the Gaussian settings were tuned by guesswork. A better palette and a deeper search would slow the encoder considerably. The format has no metadata beyond dimensions and palette, is not intended for archival, and no timescale or roadmap is given for the proposed encoder improvements. Compression is fixed at 4bpp, and no compression-ratio comparison against standard formats is given beyond that figure.