FLUX.3 Image Klein arrives in Diffusers with NVFP4 and FP8 weights
Hugging Face has published a launch post for FLUX.3 Image Klein, described as the latest open image generation model from Black Forest Labs, together with its support in the Diffusers library. Two open checkpoints sit in the FLUX.3 Image Klein collection: a distilled one and a base one. The post's pitch is one model for text-to-image, multi-reference image-to-image and bounding-box layout control, with quantized weights for smaller GPUs, LoRA training scripts for generation and editing, and delivery as a Modular Diffusers pipeline whose blocks can be removed or replaced.
Getting started takes a pip upgrade of diffusers and transformers (transformers is needed for the Qwen3-VL text encoder), then loading the distilled checkpoint "black-forest-labs/flux-3-image-klein" through ModularPipeline in bfloat16. The distilled checkpoint generates a 1024 × 1024 image in 16 steps by default, and height, width or num_inference_steps can be changed. The post advises prompting with a full scene description covering subject, setting, light, composition and style, and points to Black Forest Labs' official prompting guide.
For image-to-image, you pass one or more reference images through the image argument and write the prompt as a caption of the image you want, with the change included. With several references, the prompt refers to them as <ref_image_0>, <ref_image_1> and so on, in the order they are passed. The post's example combines a generated red fox and a wooden bench into one scene.
Layout control works by adding a JSON list of regions after the caption. Each region has an id, a bbox and a desc, and the caption mentions the region as
The base checkpoint ("flux-3-image-klein-base") runs 50 steps with classifier-free guidance and is the one the post recommends for fine-tuning; the example sets guidance_scale to 5.0 as an optional setting. Black Forest Labs also ships the transformer as a single file in bf16 and in two quantized formats, fp8r (rowwise FP8) and nvfp4. They load with from_single_file and Flux3Transformer2DModel plus a BFLQuantizationConfig, and the other components are loaded by name so the bf16 transformer is not downloaded as well.
For LoRA training, Diffusers ships DreamBooth LoRA scripts for both tasks; the post also mentions Ostris' AI Toolkit as another way to train. The text-to-image example learns a tarot-card style from the multimodalart/1920-raider-waite-tarot-public-domain dataset, triggered with "trtcrd", at 1024 resolution, batch size 1, rank 16, learning rate 1e-4 and 1500 training steps. To save memory it suggests --gradient_checkpointing, --cache_latents and --use_8bit_adam. The adapters from a grid of ranks and optimizers are on the Hub at diffusers-modular/flux3-tarot-lora. An image-to-image script trains on (reference image, target image, instruction) triplets.
On cost, the memory advice is to load a quantized transformer, run the text encoder separately and free it (it runs once per image, before denoising), or turn on automatic CPU offloading, which the authors say they did not benchmark. Speed numbers for the distilled checkpoint (16 steps) were averaged over three runs after a warmup on a single NVIDIA H100 80GB with no offloading and the default attention backend. A reference image roughly doubles the time, because reference tokens join the same sequence as the text and the image being generated: a 1024 × 1024 reference adds 4,096 tokens to every transformer call.
The architecture section starts with the inputs: text, the noisy image being generated, and optional reference images. FLUX.3 is a latent-space flow model that uses the same VAE architecture as Flux.2. That VAE compresses an image 8× in each direction into 32 channels, and each 2 × 2 patch of the latent is packed into one token with 128 channels. One token therefore covers a 16 × 16 pixel patch, and a 1024 × 1024 image becomes a 64 × 64 grid of 4,096 tokens.
Key facts
- FLUX.3 Image Klein is described as the latest open image generation model from Black Forest Labs; two open checkpoints, distilled and base, are in one collection and supported in Diffusers.
- One model handles text-to-image, multi-reference image-to-image and bounding-box layout control, with boxes in [y_min, x_min, y_max, x_max] order on a 0 to 1000 scale.
- The distilled checkpoint runs 16 steps by default and makes a 1024 × 1024 image in about 2 seconds on an H100; the base checkpoint runs 50 steps with classifier-free guidance and is the one to fine-tune.
- Black Forest Labs ships the transformer as a single file in bf16, fp8r (rowwise FP8) and nvfp4, loaded with from_single_file.
- DreamBooth LoRA scripts cover text-to-image and instruction-based editing; the tarot example uses rank 16 and 1500 training steps.
Why it matters
It gives open-weights users a single image model that handles generation, reference-based editing and explicit spatial control, with support in the widely used Diffusers library. The layout feature is the distinctive part: normalized bounding boxes with per-region descriptions let you place, or move, specific elements instead of hoping the prompt does it. Quantized NVFP4 and FP8 weights are shipped by Black Forest Labs itself, aimed at smaller GPUs.
Who it affects
Developers and researchers who already run Diffusers pipelines, people who want to fine-tune an image model with LoRA, and users with smaller GPUs who need the quantized weights. Anyone building editing or layout-driven image tools gets the multi-reference and region mechanics in the same checkpoint. The post also points to a Hugging Face space for trying the model without installing anything.
How to use it
Run pip install -U diffusers transformers, load "black-forest-labs/flux-3-image-klein" with ModularPipeline in bfloat16, move it to CUDA and call it with a prompt. Pass image=... for references and use <ref_image_0>, <ref_image_1> in the prompt; add a JSON region list for layout. For fine-tuning, use the base checkpoint with train_dreambooth_lora_flux3_klein.py (or the img2img script for editing), or train with Ostris' AI Toolkit. For lower memory, load the fp8r or nvfp4 transformer, run the text encoder separately and free it, or try automatic CPU offloading. No license terms, parameter count or training-data details are given in the visible text.
How solid is it
This is a first-party Hugging Face post with runnable code for each feature, and it names Black Forest Labs as the model's developer. The headline speed claim, about 2 seconds for a 1024 × 1024 image on an H100, is for the distilled checkpoint at 16 steps, averaged over three runs after a warmup on a single H100 80GB with no offloading and the default attention backend. No benchmark or quality comparison against other models, such as FLUX.2 or competitors, appears in the visible text.
Risks and caveats
The speed figure is one hardware setup and the distilled checkpoint only; a reference image roughly doubles generation time. Automatic CPU offloading is suggested for low-memory setups, but the authors say they did not benchmark it. The distilled checkpoint is for generation, while fine-tuning is recommended on the base checkpoint, which needs 50 steps with classifier-free guidance per image. The text encoder depends on a recent transformers release for Qwen3-VL. The post gives no price information; its cost section covers memory and speed only.
“One model for text-to-image, multi-reference image-to-image, and bounding-box layout control.”
— Hugging Face blog post, TL;DR