Glonce

Apertus 1.5 adds image and audio input, thinking mode, 262k context

modelsThe Glonce editorsHugging Face🔥64Glonce rating64Impact50Novelty60Relevance100Surprise35Players40Credibility85
Apertus 1.5 adds image and audio input, thinking mode, 262k context

Project Apertus has published release notes for Apertus 1.5, a continued pretraining of the first-generation Apertus models. The team added 4 trillion tokens of text and multimodal data to the 8B model and 2 trillion tokens to the 70B model, bringing the totals to 19T and 17T respectively. As with every Apertus release, the models are described as fully open: open weights, open data, open values and full training details.

The headline changes are native multimodal understanding, a thinking mode and a longer context. Apertus 1.5 accepts images and audio alongside text, so documents, diagrams and photos can go into the conversation; processing spoken language is described as experimental. The models can be switched into a thinking mode that reasons about the input before answering, which the team says improves performance on reasoning problems. The context window grows to 262,144 tokens, a 4x increase over Apertus 1.0. The notes also list better instruction following, better tool use and stronger adherence to the Apertus Charter, which the team says makes the model's values and principles transparent.

Multimodality is built in an early-fusion way. Images and audio enter as discrete tokens in the same sequence as text, with no separate vision encoder. Images are tokenized with the Emu3.5 tokenizer (a 512×512 image becomes 1,024 tokens) and audio with WavTokenizer (40 tokens per second). Only the vocabulary is extended, so the architecture is unchanged and training runs at text-only speed. The models understand images and audio but generate only text.

Pretraining continues from Apertus 1.0 in four stages: unpaired data (text plus standalone images and audio), paired data (image-caption and audio-transcript pairs), a cooldown on higher-quality data with more math and code, and a long-context stage where the window grows step by step from 8k to 262k tokens. More than half of the training tokens are text-only, which the team says preserves the language abilities of Apertus 1.0. Text data is refreshed with sources such as FineWeb2-HQ+, FinePDFs and Nemotron-CC. All new data, multimodal included, is publicly available, permissively licensed, and respects robots.txt opt-outs retroactively; every source is listed in apertus-data-transparency.

Post-training has five stages: SFT, long-context SFT, RL, DPO and charter alignment (SDPO). SFT uses 3.7M text samples plus image and speech data. Reasoning traces are added only to prompts the model cannot already solve without them, and tool-use data is limited to execution-verified trajectories. Long-context SFT at the full 262k window restores the long-context ability that short-context SFT erodes. RL with verifiable rewards uses about 350k math, code, instruction-following, multilingual and vision prompts, mixing thinking on and off. DPO with judge-scored preference pairs targets chat quality, tool use and safety. SDPO aligns the model with the Apertus Charter through judge feedback and self-distillation. The team releases the text SFT mix, the preference pool and the training pairs.

On evaluation, the team says the release still trails frontier open-weight models, but coding and instruction following have improved significantly, and the 8B model outperforms similarly sized open-weight models on instruction-following and multilingual tasks. Thinking further improves results on reasoning-heavy benchmarks such as AIME. Multimodal performance is described as closer to Gemma 4 12B and other current open-weight models, competitive in spatial understanding among open-weight models and in document and chart understanding among open-source models. Apertus 1.5 is said to perform strongly on Swiss Legal tasks, on par with leading open-weight models. On the Last Translation Benchmark, the 8B model matches GPT-OSS-20B and outperforms Claude Haiku 4.5 and Google Translate. New evaluations cover images, audio and thinking mode, using an MLLM-eval-suite pipeline that merges lmms-eval and VLMEvalKit, plus extra text benchmarks for translation, multilingual ability, safety and tool use.

The team also publishes results in the Every Eval Ever (EEE) schema with the EvalEval Coalition. The schema records who ran an evaluation, how the model was accessed, the generation settings and how each metric should be read. Text evaluations run on a public fork of lm-evaluation-harness, so anyone can rerun them under identical settings. EvalEval found that the EEE schema can represent 56 of the 60 evaluation-reporting elements called for by AI governance frameworks, including Article 55 of the EU AI Act and the GPAI Code of Practice. Apertus also publishes a Public Summary of Training Content and, voluntarily, a Code of Practice document.

For reproducibility, recipes, ablations and data mixtures are documented in the technical report, and intermediate checkpoints (base weights, SFT, RLVR and DPO) are hosted in a Hugging Face collection. The ablations found that multimodal data mixed with text during SFT helps across all modalities, and that the benefit of multimodal reasoning traces survives later text-only RL: RL improves mathematics more when started from an SFT checkpoint trained with multimodal traces, and the advantage persists through training.

On availability, Proton Lumo and Infomaniak Euria have recently integrated Apertus 1.5. Swisscom, Infomaniak, Phoeniqs and other companies have announced API access for developers. The team says Apertus models have passed five million downloads in Hugging Face analytics. At the AI+X Summit in Zurich it announced a roadmap for Apertus 2.0: a sparse/MoE architecture, about 600B total parameters, an extended training schedule, and agents, mathematics and coding as priorities. The accompanying paper, "Apertus 1.5: From Open Text Models to Open Multimodal Reasoners" (arXiv:2610.15542), is dated October 11, 2026.

Key facts

  • Apertus 1.5 continues pretraining of the 8B and 70B models with 4T and 2T extra tokens, for totals of 19T and 17T respectively.
  • New abilities: image and audio input (spoken language is experimental), a switchable thinking mode, and a 262,144-token context window, 4x that of Apertus 1.0. Output is text only.
  • Images and audio enter as discrete tokens in the same sequence as text (Emu3.5 for images, WavTokenizer for audio); only the vocabulary changes, so the architecture stays the same.
  • The team reports that the 8B model beats similarly sized open-weight models on instruction following and multilingual tasks and matches GPT-OSS-20B on the Last Translation Benchmark, while still trailing frontier open-weight models.
  • The Apertus 2.0 roadmap points to a sparse/MoE design with about 600B total parameters, focused on agents, mathematics and coding.

Why it matters

Apertus is a fully open effort: open weights, open data, open values and full training details. Version 1.5 takes it from a text-only family to one that accepts images and audio, reasons in a thinking mode and handles 262,144 tokens of context. The team pairs this with unusually broad release material: the text SFT mix, the preference pool and training pairs, intermediate checkpoints, and evaluation results in the shared EEE format. It describes all new data as publicly available, permissively licensed and respecting robots.txt opt-outs retroactively.

Who it affects

Researchers who want to study how each training stage changes a model can use the published checkpoints (base, SFT, RLVR, DPO), recipes and data mixes. Developers can reach the models through API providers including Swisscom, Infomaniak and Phoeniqs, or run them from the Hugging Face model cards. End users may meet the model inside apps such as Proton Lumo and Infomaniak Euria. Regulators and deployers are the stated audience for the machine-readable evaluation records, which EvalEval says map to 56 of 60 evaluation-reporting elements in the EU AI Act Article 55 and GPAI Code of Practice frameworks.

How to use it

Technical instructions for running the models are in the model cards of the Apertus releases on Hugging Face and in the Quickstart guide. Intermediate checkpoints and curated training data mixes sit in the Apertus 1.5 Hugging Face collections. The text evaluations can be rerun on the team's public lm-evaluation-harness fork, and the multimodal ones through the MLLM-eval-suite pipeline. Developers who prefer a hosted route can look at the General Availability guide for API providers. Thinking mode is a switch: the models can be run with reasoning before answering or without it.

How solid is it

This is a first-party release post from the Apertus team, so the performance claims are the developers' own. The notes say the results are published in the EEE schema with the EvalEval Coalition and can be rerun on a public evaluation fork, which makes them checkable. The ablation findings come from the technical report (arXiv:2610.15542, October 11, 2026). No benchmark scores or numeric evaluation results are given in the visible text; AIME, instruction-following and multilingual results are named without figures. The comparison to Gemma 4 12B is described only as "closer".

Risks and caveats

The team itself says the release still trails frontier open-weight models. Spoken-language processing is labelled experimental. The models understand images and audio but generate only text. The comparisons rest on the team's own descriptions, and the visible text gives no numbers for them. The notes name 8B and 70B as the models that received the continued pretraining; they do not say which sizes support multimodality. The Apertus 2.0 roadmap gives no release date or timescale.

“As with every Apertus release, the models are fully open: open weights, open data, open values, and full training details.”

— Apertus 1.5 release notes, Hugging Face blog