Glonce

Agentic framework traces emotion-reasoning errors and patches prompts, lifting MER2025 F-score 0.6 to 2.2 points

researchThe Glonce editorsarXiv🔥55Glonce rating55Impact40Novelty55Relevance95Surprise25Players10Credibility70

The paper starts from a trade-off. LLM-based multimodal emotion reasoning systems do well on standard benchmarks, but improving them through instruction tuning can be computationally expensive. Agentic pipelines built from frozen models avoid that cost, but they raise a different problem: when a prediction is wrong, the system has to identify which agent or stage caused the error.

The authors propose an agentic framework for traceable multimodal emotion reasoning and prompt-space adaptation. Modality-specific perception agents analyze visual, acoustic and lexical inputs and post evidence-cited observations to a shared board. An integration agent then combines those observations with retrieved multimodal examples to produce an emotion description, a label set and supporting citations. An MLLM critic can request one revision before the output is finalized.

The learning step happens only for incorrect predictions. A separate evidence-grounded verification stage applies deterministic checks that the cited evidence and references are valid, plus an entailment check on whether the final claims are actually supported by that evidence. Detected failures are grouped by the responsible component and failure type, converted into targeted prompt patches, and accepted only after validation on a held-out set. The result is that the agent prompts adapt without any update to model weights.

On the MER2025 dataset, targeted prompt adaptation outperforms the frozen static pipeline on all four tested MLLMs, which range from 2B to 38B parameters: VideoLLaMA3-2B, Phi-3.5-vision-instruct, Qwen2.5-VL-7B-Instruct and InternVL2.5-38B. It also matches or outperforms undifferentiated reflection baselines on each of them. F-score improves by 0.6 to 2.2 points depending on the backbone, including a rise from 52.3 to 54.5 on Qwen2.5-VL-7B.

Key facts

  • The framework uses frozen MLLMs: perception agents for visual, acoustic and lexical inputs post evidence-cited observations, and an integration agent produces an emotion description, label set and citations.
  • For wrong predictions, deterministic checks on cited evidence and an entailment check trace the failure to a responsible component and failure type.
  • Failures become targeted prompt patches, accepted only after validation on a held-out set; no model weights are updated.
  • On MER2025, it beats the frozen static pipeline on all four MLLMs (2B to 38B parameters) and matches or beats undifferentiated reflection baselines on each.
  • F-score gains are 0.6 to 2.2 points depending on the backbone; Qwen2.5-VL-7B goes from 52.3 to 54.5.

Why it matters

Instruction tuning is costly, and frozen-model agent pipelines sidestep that cost but make errors hard to attribute. This work tries to close the loop without touching weights: it ties each wrong prediction to a responsible agent or stage, then fixes it through a prompt patch that has to pass validation on a held-out set. The aim is an emotion-reasoning system that stays traceable as it improves.

Who it affects

Researchers working on multimodal emotion reasoning and on agentic pipelines built from frozen models are the direct audience. The method was tested on four backbones from 2B to 38B parameters, so the idea is pitched across model sizes rather than one model.

How to use it

The abstract describes the design rather than a ready-to-run tool. The recipe: have perception agents post evidence-cited observations, let an integration agent combine them with retrieved multimodal examples, allow the critic one revision, and for wrong predictions run the verification checks, group failures by component and type, and turn them into prompt patches accepted only if they pass held-out validation.

How solid is it

The results come from the authors' own abstract on a single dataset, MER2025, across four backbones. The gains are small in absolute terms (0.6 to 2.2 F-score points), and against undifferentiated reflection baselines the claim is only that the method matches or outperforms them on each backbone, not that it strictly wins everywhere. Beyond the Qwen2.5-VL-7B figure (52.3 to 54.5), per-backbone numbers are not given. It is a preprint.

Risks and caveats

The F-score type and evaluation setup are not specified in the text, and no results on other datasets are mentioned. The size of the held-out validation set and the number of prompt patches are not given. The abstract also reports no code release, compute cost or runtime figures, so the cost advantage over instruction tuning is stated as a design property rather than measured.