From Passive Observer to Active Critic:
Reinforcement Learning Elicits Process Reasoning
for Robotic Manipulation

ECCV 2026
Yibin Liu1,2,* Yaxing Lyu3,* Daqi Gao1,* Zhixuan Liang4 Weiliang Tang5 Shilong Mu6 Xiaokang Yang1 Yao Mu1,†
1Shanghai Jiao Tong University   2Northeastern University   3Xiamen University Malaysia
4The University of Hong Kong   5The Chinese University of Hong Kong   6Xspark AI
*Equal contribution   †Corresponding author
 Paper  Model/Dataset  Code  BibTeX
Paradigm comparison: prior approaches vs. PRIMO R1
Paradigm comparison: prior approaches vs. our PRIMO R1. Existing video MLLMs act as passive Observers that describe what is happening; PRIMO R1 is trained as an active Critic that reasons explicitly about how far a task has progressed toward its goal.
01

A 7B Critic, not an Observer

PRIMO R1 turns a video MLLM from a passive Observer into an active Critic. Outcome-based reinforcement learning incentivizes explicit Chain-of-Thought for progress estimation, reducing the mean absolute error of specialized baselines by 50%.

02

PRIMO Dataset & Benchmark

A two-stage post-training corpus with Chain-of-Thought annotations: a 116k-sample SFT set and a 182k-sample RL set, paired with PRIMO Bench to evaluate in-domain and out-of-domain generalization across cross-task and cross-environment settings.

03

Structured temporal input

Anchoring the video sequence between initial and current state images turns generic temporal perception into structured state-alignment verification, and yields strong zero-shot failure detection: a state-of-the-art 67.0% on RoboFail.

Abstract

Accurate process supervision remains a critical challenge for long-horizon robotic manipulation. A primary bottleneck is that current video MLLMs, trained primarily under a Supervised Fine-Tuning (SFT) paradigm, function as passive “Observers” that recognize ongoing events rather than evaluating the current state relative to the final task goal. In this paper, we introduce PRIMO R1 (Process Reasoning Induced MOnitoring), a 7B framework that transforms video MLLMs into active “Critics”. We leverage outcome-based Reinforcement Learning to incentivize explicit Chain-of-Thought generation for progress estimation. Furthermore, our architecture constructs a structured temporal input by explicitly anchoring the video sequence between initial and current state images. Supported by the proposed PRIMO Dataset and Benchmark, extensive experiments across diverse in-domain environments and out-of-domain real-world humanoid scenarios demonstrate that PRIMO R1 achieves state-of-the-art performance. Quantitatively, our 7B model achieves a 50% reduction in the mean absolute error of specialized reasoning baselines, demonstrating significant relative accuracy improvements over 72B-scale general MLLMs. Furthermore, PRIMO R1 exhibits strong zero-shot generalization on difficult failure detection tasks. We establish state-of-the-art performance on the RoboFail benchmark with 67.0% accuracy, surpassing closed-source models like OpenAI o1 by 6.0%.

Method

Overall framework of PRIMO R1
Overall framework of PRIMO R1. Evaluated across in-domain simulations (AgiBot, BEHAVIOR, RoboTwin) and out-of-domain real humanoid environments, the model processes a video sequence (Vseq) anchored by initial (Iinit) and current (Icurr) states. It generates an explicit Chain-of-Thought to output the final progress estimate.

From Observer to Critic

We formalize robotic process supervision as a multi-modal state estimation problem. Given an initial state image Iinit, a process video sequence Vseq, a current state image Icurr, and a language instruction ℬ specifying the task goal, the model outputs a scalar progress indicator y ∈ [0, 100], where y=0 denotes the initial state and y=100 signifies success. In the standard paradigm, video MLLMs act as passive Observers, treating progress estimation as direct regression or classification through SFT, which isolates visual features at a surface level and bypasses the causal structure of state transitions.

To build an active Critic, we reformulate prediction from direct scalar regression into a multi-step generative reasoning task. A policy πθ sequentially generates a latent reasoning chain (Chain-of-Thought) followed by the final progress estimate. Rather than supervising the intermediate reasoning with dense annotations, we optimize the policy with Reinforcement Learning against a reward computed solely from the accuracy of the final prediction. Conditioning this reasoning on diverse natural language task goals connects the semantic objective to the visual execution logic, exploiting the linguistic generalization of foundational MLLMs.

Process Reasoning RL with GRPO

We employ Group Relative Policy Optimization (GRPO). For each task tuple, we sample a group of outputs and estimate the advantage by normalizing each reward against the group distribution, avoiding a separate value network whose memory overhead is prohibitive for video MLLMs. A composite rule-based reward combines a format reward, which enforces the <think>reasoning</think><answer>prediction</answer> structure and prevents collapse into direct guessing, with a bounded linear-decay accuracy reward that provides dense feedback for numerical reasoning. A KL penalty to the reference policy guards against reward hacking and language degeneration. PRIMO R1 thereby learns that detailed causal reasoning is the most reliable strategy for maximizing the accuracy reward, emerging as a robust Critic.

Results

Main Results: Generalization in Progress Estimation

Across four environments, PRIMO R1 attains the highest average Mean Relative Accuracy (MRA 82.90) and the lowest average Mean Absolute Error (MAE 15.52). Despite a 7B foundation, it surpasses Qwen2.5-VL-72B by +9.10 absolute MRA points and roughly halves the error of specialized reasoning and video MLLMs.

Model AgiBot Behavior RoboTwin Real Humanoid Average
MRA ↑MAE ↓ MRA ↑MAE ↓ MRA ↑MAE ↓ MRA ↑MAE ↓ MRA ↑MAE ↓
Closed-Source Models
GPT-5 mini 74.2724.8179.6020.0880.5218.3467.1432.5975.3823.96
GPT-4o 81.0118.9979.9220.0881.7318.2774.6525.3579.3320.67
Gemini 2.5 Flash 73.5426.4178.0121.9981.0418.5867.3732.6374.9924.90
Claude-Haiku-4.5 74.4025.5970.9329.0774.1325.8772.6827.3273.0426.96
Open-Source General MLLMs
Qwen2.5-VL-7B 77.4322.5669.9130.0667.3732.6256.4634.7367.7929.99
InternVL 3.5 8B 78.5221.4772.1927.1570.8129.1865.4434.5571.7428.09
Qwen2.5-VL-72B 78.0022.0079.4920.5075.4124.5962.2928.1073.8023.80
Reasoning & Video MLLMs
ProgressLM-3B-RL 42.9030.8346.4328.2331.8436.4832.0034.9038.2932.61
Video R1 7B 72.4227.5870.6329.3770.8629.1453.5731.8766.8729.49
Robobrain 7B 72.9925.9172.5226.9770.4128.8555.8328.5167.9427.56
Cosmos-Reasoning 7B 72.4827.0167.0632.3573.1425.8559.3931.4166.5229.12
Specialized Progress & Reward Models (own input configuration)
VLAC 84.9215.0876.8623.1467.0432.9670.7729.2374.9025.10
GVL (Gemini 2.5 Flash) 56.8135.8458.1132.3161.1827.9148.7738.3756.2233.61
Robo-Dopamine-3B 81.1818.8257.7842.2282.9617.0468.4331.5772.5927.41
Robo-Dopamine-4B 83.8816.1265.3034.7067.2232.7873.8926.1172.5727.43
ProgressLM 81.1318.7980.8915.9767.0332.9784.2415.7678.3220.87
PRIMO R1 (Ours) 87.6712.33 87.0812.90 84.5215.48 72.3221.37 82.9015.52

MRA ↑ higher is better; MAE ↓ lower is better. Best results in bold. Specialized progress and reward models (VLAC, GVL, Robo-Dopamine, ProgressLM) are evaluated under their own recommended input configurations and are therefore not strictly comparable cell-for-cell with models run under our unified protocol; the ProgressLM-3B-RL row in the Reasoning & Video group is the same model re-evaluated under our unified setting.

Fine-grained MAE across task progress intervals
Fine-grained error analysis across task progress intervals. MAE across five completion stages in four environments. While baselines exhibit severe error spikes in the final execution stage (80–100%), PRIMO R1 maintains consistently low error, avoiding premature hallucination of task completion.

Generalization Enhancement in Failure Detection

Optimizing a policy for continuous progress reasoning implicitly constructs the temporal context needed for discrete failure detection. On the entirely unseen RoboFail benchmark, PRIMO R1 reaches a state-of-the-art 67.0% accuracy — matching closed-source Gemini 2.0 Flash and surpassing GPT-4o (63.0%), OpenAI o1 (61.0%), and Cosmos-Reason1-56B (66.2%). SFT alone regresses to 51.0% through format overfitting; GRPO recovers genuine reasoning and lifts accuracy to 67.0%.

Closed-SourceRoboFail ↑ Open-SourceRoboFail ↑ OursRoboFail ↑
Gemini 2.0 Flash67.0 Qwen2.5-VL-7B57.6 PRIMO (SFT)51.0
GPT-4o63.0 Nemotron-H-56B64.0 PRIMO (RL)63.0
OpenAI o161.0 Cosmos-Reason1-7B60.0 PRIMO R167.0
Claude-haiku-4.559.0 Cosmos-Reason1-56B66.2

Accuracy (%) on the RoboFail benchmark, measuring the capability to detect and quantify task execution failures. Best result in bold.

Ablation Studies

Impact of RL on Generalization

SFT improves overall performance but primarily overfits the training distribution; RL-only struggles to discover the correct output format from scratch. The full SFT+RL pipeline creates a synergy that pushes in-domain accuracy near 90% and drastically improves out-of-domain generalization, lifting cross-environment (Real Humanoid) accuracy to 72.32%.

Model In-Domain — Seen Tasks OOD — Cross-Task OOD — Cross-Env Avg.
AgiBotBehaviorRoboTwin AgiBotBehaviorRoboTwin Real Humanoid
Qwen2.5-VL-7B (Base) 70.8369.1371.1974.4577.4761.0148.1267.46
Our Model (SFT) 83.3780.3880.6382.0282.6179.1367.3079.35
Our Model (RL) 86.0585.8273.2782.9581.3975.4352.1276.72
PRIMO R1 (SFT+RL) 87.8389.4288.15 87.6787.0884.52 72.3285.28

All metrics are Mean Relative Accuracy (MRA ↑). Best in bold.

The Necessity of Temporal Context

Varying the input modalities shows that temporal context is essential. Relying on the current state alone yields the highest error (Avg. MAE 59.50). The final PRIMO R1 configuration integrates all three signals — Iinit, Vseq, and Icurr — which is necessary for tracking progress over long horizons: on the long-horizon Behavior dataset it reduces MAE to 22.73 and raises Acc@10 to 31.83.

Input Modality AgiBot Behavior RoboTwin Avg
IinitVseqIcurr MAE ↓Acc@10 ↑ MAE ↓Acc@10 ↑ MAE ↓Acc@10 ↑ MAE ↓Acc@10 ↑
✓ 59.640.0051.918.1766.940.7759.502.98
✓✓ 43.9718.5249.599.7345.9311.4546.5013.23
✓ 27.5831.3434.8518.3347.418.0736.6119.25
✓✓ 25.0435.2927.5929.2140.2417.3730.9627.29
✓✓ 24.9433.9832.5523.0145.3711.9734.2922.99
✓✓✓ 29.3927.1522.7331.8342.1627.6931.4328.89

Iinit: initial state image; Vseq: process video clip; Icurr: current state image. Best in bold.

PRIMO Dataset & Benchmark

PRIMO dataset distribution
Dataset distribution for SFT (left), RL (middle), and PRIMO Bench (right). Charts show sample counts and domain percentages (e.g., BEHAVIOR, AgiBot, RoboTwin). PRIMO Bench highlights the split between In-Domain and Out-of-Domain evaluation sets.

The PRIMO Dataset supports a two-stage post-training paradigm. Unlike standard video QA datasets, it features fine-grained progress indicators annotated with Chain-of-Thought reasoning paths, aggregating trajectories from a real-world environment (AgiBot) and two high-fidelity simulations (BEHAVIOR-1k and RoboTwin), augmented with general video reasoning data. It is partitioned into a 116k-sample SFT set (PRIMO-R1-CoT-116k) and a 182k-sample RL set (PRIMO-R1-182k).

The accompanying PRIMO Bench evaluates robustness under distribution shift via two splits: an In-Domain (Same Task) split over seen task categories in the three training environments, and an Out-of-Domain split covering both Cross-Task (unseen tasks in familiar environments) and Cross-Environment (real-world trajectories teleoperated on a different physical humanoid, Leju KUAVO-MY, in unstructured factory and service scenarios) generalization.

Case Study

Continuous progress estimation trajectory
Continuous progress estimation. Average predicted progress trajectory over 105 episodes for the “Clear the countertop waste” task. PRIMO R1 maintains a monotonically increasing trajectory that correlates linearly with the ground truth, while baselines overshoot early or plateau near 60–70%.
Qualitative comparison of reasoning processes
Qualitative comparison of reasoning processes. Generated reasoning chains of Qwen2.5-VL-7B, RoboBrain-7B, and PRIMO R1 during the “Fold shorts” task in AgiBot. PRIMO R1 reasons via Planning, Observation, and Reasoning modules, mapping verified execution steps against a reference plan to produce a calibrated estimate.
Real-humanoid case study
Real-world humanoid case. PRIMO R1 transfers its explicit process reasoning to the unseen Real Humanoid environment, bridging the distribution gap between simulation and the real world.

Concurrent Work

Given the pace of AI research these days, it is extremely challenging to keep up with all of the work around robot task progress estimation using foundation models. We list below a few key approaches developed concurrently with PRIMO R1. VanceChu's Awesome-Progress-Reward-Model paper list is also a useful entry point for understanding this direction.

  • Robometer combines frame-level progress supervision with inter-trajectory preference supervision and curates RBM-1M (over one million trajectories spanning diverse embodiments, including suboptimal and failed data) to learn reward functions that generalize from both real and augmented failed trajectories.
  • Robo-Dopamine learns a step-aware, multi-view General Reward Model on 3,400+ hours of data and pairs it with a theoretically-sound policy-invariant reward shaping method (Dopamine-RL) for efficient real-world policy self-improvement.
  • RoboReward introduces a reward dataset and benchmark built on Open X-Embodiment and RoboArena with a negative-example augmentation pipeline, and trains 4B/8B vision-language reward models that outperform much larger VLMs on short-horizon robotic tasks.
  • ProgressLM proposes PROGRESS-BENCH and a two-stage progress reasoning paradigm, showing that the training-based ProgressLM-3B improves progress estimation even at a small scale despite a train/eval task split.
  • VLAC builds a general process reward model on InternVL that outputs dense progress deltas and done signals from pairwise observations, unifying critic and policy within an asynchronous real-world RL loop with a human-in-the-loop protocol.

BibTeX

@article{liu2026passive,
  title={From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation},
  author={Liu, Yibin and Lyu, Yaxing and Gao, Daqi and Liang, Zhixuan and Tang, Weiliang and Mu, Shilong and Yang, Xiaokang and Mu, Yao},
  journal={arXiv preprint arXiv:2603.15600},
  year={2026}
}