VQActFlow: Vector-Quantized Action Mode Steering
for Multi-Task Robot Manipulation

Humanoid whole-body manipulation

A Unitree G1 with two dexterous hands runs four pick-and-place tasks in one shared workspace; only the target object and the instruction change.

Almost all of CFG's gain here comes from fixing wrong-task errors, which fall from 41.3% to 2.5%. It helps the policy choose the right task; it does not make the motion itself any cleaner.

A CFM policy trained the same way (same DiT backbone, data, and training protocol) converges in training but oscillates in every G1 rollout and never completes a grasp, with or without CFG. The oscillation shakes the robot's torso, so we stopped its evaluation early for safety and count its success rate as 0%.

Bimanual manipulation

Two ALOHA-style arms on four contact-rich tasks, three of them needing coordinated bimanual use.

Each row is 400 real-robot rollouts, run in five sessions on different days. With CFG and the critic, VQActFlow beats Discrete Policy at the same weight by 11.7 points (95% CI [7.4, 16.0], p < 0.001). The critic raises success on every task at both weights (8 of 8 comparisons, sign test p = 0.008), though neither average gain is significant on its own at this number of trials. Ball varies most between sessions because the red ball is sensitive to lighting. These results use a revised gripper: after the change we re-collected about 100 demonstrations per task and retrained every policy.

With the same backbone, a continuous flow-matching baseline fails all four tasks. Its joints oscillate violently, roughly two orders of magnitude rougher than VQActFlow.

350 ms fits comfortably inside the 1.28 s budget per action chunk, and asynchronous inference computes the next chunk while the current one runs.

CFG on LIBERO-Goal

Guiding the discrete distribution directly works better than guiding a continuous approximation of it.

Multi-task scaling on LIBERO-90

At this scale, CFG and the codebook critic each contribute their own gains, and the gains mostly stack.

CFG helps both kinds of policy: at w=2 it adds 2.1 points to CFM and 4.8 to VQActFlow. The critic adds 2.4 points on its own and 2.7 more on top of CFG. VQActFlow's 1.0-point margin over CFM with CFG (95% CI [−0.5, 2.4]) is within noise, while its margins over the other baselines are significant.

Success peaks at K=512. Smaller codebooks limit how finely actions can be quantized; larger ones blow up the per-position classification space and make the categorical prediction harder to learn.

Results hold across training seeds. The spread across three independently trained models (0.3–1.3 points) is no larger than the spread across evaluation seeds in Table IV. Across the retrained models, CFG adds 6.5 points (95% CI [5.3, 7.6]) and the critic adds 1.7 at w=1 ([1.5, 1.9]). On top of CFG the critic helps for all three seeds (+2.0, +1.2, +0.5), though three seeds are too few to resolve that gain.

The response to guidance has the same shape as on LIBERO-Goal: clear gains at w=1.5 and 2, none at w=3, and a loss at w=4 (95% CI [−7.4, −0.9]). The w=2 used here sits at the optimum, and the plateau from 1.5 to 2 means the choice needs no fine tuning. Each benchmark uses one w for all of its tasks, picked by a coarse sweep. Benchmarks whose tasks share one fixed scene (LIBERO-Goal, the G1, the bimanual platform) need stronger guidance, because only the instruction tells their tasks apart.

Every λ from 0.5 to 3 beats running without the critic by 2–3 points in every evaluation seed, and no two values are separable, so the result does not depend on a finely tuned critic weight.

Replacing the instruction with a learned task ID, one per unique instruction (74 IDs, so both carry the same information), lowers success at every guidance setting: by 11.3 points at w=1 and 3.9 at w=2. The policy uses what the instruction says, not just which instruction it is. CFG closes part of the gap, and the critic still helps with task IDs (+1.8 at w=1, +1.4 at w=2). All instructions here were seen in training; generalization to new instructions is not tested.

What the codebook critic learns

The critic is trained only on demonstrations and constructed negatives. On LIBERO-90 we check whether its scores track real rollout outcomes, and which negatives its guidance depends on.

We scored 4,500 rollouts per guidance weight, labeling each action chunk as failed if it falls at or after the point where a failed episode goes wrong. The ranking also holds within each task: the per-task AUROC averages 0.85 at w=1 and 0.83 at w=2, and all 90 tasks are above chance.

Ranking outcomes well is not the same as guiding well. The critic trained without code-replacement negatives ranks rollouts as well as the full critic (0.83) but, used as guidance, drops success to 66.4%. Random replacement (R) and cross-demonstration swap (X) are the only negatives that put codes from outside the episode at a fixed observation and instruction, and the gradient needs both: keeping them without any conditioning negatives retains the full gain (78.7). The language and observation negatives supply task discrimination (0.97 with them, 0.68 without) but are not needed for the success gain, since CFG already conditions on the task. The critic's useful signal is feasibility: whether an action is right for this instruction in this scene.