AFFECT: Action Effect Matching

Learning What Actions Change: Action Effect Matching for Dynamics-Faithful World Models

Learning What Actions Change:
Action Effect Matching for Dynamics-Faithful World Models

Preprint
1KAIST  •  2RLWRLD

TL;DR

AFFECT is an auxiliary training objective that teaches robot world models how small action changes alter the future: generated videos are trained to respond to logged action perturbations the same way recorded videos do, making world models more faithful for policy evaluation and planning.

Motivation

To serve as policy evaluators or planners, robot world models must reflect how small action changes alter future outcomes. A slight change in reaching motion can decide whether a gripper grasps a cube, misses it, or pushes it away. Yet existing models tend to produce visually plausible videos that ignore such differences, predicting similar futures for actions with very different real consequences. We therefore treat action sensitivity as a distributional problem: a world model should capture how the distribution of future observations shifts as actions vary under a given context.

From the same initial state, actions drawn from a narrow distribution change the gripper's approach only slightly, yet produce qualitatively different outcomes: the cube is grasped, missed, or pushed away.

Method

AFFECT asks generated videos to respond to action perturbations the same way recorded videos do, rather than asking each generated video to match its recorded counterpart. This needs interaction data in which actions are perturbed in a known way, and a loss that measures the response to those perturbations.

1. Perturbation play

Robot scene Play policy producing nominal actions Random perturbation Perturbed actions executed on the robot Recorded future World model generating a future Frozen encoders and action effects Distributional matching of action effects
Step 1. The play policy proposes nominal actions, logged random noise perturbs them, and the robot executes the perturbed actions to record a future.

We train a play policy on task-agnostic human teleoperation and roll it out autonomously, which needs no scene resets or task-specific procedures. Before execution, we inject temporally correlated noise into each nominal action chunk and log the perturbation; the robot executes the perturbed chunk and we record the resulting future. Each window needs just a single recorded future, so no repeated executions from the same initial state are required.

2. Action effect encoding

Robot scene Play policy producing nominal actions Random perturbation Perturbed actions executed on the robot Recorded future World model generating a future Frozen encoders and action effects Distributional matching of action effects
Step 2. The world model generates a future from the same context and actions. Frozen encoders φ and ψ summarize the videos and the logged perturbation, and their products give the action effects.

The world model sees only the context frame and the executed actions, and generates a future from fresh noise with a short differentiable sampler, never conditioned on the recorded future. Two frozen encoders then summarize the videos (φ) and the logged perturbation (ψ) with simple, hand-specified features. After centering the perturbation features by their expected value, the action effect is the product of perturbation and video features: positive when larger-than-expected perturbations come with larger values of a video feature. Keeping the encoders frozen prevents a trivial solution, and lets AFFECT plug into existing world models without architectural changes.

3. Distributional matching

Robot scene Play policy producing nominal actions Random perturbation Perturbed actions executed on the robot Recorded future World model generating a future Frozen encoders and action effects Distributional matching of action effects
Step 3. AFFECT aligns the distribution of generated action effects with the recorded ones, so generated futures respond to perturbations in the same direction and with the same strength as real ones.

Averaged over many windows, the action effects reveal how the scene responds to actions. AFFECT trains the world model so that the action effects of generated videos match those of recorded videos:

$$\mathcal{L}_{\mathrm{AFFECT}} = \sum_{i,j}\left(\frac{\mathbb{E}\big[g_{t,i}\,\big(\phi_j(O_t)-\phi_j(\hat{O}_t)\big)\big]}{s_{ij}}\right)^{2}, \qquad \mathcal{L} = \mathcal{L}_{\mathrm{flow}} + \lambda\,\mathcal{L}_{\mathrm{AFFECT}}.$$

Centering turns each action effect into a conditional covariance between perturbation and video features, so scene-only errors (e.g. rendering a scene slightly too dark) cancel out, and the loss compares only the direction and strength of the response to actions. The full objective is a finite-feature maximum mean discrepancy over action effects. Unlike plain feature matching, it catches a model that moves an object left whenever the real object moves right.

Policy Evaluation Results

We apply AFFECT to two state-of-the-art robot world models, DreamDojo and Cosmos3, and run real policies in closed loop inside each world model. If the world model is faithful, policies that succeed more often on the real robot should also succeed more often in imagination.

Does the world model rank policies like the real robot?

Pearson correlation between policy success rates in the world model and in the real environment. AFFECT improves it in all six backbone–embodiment combinations.

Are the generated videos closer to reality?

Gain in long-horizon PSNR (dB) after adding AFFECT. AFFECT improves it in all six combinations.

Show full results tables (PSNR, LPIPS, FVD, Pearson, and other baselines)

Real robots: DROID (Franka) and ALOHA-2

Method DROID (Real) ALOHA-2 (Real)
PSNR ↑ LPIPS ↓ FVD ↓ Pearson ↑ PSNR ↑ LPIPS ↓ FVD ↓ Pearson ↑
iVideoGPT 16.16 0.52 1946 0.151 16.77 0.52 3808 0.155
CoCo 18.13 0.50 1676 0.212 19.09 0.49 3959 0.225
Ctrl-World 17.48 0.36 1141 0.236 20.32 0.11 396 0.399
Cosmos3 18.69 0.24 910 0.425 22.94 0.08 333 0.298
+ AFFECT 19.67 0.22 744 0.583 24.05 0.05 257 0.694
DreamDojo 18.53 0.16 807 0.468 24.20 0.06 183 0.325
+ AFFECT 19.04 0.14 622 0.790 24.91 0.05 188 0.490
Long-horizon autoregressive video fidelity and Pearson correlation of real vs. generated policy success rates.

Simulation: OGBench UR5e

Method Pearson ↑ Short Horizon Long Horizon
PSNR ↑ LPIPS ↓ FVD ↓ PSNR ↑ LPIPS ↓ FVD ↓
iVideoGPT 0.334 21.53 0.49 1134 20.20 0.53 1314
CoCo 0.130 21.69 0.49 1094 20.29 0.52 1331
Ctrl-World 0.310 24.99 0.10 156 24.27 0.11 196
Cosmos3 0.091 19.07 0.19 470 18.72 0.22 562
+ AFFECT 0.334 19.95 0.18 463 19.21 0.20 540
DreamDojo 0.451 26.03 0.06 85 24.87 0.08 108
+ AFFECT 0.686 26.78 0.05 73 25.58 0.07 93
Single-window (short horizon) and autoregressive (long horizon) video fidelity in simulation, with Pearson correlation on OGBench pick-and-place tasks.

Planning with Action-Faithful World Models

Five policy checkpoints propose candidate action chunks; the world model imagines a video for each, and a learned success estimator picks the best one. With the proposals and estimator held fixed, AFFECT raises average success from 60.0% to 68.3% on UR5e pick-and-place and from 72.2% to 80.6% on a long-horizon ALOHA-2 task (open a trash bin, then throw away two items).

Uniform picks a candidate at random without a world model; DreamDojo and DreamDojo + AFFECT imagine each candidate and pick the highest-scoring one.
Show planning results table
Task Uniform DreamDojo DreamDojo + AFFECT
UR5e · Vertical2 33.3 25.0 33.3
UR5e · Horizontal 50.0 83.3 83.3
UR5e · Diagonal1 33.3 58.3 66.7
UR5e · Diagonal2 50.0 58.3 75.0
UR5e · Lateral Near 66.7 75.0 83.3
UR5e · Average 46.7 60.0 68.3
ALOHA-2 · Subtask 1 83.3 91.7 91.7
ALOHA-2 · Subtask 2 58.3 66.7 83.3
ALOHA-2 · Subtask 3 25.0 58.3 66.7
ALOHA-2 · Average 55.6 72.2 80.6

Where Do the Features See Action Effects?

On recorded UR5e videos, we measure how consistently perturbation–feature products keep the same sign across episodes at each latent cell, relative to a baseline that pairs perturbations with videos from other windows. Responses are 1.6–2.2× stronger than this shuffled baseline in regions traversed by the arm, cube, or shadow, and stay at chance on the static table and backdrop, so the frozen features provide a localized action-effect signal.

Frozen-feature responses relative to shuffled pairs; brighter cells indicate stronger responses.

BibTeX

@article{won2026affect,
  title={Learning What Actions Change: Action Effect Matching for Dynamics-Faithful World Models},
  author={Won, John and Dong, Yonghoon and Shin, Jinwoo},
  journal={arXiv preprint arXiv:2611.25980},
  year={2026}
}