Robot learning: vision-language-action models and sim-to-real

You have a vision-language model (VLM) that answers questions about images, and you want it to drive a robot arm directly: two camera views and "put the cup in the sink" go in, joint commands at 50 Hz come out. The most direct approach is to write actions as text tokens too, but a quick latency estimate shows it cannot keep up. As of October 2026, the common design lets the VLM understand the scene and the instruction while a continuous flow-matching "action expert" beside it generates a whole chunk of actions at once; humanoid robots further split slow reasoning from fast control into two or three layers. Policies trained in simulation reach real robots through randomization and online adaptation, and evaluation is moving from near-saturated simulation leaderboards to real-robot comparisons.

Draft, pending review. The site owner has not reviewed this note yet; facts, emphasis and wording may change. Check the primary sources linked in the text before relying on it.

Read first: Learning actions from demonstrations (multimodal actions, action chunking) and LLaVA and DeepSeek-VL (how images enter a language model).

How actions are written: four designs

The term VLA (vision-language-action) comes from RT-2 (Google DeepMind, 2023-07). To tell VLAs apart, first ask where the action comes out of the model and in what form.

1. Actions as text tokens. RT-2's action is 8 numbers: the end effector's 6-DoF translation and rotation, gripper extension, and a "terminate" flag; the continuous dimensions are split uniformly into 256 bins, the whole action is written as a string of integer tokens, and the model is co-fine-tuned together with web visual question-answering data (§3.2). The 55B version runs on a multi-TPU cloud service at 1–3 Hz, the 5B version at about 5 Hz (§3.3). OpenVLA (Stanford-led, 2024-06) follows the same idea: Llama 2 7B with DINOv2 and SigLIP visual features; 7-D actions, each dimension split into 256 bins between the 1st and 99th percentile of the training data, taking over the 256 least-used tokens in the Llama vocabulary; about 6 Hz on one RTX 4090 (without compilation or speculative decoding).

2. Compress first, then tokenize. The observation behind FAST (Physical Intelligence, UC Berkeley, Stanford, 2025-01): at high control rates consecutive steps are nearly identical, so most per-step, per-dimension tokens can be predicted by "copying the previous one", and the next-token learning signal becomes weak: "this marginal information approaches zero as the control frequency of the training signal increases …" (§IV). FAST applies a discrete cosine transform (DCT) to each dimension, quantizes, and compresses with BPE. Tokens per 1-second action chunk as reported by the authors: table bussing (7-D, 20 Hz) drops from 140 to 28; T-shirt folding (14-D, 50 Hz) from 700 to 53 (Table I). π0-FAST matches the diffusion version of π0 with up to 5x less training time (authors).

3. A continuous action expert. π0 (2024-10) = PaliGemma 3B + a flow-matching action expert of about 300M parameters; chunk length 50, up to 50 Hz; at inference it integrates 10 steps, computing the prefix of images, text and robot state once, caching its keys and values, and recomputing only the action part at each step (§IV). π0.5 (2025-04) first predicts the next subtask as text, then the 300M expert outputs a 50-step chunk; π0.7 (2026-04) moves to Gemma 3 4B with an 860M expert and adds metadata such as quality and speed, plus subgoal images, to the prompt. NVIDIA's GR00T N1 (2025-03) trains a VLM (System 2) and a diffusion-transformer action module (System 1) jointly end to end; Hugging Face's SmolVLA (2025-06) has about 450M parameters in total, about 100M of them in the flow-matching expert. Attaching a randomly initialized continuous expert to a VLM has a cost: PI's knowledge insulation (2025-05) reports that it markedly slows training and hurts knowledge transfer; the body text (§1, §5.2) explains that gradients from the expert damage the VLM backbone, and the fix is to insulate the backbone from those gradients.

4. Neither tokens nor diffusion. OpenVLA-OFT (Stanford, 2025-02) changes OpenVLA to parallel decoding (bidirectional attention, a whole chunk in one forward pass), continuous actions and L1 regression. The authors report the LIBERO average rising from 76.5% to 97.1% (the best configuration uses extra inputs) and action-generation throughput rising 26-fold (measured with 8-step chunks).

Worked example: how long one chunk takes (teaching numbers)

The per-step costs below are teaching assumptions chosen so the orders of magnitude can be computed by hand; they are not measurements of any model:

• Prefix: images, instruction and robot state through the VLM once, $T_{\rm prefix}=40$ ms, the same for all three designs;
• Autoregressive decoding: each token from a backbone of about 3B parameters, $c_{\rm tok}=10$ ms;
• One flow step: each integration step of an action expert of about 300M parameters (prefix keys and values cached), $c_{\rm flow}=3$ ms;
• Action chunk: 50 steps × 7 dimensions, exactly 1 second of motion at 50 Hz; after FAST compression, assume 40 tokens (the authors say typically 30–60).

$$ T_{\rm AR}=T_{\rm prefix}+N_{\rm tok}\,c_{\rm tok},\qquad T_{\rm flow}=T_{\rm prefix}+K\,c_{\rm flow} $$

$N_{\rm tok}$ is the number of tokens generated sequentially and $K$ the number of flow integration steps. Plugging in:

Per-dimension tokens are about $3540/70\approx51$ times slower than flow, FAST about $440/70\approx6$ times. If per-dimension tokens skip chunking and output one step at a time, it takes $40+7\times10=110$ ms, at most about 9 Hz, the same single-digit-Hz range that RT-2 (1–3 Hz) and OpenVLA (about 6 Hz) measured.

Now compare with real measurements. The π0 paper (v1 Appendix D, Table I) measured on an RTX 4090 with 3 cameras: image encoders 14 ms, prefix forward pass 32 ms, 10 flow steps 27 ms in total, 73 ms on board; for a mobile robot running inference off board over WiFi, network latency adds 13 ms. The FAST paper (§VI-E) reports that, also on an RTX 4090, the diffusion π0 produces a 1-second chunk in under 100 ms while π0-FAST needs about 750 ms per chunk, because it decodes 30–60 tokens and each one goes through the full 2B-parameter backbone rather than a 300M expert. The authors also say this latency did not hurt performance on the static manipulation tasks they tested.

This figure needs JavaScript.

What this example does not tell you:

• Sequential steps and policy quality are separate questions: π0-FAST takes far more sequential steps, and its authors report that it matches π0 in success rate.
• Per-step cost varies widely with GPU, quantization, batch and implementation; comparisons must state hardware and configuration.
• In the flow design the prefix dominates: $40/70\approx57\%$ in the teaching numbers, and image encoders plus prefix are $46/73\approx63\%$ in π0's measurement. To go faster, shrink the prefix first.
• Real systems overlap inference with execution: on 50 Hz robots π0 infers the next chunk after executing 25 steps (0.5 s); SmolVLA's asynchronous inference cuts task completion time from 13.75 s to 9.7 s (paper §4.6, author-reported).

Slow and fast layers: why humanoids split into two or three

Balancing and grasping need feedback within milliseconds, while the meaning of "put the cup in the sink" changes only every few seconds. A model with billions of parameters cannot run at 200 Hz, so the work is split: a slow layer understands the scene and the instruction, a fast layer turns that into joint commands.

The rhythm between layers: take S2 at 8 Hz, updating every 125 ms; meanwhile S1 at 200 Hz runs $125/5=25$ times, and S0 at 1 kHz runs 5 times within each S1 step, 125 times in total. The cost: when the slow layer misreads the scene, the error persists for a whole slow cycle, and what passes between layers (a latent vector, a text subtask or joint targets) is rarely described in detail.

RL on top of VLAs: learning what demonstrations lack

Demonstrations can only teach up to the demonstrator's level, and speed is capped by the demonstrator too; demonstrations also rarely show how to carry on after a slip or a jam. In 2025–26 Physical Intelligence began running RL on VLAs with the robot's own experience (all self-reported):

• π*0.6 / Recap (2025-11-17). A value function is trained first and used to estimate the advantage of each piece of experience (how much better than average a step was, see Actor–Critic and GAE); π0.6 then learns conditioned on that signal, with an offline-RL pretraining stage (see Offline RL and OPE). The data includes demonstrations, the robot's own experience and corrections from teleoperators. PI reports more than double the throughput on some of the hardest tasks and failure rates cut by half or more, and shows the robot making espresso from 5:30 am to 11:30 pm.
• RL tokens (2026-03-19). The VLA is first trained to output an "RL token" and then frozen; a small actor-critic trained online on that token edits the actions the VLA proposes. PI reports the most precise phases running up to 3x faster with as little as 15 minutes of real-robot data; it tested 4 precision tasks (an M3 screw, a zip tie, an Ethernet plug and a power cord), with RL targeting their critical phases.
• π0.7 (2026-04-16). PI says this generalist model matches the RL-trained π*0.6 specialists on laundry folding, espresso and box folding; the conclusion of the paper also acknowledges that unseen tasks succeed only 60%–80% of the time.

For legged or whole-body controllers trained in simulation, common RL algorithms are PPO and SAC, see How much should a policy change? and SAC; Helix 02's S0 was also trained in simulation.

Evaluation: after scores reach 97%

• LIBERO (UT Austin and others, 2023-06): 130 simulated manipulation tasks in total. VLA papers usually evaluate on 4 suites (Spatial, Object, Goal, Long) of 10 tasks each; OpenVLA-OFT's 97.1% is the average over these 40 tasks. With scores this high, strong methods are hard to separate; "LIBERO is near its ceiling" is our judgement, not a conclusion of any paper.
• SIMPLER (2024-05): recreates real tabletops and robots in simulation to evaluate policies trained on real-robot data; the authors report strong correlation between simulated and real results.
• RoboArena (2025-06): distributed, double-blind pairwise real-robot comparisons on the DROID platform, with 7 academic institutions, 600+ pairwise episodes and 7 policies.

Why the top of the scale is hard to compare, by hand. OFT's protocol runs 500 trials per suite (10 tasks × 50), 2,000 over the 4 suites. At a success rate of 0.97 the standard error is $\sqrt{0.97\times0.03/2000}\approx0.0038$, so a 95% interval is about ±0.75 percentage points; for two methods 0.5 points apart, the standard error of the difference is about $\sqrt2\times0.0038\approx0.0054$, so the gap is less than one standard error. (This treats the 2,000 trials as independent; if trials within a task are correlated, the interval is wider still.) For how to compare two scores, see The score improved. What does that actually establish?.

Using world models instead of real robots as evaluators is a separate route, covered in world models as evaluators for robots and cars.

Sim-to-real: where the gap comes from and how to close it

Simulation is cheap, parallel and safe to crash in, but friction, mass, latency and images all differ from the real robot. A hand-computable example shows three ways to close the gap.

Teaching example: the arm pushes a puck, which slides $d=v^2/(2\mu g)$ from initial speed $v$ before stopping; the target is $d^\ast=0.5$ m and $g=9.8\ \text{m/s}^2$. The real table's friction coefficient $\mu$ lies between 0.3 and 0.7 and is not known in advance.

• Tune in simulation at $\mu=0.5$ only. $v^2=2\times0.5\times9.8\times0.5=4.9$. On the real table: at $\mu=0.3$ the puck slides $4.9/5.88\approx0.83$ m, 0.33 m too far; at $\mu=0.7$ only $4.9/13.72\approx0.36$ m.
• Domain randomization. During training $\mu$ is drawn at random between 0.3 and 0.7; the policy does not know this episode's value and can only pick a $v$ that is acceptable everywhere. If the goal is the smallest worst-case error, make the errors at both ends equal: $\frac{v^2}{2g}\left(\frac1{0.3}+\frac1{0.7}\right)=1$, so $v^2\approx4.12$; the puck slides 0.70 m and 0.30 m at the two ends, an error of 0.20 m either way. More robust, but rarely on target.
• Online adaptation (the RMA idea). First nudge the puck and watch how fast it decelerates (the deceleration equals $\mu g$), estimate $\mu$, then push. A measured deceleration of $2.94\ \text{m/s}^2$ gives $\hat\mu=0.3$; take $v^2=2\times0.3\times9.8\times0.5=2.94$ and the puck slides exactly 0.5 m; the remaining error comes only from the estimate.

The corresponding papers:

• Tobin et al. (OpenAI, UC Berkeley, 2017-03) randomize rendering so that the real world looks like just one more variation; an object detector trained only on simulated images localizes to 1.5 cm in the real world (authors). For dynamics randomization, see Peng et al. (2017-10).
• OpenAI's Rubik's Cube work (2019-10) uses automatic domain randomization (ADR): once the policy does well enough on a range, the randomization range widens; trained only in simulation, the robot hand completed Rubik's Cube solves (the move sequence comes from the Kociemba solver; what was learned is the in-hand manipulation).
• RMA (UC Berkeley, CMU, Facebook AI, 2021-07): a base policy plus an adaptation module that estimates, online and from recent state and action history, a latent vector encoding the environment's conditions; trained only in simulation, the authors report the A1 quadruped walking on sand, stairs, grass and other terrain without fine-tuning. Another classic sim-to-real paper for legged robots is Hwangbo et al. (2019-01).

As of October 2026, simulators are converging on GPU parallelism:

Which models can you download? (as of 2026-10-15)

GR00T N1.6's backbone is also disputed: NVIDIA's research page names an internal Cosmos-2B variant, while the GitHub repo and the developer page name Eagle.

Humanoid robots: what the companies have said (as of 2026-10-15)

For Tesla Optimus, only Tesla's own quarterly update letters (the Exhibit 99.1 copies filed with the SEC):

As of 2026-10-15, no official Gen 3 / V3 unveil appears in Tesla's primary sources, and the Q3 2026 letter is not yet out. Claims about a V3 unveil and weekly production rates appear only in press reports and are not used here.

Other companies (self-reported unless noted):

• Figure: BMW's press release of 2026-02-27 confirms that one Figure 02 helped build more than 30,000 X3s at the Spartanburg plant; Figure said on 2026-04-29 that it had delivered 350+ Figure 03 robots, without naming customers or giving a field deployment count.
• Agility: Digit moved more than 100,000 totes in a GXO warehouse (2025-11-20).
• Boston Dynamics: unveiled the product version of the electric Atlas on 2026-01-05 and said all 2026 deployments are already committed, going to Hyundai and Google DeepMind (blog); no later primary source confirming deliveries was found.
• 1X: the 2026-04-30 factory announcement says NEO customer deliveries had not started and the first units went to internal home testing, with customer shipments promised in 2026; no later official confirmation of deliveries was found.

What to watch next

• Whether NVIDIA releases GR00T N2 by year end as previewed, and with what weights and licence; whether π0.6 / π0.7 weights are opened or access stays with the API.
• Whether real-robot comparisons appear that are not run by the model's own authors, for example more policies entering double-blind evaluations such as RoboArena.
• How Tesla's Q3 2026 letter describes Optimus production.

Back to the series guide; action-head details are in the previous note; world models used directly as policies (DreamZero, 1X) are covered in the Frontier block.

References: RT-2 · OpenVLA · FAST · π0 · OpenVLA-OFT · RMA