RoboDojo Scores 32 of 34 Robot Policies Under 10% Success
Every VLA paper reports its own benchmark table, using its own benchmark, run by the people who trained the model. RoboDojo is the other thing. 42 simulation tasks and 18 real ones, run by the MMLab@HKU and Tsinghua group that built it. One published evaluation-integrity protocol, and 34 policies ranked on one page.
The distribution is the story. G0.5 leads at 14.88% success, Xiaomi-Robotics-1 is second at 13.93%, and Tencent’s Hy-Embodied-0.5-VLA third at 8.80%. That is the entire top of the table. π0.5 comes fifth at 6.91%, GR00T-N1.7 twentieth at 1.31%, and OpenVLA-OFT last at 0.02%. Thirty-two of the 34 sit in single digits, and the two that clear it do so by four points.
Set that against what the same models report on LIBERO, where 95% and up is now unremarkable. Both numbers are real. The gap between them says how much of a reported success rate belongs to the benchmark rather than the policy. You can only see it when one group runs everything under one protocol. Two caveats before you quote it. The labs submitted the top three themselves, and the RoboDojo team ran most of the rest. RoboDojo is also deliberately hard.
Then there is the leader. π0.5 and most current VLAs bolt a flow-matching action expert onto a VLM. G0.5 drops it and keeps the VLM as the actor. One decoder from Qwen3.5-2B emits reasoning and action tokens in a single stream, and a learned codec maps 14 embodiments into one 27-dimensional action space. Galaxea reports 76.7% on its own R1-Lite and R1-Pro hardware against 53.3% for π0.5, and now has an outside leaderboard agreeing on the ranking.
Its weights have been public since 16 June. The repository shows zero downloads. The license is non-commercial, which explains most of it. Check it before you read that leaderboard as a recommendation.
Thanks for reading Topic Queue! Subscribe for free to receive new posts and support my work.
Subscribe
---
ACE-Data-0 is downloadable as of 17 August: 24,810 files, mostly video and parquet, with per-session IMU and audio on the room-scale takes. The access gate came off with it, so no request needed.
The license is where it stops. It is a bespoke `ace-data-0-research-license`, so you can read this data and not ship with it. `viewer: false` means you cannot inspect it before downloading either. If you checked the repo in early August and found it empty, look again.
Connecting N policies to M evaluation environments normally costs N times M integrations. Every model brings its own dependencies, data layout, and runtime interface. XPolicyLab gets that down to N plus M. It fixes shared observation, action, and trajectory schemas plus one small adapter interface. Policy inference and environment execution run as separate processes, so each side keeps its own stack.
Wiring up one representative policy took over five hours without the standard, two hours conforming to it, and thirty minutes with the packaged agent skills. Forty-two policies are adapted already. If you maintain your own evaluation harness, read the adapter layout before you extend it again.
Predicting a sequence of actions and executing a prefix open-loop is what ACT is named for, and most recent VLAs do it too. The reasons usually given are compounding error, absorbing inference latency, and smoothing motion. A new paper from Russ Tedrake’s group tests those and lands elsewhere. Long open-loop prefixes mostly let a short-context policy imitate non-Markovian demonstrations. That means the expert acted on history the policy cannot see. Across four simulation tasks and two real ones, expert non-Markovianity shaped the relationship between success and open-loop horizon. Compounding error mattered less.
Give the policy enough context, and the effect disappears. The most reactive closed-loop version then performs best. Put that next to the dexterity pre-training paper below, which ablates short action chunks for reactive control and points the same way. If you arrived at your open-loop horizon by sweeping it, that horizon may be measuring your demonstration data rather than your policy. Lengthening the context window is the experiment that tells you which.
HiPHI landed on the Hub during the World Robot Conference: optical motion capture at 90 Hz with sub-millimeter marker tracking, from 132 performers. On the interaction half, the rig captures each object’s mesh and 6-DoF trajectory in sync. Cross-dataset joint error falls steadily from 3 hours of training data up to 300, and policies trained on it run on a physical Unitree G1.
Three details will get lost in the coverage. The 617.5-hour headline counts left-right mirrored copies, so the original capture is around 309 hours. The release is BVH, trajectory CSVs, and OBJ meshes, with no video, no tactile, and no audio until the multimodal corpus lands later this year. Access requires a request and a questionnaire under a research-only license. Note the fit, too: whole-body motion and object geometry suit humanoid locomotion and motion priors, and is thinner ground for a manipulation team.
- Hydra-0 — represents robot actions as pixel motion, so one interface spans embodiments and video backbones. Replay a policy in the world model, and the success rates correlate with the real reference at 0.96. That is on RoboLab, the same benchmark the Cosmos policies use. No weights yet.
- Post-training Cosmos 3 Edge — not a release, since the checkpoints date to early July, but it carries the first on-device numbers. The 4B policy scores 22.9% on RoboLab against the 16B Nano policy’s 36.8%. On a Jetson Thor, it takes 1.53 seconds to generate a chunk covering 2.13 seconds of motion.
- ManiGuard — scores safety separately from task success across 200 tasks, using runtime automaton monitors over simulator state. Between 6 and 21% of successful rollouts violate the specification on the way to the goal. It releases 8,000 annotated demonstrations in LeRobot format, and fine-tuning on more of them does not close the remaining gap.
- Simulation pre-training for dexterity — five operators collected 75 hours of dexterous manipulation in one week, inside a VR headset and without a robot. Fine-tuning then used one to two hours of real demonstrations on a 56-DoF bimanual setup.
- PRISM — 45 hours across more than 5,000 trajectories of contact-rich industrial assembly. The rig synchronizes force/torque, tactile and multi-view RGB-D, which covers the precision and contact regulation that pick-and-place corpora leave out.
- GigaBrain-0.7 — a three-system embodied foundation model pretrained on over 37,000 hours of heterogeneous data. GigaBrain promises training code and weights, and the Hub has neither yet.
- Flex-π — still no weights behind the frozen-VAE pointmap result, but eight datasets appeared on its Hub organization from 16 August, including two self-repair gripper sets.
- Physical prompt injection — printed signage redirected VLM-planned sorting in 27.0%, 29.4% and 5.0% of 5,670 trials, against GPT-4o, Gemini 2.5 Flash and Qwen3-VL-32B. Masking text before the planner sees it blocked everything, at the cost of any task that needs to read a label.
- The World Robot Conference runs in Beijing through 23 August, and Noitom continues at RO-MAN in Fukuoka from 24 to 28 August. Both have already released one dataset, so watch for more tied to them.
- IROS is in Pittsburgh from 27 September to 1 October, and CoRL project pages are landing.
---
LG targets 100,000 hours of robot training data by the end of the year, out of a Seoul data factory still under construction. By its own wording, that target combines data collected at the facility with data synthesized and augmented through NVIDIA’s Cosmos world models. So it is not 100,000 hours of real robot operation.
Xiaomi reached the same headline figure from handheld capture rigs in the wild, with no synthesis in the count. If you benchmark your own collection plans against public numbers, check which kind of hour they count.
Labs rarely publish this ratio. Generalist gave its per-task split: 2 to 80 hours of human demonstration against 4 to 80 minutes of robot data. One tape-dispenser task used about 50 clean robot episodes, totaling four minutes. Capture follows the UMI approach of handheld end effectors with GoPros.
Unitree listed on Shanghai’s STAR Market on 19 August. It opened as much as 629% above the offer price and raised around $905 million, with DeepSeek among the strategic investors. Its own prospectus is more sober than the pop, noting that robot hands are still not precise or durable enough for commercial use at scale. AgiBot and Leju are both queued to follow.
---
Thanks for reading Topic Queue! Subscribe for free to receive new posts and support my work.
Subscribe
Previous