Robots and humanoids

BetaWorks, but under active development — APIs and results may shift. Each model is gated against MuJoCo on the same XML. The SO-101 has a real-hardware path; the G1 and the Panda are simulation only.

noeira/envs/robots/ holds models of physical robots, as distinct from benchmark suites like DeepMind Control. The point of each port is that a policy trained on it can be run on, or compared with, the real machine — so each one names the exact upstream model it came from, and each is checked against MuJoCo stepping that same file.

Robot	Source model	DoF	Environments	GPU batch
SO-ARM100	MuJoCo Menagerie trs_so_arm100	5 + gripper	SoArm100Reach	yes
SO-ARM101	The Robot Studio’s Simulation/SO101 (new_calib)	5 + gripper	SoArm101Reach, the SO-101 task families	yes
Unitree G1	BFM-Zero’s sim-to-sim g1_29dof	29 + floating base	UnitreeG1, UnitreeG1Batched; the walkers UnitreeG1Walk, UnitreeG1WalkRP	NVIDIA only
Franka Panda	robosuite’s Panda, gripper and mount, vendored	7 + gripper	the LIBERO scenes	yes

SO-ARM100 and SO-ARM101

The two versions of the low-cost SO arm. They share a topology and a task — SoArmReachConfig serves both — and they are not interchangeable: moving mass differs by about 25% and the moving jaw by 68%, the base frames point different ways, and qpos does not transfer between them because the joint axes are spelled differently.

• SO-ARM100 comes from Menagerie with hand-authored collision shapes and an elliptic cone (impratio="10"). It needs no engine feature the others do not.
• SO-ARM101 is the arm on the desk and the one to train on for hardware. Its model collides with thirteen full-resolution meshes, about ten times the collision cost of SO-ARM100. Upstream writes its inertias as fullinertia, which the parser refuses; tests/robots/so_arm_bake.py diagonalises them with MuJoCo and the bake is gated per body, including the inertia frame.

The reach task. Observation is 27 values — joint positions and velocities, end-effector and target positions, the vector between them, and the previous action. Actions are six joint position targets, normalised to [−1, 1] and mapped onto each joint’s own ctrlrange. Reward is tolerance on the end-effector’s distance to a target sampled inside the arm’s reachable cone; episodes are 500 control steps at 50 Hz with no early termination.

from noeira.envs.robots import SoArm101Reach

var env = SoArm101Reach[DType.float64]()

pixi run build-imgui                                                     # once
pixi run mojo run -I . examples/robots/so_arm_viewer_imgui.mojo so_arm101_reach
pixi run -e nvidia mojo run -I . examples/so101/sac_so_arm101_reach_training_gpu.mojo

Run the viewer from the repository root: both models reference their meshes by repo-relative path.

Variants of the SO-101 scene.

File	What it is
so_arm101.xml	the arm alone — the reach task
so_arm101_tower.xml	the follower with the printed wrist-camera mount and wrist_cam; the base of the so101_tower task family, which adds the overhead camera stand, the desk mat, a bowl and a brick
so101_park_k*.xml	the arm plus k parked props, for measuring what an idle slot costs the physics step; a probe with zero reward, not a task

The SO-101’s tasks — reach, settle, lift, gather, cube-in-bowl — are written with the task layer, not as environment configs. The real arm, its cameras and the path from a simulated policy to hardware are under Robots; Sim twin covers how the simulated rig matches the physical one.

Unitree G1

The 29-DoF humanoid exactly as BFM-Zero trains it, so the released BFM-Zero policy can be run and reproduced here. See Forward-Backward and BFM-Zero for the agent and the reproduction.

Control	29 PD targets at 50 Hz over 200 Hz physics; torque = kp·(target − q) − kd·q̇, clipped, re-evaluated every substep
Observation	64 proprioceptive values (joint offsets and velocities, projected gravity, root angular velocity) followed by the 463-value privileged max_local_self from the simulator’s bodies
Reward	none — BFM-Zero’s agent computes its own
Episode	500 control steps, no early termination
Reset	the XML’s standing pose, or reference-state initialisation from a motion clip: a random frame of a random LAFAN1 clip, with a 30% chance of a lie-down start

The PD gains are generated from BFM-Zero’s robot config into noeira/envs/robots/unitree_g1_pd.mojo by tests/robots/g1_bake.py. Reset from motion data lives in noeira/envs/robots/unitree_g1_rsi.mojo, and the privileged observation in noeira/envs/robots/unitree_g1_priv_obs.mojo.

from noeira.envs.robots import UnitreeG1, UnitreeG1Batched

var env = UnitreeG1[DType.float64]()            # CPU, any platform
var batch = UnitreeG1Batched[N_ENVS=256](ctx)   # GPU, NVIDIA only

LAFAN1 motion data. The motion clips come in as a trajectory store, converted in Mojo with no Python dependency:

pixi run mojo run -I . examples/g1/lafan_import.mojo

It downloads BFM-Zero’s lafan_29dof.pkl (checked by size and SHA-256), converts its 40 clips to a 50 Hz store with our forward kinematics, and writes lafan_g1_50hz.h5, the file the training and evaluation scripts read. tests/robots/test_lafan_import_vs_oracle.mojo holds it against the reference pipeline’s own dump.

The G1 gates — the environment against MuJoCo, the motion store against our kinematics, the privileged observation against the reference function — run with pixi run test-g1 once those artefacts exist.

The G1 walker

A second pair of environments puts the same body, under the same torque PD, in a velocity-tracking task trained with PPO. Commands are a forward speed, a lateral speed and a turn rate. They are redrawn during the episode, with a share of exact zeros so the policy also learns to stand. Resets are random, and the robot gets pushes and observation noise. There are two reward recipes:

Environments	Recipe	Observation	Reward
UnitreeG1Walk, UnitreeG1WalkBatched	Playground’s G1 joystick terms plus RoboParty’s standing terms	70 values	17 logged terms
UnitreeG1WalkRP, UnitreeG1WalkRPBatched	RoboParty’s interrupt-task recipe (BSD-3), with per-joint kp / kd randomisation	67 actor + 15 privileged	their 26 terms and weights

from noeira.envs.robots.unitree_g1_walk import UnitreeG1Walk, UnitreeG1WalkBatched
from noeira.envs.robots.unitree_g1_walk_rp import UnitreeG1WalkRP, UnitreeG1WalkRPBatched

Both train with run_ppo_vec. The RoboParty example follows their PPO: 10 stacked frames, a privileged critic, adaptive KL, action-rate and smoothness penalties, a mirror-symmetry loss at 0.2, and 512-256-128 ELU networks. The G1 mirror maps match the model’s joint ranges, defaults and gains exactly, and a mirrored state agrees with the mirrored physics after a step.

pixi run -e nvidia mojo build -I . -D PPO_VEC_ENV_GRAPH -D PPO_VEC_TRAIN_GRAPH \
    examples/g1/g1_walk_rp_ppo_gpu.mojo -o g1_walk_rp_ppo
pixi run -e nvidia ./g1_walk_rp_ppo --steps 60000000 --seed 1 --out runs/rp_s1

examples/g1/g1_walk_eval.mojo scores a checkpoint on the CPU: falls under random commands, tracking error, lag, stopping distance and drift while holding. --recipe switch runs a walker while the command is non-zero and a stander at zero. The walk-to-stand handover is gated on the body being slow on both feet, because an immediate handover fell 5 times in 5. With a mix-recipe walker and a RoboParty stander: no falls in 20 random-command episodes, 1 fall in 8 stops from 1 m/s, and 6 cm of drift while holding.

Not a solved walker yet

Neither recipe does both jobs alone. The mix recipe walks, but drifts 0.45 to 0.65 m while holding. The RoboParty runs so far stand within 2 cm but do not step. One cause has been removed since: the G1’s upper arms hang against its torso, and RoboParty’s undesired-contact term charged a walking policy more for that than tracking paid it. The switch controller above is the working combination. The batched walkers are NVIDIA-only, like UnitreeG1Batched. Their gates are tests/robots/test_unitree_g1_walk.mojo, tests/robots/test_unitree_g1_walk_rp.mojo and their _gpu twins.

Franka Panda

noeira/envs/robots/assets/panda_robosuite.xml is robosuite’s Panda, its gripper and LIBERO’s mount, flattened into one MJCF file (a variant without the mount sits beside it). It is generated, not hand-edited:

pixi run vendor-panda          # regenerate from robosuite
pixi run vendor-panda-check    # CI: the vendored files match the tool

The LIBERO scenes attach it with <attach> and drive it with robosuite’s OSC_POSE operational-space controller — see LIBERO.

How close to MuJoCo

The arms and the G1 each have a gate that steps our engine and MuJoCo on the same XML: tests/robots/test_so_arm100_vs_mujoco.mojo, tests/robots/test_so_arm101_vs_mujoco.mojo and tests/robots/test_unitree_g1_vs_mujoco.mojo, which drives MuJoCo with the reference’s own torque law per substep. The tower rig’s wrist camera is gated by tests/robots/test_so_arm101_camera_vs_mujoco.mojo, and both of its cameras’ world poses by tests/tasks/test_so101_tower_cameras.mojo. The Panda is checked inside the LIBERO scenes it is attached to.

Beyond these, MuJoCo Menagerie’s scenes are used as the engine’s fidelity test set rather than packaged as environments: see Validation. Menagerie models can be opened in the studio.

Next

• Robots — the physical SO-101, safety, teleoperation
• Forward-Backward and BFM-Zero — the G1’s agent
• LIBERO — the Panda’s benchmark
• Tasks — how the SO-101 and LIBERO tasks are written