The stack

StableTested and in active use. Breaking changes will be called out.

noeira is organised in layers. At the bottom sit the two things a policy runs against — a simulated world and the real one. Above them, the deep-learning framework and the data pipeline; above that, the learning algorithms; at the top, the drivers that own the loops: training, recording, deployment.

learning-specificgeneral-purpose, usable standalonehardware

The lower layers are libraries in their own right. nn imports no agent. physics3d imports no learning code — it borrows nn’s Tensor storage, not its networks. robot depends only on the I/O layer, and vision only on the 3D math. Using the physics engine to drive your own controller, or the servo driver to run your own arm, costs you nothing above it.

Two edges point up, deliberately: environment packages implement the BatchedEnv trait that lives beside the training drivers, and an environment package may ship the adapter for a policy trained on it (LIBERO carries its ACT dataset and observation glue). Neither pulls an algorithm into the physics.

Generic traits let the compiler specialize the whole pipeline for the target device, which is why the same source produces a CPU loop, a fused GPU one, and a binary for the robot’s embedded board.

Package layout

noeira/
├── core/           Traits, projects and runs, logging, tabular replay, metrics
├── agents/         Tabular & linear algorithms (Q-Learning → linear SAC)
├── deep_agents/    One facade package per learning algorithm
│   ├── dqn/ c51/ ddpg/ td3/ sac/ a2c/ ppo/ ppo_discrete/ redq/ redq_ofe/
│   ├── mbpo/ tdmpc2/ dreamerv3/ dreamer4/ alphazero/ muzero/ efficient_zero_v2/
│   ├── act/ smolvla/ bc/ hil_serl/ fb/     Imitation, VLAs, zero-shot RL
│   ├── demos/      The .demo format, recorders, DAgger handover
│   ├── training/   CPU/GPU drivers, BatchedEnv wrappers
│   └── core/ primitives/ loss/ data/ zero/
├── nn/             Deep-learning framework (Module + Param), distributed training
├── data/           Trajectory store, replay buffers, LeRobot v3-format datasets
├── io/             Native HDF5, Parquet, video, JSON, HTTP, safetensors, serial
├── physics3d/      MuJoCo-inspired generalized-coordinates engine
├── physics2d/      GPU-batched 2D impulse engine
├── envs/           Environments: classic, MuJoCo-style, DM Control, LIBERO, robots
├── tasks/          The task layer: families, goals, placement, evaluation
├── planners/       Trajectory optimizers (MPPI, CEM, iLQR) + GPU tree search
├── robot/          Robot drivers — today the SO-101 over its Feetech servo bus
├── ai/             Hosted and local models over HTTP: chat, vision, speech, Jev
├── vision/         Cameras, calibration, the OpenCV shim
├── render/         SDL3 renderers, ImGui viewers, video recording
├── math3d/         Vec3, Quat, Mat3, Mat4 (+ GPU variants)
├── cuda/           CUDA graphs, the NVML intercept
└── experimental/   Research code — APIs may break

Simulation and tasks

physics3d follows MuJoCo’s design: models loaded from MJCF, simulation state in generalized (joint-space) coordinates, and the pipeline collision detection → constraint building → constraint solving → integration. It carries CRBA and RNE, PGS / Newton / CG solvers, elliptic friction cones, tendons, equality constraints, meshes and heightfields, sensors and ray-traced cameras, and is checked against live MuJoCo test by test — see validation.

physics2d is impulse-based and GPU-batched, with state in a strided [BATCH, STATE_SIZE] layout so one kernel launch steps hundreds of environments — LunarLander, BipedalWalker, CarRacing.

envs holds the worlds: tabular and classic control, 2D physics, the MuJoCo-style locomotion and manipulation suites, DeepMind Control, LIBERO, Menagerie robots, arcade engines and an Atari 2600 emulator — see environments.

tasks is a declarative layer over the physics: a family composes one scene with a fixed budget of slots, and a task names a goal, the active slots and where things start. Many tasks run in one batched scene, one per lane.

The real world

robot drives real robots — calibration, torque, synchronized reads and writes over the servo bus — with a safety rule throughout: nothing moves unless a command was explicitly armed. The first robot it supports is the SO-101 arm on its Feetech bus. vision covers camera capture on its own thread, intrinsic and extrinsic calibration, and the image preprocessing a policy expects. Teleoperation records both into datasets through data and io.

Deep learning and data

noeira.nn — networks assembled from small Module blocks, each Param owning its value and gradient tensors.

• 70+ primitives: Linear, Conv2D, FlashAttention-2 forward and backward (on the tensor cores on NVIDIA), RoPE, LSTMCell, NoisyLinear, LayerNorm/RMSNorm
• Combinators: Sequential, Residual, Parallel, Repeat
• Pre-built models: ResNet, GPT, ViT, LSTM
• Optimizers: SGD, Adam, AdamW, with grouped multi-tensor apply
• Fused primitives: LinearAct runs matmul, bias and activation as one GPU kernel — fusion is a module you choose, not a compiler pass
• Training: supervised Trainer with AMP and CUDA-graph capture; DDP and ZeRO-1/2/3 over N GPUs

data is the trajectory store (HDF5), the replay buffers the off-policy agents sample from, and datasets in the LeRobot v3 format, read and written natively. io is the native I/O underneath — Parquet, H.264 video through ffmpeg, safetensors, HTTP — with no Python in the data path.

Learning

Family	Algorithms
Tabular and linear	Q-Learning · SARSA · Expected SARSA · Double Q · Dyna-Q · SARSA(λ) · tile coding
Value-based	DQN · Double · Dueling · Noisy · C51 · Rainbow
Policy gradient	REINFORCE · Actor-Critic · A2C · PPO
Continuous control	DDPG · TD3 · SAC · REDQ
Model-based	MBPO · TD-MPC2 · DreamerV3 · Dreamer 4
Planning	AlphaZero · MuZero · EfficientZero V2
Imitation and VLAs	Behaviour cloning · ACT · SmolVLA
From demonstrations	HIL-SERL (RLPD + BC) · DAgger interventions
Zero-shot	Forward-Backward representations (BFM-Zero)

Deep agents follow a facade pattern: one package per algorithm, each exposing an XxxAgent over a trainer plus a shared training driver. Networks arrive as compile-time parameters, so SACAgent does not know or care what architecture it is optimizing.

Some agents decide by running a search at every step — TD-MPC2’s MPPI, the zero-series’ MCTS. That search lives in noeira.planners, written against callback traits, so the same MPPI drives a learned world model in training and a closed-form oracle in a test.

Drivers and deployment

The drivers own the loops so the agents do not have to. For training they come in a matrix:

	Single env	Batched
Off-policy	run_offpolicy_train	run_offpolicy_train_batched
On-policy	run_onpolicy_train	run_onpolicy_train_batched

each with CPU and GPU variants. BatchedEnv wrappers such as Phyics3dBatchedEnv adapt an environment to the batched interface. run_ppo_vec is continuous PPO over any GPU-batched environment with the whole rollout on the device; see PPO. The newer drivers (task families, ACT, Forward-Backward, the G1) record each run as a RunContext: a directory with its own record, checkpoints and metrics, filed under a project and mirrored to noeira cloud.

The other loops follow the same shape: a recorder drives the robot or the simulator and writes demonstrations; a deployment loop loads a checkpoint and its normalisation, reads the cameras and the joints, and commands the robot — on a workstation or on an embedded board such as a Jetson Orin.

The through-line: every layer runs on the GPU

On a simulated GPU run, the whole sequence stays on the device:

env step → observe → network fwd/bwd → replay sample → target update

No host round-trips; the data never leaves the device. That is only possible because the physics engine, the networks and the driver are all Mojo, compiled together. On NVIDIA the loop can additionally be captured as a CUDA graph and replayed, worth roughly 1.4× on SAC. On the robot, the same modules run the policy on its onboard GPU.

Next

• Compile-time composition — the idiom that makes this layering zero-cost
• Traits — the interfaces between layers
• CPU and GPU paths — how the dual paths are organised