Rendering

StableTested and in active use. Breaking changes will be called out. Requires SDL3. Not needed for training.

noeira.render draws environments — for watching a policy, for recording, and for playing them yourself. It is optional: training never touches it.

3D renderershadows · reflections · skybox

2D rasterizerCPU, no GPU required

Playableinteractive, keyboard input

SDL3

pixi install provides SDL3 on every platform; there is no system package to install, and a system SDL3 is not used. SDL3_LIB=/path/to/libSDL3.so overrides the library that is loaded. See Installation.

The bindings are complete FFI — 35 files under render/sdl/ — not a wrapper around a C shim.

Two renderers

Renderer2D — CPU rasterizer

Software rasterization for the classic-control and 2D-physics environments. Draws lines, filled and outlined polygons, circles and text into a pixel buffer that SDL3 presents.

from noeira.render.renderer2d import Renderer2D

No GPU required, which makes it the reliable choice for a headless box or a quick look at a policy.

Renderer3D — GPU renderer

The MuJoCo-style environments render through the SDL3 GPU API. The shader language is chosen at compile time: MSL on macOS (Metal), SPIR-V everywhere else (Vulkan). Six passes:

Pass	Output
shadow	depth map from the light
solid	Blinn-Phong shaded geometry
ground	checkerboard floor
reflection	mirrored geometry in the floor
skybox	gradient background
text	HUD overlay

from noeira.render.renderer3d import Renderer3D

Meshes are cached and draw commands deferred, so a batch of similar geoms costs one submission rather than one per body.

Picture-in-picture. Renderer3D.set_pip_views draws the scene again from other cameras into insets over the main view — each its own pass, sharing the shadow map, without reflections or HUD text. ModelRenderer.set_pip_cameras does it for a model’s own cameras, posed from that frame’s forward kinematics: a teleoperator’s overhead and wrist views beside the free camera. examples/so101/tower_demo_viewer.mojo shows the tower’s two cameras this way.

Shaders on Linux: SPIR-V

The Metal shaders are embedded as strings in gpu_shaders.mojo. Their GLSL 450 equivalents live in noeira/render/shaders/ and are committed pre-compiled to .spv. After editing a .glsl file, rebuild them:

pixi run compile-shaders

This runs glslc from shaderc, a pixi dependency.

Interactive controls

The 3D viewer is interactive. Environment demos print their bindings, and the common ones are:

Key	Action
`Space`	pause / resume
`→`	single step
`1`–`9`	switch camera
`R`	reset camera
`S`	screenshot
`V`	start / stop recording

Mouse orbits, pans and zooms.

Video capture

from noeira.render.video_recorder import VideoRecorder

var rec = VideoRecorder()
rec.start("run.mp4", fps=30)
rec.add_frame_bgra(pixel_addr, width, height)   # each frame
rec.stop()

Encodes MP4 or GIF (chosen by the file extension) by piping BGRA frames to ffmpeg from a worker thread — no Python in the path. ffmpeg is a declared pixi dependency, so there is nothing extra to install. save_frame_bgra writes a single frame as PNG, JPEG or WebP.

pixi run mojo run -I . examples/export_env_gifs.mojo
pixi run mojo run -I . examples/arcade_games/pong_playable_gif.mojo

Supporting modules

Module	Purpose
`camera3d`	orbit / pan / zoom camera
`gpu_mesh` · `gpu_types`	mesh upload and GPU resource types
`gpu_shaders` · `gpu_shaders_spirv`	MSL shaders and SPIR-V variants
`font_atlas`	bitmap text
`shapes` · `shapes3d`	primitive geometry
`transform` · `light` · `colors`	scene helpers
`stl_loader` · `obj_loader` · `msh_loader`	mesh loading — STL, Wavefront OBJ, MuJoCo’s legacy `.msh`
`skn_loader` · `skinning`	MuJoCo `.skn` skins and linear blend skinning
`png_loader`	textures, decoded natively by `noeira.io.png`
`ui`	small immediate-mode widgets drawn by `Renderer3D` itself
`imgui/`	Dear ImGui bindings — see viewers
`image_writer`	PNG / PPM output

Mesh files are dispatched by extension inside load_stl, so the renderer, the mesh inertia and the convex hull all read OBJ and .msh through the same call.

Skins are the deformable envelope MuJoCo draws over a model such as dm_control’s dog: each vertex is weighted across several bodies, so the mesh is re-posed on the CPU every frame (skin_pose, transcribed from MuJoCo’s mjv_updateActiveSkin) and re-uploaded. examples/dm_control/dog_skin_view.mojo shows one on its own.

Two UI toolkits, not interchangeable. ui.mojo widgets record into the renderer’s command buffer and are painted later; the ImGui bindings build their own draw list. Do not mix them in one panel — they hit-test against different notions of where the pointer is. New viewers use ImGui.

Rendering an environment

Environments implementing RenderableEnv draw themselves:

_ = env.reset()
env.render_frame()

For the tabular environments that renders as text — no SDL3 required at all. The physics3d environments use ModelRenderer, which adapts a physics model to EnvRenderer3D.

Playable demos

Several environments accept keyboard input directly:

pixi run mojo run -I . examples/arcade_games/pong_playable.mojo
pixi run mojo run -I . examples/arcade_games/breakout_playable.mojo
pixi run mojo run -I . examples/board_games/chess_playable.mojo
pixi run mojo run -I . examples/craftax_classic/play_classic.mojo
pixi run mojo run -I . examples/car_racing/play_car_racing.mojo

Worth doing before training an agent on something. Playing a task tends to make its reward structure and failure modes obvious in a way that reading the config does not.

Viewers and physics studio — the ImGui viewers and the MJCF authoring tool
Camera observations — the batched GPU ray tracer policies see through
Environments — what there is to render
3D physics — the models being drawn