# Fallow kitchen interaction benchmark

One Franka Panda arm, a reachable preparation station, and five task setups. This is a standalone benchmark beside the kitchen reconstruction. It does not create a shared catalogue entry or install a policy in production workers.

The [published evaluation](../review/benchmark.html) reports the actual results and limitations. The baseline uses privileged object poses to establish feasibility. It commands the robot's native joint drives: grasps use finger contacts, with no object attachments, teleports or appliance commands during an episode. A waypoint with more than 5 cm of remaining tracking error aborts the baseline. A successful demonstration is not evidence that a learned camera policy transfers to the kitchen.

## Run it

Use a Linux NVIDIA host with the repository's Isaac Sim 6.1 worker image and the complete repository, including `sim/assets/franka`. From the repository root:

```sh
bash scenes/fallow-kitchen/benchmark/run_native.sh lid /tmp/fallow-lid-evaluation \
  --count 50 --seed 1000 --split heldout --audit-reset
```

Replace `lid` with `pan`, `drawer`, `pour` or `oven`. Use a fresh output directory. `ISAAC_IMAGE` can select a compatible image. The script mounts source read-only, authors a writable task overlay and requires a completed result receipt. The image's exit status alone is insufficient because Kit can exit zero while closing after an exception.

For a sensor demonstration, add `--sensors`. This runs the camera pipeline throughout the episode and saves its final aligned samples. Add `--record-images` to save every distinct delivered camera frame; this uses substantially more time and storage. Rendering throughput is separate from the state-based baseline evaluation.

Camera runs use RTX Path Tracing at 64 samples with OptiX denoising. They require the NVIDIA driver's `/usr/share/nvidia/nvoptix.bin`, or `FALLOW_OPTIX_WEIGHTS` pointing to that file. Rendering is checked not to advance the physics clock.

The same launcher accepts `audit-objects`, `audit-controls` and `audit-inertia`, each with a fresh output directory. These run the 36-body release audit, passive dial/button force probes, and a known-torque inertia-frame test. A passing audit receipt is required. The original scene's drawer, oven and GPU particle checks remain in `source/run_native_check.sh`.

Without Isaac, run the deterministic model and geometry checks using a Python environment with NumPy, SciPy, Pillow and usd-core:

```sh
cd scenes/fallow-kitchen
python -m unittest discover -s benchmark/tests -v
python source/audit_physics.py
```

## Policy interface

`Session(stage_path, task, sensors=True)` exposes `reset(seed, split)` and `step(action)`. A step advances 50 ms and returns `(observation, terminated)`. The runner accepts `--policy your_module:factory`; `factory()` returns a callable that receives only observations and returns an action. An agentic planner can maintain a queue of actions behind the same callable. Learned policies need their own observation/action adapter; compatible Python signatures do not establish compatible training distributions.

Actions contain exactly these fields:

```json
{"kind":"ee_pose","values":[-1.2,1.6,1.3,1,0,0,0],"gripper_open":1}
```

`ee_pose` is an absolute world position in metres followed by a unit quaternion in **xyzw** order. Alternatively, `joint_positions` supplies seven arm angles in descriptor order, in radians. The scene uses right-handed coordinates with Z up. Gripper opening is 0 closed, 1 open. The boundary limits Cartesian target changes to 0.15 m/s and 0.6 rad/s, and joint-position target changes to 0.8 rad/s. These are command limits, not hardware safety guarantees or a promise of reachability. Native differential IK executes at the 240 Hz physics rate. The scripted feasibility controller separately interpolates bounded Cartesian waypoints.

Observations contain simulation time, joint positions/velocities, achieved tool pose and measured gripper opening. Fixed and wrist cameras supply uint8 RGB, optical-axis depth in metres, `[fx, fy, cx, cy]` intrinsics and `world_from_camera`. Optical coordinates are X right, Y down, Z forward. Depth zero means unavailable. Object poses, segmentation IDs, reflective masks, contacts and thermal state stay in evaluator records. This is an API convention, not a Python security sandbox. The batch launcher disables networking; a network-hosted planner needs an explicit integration outside that isolated runner.

Cameras capture at 10 Hz, 320 × 240, with 80 ms simulated delivery latency. The first observation may have no delivered image. The seeded sensor approximation adds exposure variation, a read/shot-noise proxy, 10% previous-frame RGB blending, 2 mm depth noise, 1 mm quantization, range clipping, reflective-depth holes and frame drops. These are estimated perturbations; temporal blending is not physical motion-blur integration, and the material mask is not a measured infrared response. Policy wall-clock inference time does not automatically advance simulation.

## Reset, scoring and evidence

Resets restore robot joint state, object poses/velocities, mass/COM/inertia, friction, lights, passive controls, thermal/liquid state and camera queues before settling. Seed streams differ between development and held-out splits. Variations are deliberately small: ±8 mm object XY position, ±10% mass, dry friction and lighting, with varied jug fill and water temperature. Geometry, clutter and embodiment remain fixed. Every episode records the complete configuration, initial/final states, commanded joint targets, measured contacts, task events and compressed time-series data. The run manifest records source and asset hashes. Keep it beside the results.

Scoring requires measured bilateral contact, a lift, release and stable placement. Task-specific conditions include lid seating, trivet placement, drawer opening/retrieval/closure, delivered water mass/spill limits, and oven loading/closure/control state. Tracking failure, timeout and incomplete behavior are policy failures. Nonfinite or invalid motion and unsupported liquid states are simulator failures; these stay in the success-rate denominator. Reset failure or an execution exception leaves the run incomplete, with an error receipt. Success intervals use Wilson's 95% method. Contact peaks are sampled at 20 Hz and can miss shorter impacts.

The reference scene has its own [complete physics inventory](../review/physics-audit/inventory.json). The task overlays clear the work surface, resize the lid to the selected saucepan, add a robot pedestal/trivet and move the oven into the station. Those are explicit training arrangements; the original video layout remains in the main scene.

## Physical scope

All masses, friction and thermal constants remain **estimates**. `calibration.py` validates provenance-bearing measurement records, including uncertainty; it does not silently replace geometry or apply every accepted measurement to a simulation. No real measurements have been supplied. A measured mass, friction test or heating trace should be reviewed, applied to its matching model and followed by a new evaluation.

The hot-pan model has body, handle and gripper thermal nodes with an energy ledger and contact-dependent conduction. The oven's physical dial uses a native revolute joint, passive detents and a 270° range represented as −135° to +135°. Its spring-return button toggles power at 3.5 mm travel and rearms below 1.5 mm. The measured door angle drives the heater interlock. These are task controls with estimated behavior, not identified electronics from the source video.

Pouring uses a **reduced quasi-static/ballistic model**, separate from the existing GPU PBD tests. Gravity-aligned jug quadrature updates liquid COM/inertia; emitted mass packets carry momentum and enthalpy; target capture updates the native vessel's mass properties. Steel jug heat capacity scales with its actual dry mass. The receiving aperture matches the collision walls. A receiver containing water and tilted more than 5° is outside this model and invalidates the episode. Packets do not collide with arbitrary kitchen geometry, spilled liquid does not change friction, and there is no sloshing, viscous film, foam or coupled PBD thermodynamics. Mechanical energy is not conserved during inelastic capture; the mass/enthalpy ledger and transfer momentum are audited separately.

Whole lemons and bread rolls use convex hulls of their authored shapes. Food remains rigid; the burger's patty and toppings form one body. Tongs do not articulate, bottles do not squeeze or dispense, and decorative ingredients are not independently simulated. Flames remain visual geometry; combustion, smoke, steam flow, cutting, deformable food and full-kitchen humanoid locomotion are deferred.

## Published run provenance

The rigid-task evaluation uses [its frozen source snapshot](../review/benchmark/evaluated-code-rigid.zip). Pouring uses [the corrected inertia adapter snapshot](../review/benchmark/evaluated-code.zip). The camera demonstration has [its capture-source snapshot](../review/benchmark/camera-code.zip). Per-run manifests identify the exact source and asset hashes; configuration IDs do not replace those hashes. The launch script uses the worker Python environment to author USD, then starts a fresh Isaac Python process in the mounted repository.

The [known-torque receipt](../review/physics-audit/native-inertia-frame.json) caught a body-frame/COM-frame error before publication. Published pouring trials were rerun after correction; rigid-task controllers do not use this adapter. The [protocol receipt](../review/benchmark/protocol.json) records these boundaries and the retired attempts.

To rebuild the public result page from its archived trials, install NumPy and Matplotlib and run `python -m benchmark.report` from the scene directory. The generator requires all 50 distinct held-out instances for every task; it refuses incomplete or duplicated evaluation sets.