Scripted bot (left) and FLUX 3 Action (right), playing the same seed. The model runs in real time at 79 ms per plan. The displays show each player’s controls and the model’s 32-action plan.
## The games | Game | On screen | Actions | | ------ | -------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | | GRUNT | 256×256 raycaster shooter: grunts that shoot plasma, dogs that bite, health and ammo pickups, HUD in the frame | `[move, strafe, turn, fire]` in \[-1, 1] | | VECTOR | Out Run-style racer: curves, hills, three lanes of traffic, oil, fuel, nitro; leaving the road above half speed is a wreck | `[steer, throttle, nitro]` in \[-1, 1] | Each runs headless from a seed and ships a scripted bot. The bot is the teacher: it decides from the drawn frame alone, and a test asserts the same frame always yields the same action. At play time the model gets the rendered frame, its own last action, and a constant caption.The VECTOR bot, one minute on seed 7. It slows for bends and accelerates on straights.
## Recording Follow the [episode format](/flux_3/flux3_action_finetuning#episode-files) for `index.json`, RGB frames, and action arrays. Use `grunt=...` and `vector=...` roots in the training config. The reference adapter records no state array: it derives the previous action from each episode and uses zero at its start. Keep training and evaluation seeds disjoint. 800 episodes of 16 seconds at 15 Hz per game, recorded headless in Node on one CPU node: 75 seconds for GRUNT, 33 for VECTOR. GRUNT episodes are split equally across four scenarios: standard spawn, random heading with an enemy in view, mid-episode changes to heading, and a nearby attacking dog. The bot supplies all action labels.One recorded episode per scenario, with its action labels.
## Training | | DROID recipe | GRUNT / VECTOR | | ----------------------- | ------------------------------------------------------ | -------------------------------------------------- | | Global batch | 2,048 | 126 (7 GPUs × 9 × accumulation 2) | | Peak LR, trunk / heads | 1.92e-4 / 9.6e-4 | 5e-5 / 2.5e-4 | | Trunk schedule | Frozen through update 1,000, then 2,000 warmup updates | Frozen through update 200, then 600 warmup updates | | Steps | 30,000 | 2,000 to 3,000 | | Canvas (width × height) | 736×544, 3 cameras | 512×512, 1 camera | | Action / state | 8 joints / joint positions | 4 or 3 / last action | | Starting point | action-midtraining checkpoint, fresh DROID heads | the same checkpoint, fresh game heads | The reported result run used 7 H200s, at 9 to 10 seconds per update: about 5 hours for 2,000 updates, or 35 H200 GPU-hours. The separate timing in the [fine-tuning guide](/flux_3/flux3_action_finetuning#reported-training-costs) uses 32 H200s, global batch 128, and 3,000 updates. These are distinct configurations; neither is the eight-GPU command's measured runtime. Use the [fine-tuning guide](/flux_3/flux3_action_finetuning) for a new run. The release does not include the original run manifests or evaluation code. ## Results Each evaluation runs for 60 seconds on seeds excluded from fine-tuning. The model, scripted bot, and random controller use the same seeds. Both tables use one set of weights, trained on GRUNT and VECTOR together for 3,000 updates. The caption tells it which game it is playing. Each run lasts 60 seconds on one seed, with the bot on the same seed. GRUNT: | Policy | Kills | Deaths | Shots on target | | ------------- | ----- | ------ | --------------- | | Scripted bot | 13 | 0 | 96% | | FLUX 3 Action | 15 | 0 | 86% | | Random | 8 | 2 | 13% |Step 250: moves, fires at nothing.
Step 1,000: turns toward enemies, walks into dogs.
Step 2,000: trained policy.
VECTOR at step 1,000, seed 7. The predicted steering changes before the car reaches the bend.
## Play it Complete [training and export](/flux_3/flux3_action_finetuning#export-and-run) first. The joint game export returns four channels for either game. Preserve `single_frame_encode=true`, one sampler step, and the instruction associated with each environment. ### Choose the execution horizon `configs/games/train.json` saves `n_action_steps=8`. The shooter playback experiment instead executes **2 actions per plan** and reports better aim than executing 8. Both use a 32-action prediction. At 15 Hz: | Executed actions | Simulated/control time covered | | ---------------- | ------------------------------ | | 2 | About 133 ms | | 8 | About 533 ms | | 32 | About 2.13 s | The 79 ms plan computation is additional work. If the game waits for inference, wall time includes that pause; if it keeps running, the observation ages during inference. Record which mode you evaluate. The tables do not establish a single universal latency/accuracy tradeoff for other games. The loop below explicitly executes two rows from `predict_action_chunk`, so it does not use the export's queued `select_action` horizon. To use `select_action` with a different horizon, [save and reload the configured export](/flux_3/flux3_action_inference#cameras-and-sampling-settings). ### Connect a shooter environment Your environment adapter supplies four functions: `read_frame()` returns a uint8 HWC RGB image; `execute_action(command)` applies four controls; `wait_for_tick()` maintains the 15 Hz action rate; `episode_done()` reports the terminal state. The application must handle inference overruns if the game runs continuously. ```python theme={null} import torch from flux_action.policy import FluxActionPolicy policy = FluxActionPolicy.from_pretrained("outputs/games-export", device="cuda") def play_shooter(read_frame, execute_action, wait_for_tick, episode_done): policy.reset() last_action = torch.zeros(1, 4, device=policy.device) while not episode_done(): frame = read_frame() # (H, W, 3), uint8 RGB observation = { "images.game": torch.from_numpy(frame).permute(2, 0, 1)[None] .to(policy.device).float() / 255, "state": last_action, "task": ["play the shooter: hunt the grunts, dodge the plasma, stay alive"], } with torch.inference_mode(): plan = policy.predict_action_chunk(observation) for i in range(2): command = plan[0, i].clamp(-1, 1).clone() command[3] = (command[3] > 0.5).to(command.dtype) # fire execute_action(command.cpu().numpy()) last_action = command[None] # the action actually executed wait_for_tick() if episode_done(): break ``` Fire is binary, so decode its continuous prediction with a 0.5 threshold. Update state with the executed action after clipping and decoding. Reset state to zero at the start of each new episode. Over a network, timestamp each observation and discard commands whose intended control step has already passed. ### VECTOR Use the task instruction: ```text theme={null} drive the racer: read the road, brake before the bends, pass the traffic, take the fuel ``` The joint export still returns `(1, 32, 4)`. Send only the first three channels as `[steer, throttle, nitro]`, using the game's control decoder. Set the fourth state channel to zero. VECTOR's fourth channel is padding and was masked out of the training loss. The two-action recommendation above is from the shooter experiment; evaluate the horizon separately for the racer. ### Measure a run Record the checkpoint, seed, execution horizon, and whether the game pauses for inference. Measure latency after compilation and warmup; the 76 ms export measurement and 79 ms game result came from separate runs. See [inference performance](/flux_3/flux3_action_inference#memory-and-performance) for serving options. Compare the model and baselines on the same held-out seeds. The reported results cover one seed per game. The release does not include the complete evaluator, so document your scoring and termination rules.