OpenAI's latest org shake-up directly affects how robotics engineers evaluate control stacks. According to an exclusive Fortune report published on November 13, 2026, OpenAI disbanded its stand-alone robotics team after finding that general-purpose frontier models such as GPT-6 Astra can command robots directly without robotics-specific training. Skipping trajectory collection and policy fine-tuning in favor of a multimodal API call sounds tempting. Then the clock starts. Before planning around cloud robot control, check inference latency, token accounting, and current reproduction gates.
The clearest engineering benchmark available right now comes from Astranomy, an empirical study released for CoRL 2026 by the NVIDIA Seattle Robotics Lab, where researchers connected GPT-6 Astra (set to medium reasoning effort) zero-shot to a simulated DROID robot arm across all 120 pick-and-place tasks in RoboLab-120. At its peak configuration, Astra cleared 74.8% of tasks, beating the top specialist policy on the leaderboard, Lakeside (51.0%), by 23.8 percentage points. Direct API calls, however, averaged 18.56 seconds apiece.
Three Low-Res Camera Feeds Work Zero-Shot, Though Stateless Calls Favor End-Effector Poses
Traditional vision-language-action models (VLAs) like π0.5 bake their action format into training weights. GPT-6 Astra defines its entire control contract inside the prompt. In NVIDIA's setup, each call sends three camera views -- left shoulder, right shoulder, and wrist -- resized to 384 x 216 pixels alongside robot and camera geometry descriptions, capped at 8,192 completion tokens covering both reasoning and output actions.
Action space sensitivity depends heavily on whether the model runs statelessly or inside an agentic loop. Under stateless Direct Policy across all 120 tasks in RoboLab-120, prompting GPT-6 Astra for absolute end-effector poses (EEF) yielded a 66.3% success rate (76.7% completion score), while joint positions still reached 50.2% (63.7% score) and joint-space B-splines dropped to 38.8% (53.7% score). That gap does not mean Astra cannot operate in joint space: on the study's 30-task ablation subset, Direct Policy hit 72.7% with EEF, whereas switching to the persistent Agentic Policy narrowed the gap between EEF (86.0%) and joint positions (84.0%) to just 2 percentage points. Locking in absolute EEF is primarily a safeguard for stateless Direct Policy calls.

Variable-length action chunking carries its own trade-off. Instead of emitting a fixed step horizon on every tick, Direct Policy lets Astra choose how many actions to output within a prompted bound, defaulting to 3 to 15 steps. Stretching that bound to 16 to 32 steps is not a free speedup: on the 30-task ablation subset, widening the dynamic horizon from 3-15 to 16-32 steps cut inference calls on successful episodes by 62%, yet average task completion time stayed roughly unchanged and success rate fell from 72.7% to 66.7%.
Three Wiring Modes, Three Steeply Different API Bills
NVIDIA tested three closed-loop setups, exposing a sharp split between error recovery, call latency, and token consumption:

- Direct Policy (stateless API calls): Maps sampled observations straight to an action chunk with no persistent conversation history across calls. Result: 66.3% success, 18.56 seconds per call, and 160k tokens ($1.96) per successful episode.
- Agentic Policy (persistent Codex session): Runs GPT-6 Astra inside a persistent Codex session that retains full text and visual interaction history and returns motion feedback after holding each target for 1 to 5 control steps. Although the harness supports optional shell and image tools for ablations, the benchmark configuration that achieved 74.8% success (83.6% score) explicitly disabled optional tools as well as built-in shell, image-view, and code-execution tools, keeping only the policy-output tool and
memory_*observation-archive tools. Calls averaged 13.37 seconds, while token usage surged to 30.9M tokens ($36.40) per successful episode. - Orchestrated Policy (cloud supervisor over a local model): Uses Cosmos3-Nano-Policy, a world action model fine-tuned on DROID data, to propose 15-step trajectories while GPT-6 Astra reviews progress inside Codex with bounded memory and optional native tools disabled, overriding drifted proposals or issuing end-effector targets when needed. It scored 66.5% success (79.1% score) with 11.50 and 2.74 second latencies, averaging 11.0M tokens ($17.72) per successful episode.

Those gains in Agentic mode come from persistent cross-step history, memory_* observation archiving, and 1-to-5-step motion feedback rather than external code or image tools -- and carrying that multi-turn visual context pushes mean spend from $1.96 to $36.40. Note an internal reporting discrepancy in Astranomy as well: the key takeaways summarize these figures loosely as cost "per episode," whereas Figure 2 and the cost section specify "API cost per successful episode" based on October 1, 2026 pricing. Failed runs that exhaust step limits without completing the task sit outside that average, so $1.96 is not a guaranteed per-episode debugging cap. Because of inference expense, NVIDIA's team also limited their full 120-task evaluation on NVIDIA L40s GPUs to 5 fixed-seed episodes per task.
Simulators Pause Physics During Cloud Inference, and Reproduction Still Has Access Gates
One sentence in Section 2 of the Astranomy study frames every runtime number: the 15 Hz controller runs with simulation paused during inference.
Real hardware has no pause button. While GPT-6 Astra spends 13.37 seconds or 18.56 seconds per call, simulated objects wait frozen in place. Local specialist models on the same table return action chunks in 713.76 milliseconds (Cosmos3-Nano-Policy) and 46.96 milliseconds (π0.5), though those figures still measure per-call policy inference rather than step-by-step synchronous execution at the 15 Hz controller rate, and the report includes no physical robot timing validation. Carnegie Mellon University Robotics Institute assistant professor Guanya Shi noted the same constraint to Fortune, pointing out that cloud language models run slowly and require uninterrupted connectivity, leaving a clear need for smaller, specialized models running offline on onboard computers.
Task-level results in Table A1 also require separating binary task success rate from partial completion score. On instruction-heavy sorting such as stacking blocks in red, blue, green, yellow order (BlockStackingSpecifiedOrder), Cosmos3-Nano-Policy and π0.5 both recorded 0.0% success, whereas Astra hit 100.0% success in Direct EEF and 80.0% success (93.2% score) in both Agentic and Orchestrated modes; on clearing small toys while leaving a birdhouse alone (CleanUpToys), Direct EEF and Agentic reached 80.0% success, while Orchestrated dropped to 0.0% success (42.2% completion score) alongside both specialist models. Conversely, on contact-sensitive tasks like BagelsOnPlate and PickDrill, all three Astra policy forms posted a 0.0% task success rate -- even where partial progress registered non-zero completion scores (20.0% for Agentic and 30.0% for Orchestrated on BagelsOnPlate, and a 100.0% stage score despite 0.0% final success for Orchestrated on PickDrill).
If you want to reproduce these benchmarks today, NVIDIA has published the ai_native policy directory in NVlabs/RoboLab on GitHub. Both Direct-API (defaulting to gpt-6-astra on OpenAI's Chat Completions endpoint) and Agentic (requiring @openai/codex@0.149.1) are available now, while Orchestrated remains marked "Coming soon." Two practical gates apply: you need a Linux GPU workstation with IsaacSim, IsaacLab, and RoboLab assets installed, and the repository explicitly notes that your OpenAI account must already have access to GPT-6 Astra -- a bundled client model catalog is not proof of account access.
Once model access is in hand, treat deployment architecture as a hypothesis to validate on hardware. For quasi-static tabletop setups or offline task decomposition where pauses between motions are harmless, start with Direct-API using absolute end-effector poses and default 3-to-15-step chunks to avoid multi-turn context bloat, while budgeting extra tokens for failed episodes and retries. Where real-world tasks cannot freeze for 13.37 to 18.56 seconds per cloud call, test decoupling policy update rate from the 15 Hz control loop via action chunking or pairing a 46.96 ms to 713.76 ms onboard policy with asynchronous GPT-6 Astra supervision, and verify actual closed-loop latency on physical hardware.