A coding AI Agent inspects the current state, calls tools, writes and runs a short program, and reads what came back. That loop maps unusually well onto the boundary between a frontier model and a robot's control stack, because what a robot needs from a model is rarely a single instruction and often a small procedure.
Robot tasks involve a sequence of geometric operations with checks between them: perceive, verify held, move, verify placement. Expressing that as a short program the AI Agent writes — rather than as a stream of separate model calls — reduces round trips, makes the sequence inspectable, and lets deterministic code run the parts that do not need reasoning.
It also gives the AI Agent a natural way to handle state: a program can hold intermediate results, and its output is something the AI Agent can read back.
This is not a new idea in robotics. Program-as-policy approaches had a model write a policy expressed as code, with perception primitives supplied as callable functions. What changed is the capability of the coding model and the quality of the tool interfaces it is given.
Generated code assumes its preconditions hold. A program that assumes a successful grasp will continue into a failed placement rather than stopping, unless the primitives it calls report the effects it depends on. That is why the interface matters more than the code: a program is only as correct as the evidence its tools return.
In Teach-and-Grow Learning, Codex connects the AI Agent to the robot tools, and each subgoal is wrapped in a Skill Block with an outcome test rather than being assumed to succeed. Generic verification — “did the gripper close?” — is not treated as proof that the intended physical effect occurred.
A model that inspects the robot's state, calls tools, writes and runs a short program, and reads the result. It fits robotics because a robot usually needs a small procedure rather than a single instruction, and because a program can hold intermediate results between steps.
They can drive one through tools, but they do not produce control-rate joint commands. In TGL, Codex connects the AI Agent to the robot tools, while detection, RGB-D geometry, Contact-GraspNet, MPLib and controllers do the physical work.
Codex is the coding AI Agent in TGL's implementation. What TGL adds is the contract around each call: a subgoal is wrapped in a Skill Block with a stated effect and an outcome test, so running a program is not the same as assuming it worked.
