Six robot-control AI models tested in simulation all started moving when researchers gave them an empty instruction, and five of the six often completed a task nobody had specified, according to an arXiv preprint submitted Oct. 30. The preprint is not peer reviewed, and the robot AI models were tested only in simulation. The authors, Minsu Jang, Jae-Woo Choi, Youngwoo Yoon and Dohyung Kim, call the problem unintended task execution.
How the Test Worked
The paper, “VLA Policies Do Not Know When Not to Act: Unintended Task Execution and Start Suppression,” tested six vision-language-action checkpoints, the class of models that turn a camera view and a text command into robot movement. The researchers ran them in the LIBERO-Goal simulation with the starting scene fixed and changed only the instruction. They tried empty, neutral, nonsense, infeasible and wrong-task instructions. Every checkpoint began moving on the empty instruction. Two GR00T checkpoints also moved on infeasible instructions.
Which Fixes Worked
The authors tested three remedies. A text prefix telling the robot not to move stopped none of the six, a result of zero out of six. A gate that holds the robot when its action output matches the output it produces for an empty instruction worked on five of six, about 83%. Fine-tuning kept the models from completing a task on non-task inputs but did not reliably keep the robot still, and it failed to suppress movement on infeasible instructions. The prefix result is the one with a practical lesson for anyone building these systems. Telling a model in words to stay put did not work in any of the six, while a check on the model’s output did on most.
What the Authors Warn About
The authors wrote that unintended actions can cause risk or inconvenience to people and the environment. The paper runs 17 pages with 3 figures and 13 tables, and the code is posted to a public repository.
The limits are plain. A simulation does not have a person standing next to the arm, and one benchmark is not every task. The gate’s five-of-six result also leaves one model that no tested fix handled. The authors describe no hardware tests, so the next result to watch is whether the same behavior shows up on physical robots.
