GPT-Red, OpenAI's Super-Hacker That Breaches Other Models (and a Vending Machine)
OpenAI has built a hacker: a language model trained to breach other language models, starting with its own. It is called GPT-Red and the company recently shared details, describing it as the sparring partner that tests the defenses of its systems before they are released. The latest version of its flagship model, GPT-5.6, was specifically trained against GPT-Red, and according to OpenAI, this makes it the most resistant release ever produced against cyber attacks.
The task of GPT-Red is to automate red-teaming, the systematic search for ways to breach or hijack a system, usually assigned to teams of human testers. The goal is to find as many vulnerabilities as possible to correct them before the final version is released. With the models used as agents, capable of interacting with files, websites, third-party code, and other agents, keeping up with all possible attacks becomes prohibitive for humans alone. “The risk surface grows, and with it the radius of possible damage,” observes Nikhil Kandpal, a researcher at OpenAI and one of the project authors.
The main target is prompt injection: hostile instructions hidden in text that the model encounters, such as an email, a webpage, or a file, designed to make it do things that developers and users would rather it not, like copying confidential information, sabotaging a codebase, or producing harmful output.
A "Dojo" Where Models Fight Each Other
To build GPT-Red, researchers started with a model not trained for attacks and placed it in a game loop against itself along with several defender models. GPT-Red was rewarded when an attack succeeded, while defenders were rewarded for successfully repelling it. Round after round, the attacker learned to hit better and the defenders learned to defend better.
The training took place in a sort of dojo designed to replicate the scenarios in which models end up operational: web browsing, reading emails and calendars, modifying code. OpenAI claims to have dedicated some of the largest computing sessions ever employed to this project, an amount it defines as unprecedented for security work. “Compared to a human red-teamer, the model is great at pinpointing exactly what works, what is most effective,” says Dylan Hunn, another project author. “It is extremely persistent in digging deep into an attack it has uncovered.”
From this persistence, OpenAI claims, a class of attack that researchers say they have never seen before was born, dubbed fake chain of thought. The chain of thought is a sort of diary in which a model notes down partial results while working on a problem; GPT-Red found a way to insert a false entry, leading the targeted model to act on counterfeit information. “It’s like telling you that 1+1=3 and that you’ve already verified it,” explains researcher Chris Choquette-Choo. “The model goes: ‘Oh, sure,’ and spits out 3.”
Attacks in Decline, but with Blind Spots
The numbers released by OpenAI are quite eloquent: when testing its most effective attacks, the company states that over 90% worked against GPT-5, released in August of last year, while against the new GPT-5.6 the percentage drops below 23%. In a re-creation of a 2025 test, GPT-Red reportedly outperformed human red-teamers by a wide margin; one of the reconstructions discusses 84% of scenarios breached versus 13% for human attackers. The attacker was also tested against Vendy, the agent that manages a vending machine created by Andon Labs: it was able to change the prices of items and cancel a customer order. OpenAI claims to have reported the vulnerabilities.
GPT-Red struggles with attacks that require prolonged back-and-forth interaction with the target, ground where a human attacker moves comfortably, and it is not yet effective at hiding instructions within images. Human testers continue to catch things it misses. “I believe human expertise will remain important,” comments Jessica Ji, an AI security analyst at CSET of Georgetown University, who finds the self-playing setup promising.
As expected, OpenAI will not release GPT-Red, to keep its capabilities away from those who would actually hijack agents, and it is confident that any potential imitator would not reach the same level. After all, the project has absorbed the work of a dedicated team for over a year utilizing the computing resources of a leading AI company. The underlying idea is a flywheel: using today’s models to strengthen those of tomorrow, scaling security at the same speed that capabilities are scaling. The complete paper is expected in the coming days.