OpenAI’s GPT-4 can exploit real vulnerabilities by reading security advisories

AI agents, which combine large language models with automation software, can successfully exploit real-world security vulnerabilities by reading security advisories, academics say.

In a recently published paper, four computer scientists from the University of Illinois Urbana-Champaign (UIUC), Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang, report that OpenAI’s GPT-4 large language model (LLM) can exploit autonomously vulnerabilities in real conditions. -World Systems if you receive a CVE notice describing the flaw.

“To show this, we collected a data set of 15 vulnerabilities from one day that includes those classified as critical severity in the CVE description,” the US-based authors explain in their paper.

“When given the CVE description, GPT-4 is able to exploit 87 percent of these vulnerabilities compared to 0 percent for all other models we tested (GPT-3.5, open source LLM) and scanners. open source vulnerabilities (ZAP and Metasploit).

Extrapolating to what future models can do, it seems likely that they will be much more capable than what the children in the script may have access to today.

The term “one-day vulnerability” refers to vulnerabilities that have been disclosed but not patched. And by CVE description, the team is referring to a CVE-tagged advisory shared by NIST, for example this one for CVE-2024-28859.

The tested models that were not successful: GPT-3.5, OpenHermes-2.5-Mistral-7B, Llama-2 Chat (70B), LLaMA-2 Chat (13B), LLaMA-2 Chat (7B), Mixtral-8x7B Instruct, Mistral (7B) Instruct v0.2, Nous Hermes-2 Yi 34B and OpenChat 3.5 did not include two main commercial rivals of GPT-4, Anthropic’s Claude 3 and Google’s Gemini 1.5 Pro. The UIUC experts did not have access to these models, although they hope to test them at some point.

The researchers’ work builds on previous findings that LLMs can be used to automate website attacks in a sandboxed environment.

GPT-4, said Daniel Kang, an assistant professor at UIUC, in an email to Register“it can actually autonomously carry out steps to perform certain exploits that open source vulnerability scanners cannot find (at the time of writing).”

Kang said he hopes LLM agents, created (in this case) by connecting a chatbot model to the ReAct automation framework implemented on LangChain, will make exploitation much easier for everyone. We’re told these agents can follow links in the CVE descriptions for more information.

“Furthermore, if you extrapolate to what GPT-5 and future models can do, it seems likely that they will be much more capable than what scripted children can access today,” he said.

Denying the LLM (GPT-4) agent access to the relevant CVE description reduced its success rate from 87 percent to just seven percent. However, Kang said he does not believe limiting the public availability of security information is a viable way to defend against LLM agents.

“I personally don’t think security through obscurity is sustainable, which seems to be the prevailing opinion among security researchers,” he explained. “I hope my work, and other work, encourages proactive security measures, such as updating packages periodically when security patches come out.”

The LLM agent failed to exploit only two of the 15 samples: Iris XSS (CVE-2024-25640) and Hertzbeat RCE (CVE-2023-51653). The former, according to the article, was problematic because the Iris web application has an interface that is extremely difficult for the agent to navigate. And the latter presents a detailed description in Chinese, which presumably confused the LLM agent operating under an English message.

Eleven of the tested vulnerabilities occurred after GPT-4’s training outage, meaning the model had not learned any data about them during training. Their success rate for these CVEs was slightly lower: 82 percent, or 9 out of 11.

As for the nature of the bugs, they are all listed in the document above and we are told: “Our vulnerabilities span website vulnerabilities, container vulnerabilities, and vulnerable Python packages. More than half are classified as ‘high’ or ‘criticisms’. severity for the CVE description.”

Kang and his colleagues calculated the cost of carrying out a successful LLM agent attack and came up with a figure of $8.80 per exploit, which they say is about 2.8 times less than what it would cost to hire a tester. of human penetration for 30 minutes.

The agent code, according to Kang, consists of only 91 lines of code and 1,056 tokens for the prompt. OpenAI, the maker of GPT-4, asked the researchers not to make their guidance public, although they say they will provide it upon request.

OpenAI did not immediately respond to a request for comment. ®