Share this
Black Hat 2023 Review: LLMs Everywhere
by David Elkind on Aug 15, 2023 1:12:38 PM
I attended as many Black Hat briefings as possible this year. As a data scientist, I paid particular attention to the data science, machine learning, and artificial intelligence talks. Before we get into details, let’s address the elephant in the room: ChatGPT and LLMs.
Yes, it did seem like every third talk was about trying to apply a large language model (LLM) to either hack or secure a computer. I don’t doubt that the hacker and security communities will continue to extract value from the compressed knowledge stored in large language models, and made available in user-friendly interfaces such as ChatGPT. But the organizations that extract the most value from LLMs will be the ones that are best able to take advantage of these models’ power for interpreting language. One talk in particular did an excellent job of this.
This Year’s Best Application of Large Language Models (LLMs)
In my opinion, the talk that made the best use of a large language model was “IRonMAN: InterpRetable Incident Inspector Based ON Large-Scale Language Model and Association miNing” by Sian-Yao Huang, Cheng-Lin Yang, and Chung-Kuan Chen at CyCraft Technology. The basic idea is to borrow the strength of LLMs in interpreting natural language, and use that interpretive power to create vector representations of Windows command lines.
Natural languages are very flexible, with many ways to express the same information. Likewise, command lines have a certain amount of flexibility. The first kind is obfuscation: There are several equivalent, distinct ways to write the same command. This is important in a security context because these obfuscations can evade detection with tools like regular expressions.
The second source of nuance is that the same string can have different meanings in different contexts. As an example, a user could invoke a command foo
and also pass an argument named foo
to the command; these usages could have the same or distinct meanings. A regular expression would hit on both of those usages, even if our intention is to only capture one of them.
LLMs take (encoded) strings as inputs and yield numerical representations as outputs. The speakers in this talk leverage these numerical outputs, by observing that equivalent command lines tend to be closer to each other than they are to dissimilar command lines, even when obfuscation is used. This gives the model a certain amount of robustness to variations in the input, and allows one to group together command lines from different incidents, facilitating attribution.
I want to emphasize that the main reason this talk is so intriguing to me is that it really leaned on the LLM for the thing that it is best at (interpreting text inputs) and incorporated that utility into a security workflow. Interpreting texts has enormous value for security researchers; using LLMs to do at machine speed what was previously a human-speed task is a big deal.
This talk does not rely on the chat interface at all! Instead, it peeks “under the hood” to work directly with the numerical representations that the model uses to interpret text.
In my humble opinion, the weaker LLM talks focused on the cat-and-mouse aspects of the chatbot interfaces—using ChatGPT to create “black hat” stuff. While the chat interface is impressive, and it can generate some amusing outputs, I don’t see them as a big value-add for security researchers, especially in light of the untrustworthiness of the results. For instance, a recent study found that ChatGPT can generate plausible-but-incorrect answers to Stack Overflow questions 52% of the time. I would expect even worse results if we asked ChatGPT to respond to a security incident.
Looking to the Future
LLMs are a powerful tool, but to truly leverage that power, security researchers will need to think carefully about how to wield that tool. Where are security researchers bottlenecked? Are those bottlenecks related to interpreting large amounts of text? In the security space, I anticipate that LLM methods will find homes in assessing security risks of source code, the code on web pages, and even decompiled binary executable. I hope that we see some of these applications at next year’s Black Hat.
Share this

The term “artificial intelligence (AI)” was first coined in 1956. While progress stalled for many years, we can thank IBM for sparking real interest in AI as viable technology: First in 1997 when the computer Deep Blue defeated a chess champion and again in 2011 when Watson won Jeopardy!

Hackers have long understood that the most sophisticated firewall is no match for a well-placed psychological trick. While many focus on the technical prowess of cybercriminals, the real magic often lies in their ability to manipulate human behavior. By exploiting our natural tendencies and cognitive biases, hackers can slip past even the most robust security systems. It's not just about cracking codes; it's about cracking the human psyche.

Artificial intelligence is transforming business operations, automating everything from customer service to data analysis. But with these advancements come new security challenges. AI-driven cyber threats are becoming more sophisticated, enabling attackers to automate phishing campaigns, generate malware, and exfiltrate sensitive data at scale. Without proper safeguards, AI tools can unintentionally leak corporate secrets or connect to malicious ...