Share this
Black Hat 2023 Review: LLMs Everywhere
by David Elkind on Aug 15, 2023 1:12:38 PM
I attended as many Black Hat briefings as possible this year. As a data scientist, I paid particular attention to the data science, machine learning, and artificial intelligence talks. Before we get into details, let’s address the elephant in the room: ChatGPT and LLMs.
Yes, it did seem like every third talk was about trying to apply a large language model (LLM) to either hack or secure a computer. I don’t doubt that the hacker and security communities will continue to extract value from the compressed knowledge stored in large language models, and made available in user-friendly interfaces such as ChatGPT. But the organizations that extract the most value from LLMs will be the ones that are best able to take advantage of these models’ power for interpreting language. One talk in particular did an excellent job of this.
This Year’s Best Application of Large Language Models (LLMs)
In my opinion, the talk that made the best use of a large language model was “IRonMAN: InterpRetable Incident Inspector Based ON Large-Scale Language Model and Association miNing” by Sian-Yao Huang, Cheng-Lin Yang, and Chung-Kuan Chen at CyCraft Technology. The basic idea is to borrow the strength of LLMs in interpreting natural language, and use that interpretive power to create vector representations of Windows command lines.
Natural languages are very flexible, with many ways to express the same information. Likewise, command lines have a certain amount of flexibility. The first kind is obfuscation: There are several equivalent, distinct ways to write the same command. This is important in a security context because these obfuscations can evade detection with tools like regular expressions.
The second source of nuance is that the same string can have different meanings in different contexts. As an example, a user could invoke a command foo
and also pass an argument named foo
to the command; these usages could have the same or distinct meanings. A regular expression would hit on both of those usages, even if our intention is to only capture one of them.
LLMs take (encoded) strings as inputs and yield numerical representations as outputs. The speakers in this talk leverage these numerical outputs, by observing that equivalent command lines tend to be closer to each other than they are to dissimilar command lines, even when obfuscation is used. This gives the model a certain amount of robustness to variations in the input, and allows one to group together command lines from different incidents, facilitating attribution.
I want to emphasize that the main reason this talk is so intriguing to me is that it really leaned on the LLM for the thing that it is best at (interpreting text inputs) and incorporated that utility into a security workflow. Interpreting texts has enormous value for security researchers; using LLMs to do at machine speed what was previously a human-speed task is a big deal.
This talk does not rely on the chat interface at all! Instead, it peeks “under the hood” to work directly with the numerical representations that the model uses to interpret text.
In my humble opinion, the weaker LLM talks focused on the cat-and-mouse aspects of the chatbot interfaces—using ChatGPT to create “black hat” stuff. While the chat interface is impressive, and it can generate some amusing outputs, I don’t see them as a big value-add for security researchers, especially in light of the untrustworthiness of the results. For instance, a recent study found that ChatGPT can generate plausible-but-incorrect answers to Stack Overflow questions 52% of the time. I would expect even worse results if we asked ChatGPT to respond to a security incident.
Looking to the Future
LLMs are a powerful tool, but to truly leverage that power, security researchers will need to think carefully about how to wield that tool. Where are security researchers bottlenecked? Are those bottlenecks related to interpreting large amounts of text? In the security space, I anticipate that LLM methods will find homes in assessing security risks of source code, the code on web pages, and even decompiled binary executable. I hope that we see some of these applications at next year’s Black Hat.
Share this
Categories
- Featured (265)
- Protective DNS (22)
- IT (15)
- IndyCar (9)
- Content Filtering (8)
- Cybersecurity Brief (7)
- IT Challenges (7)
- Public Wi-Fi (7)
- AI (6)
- Deep Dive (6)
- Malware (4)
- Roaming Client (4)
- Team (4)
- Compare (3)
- MSP (3)
- Phishing (3)
- Tech (3)
- Anycast (2)
- Events (2)
- Machine Learning (2)
- Ransomware (2)
- Tech Stack (2)
- Secure Web Gateway (1)
Earlier this month I joined Mikey Pruitt, our Global Partner Evangelist, on the DNSFilter podcast dnsUNFILTERED to discuss my 2025 cybersecurity predictions. We had a lot of fun and covered all of the points I’ll outline here, but I wanted to go deeper. My 30 years of cybersecurity experience have given me a strong sense of where we’re heading as an industry—the shift to the cloud in many ways is a precursor in the adoption of AI and the future...
Most businesses only think about DNS security after an attack has already occurred. By then, the damage is done - downtime, lost revenue, compromised data, and a tarnished reputation. In an environment where cyber threats are constantly evolving, a reactive approach to DNS security simply isn’t enough.
Customer experience is the secret sauce that sets successful Managed Service Providers (MSPs) apart from the rest. In a market teeming with competition, you need to offer more than the best technology or the lowest prices. It's about how clients feel when they interact with your services. A stellar customer experience can transform a one-time client into a loyal advocate, while a poor one can send them running to your competitors. According to a ...