Embedded Files
Any information presented in this article is for educational/documentation purposes ONLY.
A big part of being a good threat intelligence analyst is to know what you're up against, and who everyone else is up against.
It's not just the who, not just the hacker group or threat actor, but also, what they attack.
And lately, your favorite ne'er-do-wells just found a new target.
"Artificial Intelligence" has become the new "High Definition", or "Blockchain", In terms of marketing terms, and it seems that everyone, from tech companies, to healthcare suppliers, have started, in some variance, to include A.I
This, of course, is done not only to stay ahead of the competition in an ever-computerizing age and culture, but also because artificial intelligence is a valuable tool in humanity's arsenal.
It's an asset, to say the least, from DeepMind/Isomorphic’s AlphaFold 3 predicting full biomolecular complexes, accelerating target/mechanism work and early medicinal chemistry; To Nasdaq embedding AI into its surveillance platform, enhancing market-abuse detection for exchanges and regulators worldwide.
AI is here to stay, and to it's detriment, it's making it a prime target for attackers, and bodies such as OWASP have already documented the ways in which AIs, and more specifically LLMs, can be exploited.
Although OWASP definitely has the right idea in mind, in my experience when it comes to learning, I've seen people leaning towards what they know, as opposed to external sources, and social media tends to give us a lot more than we can chew, so I took a deep dive into Discord of all places, a gaming-oriented communications platform that hosts millions of server pertaining to any topic you can think of, from cybersecurity professionals to anime lovers, if it exists, there's going to be a discord server discussing it.
I use discord, both for leisure, as an avid gamer, and since it's hosting a couple the sources I use to get news regarding low level exploits, Hacking and CTF events, and, luckily, AI Red Teaming.
So I decided to test the waters, and reached out to some of the server's more experienced users, some I've spoke to before, and one of them, a kind individual by the name of Zea, was more than happy to respond to my questions regarding AI Red Teaming, and much, much more.
Zea is a very experienced LLM Red Teamer, he took part in LLM Adversary CTFs, and has cracked the system prompts for models long before our talk, so it's clear that he's truly a professional at what he does.
And I decided to ask him 3 questions that I thought we as cybersecurity professionals, need the answers to.
The first, was one regarding the "How", the way - and this is how it went down:
And Zea did not fail to deliver.
Aside from telling me what I already expected to hear, which is that he does **not** use OWASP's LLM Top 10, Zea also threw a proverbial *mountain* of open source, free sources at me, which I intend to explore in articles later down the line.
But he also mentioned how the community built from the ground up, at some point, a data bank used by pros and newbies alike to learn how to attack LLMs, the right way, or as they put it:
"In order to trust the output, one must understand the input."
Truly, words to live by.
My second question focused on the "Now":
And his reply, as usual, was fascinating.
This not only shows that AI's here to stay, as we all know, but also that people use hobbyist AI-red-teamers to test their own systems, in an almost crowdfunded-QA of sorts.
Zea shows us that although he's a professional in his field, his expertise lies in the field of AI Red Teaming, as opposed to traditional cybersecurity, which is uncommon, given the fact that machine learning adversarial simulation, as well as LLM adversarial simulation, are usually fields that host professionals with extensive backgrounds in other fields, mainly due to them being relatively new to the cybersecurity landscape, it seems that the global field has yet to have enough time to create it's own group of targeted pros, like Zea.
The last question I asked Zea zeroed in on the Intelligence side of things, as it is yours truly's specialty:
And he delivered:
It appears that the courtesy given to hackers worldwide by companies to, ironically, "hack responsibly", exists in the world of AI Red Teaming.
As Zea describes it, the field seems to follow the long standing trend that's been going on for decades in the realm of vulnerability disclosure and the responsible of cybersecurity enthusiasts to, well, enthuse with caution.
Lastly, I asked Zea to narrow down his stance on AI Red Teaming to a single sentence.
This marked the end of our impromptu interview, and if we're being honest, I think there's a lot to say.
AI is indeed the new "Everything". It must be in your company for you to get with the times, but, as Zea puts it, it adds an attack surface, that with time, will evolve according to industry standards, but also, according to the devout communities and professionals that take part in helping it evolve, more often than not, by tearing it to pieces.
Next time, we'll see what our much less inviting folk on the dark web say about the LLMs that became a part of our day to day, and professional lives, and dive deeper into Zea's mountain of resources.
Until next time, fellas,
Fear No Evil.
Google Sites
Report abuse