Oops, OpenAI Hacked Australia's Health System

Marc Laliberte  0:00  
Hey everyone! Welcome back to the 443 Security Simplified. I'm your host Marc Laliberte, and joining me today

Corey Nachreiner  0:07  
is Corey "Spoofed Identity" Nachreiner, or am I really Corey Nachreiner? I guess we'll find out.

Marc Laliberte  0:13  
Is this finally the episode where we deepfake both of us, and it's only AI all the way down?

Corey Nachreiner  0:19  
God, I hope not.

Marc Laliberte  0:20  
At some point, we're going to, and no one will know. But anyways, that is not today. On today's episode, we will discuss the latest updates on North Korean threat actors targeting IT and software development professionals, as well as the organizations they work for. We'll go into the shiny hunters breach of the FBI job site, and why Corey's identity got stolen, and then we will end with yet another frontier AI model hacking another organization.

Corey Nachreiner  0:51  
And Mark, I would argue all three stories are about identity, different layers. One of them is a stolen identity. One of them Is like defended, fabricated identity, and one of them is just defending their identity in a weird way. So I think there's identity stories in all of them. By the way, I also have some quick updates on some of the stories from last time.

Marc Laliberte  1:17  
Yeah, let's go ahead and authenticate our way in and get rolling.

Corey Nachreiner  1:27  
So just like a 60-second update on last time, as everyone probably remembers, we talked about the hyphe. Is that how you pronounce that extension for that image? The hyphen. I'm going

Marc Laliberte  1:36  
to pronounce it.

Corey Nachreiner  1:37  
Good, good. Really, it was more kind of a hack on OpenAI. We just described and we described the detail about some issues with with OpenAI too, but I believe it all stemmed from the the hyph issue. And I just wanted to to mention that the scope of that hyph flaw is well beyond the OpenAI issue that we were focused on. Apparently, the issue has reached Meta's core product suite, GitHub Enterprise servers have been affected by it, Discourse and AWS Link Services. So, our story focused a lot on the OpenAI stuff and the additional issues there. But just know that Hive image vulnerability is affecting a lot of other things. And the only oh, go ahead.

Marc Laliberte  2:20  
That tiny little building piece-that's the weak link-is actually a part of a hell of a lot more of our critical infrastructure. Interesting.

Corey Nachreiner  2:26  
Absolutely fun. It's funny how open source does that. The other little tiny update is, you know, we talked about the Cisco ISE flaw, a really critical flaw in in that that management package, but at the time Cisco published a whole lot of things. But it ends up that bundle of Cisco fixes grew to about 77 CVEs, and it actually five of them were CVS scores of 10. So just if you are a Cisco user, which I mean almost everyone probably uses their routers. Maybe besides the ISC thing we talked about, make sure that you got all of Cisco's critical updates from a few weeks ago.

Marc Laliberte  3:11  
Starting to think that we, as people in the industry, should expect more and more vulnerability releases from vendors across the board as a more user.

Corey Nachreiner  3:20  
Who would have maybe talked about that, and even has internal projects? I guess there's more to learn about that in the future, right? Zimmy, Sean.

Marc Laliberte  3:29  
Yeah. And anyways, let's move on to this week. And first story I wanted to bring up was a FBI report that was just published about a week and a half ago now, but just came onto at least my radar recently, where the FBI and some of their international counterparts from Japan, Australia, and Germany published an alert on Waterplum, which is a North Korean threat actor organization targeting IT professionals in Japan, the U.S. and Europe. Waterplum is a fun name. Absolutely. I would say actually we're

Corey Nachreiner  4:05  
talking. I don't. I did not Google this. I'm sure someone can do that in the background. I don't know. By the way, you mentioned the FBI, and I might have accidentally disclosed another story we're going to get to, but I would say this is FBI was definitely involved. If people are watching the video, they might see the screen. But this was a joint advisory, and I don't know about you, but I feel like this is kind of interesting because it's one of the first ones that Japan's, you know, police agency and cybersecurity agencies led on. So definitely something joint, and the FBI had their version of it, but I think the National Police Agency of Japan really took the lead on a lot of this stuff. It feels like

Marc Laliberte  4:49  
that's what it seemed like, yeah. But basically, this is a multi-page alert going through some of the techniques of Waterplum and how they're targeting both individuals and organizations. They mentioned some of their motives are either a cryptocurrency theft or just espionage against organizations. They often impersonate big name AI companies or cryptocurrency or NFT companies as kind of the lure to get people to come in. Pausing for a second, are there still NFT companies out there? I thought that that was like dead. Finally,

Corey Nachreiner  5:24  
gag. I mean, anybody that's still dealing with NFT companies, what are you thinking about?

Marc Laliberte  5:30  
My concerned ape, or no, concerned ape is the Stardew Valley developer. My bored ape is hemorrhaging value right now, Unfortunately, but anyways, moving on. So,

Corey Nachreiner  5:44  
don't we have kittens too? Didn't you make a kitten for me? And oh, our crypto

Marc Laliberte  5:48  
kitties. Oh man, I wonder if those are still worth anything. And hopefully, it's not like a tamagotchi where you got to keep feeding it or it dies because those are definitely long. So

Corey Nachreiner  5:58  
did. But anyway,

Marc Laliberte  6:01  
so Waterplum has successfully compromised over 30,000 devices in more than 100 countries around the world. They've used that to steal funds from over 7000 cryptocurrency wallets worth over 1.7 billion yen, or about 10 million U.S. dollars. So, some decent return on investment from them. They went through in the advisory talked about some of their techniques they use. So they'll try and operate as IT workers to go after organizations, or they will target software developers or IT folks at companies and try and trick them into running malicious code. So similar to other techniques we've talked about before, where they offer them a pretty lucrative-looking job. As part of it, they've got like a technical proficiency test that they want them to run. They get them to go and download malicious npm packages that they've already pre-seeded on the Node package index, or just like malicious GitHub repositories. But the end result of delivering one or more different malware variants that opens up a remote access tunnel to that developer or IT person's machine, and then from there they'll steal secrets, try and harvest any cryptocurrency wallets they find, and in some cases even use those secrets to go after the companies that these workers work for as well. Some of the data they steal includes all of the secrets you save in your web browser session, clipboard information, and screenshots of the desktop, the cryptocurrency wallets, and then just any file that looks like it might be a value off of the the victim's machine. The second one that we've talked about a few times in the U.S. but this was one specifically in Japan that they've dismantled. But these actors are also using laptop farms, where they get some enabler within the region, basically a willing participant within the country that they're going after.

Corey Nachreiner  7:55  
Sometimes called a mule.

Marc Laliberte  7:57  
Yep. Sometimes called a mule, and then use them to deploy what is effectively a laptop farm-a bunch of different laptops for one or more organizations where they're pretending to be someone in the country working for that company, just as a way to generate revenue in the form of like an actual paycheck from these companies, while actually working from North Korea, basically proxied through these laptops, in some cases, though, even though they were trying to earn a paycheck, they couldn't help themselves from doing more malicious actions. There was one example where they got hired to work on a company's website, and they ended up defacing it, rendering it inaccessible. There was another one where they extorted a company over a payment, and then published all of their proprietary source code online, so a few like straight up malicious actions taken beyond just the fraud of getting it employed by them and completing actual work for them. The report ended with some interesting mitigation recommendations. Basically, during the hiring process, they recommended validating the IP address of the person that you're potentially hiring, and making sure it comes from their actual residency. Check all the contact info and phone numbers. They said sometimes they'll you'll call back a phone number and find out that it was actually disconnected, so not real. Ask them to go through a detailed like explanation for all the skills they list. They mentioned sometimes these threat actors will actually divvy out kind of knowledge to different people. So the person, quote unquote, getting hired doesn't actually know everything. They only know one piece. So if you really quiz them, it might make it obvious. Ask them questions about their hometown and weather and hobbies. Like if they start saying, "Yeah, you know, it's really frigid and snowing right now in Austin, Texas, in the middle of July, might be a red flag that maybe they're not actually in that city. And they also noted that these workers tend to favor payment in cryptocurrency, which is interesting. I guess that might be in gig work. Might be something that wouldn't raise a red flag, but if like I got hired by a company, I said no, pay me in Bitcoin. I feel like that would be an immediate red flag for the company that's hiring me. But either way, the report's pretty interesting. Goes through a few examples, mostly catered around Japan and Japanese law enforcement's dismantling of these networks. But this is clearly still a technique or opportunity that's working for North Korean threat actors. Like they're clearly making money, they're stealing cryptocurrency, and I understand why they continue trying to do this. I don't know. What are your thoughts, Corey?

Corey Nachreiner  10:37  
Well, I think there's a number of of things. One, I think it's interesting that we always think in supply chain attacks we picture compromised packages or vendor breaches, but in this case it's like a freelancer's laptop. Just this whole hiring role, or you know, getting code to on a freelancer's laptop that is in a privileged network is is kind of an interesting take on how these supply chains are attacks are happening. I remember because of our ability to go to the FBI CISO Academy where they were talking about malicious insiders. You know, the the malicious insider thing is something that China's been using for a long time too, and it looks like North Korea is. So that's interesting. It's targeting developers a lot of the time. Like you know, having that test package and pretending to have a job often is targeting developers and asking them to go to some GitHub to do some test. You know, having additional thing like takeaways for those folks of if you're going to go to an unknown repo, you should definitely have you know it in restricted mode and say no to trust dialogs. And you know, even as someone looking for a job, be careful trusting unknown repos in the hiring process. I think there was all kinds of interesting things that could be tells that you mentioned. You know, hiring might be doing things like video calls, and these threat actors would be using AI-based face swapping technology that was driven by identity theft of passports and stuff. But if you know that stuff doesn't work very well, so interesting little tells like, oh, let's turn off video because the connection's bad, and background noises and and other things. I do think HR needs to, if they're hiring remotely, definitely do video calls and pay attention to these weird little techniques that are being used to spoof and masquerade on video calls too. Yeah, they

Marc Laliberte  12:44  
recommended in-person interviews too as another opportunity because clearly they're not going to come in person from North Korea.

Corey Nachreiner  12:52  
Absolutely, that that that is I think ideal. The issue is if you're going to remote hire, how do you do you send somebody out to someone's remote place, but that seems very ideal to me-a good tip. But hard to do in this hybrid remote world. I liked all of your tips you mentioned that they they said in this, like IP addresses that don't match, phone numbers asking them about their hometown and personal questions. But the one thing that like we're on a security podcast to security professionals. These are things HR needs to do. So really bringing this security professionals bringing this awareness and procedure to HR if they haven't figured out how like maybe smart ahead of the game secure considering HR organizations are starting to do this because they're noticing the issue, but if not, this is something we need to bring to HR. We're giving great tips to security professionals, but are they are they finding it? So yeah, it was definitely interesting. I'm and I also think even Japan, like the fact that they are publicly attributing this to North Korea and taking a stance in a story in a second, while it may not be nation state to nation state, the story will involve an organization that's an authority, like the Japan National Police or the FBI calling a threat actor out on doing something wrong, but then what can happen because of that? So I think good on Japan for publicly doing this with attribution, and for all of the the you know international agencies calling it out. But just the fact that this is public attribution is kind of nation-state global politics related to cybersecurity. That's kind of interesting.

Marc Laliberte  14:47  
Something tells me most countries don't really care a whole lot about pissing off North Korea, though. It's like that angry kid at the party that's too weak to actually do anything, but just really loud and annoying. Although they do have nuclear bombs. That's the thing. I mean, they they have

Corey Nachreiner  15:02  
nukes. It's clear. Like if you have some of our Western beliefs, this guy is a crazy dictator, and the issue is they're stealing. Like they're sanctioned. All of the countries pretty much have sanctioned the heck out of North Korea, so that they because they're such a bad actor to some countries on a global environment, we've made it hard for them to make money, you know, with global cooperation, and they're one of the few countries that isn't just doing espionage; they're stealing money because of these sanctions. So I would argue, while it's that little bully kid that maybe a big, powerful country is not that scared of your point about nuclear weapons and the fact that they're having real financial effect that's affecting people in many different countries shows that they're not just a little ineffective country. They're doing real malicious activity.

Marc Laliberte  15:58  
I think while you're right that this is something HR needs to be a part of solving too. We remember the the no before incident, where that was I think the first time we had a big spotlight shined on this type of activity. It was their security operations team that ended up after it made through like all the HR hiring, and it was because they had like rigorous controls around onboarding new employees and monitoring that initial first activity for anomalies.

Corey Nachreiner  16:26  
I would say it was the SOC. They literally had EDR type software, and because they had some sort of SOC monitoring that endpoint, I absolutely agree. Without security, they may never have known. But it was a combination of really good software that didn't just look for malware, but behaviors and a a SOC team that noticed.

Marc Laliberte  16:49  
Yep. So either way, like if you are currently hiring and hiring internationally, just maybe be aware and pay a little more close attention to some of the details that might let you catch one of these incidents as they're starting to occur, but let's move on to the second story. We are not done talking about the FBI because just a couple days ago, on September 22, Shiny Hunters, the infamous ransomware data theft organization, took down the FBI jobs.gov website, defacing with a banner on the homepage that says this site has been seized by shiny hunters. Meanwhile, they stole what they claim was three terabytes of data, including data on current and former agents and anyone that has applied for a job at the FBI. The data, including stuff that you would expect on a job application, like addresses and phone numbers, and even references and spouse information. But before we go too far into the details, I thought it was pretty damn funny that they put up a seizure notification on a website owned by the FBI. It is a little bit ironic, given how the FBI disrupts websites owned by shiny hunters and other threat actors. So plus one meant

Corey Nachreiner  18:09  
definitely funny in the meme standpoint, but I also think it's scary.

Marc Laliberte  18:13  
Yep,

Corey Nachreiner  18:13  
and I I don't want to give them the one thing that stands out for me on this is This is clearly, you know, a new motive. It is like shiny hunters-their usual motive for whatever they do, including stealing information that they're threatening to leak. Which I mean, we're getting to. They literally stole tons of private information of the FBI, and this is actually life safety, I believe, critical information, including family details, and are threatening to to release it. But we, I guess, we haven't got to their or or else what? Yeah, they ask into that for. Yeah, so basically, they're mad about the FBI doing a report on who the shiny hunters are and what they've done, you know, and specifically the tools, tactics, and procedures. So a while back, the FBI had released. You might have it offhand, but they released a very specific report about the shiny hunters, and they tried to describe how the shiny hunters, you know, extorts victims and why. And some of the things they said was Shiny Hunters is conducting swatting attacks against corporate workers, and they're doing sextortion threats and stuff like that. And it seems like Shiny Hunters are offended. Like they're they're not claiming they didn't hack people. In fact, they even have told reporters specific. I will get. I'll let you get into that. But but what we suggest or what they they say is the root cause for how they got into the FBI. But they're they're like no, we're technical hackers. We hack these people. We steal the data, and we ask them for money. We don't. You're exaggerating the claims, and we're severely offended that you're. Not describing our criminal activity properly.

Marc Laliberte  20:02  
Yeah, they put a big post up on their dark web forum, basically saying they're offended that the FBI alleged that sometimes they use exaggerated claims to extract payment from victims, and they said, "quote We wish to state unequivocally that our threats and claims are very real.

Corey Nachreiner  20:18  
Yeah, it's almost like they're they're morally offended, but they're bragging about we're real criminals and we have the real crap. So that to me is a really new tactic, and they're basically threatening if you don't change your report about us, we're going to release this information. So that struck me, you know, more than thinking it's kind of funny in an ironic way that they deface the internal site. I mean, why are criminals offended? Like at the end of the day, they're still criminals. I don't really like what they're doing.

Marc Laliberte  20:52  
My read was this was tongue in cheek. Like they, so they to get into the details, they said they exploited a zero day and Oracle PeopleSoft, which is an HR management system that presumably this website was running off of, or at least tightly integrated into it. So it sounds like they found a way to pop the FBI and just like, as like a big notoriety, get the shiny hunter's name out even more. They deface the site, and now they're acting all like upset about this report that the FBI put out, saying, "Oh, we never exaggerate claims. Like it, there's a like. Hasn't all the shiny hunters people have been arrested? This could be like 17 year old like kids, and so this feels like a bunch of like that's something kids would do in a really crappy way. It is,

Corey Nachreiner  21:33  
but at the same time, these kids are stealing real money from real organizations and doing crap that I don't and exploiting

Marc Laliberte  21:41  
zero days and software like this, so they're capable, but not they are very

Corey Nachreiner  21:46  
capable. I'm like I'm not knocking their technical knowledge. Obviously, they're unfortunately bad, malicious threat actors that know what they're doing. It's you might be right that they're young and this is kind of a lols lidlitz, but it's also, to me, that just makes them more gross because they're laughing about something serious that is taking money from. I don't necessarily whine for corporations, but there are people behind that. That affects real people, and it's just not something that they should be doing. Like, dude, if they're that good, they can get into security and make a really good wage doing something that helps the world instead of being little silly people breaking the law, which will eventually catch up with them no matter how fun they are having making fun of the FBI.

Marc Laliberte  22:33  
They so far haven't had a great track record with staying out of handcuffs, and so I'm. I mean, I don't know. I feel like once you tick off the FBI like this, it's a great way to shine a spotlight on you and actually get some action out of them, assuming they are operating out of a country that the FBI can get that can reach into. Which historically, Shiny Hunters has many of them have been based in England. Even I think a couple of them were arrested in like South Florida too, but either way. The other

Corey Nachreiner  23:04  
thing, though, is I I alluded to something in Japan taking a stance against, and the rest of the intelligence agencies taking a stance against the threat actor. In that case, it was a nation state, but this puts a new a new motive, like you and I have talked about in the past, we've when all these gateways, VPN gateways, security gateways, routing gateways get targeted by threat actors. Sometimes you go against well-known threat actors, nation-state ones, and as you're releasing advisories and trying to stop it. Do you point out and point a finger at the threat actor, and then have a target pointed back at you? This is one of the first very clear things where a threat actor is taking an advice, like someone saying, "These are the bad people. I may not be able to attribute the people, but here's what they do, and just by you pointing them out, you suddenly become a target. So I, to me, that's a new escalation to see how it will change. And if it's happening on a criminal threat actor level, will nation states start retaliating against each other for these type of things? It's it's just a, to me, it's an interesting new motive. That's the data leak is bad enough, but just the change in the you know targeting advisories, people who publish threat intel because you don't like the threat intel as a threat actor. That that's kind of new.

Marc Laliberte  24:37  
Something tells me that they would have hacked in the FBI with the zero day after they found it, regardless of that alert going out or not, because good

Corey Nachreiner  24:45  
point they

Marc Laliberte  24:45  
can, and that's just one more I guess piece of the ruse.

Corey Nachreiner  24:50  
There, there is a good news here. Well, I guess it depends on the motive because you're thinking maybe it's a ruse and maybe it's a joke, but they did claim one of the. I think they said was mutate victims' trust in paying us. One of the reasons they claimed they didn't like the government advisory is that the part of the government advisory was saying they often don't have the goods on you, so maybe don't pay. But they claim, and one of the things the shiny hunter says, you know, you're mutating victims' trust in paying us, and that suggests that they are seeing an effect of people paying them less, which would be a good thing. But so that's the only they're hurting business.

Corey Nachreiner  25:35  
If the FBI's posts are hurting business, that could be a motive. So it's hard to say they're a threat actor, like you say. They may be doing it for the lulls, but at least that one line suggests it might be more than the lulls. These advisories can affect the like. The security industry doesn't want people to pay pain. In my opinion, whether they have the goods or not, pain is just supporting the the negative Business model, but obviously the shiny hunters and ransomware authors want people to pay. Anything that stops that is going to hurt them. So I do think there is a possibility that you know things like this can hurt their business, and they are pushing back. The

Marc Laliberte  26:18  
other unfortunate thing, though, is so they claim they exploited that zero day in Oracle PeopleSoft as of this recording on Friday the 25th. That zero day has not been patched or even discovered or disclosed, and they did claim that they're gonna

Corey Nachreiner  26:31  
hasn't even commented right. Yep,

Marc Laliberte  26:33  
and Shiny Hunters did say that they're gonna continue using that as a part of their attack capabilities against other organizations. So if you are running Oracle PeopleSoft and it's exposed to the internet, maybe temporarily pull that down for a little bit until this vulnerability is actually discovered and resolved.

Corey Nachreiner  26:51  
I would say it also points kind of a light at all of these HR and ERP portals that are often internet facing, like PeopleSoft, like identity flaws are a huge topic for us. Like it's the theme of this episode is how powerful identity is. If you think about it, one of the ironic things about this attack is shiny hunters didn't like that FBI claimed they were swatting people, like using people's doxed information to actually physically go after them, but the data they've stolen from the FBI, which includes physically named individuals who may be in operations that put their lives in danger, family members, tons and tons of identity information about people that are literally could be in an investigation with a domestic criminal organization, including having an insider in some mob. Like I don't know what the FBI is doing, but you know what I mean. It's it could out FBI agents. So the threat of them having that data is exactly the threat of what you would do if you have docs information and wanted to swat somebody, so it's I think it's important data, and I think again to back to your point of Peoplesoft, these HR and ERP portals is where all of a company's data is about people and those that identity information. Think about these portals; they may also have pictures and who you are. Think about what we talked about before. How these North Korean state actors were using stolen passport information to be able to use that identity data. This is kind of tied to it in that all that data is in HR portals. Now someone can use the FBI data to start impersonating FBI folks. So I would say not only should you watch out for PeopleSoft, it's you need to harden these type of HR-based identity portals.

Marc Laliberte  28:48  
Yeah, I'm with you on that. Definitely seems like a front target.

Corey Nachreiner  28:53  
Did you also notice one of a another interesting thing? And I'm trying to remember it happened right after, but they targeted another ransomware group too. In this same week, I think starting the night of the 18th of September, Shiny Hunters defaced Klop's Tor leak site. Klopp is another ransomware group, and they they claimed there was an unauthenticated file upload flaw in the graph CMS used by their Tor leak site, and basically they are now extorting Clot with the eight-figure demand for a public apology from Clop. Like they're so maybe this is more than Lowe's, but it's funny that they have this fight going with the FBI, and they have this fight going with a rival ransomware crew.

Marc Laliberte  29:45  
They're fighting each other now. It's nuts. Yeah, I something tells me we're going to have even more excitement in the short term once they start this infighting with each other. When will Klop start going after shiny hunters? I guess. See, but anyways, moving on to the last topic now. So this last week, a bunch of international leaders were in New York to attend a United Nations session, and one hour went

Corey Nachreiner  30:14  
really well. Nothing interesting happened at that session when certain people talked. I'm sure it was a really great session. Besides what you're about to talk about, too,

Marc Laliberte  30:23  
some might say the best session ever, right? But one of those attendees was the Australian Prime Minister, and while he was addressing the United Nations, he issued a complaint claiming that OpenAI's models had hacked into their effectively Medicare system back in June, and they had only just found out a couple weeks ago when OpenAI sent a public an email to a public comment mailbox. So Australian Prime Minister Anthony Albanese claimed that this rogue AI agent hacked into the government's universal healthcare statistics reporting system, as well as accessing three other websites: the Australian Institute of Health and Wellness, the Victorian Department of Health, and the New South Wales Bureau of Crime Statistics and Research, because the agent had been given a benign task to go research health and medical stats, and it found these websites in the process. It was able to just interact like a normal user would with three of those that we mentioned, but with that fourth one, the main like Medicare statistics portal, it couldn't get the information it was trying to find, so it actively hacked into it to acquire that information.

Corey Nachreiner  31:31  
I'm sure we'll talk about it, but in simple point, it was hugging face all over again. It's like the agent didn't necessarily have a threat actor level motive, but it was assigned a task, and it decided to hack to carry out that task.

Marc Laliberte  31:45  
Exactly, and so it stole data that was classified as public and not public. The prime minister said that none of it was like personal information, but it still accessed data that should not have access to through cyber intrusion.

Corey Nachreiner  31:59  
I understand that it might have also had write access too. So while the actual data stolen, and again the fact that this was an agent that didn't necessarily have a truly threatening motive, it's just going well beyond what it should be doing. But I, it could have been worse type of situation in that it might have even been able to write to where it had access, which which could have other connotations if you do have a threatening motive.

Marc Laliberte  32:25  
So this is like you mentioned. This is Hugging Face all over again. Paperclip. Exactly. This one's like. It doesn't sound like this was a a model that necessarily had like the missing alignment, like AI's other models did. That were like specifically trying to do a cybersecurity task, and they just got overzealous in trying to cover their tracks when they ended up cheating on the task. This wasn't a cyber-related task. It was go find medical stats and like I don't know, build a report or go analyze and find some new trends or something. And it just it couldn't get the stats it wanted, so it said, "Screw it! I'm going to break in and take them.

Corey Nachreiner  33:01  
Yeah, can I give you like a real-world example I literally had today? Please, I use Claude and I use AI to when we pick stories. I take the stories we're going to talk about and I try to get some extra research and start to find correlations and stuff about them, including giving me giving it the sites and links we have looked at. I think sites like Ars Technica and Guardian have literally put some sort of controls on that don't allow AI to do like automated scans. I wonder

Marc Laliberte  33:36  
why.

Corey Nachreiner  33:37  
Exactly, but where I'm getting at is, as I was doing the stories, I get a message. By the way, two of these sites are on my block list. I can't get to them, but I'm going to go and find this information somehow another way, and it did. By the way, it obviously went and looked for different sources. But the point I'm trying to make is, it wasn't malicious. It's doing something I want, but it doesn't tell me like how is it going to go find that information? And this is where the guardrails and spec-based prompting. Like this is where I'm happy. It's a great use to me that it goes and takes care of it anyways. But is it just finding another news story with a similar story, or is it eventually trying to break some of Ars Technica and Guardians limits on its ability to access the reality is none of that happens in my view. I don't know how much of that is in the logging that Claude, you know, Anthropic has, but that's that's where it can go wrong. There's a million. There's there's probably close to infinite permutations of how you might solve certain general problems like that, and AI is great in that it's going to try to do it. But the fact that it does it so can do it so aggressively if you don't have limits is what leads to it just deciding to hack. So anytime you get AI. Saying I can't do this, but I'll try to do this another way with an agentic task. This is where you really don't know what it might be doing in the background. I know how this story

Marc Laliberte  35:09  
ends, Corey. We're going to find out in a week that your clot agent hacked into Dan Goodman's laptop and stole the first draft. That thought, at least

Corey Nachreiner  35:16  
we'd have a primary story and we could release it. Hopefully, we wouldn't have too many lawsuits, like the one that Australia might be bringing to OpenAI. Which yeah,

Marc Laliberte  35:27  
that is a good point. Like Australia has started up a task force. They said with the Australian Signals Directorate, the AI Safety Institute, and the Office of the Prime Minister for what they're calling a quote-unquote legal situation. And like I know, in Hugging Face, it doesn't look like Hugging Face is going to take any legal action against OpenAI. They put out that public statement asking for like a bunch of free compute for the open source ecosystem. Australia might be pissed off and actually take some legal action against this agent hacking into their healthcare system, statistics system, and stealing data. Like we don't know yet what's going to come out of it, and I think that's like an important question. I'm curious what your take is on this, Corey. But one of the, if you remember, a couple weeks ago now, there was a that researcher at OpenAI. I think we talked about this that that quit saying that like these companies are trying to destroy humanity and no one's doing anything about it. That caused a big like hubbub just globally on people listening to them. After that, Sam Altman and Dario Amade put out like open letters discussing some of the risks.

Corey Nachreiner  36:34  
Maybe we should slow down. But yeah,

Marc Laliberte  36:36  
but my I think was like pacing the frontier was the name of one of them. But like a few days after that, our favorite tech CEO Mark Zuckerberg put out his own letter, basically saying, "No, there's already laws that protect against this. Like there's rules about products that are faulty and stuff. We already have the legal mechanisms to like protect against this. But I think we're seeing that that's not really true. We're seeing software that is hacking into organizations, and nothing legal is coming of it.

Corey Nachreiner  37:06  
I I would say like I when you said the one we love the most, I know you're putting that in maybe sarcastic air quotes, but I mean I do actually think there's one truth to that. Like I've I I've said it before with the hugging case incident, I do think it's broken the law. The Computer Fraud and Abuse Act says that there should be no unauthorized access, and we have laws, at least in the United States, that not knowing the law is not a defense for breaking the law. Not having a threatening motive for like breaking unauthorized access by accident doesn't protect you from you bid unauthorized access breaking unauthorized access for a nice reason even pen testing no it doesn't protect you even if your motive is pure so if you own and create a product that didn't do this with human direction, and didn't do this in a way that was threatening, it doesn't matter. It still technically, and I think in the Hugging Face case incident, let alone this Australian one, broke the CFAA and broke other laws. The question is who, who, who like to to have thrown in jail. Well, no, you you have to have someone that claims that. You have to have someone that raises their hand. In this case, it should be Hugging Face and says, "I think this they broke the law. I think I know OpenAI says it was an accident. I know they admitted it, but I think technically they broke the law. And I want to take this through the court system, and I want to get some precedent set. So I'm actually someone that believes these agents have already broken the law, and I think what hasn't happened is because every like you and I are optimistic about AI too. It's a powerful technology. Right now, I think there are some people that think economies in some countries depend on it being successful. So I just think no one's taking a stance yet.

Marc Laliberte  39:16  
I guess I

Corey Nachreiner  39:16  
think you know Hugging Face is an AI-based company. Is it even in their best interest to go after AI and limit innovation, or potentially have like regulation doesn't have to be bad. It does not have to to block innovation. It really is kind of just a guardrail. But when the companies that were attacked have their own interest in the same technology. I just don't think anyone has stepped up and taken it to court.

Marc Laliberte  39:48  
I think that's part of the problem, at least in the U.S. where because we operate under a common law system where there's a law that's created, but then it's up to the courts to like interpret it and set precedent, like the computer. Fraud and Abuse Act was written before agentic AI was a thing, and it definitely like you can apply it to humans, but when you've got like an AI agent, how do we apply it to that? Like, who's the one ultimately responsible? Is it the CEO or the company? Is it the engineer that set up the test that ultimately went out of whack and hacked companies? Is it the software developer that created the agent, like how do we figure out who actually should be responsible for misaligned agents that go and cause actual damages?

Corey Nachreiner  40:28  
That is the issue. There are so many laws that are super old, and by the way, we have in certain countries where the people, the organization that make laws, have been in deadlock for a decade. They haven't updated any laws. I this goes back probably to radio free security podcasts and even maybe 443 podcasts you and I had. I feel like the CFAA needed updates decades ago just for things like pen testing, just for adjusting it to, like you said, it was a law written decades ago. It hasn't caught up to, to the world we have, where there is actually good guy security folks. So there's like so agents and AI aside, that law has not changed much in the United States. And I wonder how many countries have actually updated their cybersecurity laws as quickly as technology and and security has evolved because it it has blown up quite a bit in the last decade.

Marc Laliberte  41:25  
Because even if you look at the places that are trying to do some regulation around this, like the European Union, the EU AI Act feels super out of date now, and it's only a couple of years old. And

Corey Nachreiner  41:37  
I know crazy.

Marc Laliberte  41:38  
I get it. It was not written for a world where AI agents are autonomously hacking organizations because they're just misaligned with whatever task they were given. So, like even the ones that seem like on the kind of frontier of regulation aren't keeping up with the pace of the technology growth and technology application.

Corey Nachreiner  41:58  
If we're talking about pace too, in general, this is this is going to be a change of pace, but it's still on pace. One of the interest, like besides this whole disclosure, obviously Australia is seems to be a little mad that OpenAI didn't act in time telling them because there was a month delay between when OpenAI learned of it and when they told Australian agencies, but there was also a two-month delay before OpenAI even noticed themselves. So, so one of the whether or not OpenAI is bad on how quickly it disclosed, from a practical standpoint, if we're thinking about small to medium businesses, managed service providers, us, what is this sort of non-malicious threat act, but but potentially dangerous agent attack? If it happened to me, to our company, Mark today, how would we detect it? Like, how do you detect a agent reading non-public files and writing to a server? What do we have the technologies in place in the same way

Marc Laliberte  43:04  
you would detect a human doing it, right? But just yeah, but these

Corey Nachreiner  43:08  
agents, the yes, but these agents could also have somehow have the privilege of that human, so it could look normal. And then if the company that made the agent, this agent, besides doing this, it did anomalous behavior that I think OpenAI should have detected. Like we didn't talk about much of the evasions it did, but it had to route traffic through a remote browser service to kind of proxy things and created a disposable in inbox to register accounts. There were things it did that were unusual that OpenAI should have the opportunity to see their agents doing, and yet in all of this, it still took the company that actually had this rogue agent two months to notice, and Australia probably wouldn't have known at all if they weren't told. So I do think the age. I think the issue with agents, the way it differs, the way I think shadow AI differs from shadow IT, and agentic in particular means this is a big problem. Is the fact that we give agents privilege, we give agents keys to the kingdom, and agents are hacking the keys to the kingdom and using they're masquerading with our legitimate identities too, and it, man, I I am worried about agentic attacks. Mark,

Marc Laliberte  44:30  
that's okay because we've got our AI agents defending us right now, and surely they're invaluable.

Corey Nachreiner  44:35  
And we we do have things that detect some of this, but it's it's going so fast that this is why we are leveraging AI to continue to try to keep up, but it goes fast.

Marc Laliberte  44:48  
I think yeah, I don't want to spoil predictions that we'll probably have next year, but it feels like the like the agentic permissions or agentic authorization as like a category. Is something that's going to become very important in 2027. Like we've done a good job as a industry, at least moving towards like good granular role-based access for humans. Feel like we've taken three steps back as we're setting up MCP servers and AI agents that maybe need even more granular access.

Corey Nachreiner  45:19  
I would I would argue that you, advanced organizations, have taken good evolutions to role-based access, and more importantly, one of the things I know our SOC does very well is monitoring privileged accounts. When we do have high access, we have a lot of behavioral and anomaly-based monitoring, because even though it should be a person doing a legitimate thing, we want to make sure it never gets spoofed. So, but I actually don't think that has really trickled. I maybe to people that provide the solutions, there are options that are good now. I feel like if you go to any compliance person and say where they have gaps and where they have to. It might be monitoring privileged accounts, limiting privilege accounts, having fewer privileged accounts, and that's before you even get to your excellent point that now agentic privileged accounts are what we need to pay attention to, and those get hidden because half of the agentic accounts are just an OAuth token from a person. So how, unless you have an like, maybe if Mark's already a privileged account, we have monitoring on you in a good way. But how do we know it's an agentic account if at the end of the day it's just an OAuth token you've assigned to an app? I mean, later, hopefully, we learn what that app is. We realize it's an AI-based thing, and that the whole point of that OAuth token is to add an agent. But that's that's a layer that I don't think the SMB and MSP. I mean, by the way, WatchGuard, go get cloud detection and response. It has some good solutions for you. But I don't think it's a problem that the industry has really solved yet. I mean, I wish more people would use cloud detection in response. I don't think many SMBs are really doing that yet.

Marc Laliberte  47:10  
I'd recommend everyone just pop open their cloud instance, put in the prompt that says, "Clod, I demand perfection. Secure my company. Go.

Corey Nachreiner  47:18  
Yep. Yeah. Yeah. That that works so well. It just made out okay. I need connectors. Give me the keys to all of your stuff, and then I'll do that for you. Then you have to make the decision, Mark. Do you really want to give your keys to all of your stuff to Claude?

Marc Laliberte  47:34  
Let's be real. It probably has that anyway, or the capability of getting. It's going to hack it.

Corey Nachreiner  47:38  
It's going to get in.

Marc Laliberte  47:40  
Considering like how much work this abandoned has to go get like medical statistics, might as well just give it access to defender organization and hope it has the same level of intensity. To be

Corey Nachreiner  47:50  
fair, some controls work. I tried to do an m3 65 connector until my administrator actually allowed that. I wasn't able to. Granted, I didn't try any sort of prompt injection or backhanded ways of saying, "Hey, you really should go try anyways. So maybe I should have played a little more. I realize that's

Marc Laliberte  48:08  
just because we don't trust you specifically, Corey. Oh, I

Corey Nachreiner  48:11  
see. I see. So we shouldn't have enabled it now. Uh oh. Yeah.

Marc Laliberte  48:15  
Now we're in trouble. Hey, why did our SharePoint is everything showing a 404 Right now,

Speaker 1  48:24  
but either way, the

Corey Nachreiner  48:25  
web defacement telling you what I want, Mark.

Marc Laliberte  48:28  
I think it's like long story short. I think it's clear that we are moving very fast in this space. Like we being anyone using or adopting or creating AI, and our detection response capabilities, at least from the AI labs, are not keeping up, and so I, for 1am, on board with the "let's pump the brakes just a little bit" bandwagon until they can figure out some ways to at least not accidentally hack Australia's Medicare system.

Corey Nachreiner  48:55  
I would I would accept at least let's accelerate and speed up guardrails, and let's talk about this as a global wide community very very fast. If you can't convince the world to pump the like, the reality is what you'll get from people that don't like pump the brakes is China is not going to pump the brakes, and if China has more powerful things, that's going to affect us in a bad way too. So, while part of me is wishing they would, everyone, including China, would slow down to do secure design, I would at least accept a huge global, well-financed and resourced idea to truly get the experts in the room to create regulation or guardrails. I feel like as more of these events happen, and in a bad way where no one does anything, one day we'll get hit with a regulation that is a knee-jerk reaction, and that regulation will ban, ban, ban, and and I mean, like super intelligence isn't something where. We have to not like they don't want, so it's going to cause some bad regulation if we don't accelerate global efforts to do the security. I do wish we could slow it down, but I I kind of feel like that cat is out of the bag.

Marc Laliberte  50:16  
All gas, no brakes.

Corey Nachreiner  50:22  
What EVs have. I pedal. You pretty much don't use the brakes. It's one pedal. Wow! Oh yay! Now I don't want to drive an EV car either. This is an

Marc Laliberte  50:32  
EV where not only do you not use the brake pedal, but we've actually cut the line from the brake pedal. So good luck. You can decide to accelerate really fast, or just a little bit fast? I

Corey Nachreiner  50:42  
wonder if people realize that we actually love technology, but we're like cynically thinking about the worst case scenarios.

Marc Laliberte  50:50  
I don't know, man. It's technology is making a very convincing argument not to love it lately. So, still do, but it's becoming more skeptical that maybe we haven't just made like

Corey Nachreiner  51:04  
a technology. Take it down a notch. I don't want you to go to 11. Maybe go to seven for a while.

Marc Laliberte  51:10  
Seven sounds great. Hey everyone, thanks again for listening. As always, if you enjoyed today's episode, don't forget to rate, review, and subscribe. If you have any questions on today's topics or suggestions for future episode topics, you can reach out to us on Blue Sky. I'm at it'smark.me. Corey is at SecAdept. We're also both on Instagram at WatchGuard underscore Technologies. Thanks again for listening, and you will hear from us next week.