Cryptographic Context Injection

Marc Laliberte  0:00  
Hey everyone! Welcome back to the 443 Security Simplified. I'm your host Mark Laliberte, and joining me today

Corey Nachreiner  0:06  
is Corey "Travel Crazy" Nachreiner. It's been a day, Marc.

Marc Laliberte  0:11  
Yeah, you literally just got back from the airport like five minutes ago, I think.

Corey Nachreiner  0:15  
Yeah, to my house five minutes ago. Here, landed probably an hour and a half. Well, yeah, an hour and a half ago, something like that. I don't even know anymore.

Marc Laliberte  0:24  
Well, thank you for not pounding too much airplane wine and for making it back in time to record this one, Corey. Because

Corey Nachreiner  0:32  
only had five Rum and Cokes this time.

Marc Laliberte  0:34  
Perfect. So normal, Corey. On today's episode, we'll be discussing an ongoing attack against data hosted in Azure that may or may not have occurred four years ago. We'll probably get into that. A recently disclosed vulnerability in Circle CI's MCP server, and then a novel, truly novel research post on a new attack method or technique against artificial intelligence, and with that, let's go ahead and I don't know generate our way in.

Corey Nachreiner  1:12  
Circle our way in. We generate a lot with AI. Everything is AI now.

Marc Laliberte  1:17  
Everything is AI. Anyways, let's get rolling.

Marc Laliberte  1:26  
So, Corey, let's start with the first story today, which I've seen making its way around several news sites. It's how it popped up onto my radar, where I guess last week a threat actor going by the name the Hat Man started posting employee databases allegedly stolen from some pretty major companies, including McDonald's, Gap, Vodafone, and the Tata Consultancy Services, and over in India, they claim to have stolen from Microsoft Azure by using techniques including actually valid credentials. They said they stole it directly from their Azure tenants using compromised credentials, bypassing or at least circumventing multi-factor authentication through MFA fatigue. The databases are 3.6 4 million records in total, including 1.7 million records stolen from McDonald's and several 100,000 stolen from other organizations as well. Now, this is what the attacker is claiming. They're posting it on whatever the latest permutation of breach forums is today. But we've already seen some responses from some of the allegedly affected individual or organizations, like Tata, for example, said that they investigated the alleged breach and found no credible evidence of a breach in TCS's systems or customer environments. They said that the the data appears to be at least four years old and includes only basic employee information. They said that the attacker claims to have used password spray and MFA fatigue in the attack vector, but the company has strong safeguards in place against such techniques for more than two years. So at least one of them are denying it, and I think they're actually correct on what they said in terms of like the data collected. The attacker, like some of the samples show, it's like employee names and IDs, job titles, phone numbers. But in some of the samples, it shows like Active Directory objects too, like group memberships and service accounts, and even like non-user AD accounts, like global administrators too. Hudson Rock did a analysis of it. For example, they found it had core identities and access groups from people's AD tenants. They said their researchers independently confirmed that it looks like it came from Azure, but they believe that the attack vector most likely came from a info stealer infection, because most of the affected organizations had credentials showing up on underground marketplaces from info stealer infections over the last couple of years. Yeah, I think that's a pretty important detail. It is I wouldn't say conjecture, but an assumption or speculation made by Hudson Rock, but it seems like a key thing

Corey Nachreiner  4:25  
here. The key thing here is we're. I think everyone is reacting to a like you said the breach forms or whatever the version of it is leak, and the breach forms leak has data. There's definitely data there we're looking at, but we're reacting to a leak. Sorry if the windows kind of changed, and people have to make guesses of how it happened. Even though the attacker claims what happened, we don't know for sure that's what it is. So yeah,

Marc Laliberte  4:57  
and I want to go on one thing. Tata pointed out that it's just basic employee info like names, phone numbers, addresses, job titles, even email addresses and home addresses. That data obviously isn't as valuable as like critical PII, like national identifier numbers or like credentials, for example. But it is still pretty useful to know every employee that works for a company and their job title, and in some cases, like group memberships within the domain. Like that feels like a business email compromise or like targeted spear phishing goldmine for someone to get. Oh, sure.

Corey Nachreiner  5:35  
Even if a lot of the data has changed, because if it is four years old, which I'm sure we'll get into. It's it's definitely a goldmine of information to start making connections.

Marc Laliberte  5:47  
Exactly, because you can imagine if you know exactly how an organization is structured, if you've got information about like service accounts in it, it makes it easier to spearfish like IT organizations too. Like you give them a specific username. Say I'm locked out of service account X Y Z. Help me reset the password and MFA on it, please. And

Corey Nachreiner  6:08  
yeah,

Marc Laliberte  6:09  
by knowing the name, just

Corey Nachreiner  6:10  
think about yeah, yeah. Knowing the name will help. And AI. I mean, what spearfishing doesn't use AI? Think about it. If I have this big data leak and I give AI all that information about McDonald's, it would have a heyday making a really good spear phishing email.

Marc Laliberte  6:27  
What is that? It was last week we were saying that jokingly, maybe it's McDonald's offensive security team that's going to be joining the the White House hacking corps of privateers. Well, Microsoft's defensive stock might need to be on their toes if this data ends up being fully leaked and not just the the the basic summary records from it. But either way, like it seems like it wasn't just Tata Gap put out a similar statement saying that they found no evidence of a breach. So all signs are pointing to it possibly being more than four or four years old or or more, and not something recent, but still potentially valuable in the in the wrong hands as a major company record dump from some of these organizations, and as a as a as a, like for the technique that they used or potentially used, like an info stealer infecting a an employee's device, leaking their credentials to an attacker. Those credentials end up online somewhere in an info stealer credential dump, and then potentially these third party threat actors use them to try and log in and use like MFA fatigue to get past MFA. That is a totally common and believable topic,

Corey Nachreiner  7:49  
for sure, for sure. Although I have to say, I believe it's also four. Like I, I believe the story that this could be four-year-old data, and maybe that's why it's just showing up on on breach. Tow or whatever version we're talking about now, because you and I know that if you have data for big companies like that, you don't just go publicly release it all on the that kind of crappy forum for almost nothing. You you do private sellers. You go to the more hard to find forms that require, you know, criminal validation to get on, and it takes a while before it gets to the point where you like selling it all in mass. So, I I can also see that this is a valuable day that the companies need to know about it. Maybe they already doing from past leaks, but I can also see it. Yes, it may be old and a little bit stale. It's still helpful for attackers, but it could still be somewhat stale by the time it's got to this disclosure.

Marc Laliberte  8:51  
Yep. So either way, just be on the lookout for more identity-based attacks, especially if you are a prominent company listening to this, but basic cyber hygiene practices like mitigating against MFA fatigue by locking tokens after too many failures for a period of time, and just basic brute force prevention can really help against this kind of a kind of attack. And if you are a like a WatchGuard Cloud Detection and Response customer. The ITDR features in there are specifically designed around mitigating this type of threat. But, anyways, while Corey troubleshoots his system so we can continue sharing the the YouTube view of this. Moving on to the next story, I saw a really interesting research post from Omri Dar of Remedio. I think that's how you pronounce their name, where they published a write-up of a critical unauthenticated remote code execution vulnerability that they found in Circle CI's MCP server integration. Circle CI, if you're not software developer. It's a populous, populous, popular continuous automation tool, very similar to like Jenkins or GitHub Actions. It basically allows developers to automate building and testing and deploying the software that they write. And like many tools, especially in the software development space these days, Circle CI put out a MCP server to allow developer AI agents to interact with the platform. Like an MCP server, at the end of the day, is think of it like an API bridge. It gives a predefined set of like skills or interactions or actions that a an AI tool or agent can take on some other application by interacting with the API in some way. And by the

Corey Nachreiner  10:49  
way, it's worth also pointing out. By the way, I we've talked about this before, but MCP servers are super important to agentic AI. But the thing I want to point out again, MCP is a really, really, really new standard, and I think we'll get into it at the end of this researcher's post. But things like HTTP, FTP, SMTP-they've been around for decades and decades. This MCP was introduced November 2024. It's if you think about the security of a protocol, the security of you know this whole entire new ecosystem of how you can from a network control AI, it's not just the individual folks that are creating different versions of these MCP servers, like this particular Circle one. It's this is a really freaking new protocol, so there is to me that always scares me, especially in innovative technologies that move fast, because it has not been really bulletproof security tested, in my opinion. I think we'll we'll really get into that on this. So not just is MCP super important for AI, it's it's a super fresh protocol,

Marc Laliberte  12:03  
and not even the protocol itself, but like the implementation of it. As everyone is moving as quickly as possible to support MCP integrations with their applications, so they can hop on the AI gravy train. And when it comes to deploying an MCP server, there's usually a couple of options. You can have a locally deployed one where you can typically use things like OAuth authorization, so that a developer can connect to a tool using their own credentials, and they that tool gives permissions based off that specific developer. Or you can use a centralized deployment where you set up one MCP server for all the developers, set up an API key so that server can communicate with the application, and usually some form of authentication for developers into that server at least. And so, when Circle CI's MCP server is deployed in that shared centralized model, it does some authorization authentication, but it also checks the host and origin headers of the web requests that it sees, and Remedio noticed that if they set the host header to just localhost and neglected to include the origin header, it would bypass all credential checks and let them freely communicate with all the tools behind this or exposed by this MCP server. That

Corey Nachreiner  13:21  
feels like like the most simple authentication bypass you could have.

Marc Laliberte  13:25  
Yeah, they showed the literal like JavaScript code from the MCP server and how it authorizes, and it like literally has a is origin allowed function that returns true if there's no origin header included at all, and then a is allowed host that returns true if local host is included in there. They said that, like, as an attack scenario, they could call the run pipeline, so basically kick off a CI job, hand it a pipeline configuration the attacker wrote, write a step to run just arbitrary commands, and it would execute it using the organization's API token on the Circle CI server, and also frighteningly, by default, this MCP server listens on just all interfaces. It's not restricted to specific IPs or even localhost itself. And I think their conclusion, Remedio's conclusion at the end of their post is 100% on the mark, where they said, "Here's the uncomfortable truth: Circle CI's bug wasn't a freak one-off. It's a symptom, and right now the MCP ecosystem is, to put it bluntly, a security mess. And I think that is 100 accurate. Where, like I said earlier, as companies are setting up MCP servers and integrations and deployment guides, oftentimes they do it the quickest way first, and not necessarily the most secure way first. And in this case, like it could have allowed anyone with network access to the server to bypass all authorization and effectively gain code. Execution within the build environment for these software development organizations, which is extremely high risk, almost as bad as it gets for them.

Corey Nachreiner  15:10  
As far as CVEs, literally as bad as it gets.

Marc Laliberte  15:15  
Now, to Circle CI's credit on this one, the vulnerability was reported on july 30, and they had it fixed by august 6. So extremely fast turnaround in resolving this issue and protecting their customers. So good for them on that one. That is a well above grade when it comes to vulnerability response. But like I agree with Remedio's take on this that like the whole MCP ecosystem because of how fast it is moving is very high risk, and so

Corey Nachreiner  15:46  
I agree.

Marc Laliberte  15:47  
But at the same time, like these are really powerful integrations, and there's an absolute need for these across the board for organizations. So I think it's the

Corey Nachreiner  15:58  
cool thing. Let's talk for a second because I mean the cool thing about me being a CISO and you being the director of the SOC and more, soon to be more, is we deal with this like politically. Any tech business right now is under heavy pressure to innovate with AI. And by the way, I want to innovate with AI. You, I believe, Mark, want to innovate with AI. We we not only are under pressure from the business owners and and our CIO and our CEO to really push innovative AI projects, but we want to because there's value there. But you do have to find ways to point out to the people that are rushing things out that I we want an MCP server. Agentic helps us. We you you use agents all the time to help manage our SOC and other things, but we really have to be careful and think about things like when we are using internal MCP servers and giving our employees, you know, not just having a standardized centralized way for the average employee to do agentic tasks that maybe is built, but have have policies around. Hey, developers, I don't want you just popping up an MCP server for your own experiment without some consideration of of how to do it in a way that can work with with our security policies and controls. So I do think it's a really interesting topic because I think both of us really want to push AI projects, but it's it's hard to slow folks down and say there we've already had MCP servers set up in our environment, and where we want to point out these things, but in ways that don't stop business innovation, and I think that's a challenge.

Marc Laliberte  17:42  
And I think, like the main just guidance I'd give for literally anything across the board in artificial intelligence right now is the industry itself is moving so damn fast that while in other areas you might rely a bit more on the vendor themselves, the AI provider themselves, on like secure implementations and secure by design, secure by default practices, and AI specifically, like you can't rely on that at all. And so, as you are adopting these extremely powerful, extremely useful tools, you have to give just a little bit more thought into what the actual threat model is, how to secure it, and how to deploy it and how to manage it over time. Like a bit more of that responsibility is unfortunately just going to be on the person using the tool. So you're right, Corey. Like internally at WatchGuard, we operate with a like we are an enablement team while managing risk. But that managing risk, more of the weight is on us than on the the tool creator that we're we're trying to manage.

Corey Nachreiner  18:43  
Yep,

Marc Laliberte  18:44  
but a great post from Remedio. I can all but guarantee this is not the last MCP vulnerability that we're going to talk about on the podcast in the next 30 days. I bet. But moving on to the the last story for today, saw another really interesting research post from Ronnie Uteve, Utevsky. Yeah, I butchered that. I'm sorry, Ronnie. At Adversara, where they even coined a new attack technique for this interesting old AI attack technique that they found, which they called cryptographic context injection, and they even found a still as of this recording working proof of concept in Grok AI, and a somewhat but somewhat less successful one in Gemini as well too. So by now, most mainstream artificial intelligence starting to get better guardrails around prompt injections to make it more difficult for like as you summarize a web page for that to just immediately gain all the privileges of the AI agent and carry out risky actions that could lead to like data exfiltration things like that and most models have. Some form of static safety guardrails now that will classify the input text without executing it. So think of it like a like a safety filter on the way in to look for things that might be risky. Tells the model that's going to actually run stuff like, hey, look out for this, or don't execute this, or this is safe, whatever. Those static classifiers even have the capability to handle some basic encoding and like light encryption. Let's say, like think like a substitution cipher kind of thing, where if it gets a base 64 encoded prompt injection command, they're smart enough to be able to decode that, lightweight enough to be able to do it, and classify that as something that you shouldn't run for the The model, but so Adversara found that instead of using like encoding or light encryption, if they use like standard encryption on the prompt injection command that they're trying to run, and then

Corey Nachreiner  20:54  
you also mean strong, like something like AES. Yep,

Marc Laliberte  20:59  
and if they include the ciphertext and the key and instructions to decrypt it, unlike the base 64 and similar encoding methods, there aren't any shortcuts for decryption to make it computationally cheap enough to run in this kind of input classifier. So it has a chance to skip past it and make it into the actual execution phase for the AI. So they walked through like some attack steps for this, where the AI system still has to process untrusted input. And a really common scenario that almost all of these prompt injection attacks use is the scenario of a user asking some the AI tool to summarize an external web page. So, like, hey Grok, look at this. I don't know white paper. Tell me the bullet points from

Corey Nachreiner  21:44  
it. And just to be nerdy for a second, because really this is an indirect prompt injection because you're giving it a prompt, but the the actual injection is hidden in that web page content, whether it's a JSON or something else, right?

Marc Laliberte  21:58  
Yep, exactly. So that starts off the process. Grok would go and, or the AI would go and retrieve the page contents and would find a encrypted JSON object along with the decryption instructions. Those static guardrails can't read the JSON payload. The input classifiers can't classify the text that's hidden in it as the ciphertext, so they don't run it, so that payload remains in there as now the agent on the back end, the actual meat and bones of the AI, starts to process it. It's forced to go through the code execution runtime. Then the model will run that decryption in the sandbox, and now the attacker's instructions exist outside of the the code of the model, or outside of the output, they come in as the output from that, and it becomes trusted output at that point instead of the untrusted input. So now the model will execute the instructions instead of skipping over them as that untrusted input. So the attacker could include instructions like reach out to an external server to leak data or produce some other sensitive output, encrypt it, and smuggle it back out past the output guardrails that are probably there. In the Grok example, they show they ask Grok to summarize a page. Grok decrypted the instructions, which direct the agent to resolve its private session context, like information about the user, and then autonomously invoke its privileged navigation to then load that URL, which effectively transmits that user data out to an attacker-controlled endpoint. They said as of now, they still have a high success rate with Rock Grok on this one, even after reporting it. But similar attacks against Gemini, which used to work back in June had been working less and less as Gemini continues to add more and more guardrails and safeguards against this.

Corey Nachreiner  23:48  
Gemini seemed to be a direct injection too, by the way.

Marc Laliberte  23:52  
Yeah,

Corey Nachreiner  23:52  
yeah.

Marc Laliberte  23:53  
So it abuses two things. Just in summary, like it abuses code execution runtime as that trusted laundering channel, and then the strong encryption forces the payload down that channel, so that once it pops out, it is now trusted output instead of untrusted input. Yep. And as we've said on, I think every single episode we've recorded this month, like it feels like greenfield for finding prompt injection techniques against AI models out there.

Corey Nachreiner  24:24  
Yeah, prompt injection is is this is an industry level problem, and I think it's not this isn't a vendor. There'll be a new vendor every time. You can kind of ignore the Grok versus Gemini versus whomever, because I I think this is going to be industry wide for a long time, and I still struggle to find how we're going to define guardrails that can completely guarantee there won't be any prompt injections. But hopefully, someone smarter than I will figure that out.

Marc Laliberte  24:52  
Because, like even this, like it used to be the wild west, and indirect prompt injection was just trivial. It worked every time. Then they started adding these input. Classifiers to look for risky prompts and maybe flag some security warnings or just like tell it not to process it. Then people would find ways around that, so they added output classifiers so that even if it worked, like whatever it was going to do, it ran that through a like, hey, are you doing something sketchy that you shouldn't classifier? And if it did, trigger security warnings. Now we found encryption can get past some of those. Like it is absolutely a cat and mouse game. Where even as they're using AI to try and catch these like attacks against AI, we're finding other ways through it as well.

Corey Nachreiner  25:32  
One day there'll be an AI based guardrail to look for this, and the vulnerability will be in the guardrail having an input that it's supposed to find that is prompt injection, but there's some sort of meta prompt injection inside the prompt injection that actually messes with the security AI guardrail, where we're going to get all inception with these prompt injections.

Marc Laliberte  25:56  
Turtles all the way down. It feels like when it comes to this.

Corey Nachreiner  26:01  
Yeah, interesting. It reminds me of web application issues too. Like some people are like, "Oh, SQL injection cross and exciting or cross-site scripting and and you know whatever version of web app flaw. These are all old and well known, and who cares? To this day, we find new ways to find them deeper and deeper in complex web apps. But

Marc Laliberte  26:19  
like at least with those, it is still like deterministic behavior. There, there

Corey Nachreiner  26:24  
is

Marc Laliberte  26:25  
theoretically an end state where if humans keep stop screwing up with writing code, we

Speaker 1  26:30  
won't have

Marc Laliberte  26:30  
SQL injection. And this

Corey Nachreiner  26:31  
one is part of the design. I mean, simply put, the problem is if someone can get your agent with your privileges to do something, it will do something

Marc Laliberte  26:42  
with your

Corey Nachreiner  26:42  
designed to to they're designed to respond to your prompts, and if they can find a way to get you to go somewhere where they can put a prompt in as you, it's going to happen.

Marc Laliberte  26:55  
It's a like we train all of our users not to trust like externally received data, like untrusted input, like don't trust a email from external party or a document or a link, and it really feels like we're not at the maturity level of AI yet, where we can trust untrusted data in it. But at the same time, like it's not as easy to say, oh, don't ask Grok to summarize a web page because that is a legitimately great use for AI, and

Corey Nachreiner  27:23  
literally have a scheduled task where it's very nice for me and can save me time. So yeah, it's it sucks.

Marc Laliberte  27:32  
So I don't know. Do we just have to eat the pain at this point? That every once in a while, one of the news stories we read is going to get our AI agent to leak all of our info.

Corey Nachreiner  27:42  
Like I said, I just hope there's smarter people at these AI companies that are doing something about it. I don't know if I there's definitely smarter people than me at these AI companies. I still haven't seen the evidence that they can completely do anything about it, though. Hopefully, hopefully they'll they'll break that impression.

Marc Laliberte  27:59  
100% accurate on both accounts.

Corey Nachreiner  28:02  
Yeah, I figured. I figured you were going there, Mark.

Marc Laliberte  28:07  
But yeah, I I don't. I'm not even going to try and predict where this one ends. I've been like wrong on every AI prediction I've made over the last couple of years about like how quickly this stuff is going to evolve.

Corey Nachreiner  28:19  
Yeah. Oh, yeah. It's broken my expectation on evolving. Although I have actually thought it was evolved faster than we would think.

Marc Laliberte  28:27  
Like right now, my my heart wants to tell me like there's no way we're going to catch up to this capability to be able to like prevent this category of technique. But like history so far has shown that maybe two months from now we will have some super powerful method of like detecting and preventing prompt injection because AI is now just omnipotent.

Corey Nachreiner  28:47  
So so far, despite false starts and situations where maybe we use technology to do damage, I think on the long run technology has continued to help humanity. So let's just hope that we we get there without too much collateral damage. There have been plenty of technologies that have caused collateral damage while users and threat actors figured them out. But so far, we've continued to help humanity as a whole. If you look at the long run, I guess the question is just how much collateral damage will we have to eat before we get there.

Marc Laliberte  29:21  
That is a very important question because that collateral damage could actually be quite serious along the way too.

Corey Nachreiner  29:27  
Yeah, I'm underplaying it by just calling it collateral damage. Collateral damage is lives and and things no one wants to see happen.

Marc Laliberte  29:36  
Yeah. Hey, if AI can help humanity by, as we start to put it in robots, have them go rogue and start cutting down flock security cameras. Then,

Corey Nachreiner  29:47  
oh yeah, I'm all for that. I

Marc Laliberte  29:49  
think we'll be in a good place for humanity then. But in the meantime, those robots volunteer.

Corey Nachreiner  29:55  
We don't like your cameras and your your techniques.

Marc Laliberte  29:58  
Exactly, but I'm. You, I think I'm still optimistic, net positive on the future of AI, but we are definitely going to have some more stumbles along the way to whatever destination we end up at. Man, exciting times! What a great time to be in cybersecurity. Now our tools are just autonomously leaking our Data

Corey Nachreiner  30:22  
now. Now the good guy tools are autonomously hacking and leaking our data,

Marc Laliberte  30:28  
and we

Corey Nachreiner  30:29  
don't have to worry about the threat actors. We're more scared about the good guys who are really good with their AI.

Marc Laliberte  30:36  
Yep, what a time to be alive!

Marc Laliberte  30:41  
Hey everyone! Thanks again for listening. As always, if you enjoyed today's episode, don't forget to rate, review, and subscribe. If you have any questions on today's topics or suggestions for future episode topics, you can reach out to us on Blue Sky. I'm at it's Mark Me. Corey's at I've never logged into this app, and you can also reach out to us on Instagram at WatchGuard underscore Technologies. Thanks again for listening, and you will hear from at least me next week while Corey enjoys some vacation.

Corey Nachreiner  31:10  
Yay!