PHP2Shell Breaks the Internet

Marc Laliberte  0:00  
Hey everyone, Marc here. I am excited to share that we're consolidating all of our threat intelligence and security news content under our new brand, WatchGuard Red Cell. That includes this podcast, which we're renaming Inside the Red Cell. You'll still get all the same great threat content that you've come to expect, but the new brand will tie everything together with the rest of the threat intelligence created from the team formerly known as the WatchGuard Threat Lab. If you're currently subscribed, don't worry; everything will be transferred over. Just don't be surprised when you see the 443 disappear from your feed and inside the red cell pop in. Now, here's this week's episode. Hey everyone, welcome back to the 443 Security Simplified. I'm your host Marc Laliberte, and joining me today

Corey Nachreiner  0:50  
is Corey, the super secure single-digit elliptical curve certificate knock-writer. Say that five times fast. I don't think I could say that. Super secure single-digit elliptical curve certificate.

Marc Laliberte  1:04  
Not right now. Okay, I promise that will resonate after our second story today that we talk about.

Marc Laliberte  1:11  
But to start with today, we are going to dive into PHP two shell, the vulnerability that took the internet by storm, which, at least as of this recording, has only been 3% mitigated.

Marc Laliberte  1:23  
After that, internet

Corey Nachreiner  1:25  
like PGPHBs anywhere. It probably won't take long. There's probably not a million old versions that don't have a fix.

Corey Nachreiner  1:32  
There's probably not in every IoT device in every development with CIDC.

Marc Laliberte  1:38  
Sarcasm is overflowing today.

Marc Laliberte  1:41  
After that, we'll discuss yet another good job internet with some CA certificates that did not have quite as much entropy as they should have,

Marc Laliberte  1:52  
and then we will end with a whole bunch of tech or security executives getting arrested in relation to Shiny Hunters, the infamous ransomware slash extortion group. Yeah, I love this. No, the security industry has nothing to do with the bad guys. It's it's great. I'm glad to see that you're still employed and not in prison, Corey. So thank God. Hey, hey,

Corey Nachreiner  2:14  
I'm not a ransomware negotiator, or

Marc Laliberte  2:19  
is he? Anyways, let's go ahead and I don't know extort our way in,

Marc Laliberte  2:30  
Corey. We have to start with the first one, which is kind of started, I guess, started exploding on Wednesday of this last week on the 21st when the PHP Foundation, who's responsible for maintaining PHP, the most popular server-side scripting framework for internet websites, published a handful of new updates in response to two critical pre-authentication remote code execution vulnerabilities. Now,

Marc Laliberte  3:00  
to your sarcasm at the start of this episode, Corey, I think roughly 70% of websites on the internet use PHP in some form or another.

Marc Laliberte  3:09  
In fact, like most major content management platforms like Drupal or WordPress, all use PHP under the hood. Many websites run as a PHP stack. I mean, crap! I remember back in the '90s making lamp stacks for websites, which include PHP on the backend for server-side scripting. Extremely popular framework for web content and websites. Yeah, it's only it's only a few sites out there, not too many as these numbers show. The graphics that Corey are showing is there's roughly 41 million websites that were vulnerable to these issues that we'll talk about in a second, and only 1 million have been patched as of right now.

Corey Nachreiner  3:50  
Just not great. Took it off, but yeah, yeah, 1.3.

Corey Nachreiner  3:54  
Man, so and all pretty much in the last 24 hours. Exactly. I guess it is Thursday where you guys are hearing this Monday.

Marc Laliberte  4:03  
We are recording this on Thursday. As of Wednesday, when this all started going down, like we started seeing websites just get taken offline by their providers as a preemptive measure while they

Marc Laliberte  4:13  
investigated their exposure and

Marc Laliberte  4:17  
and added patches to the websites. Yeah, yeah. Bamboo HR was one of them that we we we

Corey Nachreiner  4:25  
suppose they're not breached, but they they did go down, which

Corey Nachreiner  4:29  
kind of a kudos I got to say. You know, Bamboo is a well known HR platform for a lot of companies that has you know PII for employees. I actually I think CISA and others later in the day, recommended people do that. But the fact that big sites and applications, Bamboo is a web-based SaaS application, took the business hit of making their application not work for the time that they were making sure that PHP was sorted out was,

Corey Nachreiner  4:57  
I guess, required. But but.

Corey Nachreiner  5:00  
A pretty bold move.

Marc Laliberte  5:01  
It's a bold move, but it feels like in this case, and we'll talk a bit more about the vulnerability specifically in just a bit. But these were two like genuinely critical issues at face value,

Marc Laliberte  5:12  
widely used across the internet, and it makes sense as a business that when you don't know your actual exposure, when you don't know like how exploitable are they really? And all you see are the like worst case scenario. Oh my God, these are serious. Makes sense to err on the side of let's not create a security incident. Let's take our site down for a couple of hours

Marc Laliberte  5:33  
and just get it updated. So hats off to

Marc Laliberte  5:36  
Bamboo HR and other companies that did that. But like under normal circumstances, this would have already been chaotic. Like these are critical vulnerabilities in extremely widely used software, ubiquitous software. Yeah, yeah.

Marc Laliberte  5:51  
Like even more so than the log for shell issues that we saw a few years ago. Like the majority of websites use PHP, so on its own, it would have been chaotic, but there were even more chaos factors that went into this as well too. Like PHP, it's distributed through several several partners. Think like every Linux version ships its own PHP package. Applications like Drupal and WordPress ship their own packages. This is a open source

Marc Laliberte  6:20  
framework that is developed by a foundation that gets some funding from

Marc Laliberte  6:24  
private companies, but it's a small team with an extremely wide reach, and so coordinating vulnerability disclosure is very important.

Marc Laliberte  6:34  
And the PHP Foundation published a an update pretty soon after, like all of this started going down, telling their side of the story and some of the timeline around this, where they actually found these issues back in 2024. At the time, they chalked them up as like extremely difficult to weaponize and also extremely difficult to fix without some major disruptions. They spent like a bit of this post talking about some of the difficulties in that and kind of explaining why it took two years to to resolve them, while also accepting culpability for it, and that like part of the the delay was just indecisiveness from their security engineering team and some of the people involved. I think we'll get into it too. AI will be involved later, but I think part of why it was maybe okay it took a while for them to break these. These were complex vulnerabilities that,

Corey Nachreiner  7:24  
at the beginning, were not easy to exploit. Even now, by the way, it has a pretty complex exploit chain that we're finding. You know, the proof of concepts without an LLM to help out.

Corey Nachreiner  7:37  
It's like if you remember Heartbleed. Heartbleed. Some of these vulnerabilities are RCs that are kind of leaking bits of memory,

Corey Nachreiner  7:45  
but you have to kind of wait for good memory. If you remember Heartbleed, Heartbleed was kind of trivial though.

Marc Laliberte  7:49  
It was like you send a like a specific packet with a saying you've got a very large header, but only sending a small header, and so it.

Corey Nachreiner  7:56  
But you wouldn't always get the memory you wanted, right? Correct. Sometimes take a crap load of packets, and there are hardening things you can do against the ones that are leaking memory. Where you know, after so many sessions, you can. So it's it might be easy. Anyways, either way, between the getting the right bits of memory at the right time and the complexity of the exploit chain, you know, maybe they thought during finding these complex vulnerabilities, that they didn't have to rush the fix until later. I'm sure we'll get into what changed to make it exploitable. But, anyways, it it is a complex exploit chain, very complex exploit chain. But

Marc Laliberte  8:36  
towards I think it was August September timeframe, according to their disclosures here. Some independent security researchers reported the same issue to them. I guess it was end of July, early August, where they started reporting to it, largely thanks to AI assistance and actually weaponizing these bugs, these vulnerabilities into something actually exploitable. And so that's when the PHP Foundation started work on developing the fix. Now that they had a clear path to actually exploiting these weaknesses, and they started working through coordinated vulnerability disclosure. And there were several independent organizations that reported it, like Wiz, the really popular web application security platform. Did OpenAI was another one. Another security firm called Caliph did as well. All within a pretty short period of time of each other. Which quick tangent. So they're all clearly using AI to help with this. One of them, let's say, discovers it using AI.

Unknown Speaker  9:33  
What do you think? How much of a factor you think like training data was in this, Corey? Where one of them discovered it, and now suddenly multiple people, multiple people are discovering it similar time.

Unknown Speaker  9:45  
I don't know. It's kind of smells like too much. Maybe using a mythos model and something was trained in. Yeah, it it's we're all using the same foundation models once you get in. So I wonder

Unknown Speaker  9:57  
if as people like Open Air.

Unknown Speaker  10:00  
Like yes, I could see that being part of it. It's interesting. But anyways, started working on the fix, and they coordinated originally what was supposed to be the disclosure and the fix with their partners on october 22 at 2pm. UTC, but on Tuesday the 20th, a couple days before at 3pm. UTC, the two CVE IDs started circulating on social media with incorrect scores. They were tagged as 10 out of 10s instead of 9.2 out of 10s. But I mean, mind you, 9.2 is still pretty damn serious.

Unknown Speaker  10:33  
But did not include any actual vulnerability details at the time.

Unknown Speaker  10:38  
Even then, because the IDs were out there, because people knew there was a vulnerability that was going to get resolved soon. I guess two vulnerabilities. The PHP Foundation brought up their pre-announcement and published it at 7:30 UTC on the 20th, and then also moved up the fix from the 22nd to the 21st at 2 p.m. UTC. They noted that like distributors were still trying to repackage the fix. PHP is distributed through a wide variety of platforms. They needed a bit of time to get it ready to go, so that everyone could be fixed and safe at the same time.

Unknown Speaker  11:10  
At 8pm. UTC on the 20th, so still on that Tuesday, five hours after just the IDs started circulating, one of the security firms that had reported the issue, accidentally posted their proof of concept video, which included some of the vulnerability details. And this is when the cat starts getting out of the bag.

Unknown Speaker  11:29  
Three hours later, the PHP Foundation started providing mitigating advice to some of their providers, which also leaked pretty quickly.

Unknown Speaker  11:38  
And then finally, on the 21st at 7:41 UTC, a company called ChainGuard unilaterally broke the embargo that the PHP Foundation had with many of their partners, and published their images containing the fix six hours before that plan coordinated disclosure.

Unknown Speaker  11:58  
The report that the PHP Foundation put out about this the blog post that Corey's showing for our video viewers.

Unknown Speaker  12:05  
It like kind of reeked of just disappointment from them of like we're a small team. We've gone through this effort, this Herculean effort to try and resolve these issues, coordinate disclosure. Things keep leaking, and then a company just unilaterally decides they're going to go ahead and publish the details, which is important because as soon as a patched version is out, anyone can go diff them and find what changed and get basically a big shining laser right towards the issue.

Unknown Speaker  12:33  
And with AI-based coding and AI decoding, like it AI has made how long it takes to diff a patch into a working exploit, kind of a fraction of the time it used to be too.

Unknown Speaker  12:45  
Yep. Now, ChainGuard CEO Dan Laurent

Unknown Speaker  12:50  
told his side of the story on LinkedIn. Said they got a heads up from the original researchers on the Friday before this week. They independently discovered the vulnerabilities with their own large language model, and he said with a basic prompt, something like "Find bad vulnerabilities in PHP." I'm assuming it wasn't that exact prompt, but probably something similarly benign.

Unknown Speaker  13:11  
They were read in on that Saturday and confirmed that what they had found with their large language model matched up with what was going to get fixed. They noticed that leak on Monday and leading into Tuesday, they said they found confirmed exploits in the wild,

Unknown Speaker  13:27  
and that's when they made the call to go ahead and publish. By the way, they never shared them. Like I, they never shared them, so

Unknown Speaker  13:35  
they're they're saying all this stuff, and I'm you know

Unknown Speaker  13:39  
possibly, but and and I kind of in some ways I trust them, but a lot of people hadn't seen those exploits yet, and especially working. We'll talk about working exploits later, probably. Yeah.

Unknown Speaker  13:54  
And so they said, following their coordinated vulnerability disclosure process guidelines, they released after evidence of active exploits in the wild, they made the call to release their patches without coordinating with the other vendors or with the PHP Foundation themselves.

Unknown Speaker  14:10  
This was interesting, where

Unknown Speaker  14:14  
I can kind of see where they're coming from. Where if they do strongly think they have evidence that there's active exploits, it kind of makes sense for them to protect their customers and people out there on the internet, but at the same time, like suspected active exploit or attempted active exploit doesn't necessarily mean widespread weaponization. And when you publish the fix like that, it makes that more serious scenario all the more likely to occur very quickly.

Unknown Speaker  14:39  
I will actually also say that to be fair to Chain Guard,

Unknown Speaker  14:44  
I'm trying to find where I remember

Unknown Speaker  14:48  
they they do have a published principle on their disclosure and talking about vulnerable disclosure, and in in part of their published coordinated disclosure, they had before.

Unknown Speaker  15:00  
This even happened a line that said unless there's evidence of active exploits in the wild. So, to be fair to them, like it, it that is a caveat you might have to think about. So, and if they if it is part of their own policy that you know we are trying to protect the world and coordinate disclosure, but if there are active exploits, we, you know, we have to do the right thing to protect users. There's an argument here, but it definitely was. I mean, you can clearly see that PHP Foundation did not love

Unknown Speaker  15:32  
it. The because then the counterpoint to that that I'm going to make and just throw a little bit little bit of shade at ChainGuard

Unknown Speaker  15:40  
is so like last year, they were a $40 million company. They are peanuts in the in the space of like security, especially compared to something like the PHP Foundation and the entire reach that they have across the entirety of the internet. And it does not feel like the right call for a smaller security company like that to just decide they're going to publish something of this importance, at least without consulting or warning or coordinating with other vendors in the space too. Like, I guess maybe it's me being unfair. Like, I could see maybe a little more credit if it was Google or someone like that, where like they've got all the the threat intelligence in the world. They believe there is a serious threat of active exploits, so they're protecting the internet. Bigger than PHP Foundation, and they they're smart. I agree with you there. I'll also say, by the way, PHP. We've talked about this in the past. Open source surviving AI vulnerability disclosure and threats is is hard. PHP is lucky, and it's actually a big enough open source project that has a foundation. It has donors. It has money, some money coming in, but even they are already kind of as they've claimed, not keeping up with the new, you know, with vulnerabilities. They get a lot of them a day. They're also dealing with slop, hundreds of them that aren't even good. So

Unknown Speaker  16:57  
I'm with you. These open source foundation, it's a big one. It's a ubiquitous tool. ChainGuard might not be the right organization out of all the ones coordinating to

Unknown Speaker  17:08  
decide to leak that early. Yeah, you mentioned like in that same post from the PHP Foundation, they said that thanks to AI-assisted research, they are getting more reports than before. They've published 15 CVEs since july 1 of this year, but they've had to go through more than 100 reports that turned out to be invalid or not reproducible, and so they're clearly getting a lot of

Unknown Speaker  17:31  
time-wasting reports. And I guess if they're not using AI to triage them, that is human time wasted. That for an organization of that size, I'm willing to bet they don't have a lot of time to waste. No,

Unknown Speaker  17:44  
but I mean this is the reality we're in. Like, what was it? October patch Tuesday. Microsoft patched 1400 vulnerabilities, which was more than all they patched in all of 2025. Like there are the one after August. I haven't even caught up to that one. Yeah. So this is after the 999 one. I just exactly 1400 in the most recent lovely.

Unknown Speaker  18:05  
So keep going up. Like there are clearly more and more vulnerabilities being discovered and fixed by all vendors out there across the ecosystem. Like this feels like a we're reaching an inflection point with vulnerability management now where something's got to give, and I'm not sure what's going to give, whether it be patch management or

Unknown Speaker  18:26  
just our ability to effectively respond to and fix these issues. Because it's crazy. There's so many,

Unknown Speaker  18:35  
but at least into detail on the flaws, or yeah, let's go into it. So

Unknown Speaker  18:39  
Wiz was one of the research firms that found the issues.

Unknown Speaker  18:43  
They published a write-up on I think was on Wednesday with some of their analysis of them. They started out by talking about how they leaned heavily into frontier AI in order to discover them. Basically, in order to turn the flaws into a weaponizable, reproducible exploit, they had to chain together a whole bunch of different, very complicated exploits, and even they were only successful in a few different specific distributions of PHP. Like they got the the remote code execution one working in I think it was Drupal, WordPress, and Joomla. They got the memory leak working in a few others, but they noted that they would all probably work in other flavors too, and other distributions. It's just everything's a little bit different. And one thing that stood out to me was, like in the past, like so, this code's been there for a decade. And in the past, turning these flaws into exploits required a lot of manual effort from humans for each specific version, and each specific platform, and each specific distribution, and if any of those things vary, you basically start from scratch or at least pretty far back down your exploit chain. But with artificial intelligence, that makes it easy and scales, and that's no longer a barrier. Is the word? I think one of the reports basically said that.

Unknown Speaker  20:00  
What would have taken a world-class team months can now be done by one person in a week. Yes,

Unknown Speaker  20:07  
something like that. That's not my favorite quote when it comes to

Unknown Speaker  20:13  
protecting weather.

Unknown Speaker  20:16  
But

Unknown Speaker  20:17  
it makes sense. Like these tools are extremely effective and extremely powerful. Like we've seen how capable they are internally at WatchGuard, with even Mythos class models being used to find issues. And this is exactly why you're and our company is doing Firebreak is is to make sure we're the ones doing this instead of threat actors and and companies external organizations. Yeah, and so like instead of walking through each of these chain flaws that creates the vulnerability, like at a high level, it was pretty complicated. Basically, it in order to exploit these vulnerabilities, you have to trigger a fatal error, usually by

Unknown Speaker  20:56  
creating an out of memory condition on the web server. So, like you send an extremely large request, usually something with nested children under it, creates a out of memory condition, which causes the PHP server to crash and create a a dump of that error,

Unknown Speaker  21:14  
and then that doesn't correctly clear up memory that was set by the web request you sent, and so basically, by the way, PHP is literally designed to do that, right? Like each incident of a PHP, whatever you want to call it, worker is supposed to, like if you have a site with tons of people going to it, it's supposed to like build this little, like sandbox, use its memory, then stop doing something, and then memory's cleared for the next one. So the fact that you can find a way to keep this crash prevented it from clearing that memory in a way that you can start to pull it for one type of default. There's also the RCE ones, but the one we're talking about right now is the memory disclosure one that could

Unknown Speaker  21:56  
lead to. So by triggering the error, doesn't clean up all the the memory. It leaves some of it partially initialized, which then a subsequent request can potentially leak data out of

Unknown Speaker  22:06  
it. For the other one, it is a chain of like memory corruption issues, including using a huge amount of memory that ultimately resolves in the attacker being able to control something that gets deserialized unsafely and then executes code,

Unknown Speaker  22:22  
but these were like, like there's different flavors of memory corruption issues. There's ones where a single request, or at least a couple of requests, can kind of spray the stack, get everything ready to go, and then exploit the issue. Like relatively easy ones to exploit. This one feels like a, it's a long running one that takes a lot of persistence and like specifically crafted requests for the specific version and implementation for it to even work,

Unknown Speaker  22:49  
and that is where artificial intelligence shines. Like Wiz even mentioned that like a human trying to create different variants for this would have taken a huge amount of time, but AI can recognize the differences in the applications it's targeting, custom tailor that exploit towards those specific applications, and then have a much higher success rate than a human would have, or at least a more

Unknown Speaker  23:12  
a faster success rate than a human would have.

Unknown Speaker  23:16  
This feels like a I don't know about you, Corey. It feels like a warning shot across the bow, where a widely used application that the entire internet is built on has some serious issues that humans could not catch and fix on their own over the last decade, but advancements in AI are making I wouldn't say trivial. They're still complicated, but significantly easier to find and exploit.

Unknown Speaker  23:40  
At what point do we just turn off our computers?

Unknown Speaker  23:45  
I think everybody should be doing. If you're not doing what we are, and pointing it at your stuff to try to fix it, I mean, the good news is AI can help with the fixes. We know this. The fixes, like when you find this stuff, you can help generate secure code quicker with AI, probably to start fixing it, and if we point these powerful models at our own code ourselves, I think you've even said this internally, Mark. Like in a year and a half, we we might have releases like Microsoft, where our average release has four times, 10 times, a full year's. Maybe ours won't have a full year's worth of vulnerabilities that are fixed, but it's eventually going to plateau. It's going to result in secure code. So I do think we're going to have a disruption period for the next year and a half. That's going to be,

Unknown Speaker  24:32  
I was going to say scary, but to be nice, I'll just say disruptive.

Unknown Speaker  24:36  
But the good news is things can get more secure because of this. There is a silver lining. That is my hope. Where it's felt like historically,

Unknown Speaker  24:45  
companies find and fix issues, but the rate that they can find and fix them isn't fast enough to like actually meaningfully clear up their technical debt. But now, thanks to AI, like it can, and there is a future where we will have found and fixed nearly every.

Unknown Speaker  25:00  
Across my worry is the the laggards and the and the fact that many people need access to the frontier models. Eventually, there's a long chain of software companies, and there's a lot of companies. It feels like that have barely understood how to use AI for their business, let alone implemented strong AI-based software security development lifecycle, like like there's a good and a bad way to even implement AI encoding. So, if you have a development organization and within your company or organization, you you really should have been doing this a year ago. And if not, you need to start now because

Unknown Speaker  25:38  
otherwise you're going to be overwhelmed by things. And if there's still,

Unknown Speaker  25:42  
I mean, there's still going to be humans in the loop for validation of all this stuff. But if you're human-based for most of that development lifecycle for fixes, you're you're going to run into issues. Open smaller open source products are already underwater with a flood.

Unknown Speaker  25:58  
Yeah, agreed. So this is like I said, we're recording this on Thursday. There's even some evidence now that

Unknown Speaker  26:06  
the fixes may not have even been complete. Seen some reports from one of the original researchers that a couple of different paths are still exploitable. So maybe by the time that this episode airs on Monday, we'll have yet another round of patches to go out and try and do across the entirety of the internet.

Unknown Speaker  26:25  
And by the way, there's a lot of issues. There's old fixes that won't have patches. Like there's going to be some updating from PHP versions. There's going to be IoT devices that are built on Linux distros with PHP in them. There's going to be

Unknown Speaker  26:39  
a not even IoT like ton. This this is

Unknown Speaker  26:42  
definitely log for shell. It's going to take the industry months to really patch this, not just out of the web server software we're looking at right now, but the entire infrastructure we have that leverages open source and. I'm going to say that months is even ambitious, but years. Yeah, you're right. The good news is that, like,

Unknown Speaker  27:02  
maybe not the good news, but the the hopeful, the optimist in me hopes that, like, yeah, IoT is probably like a disaster case for this, but those should hopefully be buried somewhere in the bowels of a company and not actually exposed. Hopefully, not exposed externally. We hope, but man, like, we both. Oh wow, the site refreshed. Look at this. We're we're up to 2 million now. Dang! Look at that. We're at what 5%? How did that pop up so quickly? That's good news. Maybe.

Unknown Speaker  27:29  
Wow. 5%. Wow. Look at that. Happened two whole percent improvement in the 30 minutes we've been recording this. Maybe there is hope for the internet after all.

Unknown Speaker  27:39  
That's crazy. And or the page was broken and they just fixed it. Maybe the page is running off PHP. Maybe all this is made up.

Unknown Speaker  27:49  
Who knows?

Unknown Speaker  27:51  
I don't know. Any other parting thoughts, Corey?

Unknown Speaker  27:55  
We've we've hammered this one home. So let's move on to the next interesting. Yeah, patch your PHP stuff. So moving on to the next story. Also on Tuesday, researchers discovered eight intermediate CA certificates belonging to an India-based certificate authority called EMUDA,

Unknown Speaker  28:13  
which were compromised. And compromised is, I think, putting it lightly.

Unknown Speaker  28:18  
So

Unknown Speaker  28:20  
these were discovered after a an internal reporter signed a message saying we have a problem here, signing it using the private keys for all eight CA certificates. Now, without getting too in the weeds on certificate cryptography, a certificate or I guess a certificate authority like this is used to sign other certificates that might be assigned to a website or some other application. When you go to Google.com, it's signed by a CA certificate, for example. Your web browser includes hundreds of CA certificates that it trusts by default. Up until Tuesday of this last week, it included these eight CA certificates. Almost certainly,

Unknown Speaker  28:59  
certificates are basically made up of a public key and a private key. The intent being the private key is secret; it's used to sign other certificates. The public key is what you advertise along with it. Madra has it. Yep, is what you advertise as proof that it's related to that private key that was used to sign them. There's different flavors of cryptography. We used to have RSA as the the primary key method for most certificates. Now many are moving towards elliptic curve cryptography. There's a very popular proof in many cases. Yep, there's a very popular elliptic curve

Unknown Speaker  29:35  
I don't know number or variant called p3 84, which is supposed to include 384 bits of entropy. Think of it like the number is supposed to be two to the power of 384 possible numbers. So that that by the way, that's just doesn't help. Your brain cannot imagine anything that big. It is a number with 116 digits.

Unknown Speaker  30:00  
So noting that a million, the number 1 million is nine digits. This is a 116 digit number. So that's how much entropy it's supposed to have.

Unknown Speaker  30:10  
And millions 10 digits, right?

Unknown Speaker  30:13  
No, wait.

Unknown Speaker  30:14  
Oh, yeah, because you have to include one before you can get to a million. So it's it's nine zeros. Like I said, math was never my strong point. But moving on,

Unknown Speaker  30:25  
so it's supposed to have a huge amount of entropy and very large numbers. But it turns out that the private key sizes for these eight certificates range somewhere between three bits and eight bits. That's the best cryptography. Four. What a great, what a great private key. So like private keys are made up of a couple of different numbers that are used to like effectively derive that key material. It's supposed to be a 116 digit number, but yes, one of these was literally the number four as the big primary component for the. I mean, I'm trying to find what story saw it, but in a way, it was, it was 00. Like if you look at it in hex, it's 0x 0004 at the end. But ultimately, it was still four. Yep. Another one was 56. One of them was 66. None of them were more than three digits. These these are literally the ones that were used for the eight certificates. If you're watching the video version, so if you're supposed to have a 116-digit

Unknown Speaker  31:23  
number and you come up with the number four, there's something fundamentally broken in your key generation algorithm. And and and the biggest one, 237, that's still less than my back bank pin. Which, by the way, a four-digit any computer can break a four-digit key in

Unknown Speaker  31:40  
milliseconds, probably. And live into bed. that a rainbow table do it. A rainbow table probably exists for every single four-digit number already. Yeah, but so this is like bad. This is as bad as it could be. Not only for just a certificate, like a private key for a specific website, but this is a certificate authority that can sign a certificate for any domain that's out there. So before these were revoked out of every web browser and every application's trusted CA certificate store, like these private keys could have been used to sign a certificate for Google.com or for Microsoft.com or WatchGuard.com or any company out there, because it's a certificate authority, and we quote unquote trusted them up until here. And think about it. This is another thing that affects everything. Because we're like when we talk about this stuff, you're probably just thinking, "Oh, I'm right now at Ars Technica's HTTPS site, and that's telling me that this is really their site." But these are the same certificates used in in source code repositories and development,

Unknown Speaker  32:45  
you know,

Unknown Speaker  32:47  
pipelines. Like everything that we verify to make sure we're getting a piece of open source or or something from the right place is using this from these authorities. So it affects everything. Our email encryption, our our connection, like it's a huge deal.

Unknown Speaker  33:05  
And so you're looking at the beginning. They say this is going to be a bigger cleanup than Heartbleed. And because like some things can respond quickly, like every major web browser has already revoked them. If you're a WatchGuard customer, the Firebox firewall appliance has a built-in CA certificate store. We've already revoked them out of there immediately after seeing this, but those are like the easy things. There's the long tail of like applications that have their own CA certificate store, both like mobile apps and desktop apps and things that use like cert pinning, where they're more difficult and you have to update each of them individually. Or your example, Corey, an IoT device that never updates its CA certificate store, and so these are permanently in there.

Unknown Speaker  33:48  
And yes, like what this means is someone can like man in the middle of the connection and sign arbitrary certificates to make it look like it's still encrypted. They could use like DNS spoofing to force you to go somewhere, sign a CA certificate to make it seem legitimate. It does still take some

Unknown Speaker  34:08  
like attacker effort to actually exploit this type of weakness or flaw.

Unknown Speaker  34:13  
But by the way, it also takes a man in the middle. We we do forget this. I mean, but those can be pretty arbitrary nowadays. So it's not a huge step. But you do have to intercept some sort of man-in-the-middle attack to be able to take advantage of this. But on the, I guess, on the happier side, or maybe for the hackers and tinkerers out there, I'm a I follow the home automation subreddit on Reddit, and one of the biggest issues for people that like to participate in home automation is many of the tools and stuff that we the IoT devices we deploy in our houses are starting to move towards a model where you have to use that manufacturer's website and SaaS infrastructure to manage it, and they like lock you out of being able to do any local management. This.

Unknown Speaker  35:00  
Type of flaw with those IoT devices when these CA certificates baked into them, where now we've got the private keys. It makes it really easy to intentionally hack those and basically homebrew your own connection to mirror that manufacturer's website and do whatever you want with it. So it does open some opportunities for people that like to tinker with IoT devices. I'm excited about that. Back into it, yeah. I'm assuming that my iRobot Roomba is going to go out of business here pretty soon, considering they're under bankruptcy, and this might let me keep my my robot alive for a little bit longer, which is kind of nice.

Unknown Speaker  35:34  
But for the rest of us on the internet, this was a pretty dang serious finding,

Unknown Speaker  35:40  
and it was actually

Unknown Speaker  35:42  
a not existential threat, but basically the company

Unknown Speaker  35:46  
that like owned these certificates announced pretty soon after that basically screw it, we're done, and they're shuttering their CA certificate business and no longer going to provide that.

Unknown Speaker  35:58  
I also find it interesting that this is a perfectly valid certificate. There is literally not like mathematically there was nothing. It was weak. It was stupidly weak because it was generated with no entropy, and they they should have a setting saying you can't create a certificate that has less than X amount of that that that uses for, let alone through. Feels like something that could be automated and checked before it ends up in like Google. I think there's a setting, and one of the things I read there's actually setting like this was a mistake they made in configuring where you can actually set the at the minimum number of those 116 digits to use or whatever.

Unknown Speaker  36:40  
But but either way, it was it act. There's nothing. It was a valid one little through line to some of our stories is when we get to the next story too. There's supposedly valid people that are part of a security chain. But anyways, it's a perfectly valid certificate. It's just a very badly created one. It is crazy that

Unknown Speaker  36:59  
Mandra is no longer making. I mean, probably good that they're no longer making certs. So,

Unknown Speaker  37:04  
yeah, I think that was forced. I wonder if I

Unknown Speaker  37:09  
or or if other certificate, yeah, I can right. They help with all of certain. They handle. It is possible that like Google said, screw you. We're not adding any more of your CAs back here. That was your one and only chance to be trusted, and now you're not trusted. By the way, are there firewall implications? Because we're part of CA trusts a lot, but we have the ability to adopt. I mentioned that on

Unknown Speaker  37:34  
immediately after the disclosure, we yanked them out of the CA store on the Firebox, which updates every 12 hours or so. And so, everyone that is a Firebox customer that should already be removed from there.

Unknown Speaker  37:46  
Good, good. It,

Unknown Speaker  37:48  
yeah. It it also shows like a fundamental weakness in the internet, where like how we do authentication for websites is built off this layer of trust, and like if trust can be broken that easily, like it's,

Unknown Speaker  38:03  
it's an issue.

Unknown Speaker  38:05  
Yeah, we've talked about it before. Cert authorities matter, DNS authorities matter, and BGP authorities matter.

Unknown Speaker  38:12  
And the internet is that those are the core of everything we do on the internet.

Unknown Speaker  38:18  
Good thing that AI and quantum computing, and none of that is a major risk for this technology we rely on.

Unknown Speaker  38:27  
That's

Unknown Speaker  38:28  
decades and decades old, and probably has no issues with it, so we're fine.

Unknown Speaker  38:33  
Just totally fine. Speaking of people that are fine while probably sitting in jail,

Unknown Speaker  38:39  
moving on to the next story. So Shiny Hunters is one of the I'd say most prolific and one of the longer running extortion organizations that are out there. And I think at this point, like I don't know about you, Corey. I feel like they've kind of reached

Unknown Speaker  38:54  
the they're kind of like anonymous, the hacking organization where there isn't really a when you steal the FBI's data and start to extort them to stop lying about your threat actor practices. You you you have reached the pinnacle of you're you're now a historical part threat actor. That will be noted, I think. So yeah, and we did we did talk about that at the end of September where they had hacked into FBI jobs. Gov and stolen effectively all the data in there, which immediately put them on the FBI's radar,

Unknown Speaker  39:26  
caused a couple of arrests. But back on October 9, a couple weeks ago, the FBI arrested one individual named Ed Dubrovsky, which was a executive at a Canadian security firm called Cyber Steward, and they arrested him while he was down in Pennsylvania, attending a security conference.

Unknown Speaker  39:45  
And Ed was known for being a ransomware. By the way, I don't think they named him. I think I think it ended up, and it wasn't Krebs either. But I think it was found out through the media. But yeah, either way, he was arrested.

Unknown Speaker  40:00  
I think when he was attending the conference, he is part of Cipher, but now it's Cyber Steward. But that's crazy, right? This is someone like us, me and Mark. You go, you pay, you have a badge. You're attending a security conference. You assume you're with all the good guys, and then the FBI come and arrest the guy. Who, by the way, I think they were the biggest sponsor

Unknown Speaker  40:20  
of the actual conference. Yep. So he's a experienced ransomware negotiator. He even wrote a book called Cyber Extortion Strategic Response, where he advocates for working directly with cyber criminals during ransomware extortion events.

Unknown Speaker  40:37  
But like this was pretty crazy when I saw this article pop up a couple weeks ago. I think it started. One of our our analysts just posted randomly that someone from Shiny Hunters got arrested, and then Krebs posted his article detailing that it was a executive at a Canadian company that got arrested. But this story gets even crazier too, where this arrest came after the FBI spent a month reviewing devices that were confiscated from another arrest in the previous month from Dutch authorities, from a member of Shiny Hunters called Pepjin van der Staap,

Unknown Speaker  41:13  
where this one's nuts, they used a flashbang grenade to stun him while he was logged into all of his devices, so that they could keep them authenticated and access them to see who he was talking to on Signal and the other chat apps he was using.

Unknown Speaker  41:29  
That has to be right time. That means this guy is literally on his phone. The phone is unlocked. I'm assuming, or the computer device, but I'm guessing with Signal, it's the phone. I'm imagining they're making sure that they see him actively in the app in order to time the flashbang, so that he won't think to lock his phone. Like I'm trying to picture this scenario: they like throw a flashbang grenade, grab his phone before it automatically locks, put something on it, like a fake thumb on it that wiggles to keep it unlocked permanently. Yeah, plug it into power so I can keep it unlocked, and then use that to view every app he has in it. It's crazy.

Unknown Speaker  42:07  
Real note to self: never become a criminal that has to worry about FBI or Dutch police sending flashbangs. But other note to self: if you do become a criminal, if there's ever a big flashbang, press power on your phone immediately, otherwise you're screwed.

Unknown Speaker  42:24  
Yeah, agreed.

Unknown Speaker  42:26  
But so they got access to his phone. This is mr. Vanderstapp in the Netherlands, and they found that he had frequent communications with Dobrovsky, the Canadian executive, and a still unnamed U.S. executive from what Krebs is calling a large U.S. consultancy firm, he intentionally didn't include their name or this executive's name because they haven't been formally charged with the crime yet. But he apparently knows who they are.

Unknown Speaker  42:53  
And it gets even crazier where Krebs, like one of the premier cybersecurity journalists, says that this unnamed U.S. based consulting executive appears to have been the one that set up shinywire.com or shinywire.com, which is the extortion site that ran from May through October for shiny hunters.

Unknown Speaker  43:14  
And he goes on in his post to talk about some of Dabrowski's other previous coworkers. There was another colleague from Cipher called Daniel Tobach, which appears to be very closely related to ransomware extortion and ransomware extortion payments. Like he, there's emails or messages that Krebs showed where this

Unknown Speaker  43:36  
Tobak guy was specifically recommended by some of these ransomware operators as the person to call to help negotiate it. He's linked to some cryptocurrency wallets that had three and a half million dollars with the Bitcoin that traversed through them.

Unknown Speaker  43:51  
And my main takeaway from all of this is,

Unknown Speaker  43:54  
ransomware negotiators seem to be insane and maybe flying a little too close to the sun and getting too involved with some of the ransomware operators.

Unknown Speaker  44:03  
Are you just saying insane? I think they're corrupt. I think this is corrupt. Like I'm not sure we can make a statement on all of ransomware negotiate. I well, let's step back.

Unknown Speaker  44:14  
I have never really liked ransomware negotiation. This Dubrovnik, whatever guy Dubrovsky, he he wrote a book on it, and he said, you know, it's not all about trying to drive down the ransom. It's about delay tactics and other things to try to learn how real it is and to try to delay them while you're figuring out what to do. But I never really liked it because, as you know, I don't believe in pain. I know it's not that simple. I know hospitals and life and death and all that. I I can't make the simple choice for everybody, but I think it's stupid to pay this business model because you just ensure it exists. But I never really loved the idea of ransomware negotiators. I think it was part of cyber insurance, by the way. Like when.

Unknown Speaker  45:00  
Cyber insurers first started paying. They wanted to. I think they inserted some of this. But the first thing I think about is everyone always jokes with antivirus companies. Oh, you guys are writing the viruses,

Unknown Speaker  45:13  
and it's like a common industry joke. I don't think it's true. I do think. I mean,

Unknown Speaker  45:19  
there's a non-zero chance that John McAfee wrote a few viruses in his day. True, true, and we do know Marcus, right? Marcus was Hutchins was a good Hutchins was a good guy that had done some bad stuff. We all have had hacker stories where we've had like there's a difference between black and gray and exploring, but a lot of

Unknown Speaker  45:39  
us, you know, explored be as we learned how to secure, but I I I would like to think most defenders really are defenders. So to me, the big part of the story is it's disgusting whether like negotiators should be there or not. This is the the people you're trusting and going to to get help to try to help your business in the most critical thing, and they're literally working with the bad guys to get themselves more money. And I think the articles even talked about how how you know it is kind of a racket where they get portions of the ransom that they're negotiating. So it's even in their own interest to negotiate it down. So yeah, it's not in their interest to negotiate it down if they get a percentage. It feels kind of gross.

Unknown Speaker  46:27  
I I think this is it's the type of story that's going to affect the trust of the security industry, and really trust is what all of security,

Unknown Speaker  46:36  
you know, when even us as a vendor trying to help our MSPs and customers is all about you trusting us to do a good job for you, and I so I I hate this story, but I'm glad these jerks got caught. I hope that anybody doing this, I hope

Unknown Speaker  46:53  
there's a waterfall of these, as there seems to be other people being found out. So yeah, I think trust is an important piece on this. Like this, all kind of started, or at least exploded, because shiny hunters felt like they were losing their trust from their hate to call them customers, their victims. Where the FBI and like other organizations were saying that shiny hunters was destroying data. There was no way to recover it, so they were losing trust that if you pay, you would get your data back, and that's what caused them to go hack the FBI, and or at least at face value, what they claim was the reason they went to go

Unknown Speaker  47:29  
hack the FBI,

Unknown Speaker  47:31  
and that seems to have blown up in their face. Where now a lot of them are getting arrested.

Unknown Speaker  47:36  
Yep. At the same time, it's also kind of weird that Shiny Hunters has been around for quite some time, and only now are a bunch of people getting arrested. It wasn't until the FBI themselves had egg on their face that they went headfirst into trying to

Unknown Speaker  47:52  
get them all apprehended. But

Unknown Speaker  47:55  
I don't know.

Unknown Speaker  47:57  
I I don't like seeing stories like this where the people that are supposed to be there helping businesses recovering during a very difficult time, like a ransomware extortion event, are

Unknown Speaker  48:09  
more closely related to the ransomware operators themselves than they they let on. Now it is a developing story. I'm sure we'll hear other sides of it at some point too, but at face value, this just seems sketchy and crappy.

Unknown Speaker  48:25  
Yeah,

Unknown Speaker  48:26  
validity doesn't guarantee trust. Just because they're a negotiator, you may not trust it. And just because a

Unknown Speaker  48:33  
CA certificate is technically perfectly, mathematically, and cryptographically generated soundly doesn't mean it has enough entropy to be valid. There it is.

Unknown Speaker  48:45  
There's the through line. Well

Unknown Speaker  48:47  
done,

Marc Laliberte  48:49  
man. What a this was a crazy week in cybersecurity. With yeah, that's a lot of stuff, man. Insane vulnerabilities, people getting arrested, CA certificates turning out to be just number four. I

Marc Laliberte  49:04  
hope we can like slow down a little bit for the holidays. I'd like to be able to enjoy Thanksgiving and Christmas this year.

Corey Nachreiner  49:09  
I hope Christmas doesn't continue to be this. You know, this is the state of cybersecurity right now.

Marc Laliberte  49:17  
Did you crochet that yourself? Damn, you're talented.

Corey Nachreiner  49:20  
No, a friend, a friend. This is a gift from a friend. We we've been talking about

Corey Nachreiner  49:25  
the world.

Marc Laliberte  49:26  
You could have taken credit for that right there, but

Marc Laliberte  49:29  
very noble of you. Unlike other security practitioners out there. Well done. P.S.

Corey Nachreiner  49:36  
I'm not going to pretend to be a negotiator while taking your money. Thank you, I appreciate that.

Marc Laliberte  49:44  
Hey everyone, thanks again for listening. As always, if you enjoy today's episode, don't forget to rate, review, and subscribe. If you have any questions on today's topics, or if you have suggestions for future episode topics, you can reach out to us on Blue Sky. I'm at itsmark.me. Corey's at where are you that way?

Marc Laliberte  50:00  
Sec adept, and the both of us are on Instagram at WatchGuard underscore Technologies. Thanks again for listening, and you will hear from us next week.

Corey Nachreiner  50:09  
Peace out.