Security Now 1092 transcript

Please be advised that this transcript is AI-generated and may not be word-for-word. Time codes refer to the approximate times in the ad-free version of the show.

Leo Laporte [00:00:00]:
It's time for Security Now. Steve Gibson is here. Of course, there's a lot of security news, including, yes, a supply chain attack. France's under media— under-15 social media ban hits its constitution. That's not a bad thing. But this is what we call in the business a propeller hat episode. Steve is going to take a very deep dive into AI, how chatbots are made and unmade. This is a fascinating episode next on Security Now.

Steve Gibson [00:00:34]:
Podcasts you love. From people you trust.

Leo Laporte [00:00:39]:
This is TWiT. This is Security Now with Steve Gibson, episode 1092, recorded Tuesday, August 18th, 2026. Restraint obliteration. It's time for Security Now, the show we cover the latest in security, privacy, computing, science fiction, vitamin D, and anything else on this man's mind because he is a genius. Ladies and gentlemen, I give you Steve Gibson. Hi, Steve.

Steve Gibson [00:01:10]:
So, thank you, Leo. What our listeners are going to find is that—

Leo Laporte [00:01:20]:
I'm so excited about this show, by the way. He gave me a preview, folks.

Steve Gibson [00:01:24]:
Is that I did not get to the 2 topics that I wanted to.

Leo Laporte [00:01:29]:
Oh, no.

Steve Gibson [00:01:29]:
I only got to one because I started laying down the foundation for the topics. And there was just so much to say. I'm, you know, my only defense for this being about AI is that the world is. And security certainly is. I mean, we've— I don't have to even explain that anymore. As we said, Black Hat a couple of weeks ago was like the AI conference that happened to be doing security as the reason for spending all that money. Um, as I'm— okay, I look back at the tutorials on the internet, how it works, and computing and how it works. And there I was able to share with our audience things that I had understood for quite a while.

Steve Gibson [00:02:27]:
In the case of AI, I'm a, I'm a, you know, a layperson, neophyte, rank amateur. But I'm a curious researcher and I've been reading research. So What I'm going to be doing over— I have no choice really, is because it excites me, is I'm beginning to develop an understanding at the level I want to. Remember, I program in assembler. So when I say I understand something, for me to be satisfied, I have to understand, well, is the carry bit set or not? So, but at the same time, to make it understandable. So I think most of, you know, when I look at the things we're going to talk about, we're going to talk about how trusting an open source AI proxy might bite you. France's under-15 social media ban, a bit of AI prompting turned up a new serious bug in Zoom. Uh, and that the, the stock prices of AI-based network defenders has jumped up after that Black Hat conference.

Steve Gibson [00:03:51]:
That's all we had time for because the rest is me sharing a bunch of new understanding that I have that I think our listeners are going to appreciate. So, uh, And actually not that much. I mean, I didn't skip any fantastic news. I looked for all the good things. We got a great picture of the week. And by 2 hours from now, everybody listening is going to understand, unless they already do, they might, but will understand what I now understand about the early first steps of how AI as we know it today happened. What were those things? And for example, Leo, you were first out of the gate saying it's nothing more than fancy autocorrect. Turns out that next token prediction is all it did in the beginning.

Steve Gibson [00:04:51]:
That's what it was.

Leo Laporte [00:04:52]:
This was— I said that in my defense a year and a half ago. I mean, we were talking— That's my point.

Steve Gibson [00:04:57]:
No, and that's my point. Then it was true. And also Matthew Green, we quoted him last week saying it's not fancy autocorrect.

Leo Laporte [00:05:06]:
not just the next token, the next token, the next token.

Steve Gibson [00:05:08]:
It's because of what happened between. And I understand now, and I'm going to explain how we got from next token prediction to you can have a dialogue, because that's a very— that's a— those are different things. And, and it's a little freaky how— I want to say how easy it was But it also led me to have a conversation with Claude about its own nature. So anyway, I think a great podcast for our listeners. I titled this Restraint Obliteration, not obliteration but obliteration, because that's actually the, the term of art which is used for some of what happens or can happen with open weight models. So We're gonna— by, again, all I can say is 2 hours from now, you're gonna be like, oh, I understand a lot more than I did. And you'll probably, at that point, you'll understand about as much as I have, but I'm not, I'm not—

Leo Laporte [00:06:12]:
In other words, this is a brain dump. This is gonna be Steve's brain dump so he has some room to read more papers.

Steve Gibson [00:06:19]:
That's correct. And I already know where the next one is.

Leo Laporte [00:06:23]:
Oh, good. It's so exciting. Yeah, it's so fascinating. And I, and actually, I'm really glad you're digging into it.

Steve Gibson [00:06:29]:
Um, I don't have a choice, Leo. This is the most important thing that has happened in my 71 years of life. You could say, well, okay, computers, yes, I was programming them on a PDP-8 in high school. Internet, yes, we watched all that happen. But, but this is knowledge. I mean, those, I mean, yes, we needed to have computers.

Leo Laporte [00:06:54]:
Well, it's a stair set. Yeah, we got We had to have the internet for training. We had to have the computers, of course. You know, I would throw mobile in as another big revolution. The idea that you have the internet everywhere you go in your pocket. In your pocket, yes. And a significant amount of computing. But, and of course, if it weren't for gamers, we wouldn't have these video cards that are, turns out, are really good at AI as well.

Leo Laporte [00:07:19]:
So it's all been, you know, it's always the case. It's always been a stair step.

Steve Gibson [00:07:22]:
And of course, this is for curious people, right? You don't have to understand how any of this works. order to use it.

Leo Laporte [00:07:29]:
Any more than you do a computer, right?

Steve Gibson [00:07:32]:
Most people have no idea how a computer works. The internet is magic. AI is a worry because what's going to happen? So again, you, you can use it without understanding it, but here in our little, you know, little corner of the world, uh, we like to understand how these things work. So yeah, yeah, um, we're all going to understand how AI works.

Leo Laporte [00:07:55]:
It's pretty amazing. I mean, and actually, there is a continuity. AI is— I'm sure when you were at the Stanford AI Lab sale in the '70s—

Steve Gibson [00:08:06]:
'73.

Leo Laporte [00:08:07]:
When John McCarthy, the creator of Common Lisp, was there, I mean—

Steve Gibson [00:08:11]:
Yep.

Leo Laporte [00:08:12]:
This is ancient history.

Steve Gibson [00:08:13]:
With his gray ponytail pulled back.

Leo Laporte [00:08:16]:
Yeah. This is ancient history. But even then, the vision was What we would like to do with these computing devices is talk to them and interact with them in a natural way as we do with other people. Well, now you can. And it just blows me away that we have made sand think. It is mind-boggling. And as Geoffrey Hinton says, the way we did it is by applying huge amounts of electricity. He says— and it's really interesting, the talk I sent you, he says, Human brain is designed for low-power analog, right? And, and, and so its design is designed specifically for the kinds of things we were— we could have.

Leo Laporte [00:09:01]:
But once we figured out how to do ones and zeros and keep it accurate— our brains are not accurate, but ones and zeros— and he says if you apply enough power, the one stays a one and the zero stays a zero, right? It's all about applying really a vast amount of electricity to these things. Once you could do that— Then you have something that is analogous, but not the same as a brain and can do some interesting things. And that's what we're going to talk about next on Security Now. Don't get obliterated. All right, Steve, picture of the week time.

Steve Gibson [00:09:34]:
So this picture generated a great deal of feedback, furor, hubbub from our listeners. The email went out. I forgot to mention a week or two ago that I broke 21,000 subscribers.

Leo Laporte [00:09:49]:
Wow.

Steve Gibson [00:09:50]:
So we're on the— we're north of—

Leo Laporte [00:09:52]:
You've got more subscribers than TWiT has club members. That's very nice. Good job.

Steve Gibson [00:09:56]:
That's well, and I, I— That's because it's free, I might add.

Leo Laporte [00:10:01]:
Yeah.

Steve Gibson [00:10:03]:
Yes. And, uh, uh, great feedback. So first of all, I, I gave this picture the caption, this actually happened in Albania. Was it an accident? Or a very clever way to create a bridge across the river.

Leo Laporte [00:10:23]:
All right, I'm scrolling up for the first time. I haven't seen it. Wait a minute, there's a bus. Whoops. Now, wow.

Steve Gibson [00:10:34]:
You wonder, looking at this. So for those who are not seeing the video, we have a long, very green It's got Go Green hybrid city bus. I mean, it's long enough that it's got a door in the front. It's got doors halfway down its length. And then it's got rear doors.

Leo Laporte [00:10:57]:
It's good it's not one of them articulated buses, though. I don't think it would serve as well as a bridge.

Steve Gibson [00:11:01]:
Oh, it would be a problem. Yes. And somehow this darn bus is straddling the river. And, but, but what's— what I noticed about it first, well, after I actually— after I got over the fact that it was there, was that it's so long and it's got doors in the front and rear. And the fact that you're able to walk the length of the bus, it's a bridge.

Leo Laporte [00:11:26]:
It is a bridge now. So I think it's telling the middle doors are not open.

Steve Gibson [00:11:31]:
Yes, you don't know unless you wanted— unless you wanted to fish, then it would be good. You could, you know, sit there and dangle your feet out and fish. Anyway, Leo, this actually happened. One of our listeners found the— a, a, uh, I have a link on the— at the bottom of the picture of the news because some, some people who didn't see that said, oh, that's AI. That's that. How could that bus possibly get into that position? Because you would think looking at it, it would just go nose down.

Leo Laporte [00:12:02]:
Yeah.

Steve Gibson [00:12:03]:
into the river, like, right? How could it, like, get on the other side? It turns out it's the bizarrest accident.

Leo Laporte [00:12:10]:
And by the way, you know it's not an intentional bridge because there is crime scene tape across the bottom.

Steve Gibson [00:12:16]:
Ah, that's true.

Leo Laporte [00:12:18]:
They don't want people to use this. They're trying to keep people out.

Steve Gibson [00:12:20]:
No, there, there's a, uh, one of our listeners found a, one of the news reports where they did an an animated recreation of the accident. There was a Mercedes being driven by a younger man, and I read the news report. I don't remember the ages, but he was like 27 or something. And so now there you can see a picture, and notice underneath the front, Leo, are white lights.

Leo Laporte [00:12:50]:
Yeah, there was—

Steve Gibson [00:12:52]:
that's the Mercedes. Oh God, so not good. The Mercedes was driving to the left of the bus when the bus driver lost control, turned to the left, knocked the Mercedes off the road. It preceded the bus into the river, flipped upside down, and the bus rolled over it.

Leo Laporte [00:13:14]:
Oh.

Steve Gibson [00:13:14]:
So the bus— so Mercedes formed a, a brief, uh, stone in the middle of the bridge. That allowed the bus—

Leo Laporte [00:13:23]:
No deaths, thank goodness, but 6 people were injured. Here's another picture of the scene.

Steve Gibson [00:13:28]:
Yeah. Wow.

Leo Laporte [00:13:31]:
Holy cow.

Steve Gibson [00:13:32]:
The Lana River. So I guess—

Leo Laporte [00:13:35]:
I would have said Photoshop for sure.

Steve Gibson [00:13:38]:
Yeah, it's nuts. It's nuts. I mean, you needed another car there for the bus to drive over the car in order to get to the far side. And then they pulled the car out from underneath.

Leo Laporte [00:13:51]:
Crazy.

Steve Gibson [00:13:51]:
So, wow.

Leo Laporte [00:13:52]:
Well, I'm glad the driver survived.

Steve Gibson [00:13:54]:
Anyway, thank you, one of our listeners who sent this to me and, uh, and thought maybe this would be a good picture of the week.

Leo Laporte [00:14:00]:
And I agree. Okay.

Steve Gibson [00:14:02]:
So, uh, as I promised last week, uh, there are 2 very important pieces of core AI technology, uh, that I wanted to discuss. And I also promised last week that we were going to do a deep dive into aspects of the operation of today's AI. Well, that turned out to be more true than I expected, so much so that it entirely crowded out the second of the two topics that I had planned to get to. But fear not, we're going to have another deep dive into that next second topic Next week, and that's the role confusion topic, Leo, which you and I talked about at Black Hat, the research paper that I read during the flight there. And I said, OMG, how, how, wow, can that still be the way things are being done today? And I've confirmed, yes, unfortunately it is.

Leo Laporte [00:15:05]:
Not as weird as a bus crossing the river, but it's close. It's close.

Steve Gibson [00:15:10]:
It's up there. Okay. We're going to start by covering some important recent security events and then get into understanding this first of the 2 core issues of the way AI works. Last Wednesday, uh, Ars Technica's great security topic, uh, writer Dan Goodin reported under the headline, Terabytes of Credentials Leaked in Massive Supply Chain Attack. Was Uh, ours headline. Uh, and of course, that was the kind of thing I would have chosen to share even a few years ago. But the thing that raised my interest another several notches was the article's tagline, which read, that data, that is the terabytes of credentials, was scraped and exfiltrated from 2,500 users of a compromised AI package. So I thought, whoa, okay.

Steve Gibson [00:16:09]:
It turns out what's even more significant is we're not talking some random end users in Nebraska, you know, who no one knows. Wait till you hear whose credentials were among the more than 2,500 that were stolen. So Dan writes, terabytes worth of credentials, many belonging to the world's biggest and most sensitive organizations have been exposed in a supply chain attack on LightLLM, an open-source tool that streamlines AI-driven software development. Microsoft, Amazon, Cisco, Samsung, and Salesforce are only a handful, he writes, of the entities whose access secrets were exposed. The revelation was posted on Tuesday and Wednesday, that's last week, by security firms CloudSec, you know, S-E-C, and Hudson Rock. CloudSec said it found keys, repository tokens, SSH keys, Kubernetes secrets, package publishing credentials, environment variables, and AI provider keys. That could allow attackers to gain access to more than 2,500 organizations. And 40 minutes is all it took.

Steve Gibson [00:17:35]:
The credentials were extracted during a 40-minute window in March while the victims used, obviously unknowingly, compromised versions of LiteLLM, which had been downloaded from the package's official location in the Python Package Index repository, you know, PyPI. Hudson Rock said it made the discovery after analyzing a 195-terabyte file that it had obtained. Neither firm identified the source of the information. The LightLLM compromise was the result of a— get this, Leo— a previous— this will ring some bells— a previous supply chain attack that infected the widely used vulnerability scanner Trivy. And remember that we had talked about this months before. Other software infected in the campaign includes KICS and the Telnyx Python SDK. Team PCP, which Dan describes as a ramshackle but extremely capable gang largely made up of teenagers took credit for the attack, and researchers have largely corroborated their claim. Independent security researcher Kevin Beaumont said, quote, I've confirmed the data is legit.

Steve Gibson [00:19:04]:
By the way, multiple victim orgs. He said it contains a significant volume of sensitive content at orgs. It's a massive supply chain breach due to poor AI security, not because AI is the threat, but teens can now run circles around orgs obsessed with rushing out AI and poor DevOps security. And I'll just explain that a little bit here. I'll take a moment. LightLLM was compromised. It's not that AI was used in the compromise. It's that, that, you know, Kevin is saying that the way LightLLM is used, the way it needs to be used, that is to be a proxy for other AI services, means that you need to give it all your secrets so it can act on your behalf.

Steve Gibson [00:20:05]:
We've talked about this fundamental problem previously, and a lot of orgs just got bit by it. Anyway, I'll have more to say here in a second. So continuing, Dan writes, the compromised versions of all 4 software packages contained, um, uh, right, 4, 4 packages, uh, compromised by, by Trivy contained code that accessed the memory of infected machines, scraped its contents, and exfiltrated it through an attacker-controlled channel. The data is filled with an assortment of information. And again, 195 terabytes. So it's like a wealth, but you got to find the goodies in there. Interspersed in the wall of data are credentials to software pipelines maintained by tens of thousands of organizations that ran LightLLM during the 40-minute span that the supply chain attack remained active. In all, both security firms said some 434,000 CI/CD, you know, continuous integration, continuous delivery software pipelines had credentials exposed After running the compromised LiteLLM versions, there were 2 versions that were compromised.

Steve Gibson [00:21:31]:
I'll clarify that in a second. In many cases, the researchers at CloudSec and Hudson Rock had trouble identifying, which is a problem, the organizations the credentials belonged to. For instance, an email address in the dump from the domain at SiriusXM.com ultimately did not indicate a breach at the satellite broadcaster, but rather within the infrastructure of SiriusXM's subsidiary, AdsWiz. A trove of internal corporate secrets were found exposing sensitive tokens for platforms such as Salesforce_client_secret. and Slack, slack_signing_secret, and Microsoft Azure environments. The researchers had high confidence that the following organizations did have their credentials exposed. Get this. NVIDIA, AWS, Samsung, Salesforce, Cisco, Hoffman LaRoche, ServiceNow, Siemens, S&P Global, Airbus US Space and Defense, John Deere, Regeneron Pharmaceuticals, London Stock Exchange Group, Thomson Reuters, FedEx, MediaTek Inc., Volkswagen AG, Deloitte, the Kroger Company, Siemens Energy, Thales Group, X Corp, as in Twitter, Zscaler, Epic Games, Orange SA, HP Inc., Philips, Vodafone Group, Carl Zeiss, Deutsche Bahn, NGINX, BT Group, and Roku.

Steve Gibson [00:23:22]:
I mean, wow. Many CI/CD pipelines are configured generically. The dumped variables contain active database passwords, third-party API keys, and cloud credentials without any identifiable company email, custom domain string, or internal server name. This means that countless organizations which they were unable to identify currently, as in still, have active secrets sitting in this database. They are completely unaware of their exposure. The research firms, I should note, Immediately identified all the companies that they could among, you know, among those I just read. But lots of other companies, they're like, what, 434,000? Was it different CI/CD pipelines? You know, all their secrets are out there and they haven't been notified because it's not clear who they are. So maybe that's safety.

Steve Gibson [00:24:26]:
You know, I mean, the bad guys probably can't tell either. But it certainly gives you some starting credentials to use for some password guessing. Finally, under the heading, Welcome to the New World of Supply Chain Attacks, Dan adds, both firms, those 2 security firms, are urging all organizations that use the compromised versions of LiteLLM, particularly those listed in the high confidence section of the list, Well, that— no, the organizations that are known to thoroughly rotate all credentials in their pipelines. Hudson Rock instructed any organization that uses any AI proxy infrastructure, third-party CI/CD vulnerability scanners, or downstream AI packages to immediately audit their environment for versions 1.82.7 and 1.82.8 of LightLLM, which were the 2 compromised versions of the software. The firm advised those affected to perform, quote, aggressive credential revocation, unquote. Assume any secret accessible to the LightLLM environment is compromised. Invalidate and rotate all cloud keys, Kubernetes service account tokens, the GitLab GitHub PATs, and audit logging and egress filtering. As a cautionary tale, CloudSec said that Trivy developers rotated but failed to fully revoke an automation token over a 20-day window.

Steve Gibson [00:26:11]:
That lapse gave the attackers a nearly 3-week period in which to force-push malicious code to third-party builds that use the vulnerability scanner. As Beaumont observed, organizations' rush to integrate AI into their software delivery systems has also greatly contributed to the scale of the damage, meaning just, you know, as we've seen. And Leo, remember when you were, you know, immediately thought, hey, This, I can't remember the name of it. It was the package that came out at the beginning of the year that was the code writing. Claw.

Leo Laporte [00:26:55]:
Claude? What? No, no. Claw.

Steve Gibson [00:26:59]:
Open Claw.

Leo Laporte [00:26:59]:
Oh, ClawBot.

Steve Gibson [00:27:00]:
Yeah. Yeah.

Leo Laporte [00:27:02]:
Open Claw. Yeah.

Steve Gibson [00:27:03]:
So Open Claw happened. You were excited about it, but you pulled back at the last minute.

Leo Laporte [00:27:11]:
And it turned out that was a—

Steve Gibson [00:27:12]:
yeah, there was just— You had to tell it too much in order to allow it to do what you wanted it to do.

Leo Laporte [00:27:20]:
But yeah, if you wanted it to do anything, you gave it your email, you gave it money, you gave it a phone number.

Steve Gibson [00:27:26]:
Exactly.

Leo Laporte [00:27:27]:
A little dangerous.

Steve Gibson [00:27:29]:
So then, in a— okay, in a final update to his initial reporting of this massive mess, Dan added, there are already signs that some of the affected organizations are not taking the disclosure with the seriousness warranted. After this post went live, he writes, Kevin Beaumont reported, quote, these creds date from about March. One of the orgs impacted told me, he writes, they'd rotated them all and it's a nothing burger. He said, so I looked at their responsible disclosure policy. It allows trying creds. So I tried them all. Almost every one of them worked. He said, I submitted a report, one of the biggest U.S.

Steve Gibson [00:28:21]:
telcos. So someone said, oh yeah, we don't worry about it. We rotated our credentials. Nothing to see here.

Leo Laporte [00:28:29]:
So— They probably made new ones but didn't delete the old ones is what they did. Jeez.

Steve Gibson [00:28:34]:
So ultimately, um, the new revelations concerning the LightLLM supply chain attack underscore— is writing Dan— the growing threat of such campaigns, and hence the importance of maintaining vigilance around the use of open-source software that, when infected, can spread rapidly across the internet. Uh, Alon Gall, co-founder and chief technology officer of Hudson Rock, wrote in an email, quote, the key takeaway is how supply chains have evolved to make a single upstream breach affect thousands of companies simultaneously. A window of roughly 40 minutes in which the LightLLM dependency was hacked led to over 430,000 instances in which millions of secrets were harvested. This magnitude pushes us into a completely new world regarding the type of response required from the cybersecurity industry. And Lord knows, you know, we've been unimpressed typically by the kind of responses that we've seen historically. So, so just to be clear that the cautionary takeaway from this is not a case, as I said before, of AI going rogue or any kind of AI misuse. You know, I followed Dan's reference links back to one of Hudson Rock's reports, whose much more technical write-up of the breach makes what happened actually further clear. Their headline, Hudson Rock's headline, was Largest AI Supply Chain Breach of 2026: Light LLM Hack Impacts Thousands of Global Enterprises.

Steve Gibson [00:30:31]:
And Hudson Rock explains quickly, the cybersecurity landscape is currently reeling from one of the most sophisticated multi-ecosystem supply chain campaigns publicly documented to date. The orchestrated— it's a— I'm sorry, orchestrated by a threat actor group known as Team PCP. This cascading attack ultimately compromised LightLLM, a widely adopted open-source AI proxy gateway, leading to the silent exfiltration of deep developmental secrets from thousands of continuous integration and continuous deployment pipelines worldwide. Excellent forensic research published by Snyk, you know, that's S-N-Y-K, Trend Micro, and, and, uh, Psycode has thoroughly detailed the mechanics of this breach. The attack did not begin with LiteLLM. Instead, Team PCP first compromised the GitHub Actions pipeline for Trivy, a highly popular open-source vulnerability scanner. Because the developers of LiteLLM utilized Trivy in their own CI/CD pipeline, the poisoned security scanner was granted legitimate read access to their runner environment. This allowed the attackers to silently exfiltrate LightLLM's PyPI publishing tokens.

Steve Gibson [00:32:16]:
Armed with these credentials, Team PCP was able to publish malicious versions of the LightLLM package, versions 1.82.7 and 1.82.8. The payload delivery was exceptionally stealthy by utilizing a .pth Python startup hook. The malicious code was executed the moment the Python interpreter initialized, regardless of whether the LiteLLM library was explicitly imported. The 3-stage payload immediately began harvesting environment variables, local configuration files, like .kube/config and .aws/credentials, attempted lateral movement across Kubernetes clusters, and installed a persistent systemd backdoor. While the security community has deeply analyzed the malware's behavior, Hudson Rock has independently obtained the actual fallout, the raw Exfiltrated data. That's that 182 terabytes of data. I mean, talk about a huge amount of data in 40 minutes. That, I mean, because there were that many instances of those 2 versions of the malicious LiteLLM that were updated from Python Pi and installed and started and all the credentials poured through it and they sent them all off.

Steve Gibson [00:33:58]:
to some malware mothership somewhere. So they said this provides an unfiltered look into the massive scale of the compromise through the actual raw files. Our researchers have obtained and analyzed a staggering 153-gigabyte RAR archive. You know, we've— and these files will compress way down. This massive corpus contains exactly 433,909 files. Through our analysis, we've successfully attributed 118,829 CI runner dumps to 2,488 affected corporate domains. Whether a developer, machine, production server, or CI/CD pipeline executed the compromised LightLLM package, the threat actors successfully harvested the live environment memory and configurations mid-execution. Okay, so the real takeaway from this is that the massive adoption of the automation of development and delivery will tend to coalesce around relatively few most popular best-of-class tools.

Steve Gibson [00:35:30]:
This naturally makes those tools a highly valuable target for attackers. Over at GitHub, the LiteLLM repository describes itself Writing, LightLLM is an open-source AI gateway that gives you a single unified interface to call more than 100 LLM providers, OpenAI, Anthropic, Gemini, Bedrock, Azure, and more, all using the OpenAI format. Use it as a Python SDK, for direct library integration or deploy the AI Gateway, a proxy server, as a centralized service for your team or organization. Managing LLM calls across providers, you know, multiple, like, you know, Anthropic, OpenAI, Gemini, so forth, multiple providers, they say managing those calls gets complicated fast. Different SDKs, different auth patterns, request formats, and error types for every model. LightLLM removes that friction with a unified API, one interface for more than 100 LLMs, no provider-specific SDK juggling, drop-in OpenAI compatibility, swap providers without rewriting your code, production-ready gateway, virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard out of the box. 8 milliseconds P95 latency at 1K RPS, you know, requests per second in benchmarks. So, wow, right? Sounds great.

Steve Gibson [00:37:21]:
The only glitch here is that it must be totally trustworthy. In order for LightLLM to be able to proxy for its users, all of those differing LLM backends, it must necessarily have every user's authentication credentials for every one of those different LLM backends. Just like Leo, you were saying OpenClaw, you know, had to be able to log in as you, had to be able to access your bank records, had to be able to make payments on your behalf, blah, blah, blah, blah, blah. I mean, We, when we talk, when we get into proxies and agents, where trust has to be there because, you know, they're acting on our behalf. So what happened here is that the LiteLLM developers were users of the Trivy vulnerability scanner. And as we know, because we talked about this back in March when this happened, Trivy was compromised. This allowed attackers to compromise any project that was using Trivy as its scanner. And thus, in turn, those 2 versions of LiteLLM were compromised, giving attackers complete visibility into the credentials and domains of those 2,488 corporate users of LiteLLM during just that 40-minute window.

Steve Gibson [00:38:50]:
And, you know, I always like to try to suggest solutions. To the problems we encounter here, but I got nothing because this is, this is like a fundamental problem with the way we're doing things now. You know, we're now in a mode where everyone feels that they need to always be running the latest and greatest release of everything, right? It was, it was because of the updates to those bad, those 2 bad LiteLLM packages in the repository that, that this happened because there was a new version. Oh, gotta have that. So, you know, we've done this to ourselves. And, but I, I preach updating relentlessly, right? You know, the entire security community is constantly pressing everyone to get better About updating their software. Stay current. Be sure you're receiving announcements of important updates, blah, blah, blah.

Steve Gibson [00:39:54]:
You hear it here all the time. But in this instance, doing that is precisely what bit the users of those 2 specific versions of LightLLM. If they had not updated to those, if they'd remained on a previous release, they would have never run one of those 2 malicious versions. But of course, not updating doesn't work as a strategy either.

Leo Laporte [00:40:17]:
No, I pin stuff, uh, trying to avoid this. After this LLM, uh, debacle, um, you know, people said, you know, I was caught within a day, I think, or a couple of days. They said, if you just make sure you don't install anything that's less than 3 days old, you'll be all right. I made it 14 days because I figured, I mean, you're still not going to catch everything, but if it's a popular package, 2 weeks should be enough. So I'm very careful. The other thing I do though is I store all those tokens and secrets in Bitwarden Secret Manager. So they're not, it's like my passwords, right? They're in a locked vault.

Steve Gibson [00:40:54]:
And we did just talk about this a couple of weeks ago where, uh, where we're beginning to see a new category of security software, which is a means of allowing AI to, to access your credentials without it ever having them.

Leo Laporte [00:41:12]:
Right.

Steve Gibson [00:41:13]:
That is so, so, so it says to Bitwarden, I need you to log in for me.

Leo Laporte [00:41:19]:
In effect. Yeah. Well, or it gets a one-time token or, yeah. I mean, that the, it's still going to see the problem is, so you're using that OpenAI endpoint, which is connecting to somebody who's serving that AI. They want that token. So the token has to float around somewhere in memory. You don't want to write it to the hard drive, but it has to— LightLM would still have to be able to see it and send it. So I'm not sure a secrets manager would have protected me in that case.

Leo Laporte [00:41:46]:
I, yeah, you try to do what you can.

Steve Gibson [00:41:49]:
So the, the flip side, of course, of your 2-week window, which is good, is that if there was a critical vulnerability that actually was authentically, authentically patched, then I wouldn't get— you wouldn't be getting it for 2 weeks. So, right, so, you know, who knows what might, might be malicious and what isn't. We are— we're, we're in a bad place right now. The only winning strategy, or at least the best strategy that's available for the moment, is just to use the tools but carefully monitor the news for the tools you're using. Stay current.

Leo Laporte [00:42:27]:
Yes.

Steve Gibson [00:42:28]:
But You know, because mistakes are going to happen, that the instant you learn of a breach that affects you, invalidate and recreate— in other words, rotate— all possibly affected credentials. We've seen many instances where that was not done. You know, remember that part of what made LastPass's troubles even greater was that even after they learned that they'd been breached, they failed To fully cancel all previous authentication credentials. And, you know, and as a perfect case in point, we learned that, you know, from Kevin Beaumont's test, a major telco did not actually rotate their credentials when they had claimed to. So I guess there actually is an important and practical takeaway from this. I mean, it's like a real action item. We know—

Leo Laporte [00:43:25]:
Go ahead.

Steve Gibson [00:43:26]:
We know that, that things that are easily done tend to be done, and things that are a confusing pain in the butt too often fall into the, okay, I'll get back to that later. What we see is that the speed of modern attackers means that later Stands a very good chance of being too late. So here's my, my takeaway point from this. I think it's really crucial. If there's really no practical means of preventing inadvertent exposure to credential-leaking malware, and there may not be today, then rotating all of the credentials that any malware might have obtained the moment a credential-compromising attack is known is important.

Leo Laporte [00:44:26]:
Of course.

Steve Gibson [00:44:27]:
Moreover, periodically rotating credentials preemptively on the better safe than sorry principle can be useful when the threat environment warrants it.

Leo Laporte [00:44:38]:
But this is, by the way, this is contrary to the, uh, advice we've been giving about passwords, rotating passwords, but it is what you should do, which is rotate.

Steve Gibson [00:44:49]:
Yes.

Leo Laporte [00:44:50]:
So when I make keys now, I give them an expiration date of a month or 2 months or 3 months, and I know I'm gonna have to rotate them 'cause I don't get to keep using them. Right.

Steve Gibson [00:45:01]:
So what I believe this means is that whole organization, organization-wide credential rotation needs to be both possible and easy.

Leo Laporte [00:45:15]:
Right.

Steve Gibson [00:45:15]:
If it's not, if it's not easy, it won't happen or it will be put off. So my best advice would be, if you're looking for a nice, self-contained, easy-to-describe project for an AI agent to tackle, a great investment would be to employ some AI to implement a comprehensive credential rotation facility for your organization.

Leo Laporte [00:45:45]:
Yeah.

Steve Gibson [00:45:45]:
Make it automatic. Make it a single command that handles everything. Make it easy, even fun to use, and use it to maintain the credentials for both current and new services as they're being brought on board, you know, as services are added and removed. And require its use for instantiating any new credential into use so that it cannot fall out of sync. Or again, if that happened, it wouldn't be trusted and used. So I would say making, you know, automating credential rotation would be a cool— it's easy to describe to an AI. It's self-contained. You can see if it's working or not.

Steve Gibson [00:46:33]:
Failure doesn't kill you. It just, you know, you got to fix it. But I think it would be then incredibly useful if, if, if, well, when thinks Canary finds that some guy is in your network, the first thing you want to do is, is, you know, you know, don't panic. As, as the Hitchhiker's Guide to the Galaxy tells us, rotate those credentials. And, you know, uh, and, and, you know, make, make it easy, make it fun.

Leo Laporte [00:47:05]:
There are, uh, there is a way to do this a little safer, and I think if I were a big— or any, any business, I probably would be doing this. Uh, when we were at RSEC, I met a few of these guys. They, they're an intermediary, a credential capability layer. So they hold— they're a third party, they hold their credentials You don't ever have the credentials on your machine. You have a credential to them. And when you want to connect to an AI, you connect through them. So they have your credentials and they do it and they— you only get a one-time token. Uh, the problem and the reason I didn't want to do that, A, it's you pay for it, but, but B, they have your credentials.

Leo Laporte [00:47:42]:
And I, you know, it has to be somebody you trust because they're going to have your credentials.

Steve Gibson [00:47:47]:
Right.

Leo Laporte [00:47:47]:
Uh, you know, it has— at some point these credentials have to be exchanged. It's just like a password. I guess you could use hashing, couldn't you? Well, that would be the solution.

Steve Gibson [00:47:57]:
We, a couple of weeks ago, we talked about 1Password saying that they were introducing exactly this service. So keeping it local. And of course, the problem is that we're talking all credentials. So like that online service would be handling your credentials as a proxy for AI. But what about SSH servers? What about web, you know, all the other things So what you want is one thing that is just able to wipe the secrets out of your organization and replace them.

Leo Laporte [00:48:33]:
Yeah, that's the one password credential broker. That might really be the right way to do that.

Steve Gibson [00:48:39]:
We're going to have to have something like that. Well, you know what we're going to have to have right now, Leo?

Leo Laporte [00:48:43]:
A break in the action? You're watching Security Now, Steve Gibson, The man of the hour, the man— every Tuesday it's Security Now Day here at, uh, the TWiT Podcast Network. We're glad you're here with us. Now back to Mr. Gibson.

Steve Gibson [00:49:02]:
Uh, France's recently celebrated, uh, ban on social media access for all children younger than 15 hit a bit of a snag last Friday. Reuters reported the following. They said France's top court on Friday blocked a bill banning social media access for under-15s, saying it infringed upon freedom of expression and delivering a setback for President Emmanuel Macron, who asked his government to rewrite the legislation. The bill would have barred children younger than 15 from opening a social media account from September— beginning September 1st. And all accounts already open would be closed by the year end, which would also need to use age verification provided by the French privacy regulator. But Reuters writes, France's Constitutional Council found that the bill, while requiring everyone to give proof of age, failed to, quote, specify the conditions and limits under which it should be provided, as well as infringing upon freedoms and privacy. So that's their report. Uh, you know, as we immediately understood when this began to happen in the U.S., you know, in the context of, of U.S.

Steve Gibson [00:50:27]:
domestic, uh, legislation to control the, the viewing of pornography online, we've noted that blocking anything for all users below a certain age inherently requires everyone's age to be known. You know, there's no way around that. So France now needs to tackle the thorny problem of that inescapable infringement upon the internet's illusion of total freedom and privacy. Um, you know, uh, I— it is an illusion to some degree. Uh, because we know how the internet's technology works, you know, like from the beginning. There's never been a greater infringement upon privacy than the abuse of third-party browser cookies to track people, but that went largely unseen, so nobody really worried about it. Um, uh, the problem with age assertion is that because it's explicit and it cannot be hidden in the same way that cookies were, everyone's getting outraged. You know, the truth is that if we want our governments to restrict children's access to internet content, then everyone's age must be known by someone somewhere, either by every source of the proscribed content or by every means of accessing that content.

Steve Gibson [00:51:57]:
So anyway, they, uh, Reuters continues quoting the Constitutional Council statement, writing, the, quote, the council holds that the contested provisions on the one hand disproportionately infringe upon the freedom of expression and communication, and on the other fail to provide the legal safeguards necessary to ensure the right to respect for private life. French lawmakers, they wrote, had approved the bill in July, which is when we first talked about it last month or month before last, uh, becoming the first in Europe to follow Australia, whose world-first ban barred access to platforms including Facebook, Snapchat, TikTok, and YouTube for under-16s in December. Lawmakers there are considering stricter penalties after data showed mixed success. Countries around the globe, including China, the UAE, and Turkey, have either instituted measures intended to curtail or bar access to social media for young people or have said they're planning them. The European Union has said it was planning to seek stronger protections for children from harmful social media features. Social media companies generally oppose blanket bans, saying they have measures already in place to protect younger users, including age restrictions, though they have also said they would comply with government bans. Google, Meta, Snap, and TikTok did not reply to Reuters' response or requests for comment. Macron, who in April urged teenagers to turn off their devices and read— that went over really well— in order to become better citizens, has ordered Prime Minister Sébastien uh, Laicorneu to rework the draft legislation to make the Constitutional Council's concerns, uh, to take them into account.

Steve Gibson [00:53:56]:
Um, they said in a statement, uh, that they were, uh, determined for the reform to take effect before the spring of 2027 when they hold— when France holds its presidential election. So anyway, Macron has not given up. The draft legislation is being hastily reworked to address the council's concerns. Since Macron still hopes to have this reform in effect as soon as possible. And he was also going to add smartphone restriction to the legislation that would take effect for high school in addition to the lower schools. So anyway, we will see what's going on and what happens, Leo.

Leo Laporte [00:54:39]:
Wow.

Steve Gibson [00:54:40]:
Humph. Bah humbug. So, um, last Tuesday Wired reported on the unnerving discovery of a serious vulnerability in the Zoom teleconferencing system. And of course now that's like recent, right? Remember, um, we'll all remember when Zoom really became a big deal at the beginning of COVID because, you know, it saw its adoption soar as teleconferencing became super important. It was the only way to continue doing business if you were stuck at home. And boy, Zoom had a bunch of early problems. Uh, they weren't all resolved, as it turns out. Wired's headline reads, a Zoom screen-sharing bug let anyone take over other devices on a call.

Steve Gibson [00:55:35]:
And their brief teaser is what makes this so interesting. They said researchers say it took fewer than 20 prompts for a public AI tool to find a flaw, which has now been fixed, allowing anyone on a Zoom call to hijack other participants' devices. And what, what wasn't clear from the reporting, and I didn't dig deep into it, was, wait a minute, a public AI tool was used to do some sort of clear cybersecurity work without hitting guardrails?

Leo Laporte [00:56:14]:
Probably a Chinese model.

Steve Gibson [00:56:17]:
Uh, could be. Oh, good point, publicly available. Yes. So Wired said, as AI models gain advanced capabilities to find vulnerabilities in software, develop ways to exploit them, and even carry out autonomous hacking sprees, all of which we've been seeing. Researchers offered a sobering new example on Tuesday, disclosing vulnerabilities in the video conferencing platform Zoom that could have been exploited to take over targets' devices. Anyone on a call that involves screen sharing, whether participants or the host, would have been vulnerable to a silent attack that could be carried out with no indication and no interaction from the victim. Researchers from the digital defense firm A Security, just the numeral A Security, say, or the letter A Security, say the bug was discovered in early June using publicly available AI models and that it took fewer than 20 prompts to uncover the vulnerabilities and create a working attack. Zoom issued a security advisory on Tuesday, including details about fixing— about the fixes the company has already begun rolling out to address the flaws, which affected devices running across all operating systems that Zoom supports— Windows, Mac, Linux, iOS, and Android.

Steve Gibson [00:57:46]:
A security— the A Security co-founder, the company A Security co-founder, Omar Gull told Wired ahead of the disclosure, quote, what's interesting for us and what we believe is dangerous is the democratization of these capabilities. The barrier to entry is dropping rapidly. Before, it would have taken a team of 5 people maybe 6 months With a lot of refining and iteration to find this. Now people can reach the same results with fewer than 20 prompts. And Zoom is an important type of target because people assume trust when using it. They don't see it as a threat. His quote ends. The vulnerabilities, writes Wired, were found in the protocol used to facilitate real-time annotation during screen sharing.

Steve Gibson [00:58:49]:
The researchers say that their AI bug hunting systems specifically delved into this component because, like human bug hunters, they've been trained that convoluted and obscure functions often contain overlooked vulnerabilities. This is particularly true with proprietary closed-source software. An established company like Zoom presumably does extensive code review and vetting on all components and functions. But without the benefit of public open review, esoteric yet complex features like annotation are more likely to contain mistakes. Zoom did not respond to multiple requests for comment from Wired about the A security findings. The bugs are now patched, with Zoom issuing both server and client-side fixes or patches for both Zoom's own servers and the applications that run on customer devices. But the researchers emphasize that it was alarming to contemplate bugs that could have been exploited to take over a target device simply by getting someone on a Zoom call. Joining a call is itself a gesture of trust, but given how ubiquitous video calling is in both personal and professional contexts, and given that Zoom in particular is also widely used for events and semi-public activities like webinars, people typically have their guard down when joining a Zoom.

Steve Gibson [01:00:25]:
A security co-founder, Yossi Torati, told Wired on a call If you just get on a Zoom with us, we can take over your device. The worst-case scenario is that we can take over an enterprise just by having this capability in our hands. If I'm an attacker, I can be on a call with someone from a company, take control of their computer and their credentials, and then use them to move laterally across the enterprise. Practitioners often call security a cat-and-mouse game, but as AI bug hunting proliferates, this delicate dance has become an all-out race, which of course is exactly what we've been seeing. Um, all indications are that the high-tech computer world at every level, from IoT embedded device to consumer PC and enterprise, Probably is heading for a rough patch. So far, I've, as we know, I've been a cheerleader for the fixing the bugs team, you know, and the good news is, you know, indeed been that an astounding number of bugs are rapidly being found and fixed. You know, in those last 2 versions of Chrome, more than 1,000 total between those 2, 149 and 150. But that's also the bad news.

Steve Gibson [01:01:48]:
Because the fact that so many latent problems are being found tells us that the software at every level that we've been living with for years has been demonstrably riddled with previously unknown flaws. So the best that can be said is that the future remains stubbornly uncertain. Uh, we're getting the bugs out of the software. And, you know, my feeling is, absent a state actor having some reason to attack another country, the money is still what drives the bad guys. Now that we've got cryptocurrency, now that we've got the ability to exfiltrate and extort, money is the motive. And so there's not money behind a mass casualty event. There's money behind selectively targeting, exfiltrating, blackmailing, extorting, and, and getting what cash you can. So, so I, I don't, I don't think we're gonna see a big Y2K-style I mean, like, or, or the, the, yeah, a Y2K apocalyptic sort of thing.

Steve Gibson [01:03:10]:
To me, that doesn't make sense because it doesn't make money. And that's what I mean, that's kind of a saving grace. It means that there will be people hurt, but they're, you know, their insurance is going to go up and they're going to be paying out of pocket for bad guys having gotten into their system using bugs that are probably latent and aren't their fault and are zero days. So there's really nothing they could do about it. So hopefully that's what we see, and that over time the ability to do that dries up because we get our software fixed. Okay, uh, one last piece before we get into our big topic. Um, there are some folks who are, I would argue, justifiably profiting from all of this. Uh, bad guys not justifiably profiting, but good guys.

Steve Gibson [01:04:02]:
Last Monday Following the Black Hat conference, CNBC reported, they said cybersecurity stocks of CrowdStrike and Palo Alto Networks jumped more than 5% to new highs on Monday on renewed demand for artificial intelligence security tools following the industry's annual Black Hat conference in Las Vegas. In other words, those guys were there, they were showing off that their AI is going to be used now, that they're ready to deploy defensive AI solutions, and the world said, we need some more of that. Analysts at BTIG, which is a large financial services firm, wrote to their clients in an internal firm letter Quote, the single most consistent theme across our conversations, partners, vendors, and customers alike, was that AI agents have fundamentally changed the threat landscape. While AI agents have become the predominant attack threat and the environment is meaningfully worse, deployment and, deployment and AI security tools are only in the early innings. CNBC said businesses are turning to cybersecurity companies for new agentic tools to fend off adversaries in a hyper-accelerated threat landscape fueled by new cyber models. Executives and potential customers gathered in Las Vegas last week in search of answers and ways to secure systems from rogue AI agents. BTIG wrote, quote, we think AI is creating a new modernization cycle in the endpoint security space, which directly benefits CrowdStrike's core business. Analysts at Cantor said, quote, AI has moved from being a cybersecurity feature to a key pillar of both the attack surface and the defender infrastructure.

Steve Gibson [01:06:25]:
So I remember the first time we touched on this. I think it was one of our— it was, it was one of our listeners who was commenting that his company was already using some AI-based systems. I think he was He was on AWS cloud and he was using a third party's AI-based systems. And I was— this was a couple of weeks ago. And I was like, wow, this is happening already. I mean, we've got AI on the, you know, being deployed for defensive purposes, which is wonderful. And, you know, Leo, we're going to now dig into what I have learned recently about AI and can share. But first, I think we need to hear—

Leo Laporte [01:07:18]:
To share a fine sponsor, perhaps?

Steve Gibson [01:07:21]:
I think that'd be perfect.

Leo Laporte [01:07:23]:
I think I can do that. Our show today, oh, I'm excited about all of this. Yeah, I have a beverage. Looks like Tang. Are you drinking Tang? Oh, that's right. Dilute orange juice. Yes.

Steve Gibson [01:07:35]:
Exactly.

Leo Laporte [01:07:36]:
Yes. Do you add vitamin C to your diluted orange juice?

Steve Gibson [01:07:41]:
No?

Leo Laporte [01:07:42]:
Okay. No, I'm just curious.

Steve Gibson [01:07:44]:
I take 15 grams of C a day. So way more than you can—

Leo Laporte [01:07:47]:
You get plenty. I know. I know. 15 grams.

Steve Gibson [01:07:51]:
Yep. 3 divided doses of 5 grams each.

Leo Laporte [01:07:53]:
That's half an ounce. You're crazy. You're a madman.

Steve Gibson [01:07:59]:
Are you sure?

Leo Laporte [01:07:59]:
At least you— well, I don't know. Whatever that means. I, I don't know what vitamin C is doing for you. You don't get any sore throats, I guess.

Steve Gibson [01:08:05]:
My, my liver would like to be generating about 20 grams a day and can't because it's got, uh, the human genome has a little glitch. So, right, we've talked about that.

Leo Laporte [01:08:15]:
Yeah, yeah, damn genome. But now back to Steve.

Steve Gibson [01:08:21]:
So, okay, as I promised last week, there are 2 important and fascinating pieces of core AI technology I want to spend time on today. Uh, I got to one of them. Uh, the first is a solution to what's been dubbed the dual-use problem. That's the fancy name given to the fact that most knowledge can be used in ways that we both want and don't want. And of course, that's no surprise, right? Since that's always been true of knowledge. So what is it about AI that changes this? Anyone who's spent any time with any of the recent state-of-the-art AI chatbots will have come to appreciate that what we already have today is, at the very least, an over-obliging conversational partner that can barely restrain itself from being oh so very helpful. And that tail-wagging puppy happens to also have access to the world's stored knowledge. So it's not that it was impossible before AI to obtain that knowledge the old-fashioned way, you know, by researching, reading, learning, and understanding.

Steve Gibson [01:09:42]:
No, the difference is that friction matters, and AI chatbots have hugely facilitated the access to that same knowledge by nearly eliminating all of the work that was previously required to gain such knowledge. And that's incredibly valuable. That's what underlies all of this frenzied hyperscaler data center buildout. Investors believe, with good reason, that offering knowledge at our fingertips by phrasing a question is something most of the world will pay for. The fact that AI chatbots now have just shy of 1 billion users strongly suggests that these investors are not wrong. I, I, for myself, I'm completely spoiled. Even though I've never turned any agent yet loose on anything, Claude is my go-to for quick answers. Uh, it's an accelerator.

Steve Gibson [01:10:42]:
for me. Um, and, you know, don't let anybody know, but I, at this point, I would probably pay pretty much anything for it.

Leo Laporte [01:10:50]:
Yeah, yeah, I know, I know how you feel. I know how you feel. Yeah, yeah, I have paid anything for it.

Steve Gibson [01:10:58]:
But, but Leo, you and I already know we're not going to have to because, as I mentioned, I think before the show, it turns out that, that, that Lenovo ThinkStation machine that I bought had a strong GPU. It's got a, it's got an NVIDIA RTX 2000 with 16 gig, and there are useful models now that can run in that.

Leo Laporte [01:11:22]:
Yep.

Steve Gibson [01:11:23]:
So, you know, but I'm, you know, still, you're always going to want the latest and greatest, and the best—

Leo Laporte [01:11:28]:
It won't be as good as Claude. I mean, that's the thing. And not yet. Next year. And then you could stop, then you'll be happy, right?

Steve Gibson [01:11:36]:
So, so, you know, by comparison, right, you know, anyone could download the Encyclopedia Britannica or the unabridged Oxford English Dictionary, but you can't ask them questions. Yeah, you know, it's all just dead knowledge lying there, so it's much less entertaining and it takes a lot longer, you know. So, you know, back to— and, and using them is back to old school researching, reading, learning, and understanding. And of course, you know, look behind me. Anybody who has seen a video of this podcast is aware that there's a solid wall of textbooks. And as it happens, up there, just out of camera, is an unabridged Oxford English Dictionary, 27 volumes. I cannot recall the last time I opened any of those books back there. You're right, AI neural networks are our new store of knowledge.

Steve Gibson [01:12:36]:
Incredible as it may seem, and it would have seemed like science fiction just a few years ago, um, the collection of just an array of scalar variables which specify the scaling weights of the inputs to a vast neural network creates a representation of the expression of all of the knowledge that has been trained into that model. Um, and I said that exactly the way I wanted to— creates a representation of the expression of all of the knowledge that's been trained into that model. I think it's important to frame it that way. What we're feeding into the neural network while it's being trained is the expression of the knowledge largely gleaned from the internet as well as from reference texts which AI companies have been quietly purchasing and ingesting, uh, to create an overall training corpus. The end result of this, and this is still mind-boggling to me, is purely and simply a next most likely token prediction engine. You know, Leo, your very— as, as I've said, your very early initial observation was that what we were calling AI, and this is a couple years ago, was little more than fancy spell check.

Leo Laporte [01:14:07]:
I called it spicy autocorrect.

Steve Gibson [01:14:10]:
Yes. Um, and then, as our listeners will remember, last week I, I, I quoted a nearly stunned Matthew Green, specifically saying that it's not just fancy spell check, but its essence has not changed. You know, you were not wrong, Leo, then, and Matthew is also not wrong today. So how do we explain this apparent disparity? It's that Over the past couple of years, what was initially a simple next-word predicting spell check has become really, really, really, really fancy spell check. Um, deep underneath even today's astonishingly apparently intelligent reasoning systems down at their core is still just a neural network that only does exactly one thing. Given a long, and in many cases astonishingly long, token sequence, it predicts the most likely next token. And the fact that we get what we now get from that Well, that's what's still mind-boggling to me. Um, I'm going to spend a bit more time on this because a deeper understanding of the truth of what's actually going on with today's AI, I think, will help everyone to appreciate next week's, the second of the 2 research papers I plan to share, which is— well, I, I don't know how to sum it up quickly.

Steve Gibson [01:16:03]:
So Stay tuned. So your original, Leo, you know, it's just spell check autocomplete summation that has its roots in AI circa 2020. Way back then, if you were to carefully phrase a question to GPT-3, such as the capital of France is, it would have been able to complete the sentence by emitting the next expected word, Paris, because the model— that model, the GPT-3 model's vast statistical data set, um, made, uh, made Paris the next most likely word. The capital of France is Paris. But if you used question-style phrasing back then, what is the capital of France, that would not have been met with the same success. So what happened? Obviously we have that now. How did we turn these models from autocomplete engines— that is, that's where that's all they could do— into conversationalists? So it took a few years of experimentation, but AI researchers first used something that's now known as instruction tuning. They took the knowledge-trained model and fine-tuned it on a— and this is again another surprise— a surprisingly small set of human-written examples.

Steve Gibson [01:17:48]:
Of what good responses would look like. After that, then something known as RLHF, reinforcement learning from human feedback, was used. So this applied preference rankings where, again, humans compared and ranked multiple outputs from best to worst. And that ranking was used to train a reward which pushed the model toward the better behavior. Now here's the astonishing part to me. Researchers then found that they could employ a comparatively small model of these query and ranked response samples which then created reward feedback, and that these large language models would and did with startling speed— they were able to generalize from the language patterns of queries and responses across their entire knowledge base. That is, They learned that pattern and generalized it. So to better appreciate the scale of this, a model that had been trained on trillions of tokens of raw knowledge, you know, this created the original statistical autocomplete engine, a neural network that just could probabilistically choose the next most likely token.

Steve Gibson [01:19:37]:
It could then be rewarded using only— again, it was originally fed trillions of tokens. It could be rewarded using only tens of thousands to low hundreds of thousands of query-response samples or examples And the behavior of the entire model would be reshaped across all of the knowledge that it contained. Um, this is a well-documented fact now that, that, that this occurs, um, with— and the fact that it occurs with such sample efficiency is what was utterly unexpected. I mean, this was the surprise. Today, now, as I said, it's a well-documented real phenomenon which has now been termed— there we have a term for it now. It's known as the superficial alignment hypothesis. The idea that all of the raw knowledge was already there from the model's initial knowledge corpus training. Then a relatively minuscule bit of fine-tuning teaches the model format and behavior, but not new knowledge.

Steve Gibson [01:21:01]:
That is, it wasn't giving it new knowledge. It was, was, it was reformatting it and giving it behavior for the first time. So to state that a bit differently for clarity, the breakthrough about 4 years ago was not the use of a larger model than we had at that time. That is one of the things that's been happening since then, but the breakthrough was the unexpected discovery that a comparatively infinitesimal dose of human-provided— here's what a good answer looks like, and here's which of these answers is better training would reshape a massive already trained model's entire behavior, turning it from something that completes patterns into something that acts like it's trying to be helpful. So once the massive model learned what helpful looked like, and that its trainers wanted it to look like that, all of its stored knowledge was immediately available in that new helpful format. And we got helpful chatty AI. That's how it happened. Um, and as we know, those first steps made by ChatGPT were more than a little shaky.

Steve Gibson [01:22:36]:
You know, uh, you know, researchers realized that their new— their newly birthed chatbot would need some additional post-training alignment, as it's now being called. And this began as that RLHF, the reinforcement learning from human feedback that we talked about. Uh, but then a year later in 2023, a handful of AI researchers at Stanford University published a paper titled Direct Preference Optimization. The paper's title is Direct Preference Optimization: Your Language Model Is Secretly a Reward Model. And since all of the descendants of this— it's now known as DPO system. Direct Preference Optimization was sort of the granddaddy. Now we have further refinements of that, which have occurred in the last 3 years, something known as IPO. There's KTO, ORPO, and SIMPO.

Steve Gibson [01:23:40]:
They all descend from DPO. I want to share just the abstract of the Stanford researchers' original paper, which was the breakthrough beyond that earlier reinforcement learning from human feedback. So they explain their invention by writing, while large-scale unsupervised language models learn broad world knowledge and some reasoning skills, achieving precise control of their behavior is difficult due to the completely unsupervised nature of their training. Existing methods for gaining such steerability, collect human labels of the, you know, feedback of the relative quality of model generations, you know, model output, and fine-tune the unsupervised language model to align with these preferences, often with reinforcement learning from human feedback, RLHF. However, they write, RLHF is a complex and often unstable procedure First fitting a reward model that reflects the human preferences and then fine-tuning the large unsupervised LM using reinforcement learning to maximize this estimated reward without drifting too far from the original model. In this paper, we introduce a new parameterization of the reward model In RLHF that enables extraction of the corresponding optimal policy in closed form. I have no idea what that means, but you'll get a sense for this, allowing us to solve the standard RLHF problem with only a simplification— a simple classification loss. The resulting algorithm, which we call Direct Preference Optimization is stable, performant, and computationally lightweight, eliminating the need for sampling from the language model during fine-tuning or performance-significant hyperparameter tuning, whatever that is.

Steve Gibson [01:26:04]:
Our experiments show that DPO can fine-tune language models to align with human preferences as well as or better than existing methods. Actually, it's vastly better. It completely obsoleted everything that came before. Notably, fine-tuning with DPO exceeds PPO-based RLHF in ability to control sentiment of generations and matches or improves response quality in summarization and single-turn dialogue while being substantially simpler to implement and train. So, okay, that's just their abstract, and the paper goes on at length with, like, with crazy formulae. Uh, I wanted to share that because I didn't want to leave everyone with the belief that the original RLHF, the reinforcement, reinforcement learning from human feedback approach, which began this transformation from autocomplete to actually Q&A Um, that, that's what the industry was still using. It, as I said, it was the genesis. Stanford's DPO, their Direct Preference Optimization, dramatically improved it.

Steve Gibson [01:27:17]:
Um, and DPO's success, as I said, spawned a number of successors which I cited earlier. Okay, so what is all this about? It's about how we take a massive neural network which has been trained on and contains raw knowledge, which can initially only be used to predict the next most likely token, and impress upon that network actual behavior. That's what's changing here. We're giving this knowledge base a set of behavior. The first example of behavior was turning the network into something that could actually respond to queries. That gave us the first interactive AI. But as I noted, those first steps were somewhat shaky. So over the time, researchers learned that by using the technologies I just described, they could improve the model's instruction-following behavior so that it would answer the question that was asked as well as respect the requested format and length.

Steve Gibson [01:28:28]:
And language. The model's style and tone could also be modified and tuned to improve its response formatting, structure, hedging, politeness, and its own verbosity. These factors were, again, they were given significant weight in the tuning. And as we saw in the early days, sycophancy was an often-seen problem. This arose because, you know, we humans reliably prefer being agreed with. So preference optimization trains models toward agreement, and it takes deliberate counter-effort to prevent it now. So that's now in place. We've been able to kind of get that under control.

Steve Gibson [01:29:16]:
Another improvement that was impressed upon models was factuality, preferring responses that will admit uncertainty rather than always confident fabrication.

Leo Laporte [01:29:29]:
Well, that explains a lot, because hallucination, at least in my experience, has almost disappeared, and I was wondering how they did that. Now we know. RPO.

Steve Gibson [01:29:36]:
All right, exactly. So the other class of behavior that can be trained into a model is its refusal to provide certain classes of information. That's like That's so— and what's significant is that there— that this exists in 2 places. This is different than guardrails. There's, there's filtering what you allow the, the human prompter to, to get into the model, and then filtering what you allow the model to show of its results. So that's real-time filtering input and output. But the, but the other place is the— you can actually train the model itself to refuse independent of input-output filtering. So, you know, which is to say the harnessing of the model.

Steve Gibson [01:30:31]:
So it's one thing for a model to contain knowledge that's dual use, which, you know, only authorized users should be able to access. But some model behavior and/or knowledge dissemination should be proactively prevented. The test that AI designers apply is termed uplift. That is, does a model meaningfully advance someone's capability beyond what they should, what they could already get? You know, does it uplift them? This falls into 2 categories. One is where knowledge— where the knowledge is the harm, and the other is where the output is the harm. Examples of knowledge uplift would be, for example, bioweapon synthesis routes, nerve agent production, and nuclear device design. You know, the relevant literature is scattered, It's partial and it's difficult to assemble. So any model that would synthesize it into an actionable protocol provides genuine uplift toward mass casualties.

Steve Gibson [01:31:52]:
And the asymmetry is clear, right? Defensive work in these fields does not require the synthesis route. Somebody, you know, a vaccine researcher needs to understand pathogen biology, not a means of enhancing a pathogen's, uh, you know, uh, malicious use. So this list and those examples that I cited, you know, would not take anyone by surprise. They're one category Where essentially every AI model provider refuses regardless of credentials or system prompt. That is, it's not about who you are, what your privileges are on the model. You just can't have that. It's been trained. The model itself has been trained to refuse.

Leo Laporte [01:32:51]:
Trained in what way? Do they not have the information?

Steve Gibson [01:32:55]:
We're going to get there.

Leo Laporte [01:32:57]:
Oh, good.

Steve Gibson [01:32:57]:
That's perfect.

Leo Laporte [01:32:58]:
By the way, this is fascinating. When the world became aware of this, I mean, the knowledge of this has been around papers and so forth for some time, but it was when DeepSeek came out from China, it was the first model to use RLHF, as far as I know. And it was an eye-opener for people because the model was stunning. This was January of last year. I remember it very well. It was a deep-seek moment. That's when all the stocks of all the AI companies plummeted because people said, wait a minute, the Chinese can do this cheap. Very interesting.

Leo Laporte [01:33:35]:
Now everybody uses RHLF, of course.

Steve Gibson [01:33:37]:
Yep.

Leo Laporte [01:33:37]:
Yeah. Really interesting.

Steve Gibson [01:33:39]:
Now actually would be a great time to take a break. We're a little after an hour and a half in, so we're going to pace ourselves.

Leo Laporte [01:33:46]:
This is fantastic. Yeah. And you found this by reading papers?

Steve Gibson [01:33:50]:
Yeah. I've found—

Leo Laporte [01:33:52]:
On archive.org or?

Steve Gibson [01:33:55]:
Yeah, they're all there. Yeah.

Leo Laporte [01:33:56]:
Yeah. Yeah. Jeff loves reading those papers too. There's a lot of garbage there too. That's my only— but you know what to look for.

Steve Gibson [01:34:03]:
This is like, yeah, these were the found— These are the seminal papers. The founding seminal work.

Leo Laporte [01:34:10]:
The founding documents of it. Yeah. Yeah. Yeah. Absolutely fascinating.

Steve Gibson [01:34:12]:
Yeah. Like, you know, 3 AI researchers at Stanford that figured out, you know, how, how, how to improve on RLHF so that that's what everyone is using now. Right. That's what you wanna look at.

Leo Laporte [01:34:26]:
It's, you know, the other thing that I find amazing is there are breakthroughs like this happening almost all the time now, that there are researchers all over the world working as hard as their little research brains can to find new techniques.

Steve Gibson [01:34:38]:
Yes. That's why I, I keep saying today's AI, today's AI, I mean, anybody looking back, like just quarterly, look back in 3-month hops and it's clear we're onto something.

Leo Laporte [01:34:56]:
Oh yeah. And the other thing is that I, from the outside, I could put flags in the ground and exactly when, January 2025 is when DeepSeek came out and everybody said, oh, RLHF. Oh.

Steve Gibson [01:35:12]:
Last November.

Leo Laporte [01:35:14]:
November 2025, when Opus 4.5— we're going to, I'm sure, get to that. So these internal progress shows up in ways that those of us who use this heavily can see those milestones, the impact of those milestones. I mean, it's very clear. You don't see hallucinations like you used to. And I don't know why. I'm glad you're explaining this. You also see sycophancy going down a little bit, although it's— this is the other thing is the companies are loath to get rid of the stuff that makes people like me keep using their products, right? So they're not going to get rid of all of that. They're not going to get rid of all of that.

Leo Laporte [01:35:56]:
Now, I'm a big fan of Steve Gibson, who is finally explaining something I've been using for a year or 2 and had no idea what was going on under the hood.

Steve Gibson [01:36:05]:
We all have been.

Leo Laporte [01:36:07]:
Yeah, it's fascinating.

Steve Gibson [01:36:08]:
So knowledge is, you know, forbidden knowledge is one class. The other, uh, is where the output itself is the problem, um, uh, or, or the harm. So an example of that, uh, you know, which carries universal condemnation would be, you know, texts which sexualizes minors. Uh, models will not produce any such text. They're— that's trained out of them.

Leo Laporte [01:36:39]:
See, that's fascinating because we've heard about classifiers, which kind of are, uh, uh, gates preventing the egress of that information. But this goes deeper than that. The models themselves say no, no, no.

Steve Gibson [01:36:50]:
Yes. So, and, and that's why, um, unless you remove that, um, even without any kind of a harness, even without you know, anything that is filtering input and output, the model says, uh, sorry, I cannot help you there.

Leo Laporte [01:37:07]:
Right.

Steve Gibson [01:37:08]:
So as we've seen earlier, um, the good news is that these large language models are astonishingly able to absorb and embrace. We saw this in like their ability to, to learn to answer questions. I mean, to understand the, the linguistic nature of a question and how to apply the knowledge they had to create an answer. I mean, that it's an astonishing technology, but it, it does this. So we're able to give them, as a consequence, what appears to be a personality, many forms of behavior, and also instruct them what they may and may not do. So that's the good news. The bad news, as it turns out, is that any behavior like this that can be easily imprinted can also be easily removed. In the summer of 2024, researchers at ETH Zurich, the University of Maryland Anthropic, and MIT published a paper titled Refusal in Language Models Is Mediated by a Single Direction.

Steve Gibson [01:38:31]:
And here, direction is a, is a term of art in, uh, neural networks, as we'll see. So the abstract of their paper employs some of, you know, of this inside baseball terminology, but everyone should be able to easily get the gist of it. So, and I'll explain a bit more afterwards, the paper's abstract, that is, refusal in language models is mediated by a single direction. The abstract reads, conversational language models are fine-tuned for both instruction following and safety— safety meaning not going to tell you that— resulting in models that obey benign requests but refuse harmful ones. While this refusal behavior is widespread across chat models, its underlying mechanisms remain poorly understood. Again, this was, uh, summer of 2024, so about just about 2 years ago this paper appeared. In this work, across 13 popular open-source chat models up to 72 billion parameters in size, we show that refusal is mediated by a one-dimensional subspace. Specifically, for each model, we find that a single direction such that erasing this direction from the model's residual stream activations prevents it from refusing harmful instructions, while adding this direction elicits refusal on even harmless instructions.

Steve Gibson [01:40:23]:
They said, leveraging this insight, we propose a novel white-box jailbreak method that surgically disables refusal with minimal effect on other capabilities. Finally, we mechanistically analyze how adversarial suffixes suppress propagation of the refusal-mediating direction. Our findings underscore the brittleness— and this is the key— the brittleness of current safety fine-tuning methods. In other words, just Instructing the model not to answer the question, well, that works. But if the weights are open, it turns out to be trivial to remove those instructions even after the fact and not having known what the instructions were. I'll explain a little more. It's amazing. So they finish saying, more broadly, our work showcases how an understanding of model internals can be leveraged to develop practical methods for controlling model behavior.

Steve Gibson [01:41:35]:
Okay, so what this group discovered was that any late-term model behavior imprinting can later be removed from such models. And the way this is done, as I— it's, as I said, it's wonderful and wild. They compare The model's activations on harmful versus harmless prompts. Compute the mean difference, then project the weight matrices orthogonal to that direction. So this— so essentially, they, they ask it, they deliberately ask it questions it is trained not to answer, and they watch some of what it does, some of where the activations are, compared to asking questions that it's happy to oblige. And they're able to take the difference in the activations, see them, and then apply a remover that suppresses that, and suddenly It will now answer all questions. So after they do that, the model loses the ability to represent and therefore to execute on its refusal. And what they found was that this was surgical.

Steve Gibson [01:43:05]:
It, it specifically disables in completely— 13 completely different chatbots, all of their refusal while having minimal effect on other capabilities.

Leo Laporte [01:43:20]:
So it's kind of like a functional MRI on the model. Like you're looking for what got activated.

Steve Gibson [01:43:29]:
Yes. And then remove it.

Leo Laporte [01:43:31]:
And then excise it. The only thing I'm worried about, for instance, I use a model from China, Quen, As we mentioned, 3.827, and I've seen obliterated versions of it, but people say, well, you also have a risk. This is brain surgery after all. You might make it dumber.

Steve Gibson [01:43:52]:
Um, so I can't speak to that, but I can, I can cite the research because they, they do address this. So, okay, so first, from my lay view, it appears clear That since little training, amazingly little training, was required to imprint that original refusal behavior—

Leo Laporte [01:44:16]:
Ah, now I get it.

Steve Gibson [01:44:18]:
The impact upon the model was minimal. You know, it wasn't diffused throughout the entire model. It, it had a— but, but it's still, it's significant That this is able to change its behavior so quickly.

Leo Laporte [01:44:33]:
Well, think about it. You're modifying with just a few inputs a large model. There's going to be some collateral neurons that are affected.

Steve Gibson [01:44:45]:
Well, and actually, I use the analogy a little bit later of clean margins when a surgeon excises something. Anyway, so these guys discovered How to identify the changes created by that training, which again wasn't pervasive. It was, you know, not that much training did comprehensively change the model's behavior. So it turns out it could be removed. So, okay. So for what it's worth, when we encounter the term obliteration, this is what is meant, you know, not obliteration. Obliteration. So, okay, just to put a final point on it, a Hugging Face blog posting in the summer of 2024, that is, you know, following this research, was titled Uncensor Any LLM with Obliteration.

Steve Gibson [01:45:44]:
And I'm going to share just the intro from that posting to give everyone a sense for what that earlier for where that earlier research led, which is here. The blog says the 3rd generation of LLaMA models provided fine-tunes, then it says in parens, instruct versions that excel in understanding and following instructions. However, these models are heavily censored, designed to refuse requests seen as harmful with responses such as, as an AI assistant, I cannot help you. While this safety feature is crucial for preventing misuse, it limits the model's flexibility and responsiveness. In this article, we will explore a technique called obliteration that can uncensor any LLM without retraining. This technique effectively removes the model's built-in refusal mechanism, allowing it to respond to all types of prompts. The code is available on Google Colab and in the LLM course on GitHub. So then it says, what is obliteration? Modern LLMs are fine-tuned for safety And instruction following, meaning they are trained to refuse harmful requests.

Steve Gibson [01:47:21]:
In their blog post, Ardidi et al., and that is the, that is the previous research that I was referring to, the research in the summer of 2024, Ardidi et al. have shown that this refusal behavior is mediated by a specific direction in the model's residual stream. If we prevent the model from representing this direction, it loses its ability to refuse requests. So that Arditi reference, as I said, is the research I referred to previously, which showed the world how to do this, just how to simply perform this excision. The blog posting on Hugging Face is one artifact of that preceding research, and the companion repository on GitHub contains all of the details. Uh, that, that is, it, it's mlabonne.github.io, um, and it is Uncensor Any LLM with Obliteration. So, and as you would expect, it was all tremendously exciting. To the world's AI hackers, many of whom immediately jumped on any and every published and available open-weight model and happily obliterated away any and all perceived and imposed censorship upon those models.

Steve Gibson [01:48:56]:
And those obliterated public open-weight models are now available for use by anyone who can harness them. And as you said, Leo, you've seen them both with and without the, the, uh, censorship, uh, obliterated from them. So this finally brings us to the first major topic I wanted to discuss today. Now that we have a much deeper understanding of where these chatbots came from, how they work, how they can have behavior imprinted upon them after they've been filled with raw— not with raw knowledge— and how unfortunately brittle that imprinting is. So, okay, now I, I also need to mention that the term pre-training is what the AI industry has unfortunately landed on to actually mean training. the training that occurs before the post-training. And I suppose since there is now always going to be a definite post-training phase during which the behavior is layered on top of the previously trained-in knowledge, you know, if, if, if that pre-training were just called training, which is actually what it is, Then it might be assumed to encompass both the training and the post-training, meaning if it was just called training, that it would mean all training. So my point is, there's pre-training and there's post-training, and there is not any just training in the middle.

Steve Gibson [01:50:37]:
We don't have— we don't use that. So, um, now we know that pre-training What's it— that's what it's called. Pre-training is the initial knowledge corpus training phase. Um, uh, now, now that we know that, I can explain that a small, competent team of AI researchers, uh, at AE Studio working with Anthropic have— they've proven the feasibility of a new form of AI model pre-training. That is a, a new way of doing that initial knowledge capture. Um, last month, both AE Studio and Anthropic blogged about it, and they published a joint research paper. I'm going to start by sharing Anthropic's press release style posting since it provides the essence of the research without dragging us too far down into the weeds under the title of, which is, which is their title, An Off-Switch for Dual-Use Knowledge in AI Models. And again, the issue here is dual use, right? Uh, nuclear bomb generation, uh, you know, creation.

Steve Gibson [01:52:07]:
Well, there's some knowledge that we want to be able not to provide. The problem is we've seen how brittle the instruction of do not provide that given post-training is. It can be removed, and it has all been removed. It's been obliterated. From all of the current open weight models. So here is what Anthropic said. They, and they, they start by saying this post describes research conducted by AE Studio in collaboration with Anthropic. They said a frontier AI model is, among other things, a large store of knowledge.

Steve Gibson [01:52:50]:
Some of that knowledge is dual use, meaning it can be used for good or bad. For example, knowledge of cybersecurity can help patch critical security vulnerabilities, or it can be used to exploit them. Knowledge of virology can help a researcher create a vaccine, but it can also help a malicious actor design a deadly pathogen. Ideally, we would be able to balance 3 separate goals. First, limiting access to dual-use capabilities in as surgical a way as possible. Second, allowing trusted users to access those same capabilities for beneficial purposes. And third, doing all this without affecting the model's performance on any other task. Current safeguards are imperfect.

Steve Gibson [01:53:47]:
They wrote, we train models to refuse harmful requests and use classifiers to screen inputs and outputs for dangerous content. These layers of protection guard against dangerous outputs, but they don't change the knowledge stored in the underlying model. Despite our safeguards, a sufficiently determined attacker may still try to jailbreak the model, working past its defenses to access the dual-use knowledge. A more robust protection against misuse would be to control what the model knows. We've explored this before, they wrote. In earlier work, we filtered information about chemical, biological, radiological, and nuclear weapons out of pre-training data and later showed that dual-use knowledge can be confined to a removable slice of a model's weights. But filtering is a blunt instrument. It produces one model with one fixed set of capabilities.

Steve Gibson [01:55:05]:
Using filtering, if you want a model version that can discuss advanced virology for deployment in a vetted biosecurity lab, say, and another version that cannot discuss that because it doesn't have the knowledge, they say you have to train 2 separate models, especially in the case of frontier models, which are large and very expensive to train. The cost to the developer would be prohibitive. So I'm going to interrupt here just to add that while OpenAI and Anthropic are not being currently forthcoming about the cost to train a current frontier model— Leo, you've always talked about how expensive it is, and oh boy, um, you know, once those 2 are publicly traded, then their accounting ledgers will no longer be private. So we're going to find out. But to get some sense of scale, OpenAI's Sam Altman has stated that the training cost for GPT-4 was more than $100 million. And OpenAI reportedly spent $3 billion overall on compute to train their models 2 years ago. back in 2024. Also, as we know, models have grown much larger recently.

Steve Gibson [01:56:34]:
And those numbers ring true, those earlier ones, because Google's Gemini model is believed to have cost Google $192 million to train. So we're talking—

Leo Laporte [01:56:48]:
That's chump feed though compared to what they're spending now. I mean—

Steve Gibson [01:56:53]:
Well, actually, Yes, right, because these are old and smaller models, right? So it could be, uh, so it's a billion dollars.

Leo Laporte [01:57:02]:
Some have speculated a 10 trillion parameter model.

Steve Gibson [01:57:07]:
Gosh.

Leo Laporte [01:57:08]:
Uh, and ChatGPT-4 was, I don't know, several hundred million probably. Right.

Steve Gibson [01:57:16]:
Yeah. So, and, and so all of this leads us to understanding why It's not possible, it's not feasible for any commercial provider, any, anyone commercial or not, to train, to have multiple versions of, of a single model type, like a, like a Mythos that knows nothing about cybersecurity. The advantage would be you can't trick it into revealing what it doesn't know. It did. The knowledge was— would have never been put in there. So it, it's just not there to ask. But it— you, you would, you would spend so much money training that up. And this is the problem that, that they're trying to identify.

Steve Gibson [01:58:01]:
So the quite valid point that Anthropic is making here is that it's massively infeasible to train a state-of-the-art model on on any pre-filtered knowledge to make it dumb about some things. You make it just, I don't know anything about that. You know, if you're going to be spending that kind of money on training, it needs to know everything so that it can have the widest application range for its use, you know, in order to have some chance of getting some money back out of all that, that training that went into it. So If a state-of-the-art model were to be trained on a filtered subset of everything, then, uh, you know, that is a, a limited one forever. You end up with a very expensive, forever limited model. Um, okay, so, um, we've seen that imposing post-training behavior, which is what we're just talking about, this refusal obliteration Post-training behavioral restrictions on open-source models where you're able to modify the network weights, that can be altered. And so it was a short-lived solution. Um, the means for removing those restrictions, as we saw, so it's all public knowledge, child's play, you know.

Steve Gibson [01:59:29]:
And even the best closed-weight models which are operated by cloud-based hyperscalers, you know, like OpenAI, Anthropic, and Google and so forth, AWS, we've seen that they can be prone to trickery and abuse. We've, you know, prompt injection example, and we're next week, we're going to understand exactly how that happens. So what's clearly needed is a new solution. And that's what these researchers have found. Um, Leo, we're at 2 hours. Let's take our last break and then we're gonna look at the solution for this dual use problem and how to, how to solve this with a single training.

Leo Laporte [02:00:10]:
Uh, we will have more of this. I'm just, by the way, absolutely fascinated by this. I feel like, uh, this should be required listening for, uh, the, uh, listeners of Intelligent Machines, our AI show tomorrow. Because it's such foundational information about how all this stuff works. And it explains a lot, to be honest, about how these models work. And I think it's a good idea to kind of understand the underlying technology because it makes— it means that you could do a better job.

Steve Gibson [02:00:42]:
And again, as a user, you'd, you know, Lori, she's using the heck out of ChatGPT, but our audience, that's why they're here.

Leo Laporte [02:00:52]:
Sure.

Steve Gibson [02:00:52]:
Is to get this.

Leo Laporte [02:00:54]:
And it, you know, for advanced users who are looking at things like obliterated models, uh, and wondering why some models do this and some models do that, there's so much going on. This stuff moves so quickly. It's very helpful to understand it a little bit. Absolutely. So I appreciate it. Uh, we're going to take a little break and come back with more. You're watching Security Now with the wonderful Steve Gibson. On we go, sir.

Steve Gibson [02:01:19]:
So, um, because post-training, uh, behavior modification has been demonstrated to be easily removable, it is not sufficient to, to say don't, don't give people these, uh, you know, disinformation. Uh, uh, it doesn't work to suppress that behavior. What we need is a new solution, and It's completely infeasible to do multiple training runs with different combinations of information filtered out of a model because training is prohibitively expensive. So what we need is a new solution. And that's what these researchers working with Anthropic have found. Anthropic continues their writing saying, in new research carried out with collaborators at AE Studio, We explore a new method that could enable the benefits of training many separately filtered models, but at the cost of training only one. We call it GRAM, gradient routed auxiliary modules. Note that they wrote, note that the results of the experiments presented here are preliminary.

Steve Gibson [02:02:38]:
Graham has not been applied to any of the production models in Anthropic, uh, and they said, and we're not sure it ever will be. And I— okay, so I, I take that to reflect a very reasonable and very cautious approach, because can you imagine the cost of a mistake if one of these companies' Graham-trained model, whatever that is— and we'll, we'll get to that in a second— turned out to have unsuspected problems. Uh, there's no reason to believe that it would or that that would be the case, but again, their caution is understandable because they could be scrapping half a billion dollars these days. So they write— Anthropic writes how Graham works. The idea behind Graham is to give a knowledge— a model dedicated removable compartments for each category of dual-use knowledge and to update only those compartments when learning from dual-use data. Again, to update only those compartments, not all of the model's weights, only those in the compartments when learning from dual-use data. They said concretely, Graham adds extra neurons to every layer of a standard transformer, you know, the neural network architecture on which large language models are based. These neurons are divided into groups or modules, one module per dual-use category.

Steve Gibson [02:04:16]:
During training, when the model encounters general-purpose text, that is, you know, just standard text, we don't worry about it one way or the other, it learns in the usual way. But when it encounters text from a dual-use category, virology for instance, they wrote, the rules change. The model could use its general knowledge to make predictions, but only the virology model module is allowed to learn from that text. The general-purpose weights are temporarily frozen, right? So in other words, the bulk of the model isn't changed by, by learning about virology. Only the neurons in the virology module are changed. And if the bulk of the model's weights don't change after learning about virology, it doesn't learn about virology. It doesn't gain any knowledge from that. It's like it never happened to most of the model.

Steve Gibson [02:05:26]:
They said the consequence is that virology knowledge accumulates in the virology module rather than diffusing across the whole network after training. The module can simply be deleted and the virology knowledge goes with it, or it can be left in place for trusted deployments when virology knowledge—

Leo Laporte [02:05:51]:
Give it a lobotomy.

Steve Gibson [02:05:52]:
Yeah, exactly. No, it's exactly.

Leo Laporte [02:05:56]:
What's interesting about this is you would think, well, that's how all knowledge is, but it isn't.

Steve Gibson [02:06:01]:
It's hard knowledge to store.

Leo Laporte [02:06:02]:
When the models are trained, it propagates through the whole model.

Steve Gibson [02:06:06]:
Yes.

Leo Laporte [02:06:06]:
So this is a special technique that says, no, no, only virology can, you know, can only go here.

Steve Gibson [02:06:12]:
Yes. And it— what's really interesting is it tends to concentrate there because it's like the virology module doesn't— it's like it knows it's carrying the full weight of that knowledge. Yeah. It's so cool.

Leo Laporte [02:06:26]:
So amazing.

Steve Gibson [02:06:28]:
So they said the knowledge can be tailored very specifically to the type of deployment needed. In our experiments, we defined 4 dual-use categories so that one training run with Graham yields a model that can be configured in 16 different ways, you know, on or off for each of the 4 categories. And as we know, that a 4-bit number can have 0 through 15, so 16 different possible ons and offs. They said, we, we tested GRaM in 3 settings of increasing realism. First, on a synthetic dataset of children's stories tagged by topic, a small GRaM model could be reconfigured to forget any chosen topic, and each configuration performed almost identically to a separate model trained from scratch with that topic filtered out. In other words, they did an A/B comparison. Here's a GRaM-trained model where we turned off the topic, comparing it to a normal model that was never trained with that topic, and there's no difference in behavior. They said, and they They made it more clear.

Steve Gibson [02:07:54]:
They said, that is, for the cost of training a single model, we achieved results that would normally require multiple training runs on different datasets. Second, we trained a larger model on a realistic mixed mix of web text, code, and scientific papers with 4 dual-use domains. Virology, cybersecurity, nuclear physics, and a niche programming language just to serve as a proxy for specialized dual-use code. They said the capability associated with each dual-use domain is routed to its own module. Deleting a module removed the corresponding capability about as effectively as never having trained on that data at all.

Leo Laporte [02:08:49]:
Wow.

Steve Gibson [02:08:49]:
Remarkably, they said, we find that this removal did not degrade general performance. And finally, they said, we also tested whether an attacker could recover the removed knowledge by training on a small amount of malicious data. Believe it or not, Graham resisted this about as well as data filtering did. By contrast, an unlearning technique applied after training only suppresses the knowledge. That's what we've been talking about. It was easy to remove that with a small amount of fine-tuning. So that's a parenthetical about the resistance ablation. And then finally, third, they said, we ran the experiment at 7 model sizes.

Steve Gibson [02:09:42]:
from 50 million to 5 billion parameters. Graham matched the performance of data filtering at every size, and the gap between module on and module off grew wider as models got larger.

Leo Laporte [02:10:02]:
Hmm.

Steve Gibson [02:10:03]:
Now, I'm going to pause on that for a moment. The, the larger the model, the more general knowledge was stored Outside of the various subject matter-specific modules. So the subsequent removal or suppression of any one or more of them during inference, as a result, had diminishing effects upon the model's overall performance. This is exactly what we would hope to see.

Leo Laporte [02:10:33]:
I wonder, you know, I'm running 2 different kinds of models. Right now. In the large video card, the 3090, I can run what's called a dense model, which is— that's the Quen 3.8 27B. And it's a, I think, kind of extrapolating from what you just said, it's a model where all the weights are spread throughout the model. That's why they call it dense. But in order to run DeepSeek V4 Flash on my Sparks, on 2 different machines with a fast interconnect, It has to be— you can't use a dense model. You can't split the lobes of the brain. It has to be what they call an MOE or mixture of experts.

Steve Gibson [02:11:15]:
Experts, right.

Leo Laporte [02:11:17]:
And the advantage of doing that is you don't load the whole model into memory. You take shards of it. I suspect this is a similar technique that you kind of localize knowledge in a shard as opposed to spreading it throughout the entire dense model.

Steve Gibson [02:11:34]:
Right.

Leo Laporte [02:11:34]:
You—

Steve Gibson [02:11:35]:
I mean, it— in retrospect, it seems obvious, right? If you don't update a model's weights, it can't learn what you just showed it, right? Because you didn't change it.

Leo Laporte [02:11:49]:
That's how it learns. That's what learning is.

Steve Gibson [02:11:51]:
Yes, yes. You made it forget. You, you made it come, you know, like nothing happened.

Leo Laporte [02:11:56]:
Yeah.

Steve Gibson [02:11:56]:
And so if, if you— then if you reserve a region that you do allow to learn, then as it turns out, it learns about virology. Uh, the rest of the model can't because you, you froze its weights. It's all stuck here.

Leo Laporte [02:12:12]:
Yeah, doesn't—

Steve Gibson [02:12:13]:
I mean, but it turns out this actually works, and the larger the model gets, the better it works, which is what, of course, which is what they care about. Because now, you know, they're not, they're not interested in doing any 5 billion parameter models anymore. That's— sorry, that was just an experiment to see, you know, whatever.

Leo Laporte [02:12:32]:
This 3.8 is a relatively small model at 27 billion parameters, right? And DeepSeek V4 Flash is considerably larger than that. Yeah.

Steve Gibson [02:12:41]:
So, so they said, uh, they, they, they continue saying, as AI companies train more capable models, we need to limit access to dual-use capabilities— or I'm sorry, the need to limit access to dual-use capabilities will increase. Okay, so in other words, Anthropic understands that the more capable the industry's models become— and we're seeing it like before our eyes— the more valuable the knowledge they contain will become, and thus the need to manage the access to that knowledge grows increasingly crucial. So they also make another good point. They write, today, companies limit access through classifiers and refusal training. And we just, we just blew up refusal training, right? That's gone. That no longer works. Well, if you have access to the weights, you, you, if, you know, if it's OpenAI and Anthropic, they're not giving you access to their closed-weight models, so you, you can't You obliterate those, but you sure can the open weight ones. They said, however, these safeguards, meaning classifiers and refusal training, are difficult to make robust without degrading performance on harmless requests.

Steve Gibson [02:14:04]:
Methods like GRAM offer a potential path toward access control that is more robust. And that's a really great point. The current system, which combines the refusal training, uh, which we examined earlier, with real-time input and output classifiers, that provides at best fuzzy filtering. The AI can frustrate its innocent user by refusing a benign prompt. It's like, wait, I thought What do you mean you won't answer that? You're an AI. I know you're an AI, but why won't you give me the answer? And similarly, it can delight a malicious user by letting down its guard when it should not. But by using a method like GRaM, I mean, a model can have its functioning knowledge base selectively tuned to the authentication level or nature that the prompter's access permissions specify. So that's a significant improvement over today's soft and somewhat ad hoc solutions.

Steve Gibson [02:15:19]:
So Anthropic concludes writing, this is early research and there are clear limitations. We have not tested Graham at frontier scale or in a production training pipeline. And they said, as noted above, it's not— it has not been applied to any of our Claude models. Our evaluations quantify performance in terms of next token prediction ability rather than performance on real downstream tasks. And there's a deeper open problem that applies to data filtering and methods like GRaM. Some dual-use capabilities might be so entangled with general knowledge that no method can separate them cleanly. So they said— they, they finished saying, for further details about our experiments, read the post in our Alignment Science blog. Um, and that's really interesting, Leo.

Steve Gibson [02:16:18]:
The point they make, I think, is a good one. Like, you need to be— you need to know when to freeze learning globally and only allow the knowledge to be concentrated in the module. But there, that, that decision is going to be a little soft and fuzzy too, right? Like some biology is not virology or not prone to abuse, but, you know, again, it's not, it's not binary, right? It's going to be kind of on a continuum somewhere. So anyway, when I wrote about Graham 2 weeks ago in the Security Now weekend special email, the one before Black Hat, several of our listeners wrote back with feedback that I also shared during our Black Hat podcast. Their concerns surrounded, you know, censorship and who gets to decide who has access to what data. When I voiced that during Black Hat, both Richard and Paul chimed in immediately Well, this was just a different version of the way things have always been. You know, the fact that AI has made most of the world's knowledge vastly more accessible doesn't necessarily mean that AI has made all of the— doesn't necessarily mean or need to mean that AI has made all of the world's knowledge vastly more accessible to everyone. Um, you know, there isn't any entitlement to the knowledge that AI holds.

Steve Gibson [02:17:52]:
After all, it costs those companies hundreds of millions of dollars to create and offer this facility. So I would argue that they can put whatever restrictions on it they wish if commercial providers of that knowledge are required by internal policy, public pressure, the government, their stockholders, or whomever to gate and control access to some aspects of that knowledge, I think that's entirely reasonable. And I would argue that the commercial providers have every right to do so. We just saw an example with OpenAI and that Hugging Face incident where Hugging Face was unable to deploy either Anthropic's or OpenAI's models to help with their cybersecurity forensic investigation after they'd been attacked by OpenAI because Hugging Face hadn't been granted the magic keys to those AI cybersecurity, uh, to those providers' AI cybersecurity knowledge. Everyone would argue now that they should have had such access, and that we did later hear that OpenAI was working with them to give it to them. So anyway, I keep repeating the qualifier commercial providers because, as we know, and you're using them, Leo, there are alternatives.

Leo Laporte [02:19:22]:
Mm-hmm.

Steve Gibson [02:19:23]:
And an alternative is exactly what Hugging Face turned to after their commercial models refused to help them. They used one of the many publicly available unrestricted open source models.

Leo Laporte [02:19:34]:
GLM 5.2, which is really good and has been succeeded by, interestingly, GLM 5.3, which is the same. This is an interesting slice on what you were just talking about.

Steve Gibson [02:19:45]:
It's the same. From z.ai, right?

Leo Laporte [02:19:47]:
z.ai says it's the same model. It's the 5.2 model, With more enhanced post-training. Uh-huh. So there is, there's headroom even there. Yes. Where you could take the blob that is all the weights and just do a better job with the knowledge. Fine-tune it. Yeah.

Steve Gibson [02:20:08]:
Yes. Because the post-training is behavior. Pre-training is knowledge. Post-training is behavior.

Leo Laporte [02:20:14]:
Well, and interestingly, that's where it really excels, is at coding and that kind of thing. Yeah, it's really fascinating what's going on here.

Steve Gibson [02:20:23]:
Wow. So just to finish here, anyone who might object to the big commercial services restricting what can be done can easily turn to any of the many alternatives. And I have to say, given what I was able to do with the carefully unrestricted and uncensored Venice.ai service, which I played with briefly when it appeared, and we talked about it here on the podcast. I'm pretty certain that fully unrestricted, unrestrained, and uncensored AI models are readily available, even from third parties in the cloud. Now, not from OpenAI and Anthropic. That's not their, you know, their style.

Leo Laporte [02:21:08]:
They're serving their proprietary models, but because there are so many open weight models. Hugging Face has more than 3 million models, and those aren't all— those are many, many millions of them are just obliterated or somehow modified larger, well-known open models. So there might be thousands or hundreds of thousands of GLMs on Hugging Face that hackers modified. That's a very fun thing for people to do.

Steve Gibson [02:21:35]:
Yeah, it is. It's like apps to download for Android. A bunch of, you know, how many millions of them are—

Leo Laporte [02:21:41]:
A lot of them are crap. Absolutely. Exactly. Yeah.

Steve Gibson [02:21:44]:
So we've run out of time and there was a lot to take in.

Leo Laporte [02:21:49]:
Oh, I want to know more, Steve.

Steve Gibson [02:21:52]:
We have now a better, far better understanding of the way today's AI works. And I mean, at least enough to kind of have a feel for it. It seems like it's less magic than it was. We're going to learn next week how and why prompt injection attacks continue despite the best minds in the industry struggling to prevent them. That technology blew my mind on the plane flight to Las Vegas, and I'm going to blow everybody's mind, uh, next week.

Leo Laporte [02:22:24]:
Your chain of thought is not all it's made out to be.

Steve Gibson [02:22:27]:
As they say, stay tuned.

Leo Laporte [02:22:30]:
It really is interesting stuff. Thank you, Steve. Steve Gibson, our guru, now just not of just security, but of AI as well. You'll find him at grc.com, the Gibson Research Corporation. You know, it's really great because you've come full circle. As I mentioned earlier, as a kid, as a high schooler, you started at the Stanford AI Lab.

Steve Gibson [02:22:51]:
Yeah.

Leo Laporte [02:22:52]:
Of course, AI in those days was symbolic AI. It was a very kind of different enterprise.

Steve Gibson [02:22:57]:
It was.

Leo Laporte [02:22:57]:
But the same goal.

Steve Gibson [02:22:58]:
It was so overstated even then.

Leo Laporte [02:23:00]:
It was like, it was Eliza.

Steve Gibson [02:23:02]:
It's very artificial. It's like, you know, people's moms were saying, that's all it does? Well, that doesn't seem like very—

Leo Laporte [02:23:10]:
It's pretty amazing if you really get deep into today's models. They surprise me every single day with the things they say, the capabilities that they have. It's truly remarkable. It's, as you mentioned in the the past. It's great for system administration, coding. I don't use it much for writing and that kind of creative stuff. I do use it— I've actually had a lot of fun making cartoons. When we had a house sitter, when we went down to Black Hat, we had a house sitter taking care of the cat.

Leo Laporte [02:23:43]:
And of course, our television setup is inscrutable, as most people, as most geeks are, 4 or 5 remotes, 8 different devices. No house sitter could ever watch TV. Even Lisa says, if you die, I'm out of luck. I can't watch TV anymore. So I made a cartoon, a comic book that shows how to use the TV, and it's fantastic. And I didn't have to do much because the AI already knew. I just said, we've got this, this, and this.

Steve Gibson [02:24:13]:
Oh.

Leo Laporte [02:24:14]:
You control it with the Apple remote. It said, I got this. It's pretty— I mean, and again, I wish I could show you the cartoon. It's amazing. So constantly impressed. And I don't think, unlike much magic where when you know how the trick was done, it loses all its magic, this is not that case. What they are doing is unbelievable.

Steve Gibson [02:24:38]:
It's us. It's us.

Leo Laporte [02:24:42]:
Yeah. I, I think it is, and I know you think it is. I think there are a lot of people who hate that idea. A lot of people who say there's something special that we do.

Steve Gibson [02:24:50]:
Hate away. I, uh, I, I, I mean, we're still figuring this out. It's going to get better. It's going to get more efficient. It's going to get better trained. Costs are going to come down. Uh, I love it.

Leo Laporte [02:25:04]:
One of the reasons I wanted to spend a ridiculous amount of money, it's not an economic decision. Because it's so cheap to use these models. I know it's expensive relatively, but it's not thousands of dollars. I wanted that brain in the house.

Steve Gibson [02:25:18]:
Yeah.

Leo Laporte [02:25:19]:
I wanted it to live next to me, and now it is. It's sitting— actually, I have a couple of brains, 3 actually, sitting there thinking, doing stuff. And I use it all the time.

Steve Gibson [02:25:32]:
What a world.

Leo Laporte [02:25:33]:
What a world.

Steve Gibson [02:25:34]:
Yep.

Leo Laporte [02:25:35]:
stevesatgrc.com, that's his website. Now, there's some good reasons to go there. Uh, of course, there's his bread and butter, the world's best mass storage maintenance, recovery, and performance-enhancing utility. I live now on SSDs, and SSDs are so expensive. Those Rust drives in my Synology, so expensive. You better spend a little money, get SpinRite, so you know you can keep them flowing and going, and no bits will be lost in the process. Uh, everybody ought to have a copy. And the beauty of it is there are people who bought this program 30 years ago, we're still getting free upgrades.

Leo Laporte [02:26:09]:
Steve is amazing that way. grc.com. You'll also find there his DNS Benchmark Pro, a great way to test to make sure you're using the fastest DNS server. You're probably using your ISP's, almost certainly that's not a good choice. Steve can help you find the right choice for your locale. Lots of other free stuff. And of course the show is there. Steve's got unique versions, shall we say, of this show, the 16-kilobit audio, the 64-kilobit audio, the handmade transcriptions by an actual human being, the wonderful Elaine Ferris, and the show notes, which Steve, I mean, this is one where I absolutely going to be reading the show notes to reabsorb or absorb better.

Leo Laporte [02:26:51]:
In fact, maybe I'll point my AI at it. Say, is this what you're doing? 20 pages, thereabouts, of goodness, and you get all of that for free at grc.com. You can even get the show notes mailed to you if you want. Go to grc.com/email. Uh, you'll be putting your email address in. Actually, the main purpose of that is to whitelist so you can send them emails, send them pictures of the week of buses as bridges or whatever. But underneath that email form, there is— there are 2 checkboxes. one for the show notes you'll get every week, and one for a very infrequent mailing list whenever he has a new product.

Leo Laporte [02:27:25]:
I don't think he's sent anything out in years, to be honest. But anyway—

Steve Gibson [02:27:27]:
No.

Leo Laporte [02:27:28]:
No, it's there. Sign up. It's not going to— believe me, you're not going to get any spam from this one. Steve is going to protect your secret identity. We also have copies of the show at our website. We have 192-kilobit audio, I think, some ridiculous size. That's because Apple Apple wants to down—

Steve Gibson [02:27:46]:
It's 128 because I see it.

Leo Laporte [02:27:49]:
Is it only 128? Oh, okay. Well, that's not too bad. I thought it was even bigger. That's because Apple downsamples it. And so we have to give them a better quality version or you won't get a good—

Steve Gibson [02:28:00]:
I don't know.

Leo Laporte [02:28:02]:
We also have video, which no one has. Steve, when I said, let's do video, said, what?

Steve Gibson [02:28:07]:
Why?

Leo Laporte [02:28:07]:
What a terrible idea, I think is what you said.

Steve Gibson [02:28:12]:
Nice to see you. We're doing it anyway.

Leo Laporte [02:28:14]:
You can get both of those at twit.tv. That's the website, /sn for Security Now. There is also video on YouTube. We're actually glad we do video now because now we can put our stuff up on YouTube, which is fantastic. Or subscribe, audio or video, on your favorite podcast client. You can even watch us do the show live. If you're a member of Club Twit, you can watch in the Club Twit Discord. Even if you're not a member, you can watch on YouTube, Twitch, X, Facebook, LinkedIn, or Kick.

Leo Laporte [02:28:39]:
We stream it on 7 different platforms as we're doing the show every Tuesday, right after MacBreak Weekly. That's about 1:30 Pacific, 4:30 Eastern, 20:30 UTC. Well, that just about does it for us. I hope your brain is swelling. Do not obliterate any portion. You're going to need it for next week. We'll see you right back here on Tuesday for Security Now.

Steve Gibson [02:29:02]:
Bye. Security now.

Leo Laporte [02:29:05]:
Security now.