Amazon’s Schmidt talks China, cyber traps and the battle in the cloud

Steve Schmidt is the chief security officer at Amazon. He’s in charge of safeguarding everything from classified government contracts to your Amazon orders. Among his secret weapons? A sprawling network of decoy systems — thousands of digital traps that lure hackers into revealing themselves. It’s called MadPot, and it recently helped expose one of the most sophisticated cyber operations ever linked to China: Volt Typhoon.

The Click Here podcast spoke with Schmidt about honeypots, AI’s role in cyberdefense, and why he believes the biggest vulnerability isn’t in the code. It’s in the people. The interview has been edited for clarity.

CLICK HERE: Can you talk a little bit about MadPot, what is it and how does it work?

STEVE SCHMIDT: MadPot is a honeypot network. A honeypot is a computer system that's been specifically built to be attacked. So it's designed to attract adversaries by presenting the image that it's a vulnerable computer system. Adversaries then probe for it. They find it. They interact with it and they often deploy their tools to it in an effort to exploit the system. It's a super important part of our overall intelligence apparatus because it allows us to identify what our adversaries are interested in, the tools that they're using, the techniques that they use, and even down to which one of our customers might they be going after.

CH: How often do adversaries actually interact with these honeypots?

SS: It is absolutely constant. So to give you some relatively scary numbers, we operate some 10,000 of these things around the internet right now. It takes about 90 seconds for one of these honeypots to be probed by an adversary — from when it goes online, within three minutes adversaries are trying to exploit it. So whenever I hear anybody saying, "Oh, I'm gonna put something on the internet, but nobody will know it's there and it'll be safe because it's innocuous" — you’ve actually got 90 seconds before someone knows.

CH: Where were you when you heard that Volt Typhoon had interacted with MadPot?

SS: I don’t remember where I was — probably at my desk. What I did get notified was, “Hey, we found what we consider to be a significant threat actor.” And more importantly, we not only found them in current data, but we were able to go back several years … because we store the MadPot data. We saw them several years earlier in the data.

CH: How did you actually identify them?

SS: Volt Typhoon, like many actors, uses multiple sets of intermediate systems to try and hide where they're coming from. When they're attacking somebody, they'll use one set of systems. When they're doing their reconnaissance or their test, they'll use a different set of systems. We saw both because the signature of the tool was the same across both of them.

CH: So once you know someone fell for the honeypot, what happens next?

SS: We go through a bunch of validation steps to make sure that we're correct with what our conclusions are. The second thing is we identify anybody who we think may be targeted by the adversaries. If we have a relationship with those people — whether it's an [Amazon Web Services] customer or somebody we've interacted with before, or an industry partner — we'll notify them directly that we believe they have a problem and we'll give them information that allow them to determine: is this actor actually in their systems or not?

We then go to CISA or law enforcement in the U.S. and share the data with them and say, “We believe this is this particular threat actor: dates, times, signatures, those sorts of things. Could you please notify the industry at large?”

CH: Do you wait to go public until there's a fix?

SS: Yes, generally that's what we try and aim for. If the adversary is exploiting a vulnerability in a piece of software, we'll work with whomever the software developer is to get it patched and make the patch available before it's publicly known that there's a problem.

CH: How is MadPot different from something like traditional endpoint detection?

SS: Endpoint detection is about making sure that your laptop is safe. MadPot, on the other hand, is a computer system expressly designed to be exploited by an adversary. We want someone to break in. It's the storefront where the front doors aren't locked, the alarm system isn't turned on. The security guard isn't present because we want to see what tools do they use to break in. Do they jimmy the lock on the front door? Do they use a crowbar? When they walk in, what are they going after? What are they trying to steal? What time of day do they do it? Where did the tools that they're using come from? Who supplied it to them?

CH: And AI is helping you track all of that?

SS: That's the beauty of using AI to do this kind of analysis. … It can extract the tools that the adversary is using, but more importantly, it can say, “This tool is different than any of the other tools they have ever seen before.” That's interesting. Let's kick it out to a human who can say why. Is it different? We don't necessarily know, but it's part of the equation.

CH: Are adversaries using AI too? Are you seeing that?

SS: Adversaries do use AI. They are not using AI yet in any large scale to do the actual attacking process. They're using AI to make themselves much more skilled. So an example would be, you get an email which is phishing: “Please click on this link 'cause you haven't paid the bill” kind of thing. A lot of those are readily identifiable by a normal human because of spelling errors, grammatical errors, poor formatting. What AI can help the adversary do, though, is create a much more believable object. So it's much more likely to get someone to click on it.

CH: So what keeps you up at night?

SS: Humans. They are the single thing that we have to think about the most. A lot of people look at my job and say, “Oh, computer security — you gotta be worried about tools and techniques and all that kind of stuff.” No. It's people.

CH: Let’s talk about Amazon’s use of AI agents. What are they doing?

SS: We have a really fun set of agents inside my security org right now who are effectively dueling. There's one agent who is an adversary, and their job is to break into computer systems. Another agent is a defender. That's identifying the adversary behavior, building signatures that allow us to put into our detection systems the things that alarm on the presence of it, and take the next step, which is patching to prevent the problem from working in the first place.

CH: Can those agents patch systems automatically?

SS: I think that we will be pretty darn close to an agent doing that for us for non-production systems in the near future. So what non-production versus production means is: A production system is the system that serves www.amazon.com. If it goes down — big deal, big problem. A non-production system is the workstation that I use for my day-to-day work. If it goes down, it's annoying to me, but it's not the end of the world for the company. So I'm much more willing to take an automated action on a non-production system because the blast radius of a problem is much smaller. Whereas for a production system, like something that supports the retail website, we've gotta be exactly correct every time.

CH: Cyber is becoming part of conflicts around the world. What did you learn from something like Ukraine’s cyber response to war?

SS: So if you look at the Ukraine situation particularly, it was about taking data which was physically present in an area that was under threat and moving it into a diffuse set of cloud storage locations that gave us availability and resiliency that they couldn't get any other way. That was true for Ukraine, but it also applies in any circumstance where a natural disaster, for example, might be at risk.

CH: Has Amazon’s national security role expanded because of this?

SS: The reason I'm at Amazon is because I was in the FBI and asked Amazon to build a system to help support an intelligence mission at the FBI in 2008. So the company has always been focused on supporting the national security customers who have a need to be able to process information securely and effectively. I think that's just growing over time because of the efficiency gain that you get from using the cloud versus on-premises.

CH: Last question. If you could whisper one thing in the ear of the next generation of cybersecurity leaders, what would it be?

SS: Use AI to do your job more efficiently. It's not AI replacing humans. It's AI replacing the grunt work that none of the humans actually want to do anyway — to make them more efficient, to make them more effective, and to transfer the knowledge from the really skilled individuals at the top of the pyramid to the frontline engineers at the bottom.