The Speed of Threat

The speed of threat: Threat intelligence and detection engineering in the age of AI

How can security teams shorten the time from identifying a cyber threat to having detection code live in production?

In this episode of the mnemonic security podcast, Robby is joined by Nick Maeckelberghe, Co-Founder and Managing Director at the offensive security research lab and services company Crimson7, and welcomes Jeff Schiemann back to the podcast, now a Board Advisor at Crimson7. 

Nick and Jeff discuss how security teams can shorten the OODA loop, the time it takes to observe, orient, decide and act on a threat, with the ambition of going from understanding what an attacker is doing to having detection code live in production in a matter of hours.

They explore what it takes to keep pace with attackers as new tools and techniques emerge, and how cyber threat intelligence (CTI) and detection engineering can work together to identify and stop threats at different stages of an attack campaign. They also discuss how LLMs and other forms of AI are changing cybersecurity on both sides of the equation, as attackers move faster and defenders look for ways to analyse threats and turn what they learn into detections at the same speed.

The conversation also turns to some future-gazing: the rise of neoclouds, the concept of "headless security services" increasingly delivered by AI, and a potential shift from identity as the attack surface to trust as the attack surface. They discuss what these developments could mean for the security industry, and how customers will distinguish between good and average security vendors as AI takes on more of the work.

Transcript

Copy Transcript Copied

Speaker 3 0:03

From our headquarters in Oslo, Norway, and on behalf of our host, Robby Peralta, welcome to the mnemonic security podcast.

Robby Peralta 0:15

CTI tells us what adversaries are up to. Purple teaming tests whether we'd see it. And detection engineering makes sure we can reliably catch it. Not just once in a test, but repeatedly over time. It's the good old OODA loop that the Air Force gave us decades ago. Except when that metaphor first landed in security, we were nowhere near a supersonic fighter jet. More like a propeller plane flying into a headwind. And today's guests have an ambition to shrink that loop down to ours. From the moment they first spot a threat to having detection code live in production. Nick Maeckelberghe and Jeff Schiemann - welcome to the podcast.

Nick Maeckelberghe 1:00

Thank you, Robby.

Jeff Schiemann 1:01

Hey, great to be here, Robby.

Robby Peralta 1:03

Welcome back, Jeff. And it's uh your first time for you, Nick. Would you mind uh introducing yourself for uh for the listeners?

Nick Maeckelberghe 1:09

Yeah, sure. So yeah, Nick Maeckelberghe, based in Belgium and uh managing director and uh co-founder of Crimson 7, which is a cybersecurity uh startup um out of Belgium focusing on what we call uh the speed of threat. So we have two platforms which consists of uh two SaaS products, and we'll talk about uh a bit of that, what we do and what the speed of threat is a bit later.

Robby Peralta 1:29

Cool. And Jeff, welcome back.

Jeff Schiemann 1:32

Hi, uh great to be back. Uh my name is Jeff Schiemann, former CISO at a crypto bank, uh here for the last five years in Switzerland, one of the first. Um, also uh heavy in the crypto industry for the last uh eight years. Prior to that, uh part of the expert group at France Telecom and really working with CISOs, uh first layer of CISOs is operational response and also second layer CISOs uh now in compliance. Uh and I'm board advisor to Crimson 7.

Robby Peralta 1:58

Lovely. So last time we talked, Jeff, uh it was me, you, and uh Joe Slowik. At the time you said your, I want to call it your OODA loop, you called it your triangle, but it was going from seven to ten days of being able to see a threat somewhere and be able to feel that you had uh detection capabilities or feel safe from that specific threat. And you said your ambition was to get to hours within a five to six year range.

Jeff Schiemann 2:22

Yeah.

Robby Peralta 2:23

How's that aged?

Jeff Schiemann 2:24

It's gone very well. Um, we uh worked with a lot of companies, including uh Crimson 7, to get down our detection uh validations, what we called DCV at that time. Now people know it in Europe as continual validation and the adversary enumeration validation space. Um, we were ahead of that uh sort of you know defining area of you know what was our time loop, what was our processes for doing detection, confirmation, validation, how much threat hunting would we start uh to do, and how much cyber threat intelligence and quality would we put into blocking mode right away? Um we were able to obtain uh almost 90% ticket handling, uh having over 400 custom detection rules and really uh stopping attacks uh in their tracks, right? It's about stopping along the attack campaign, finding two or three pieces of cyber threat intelligence from previous campaigns that could be rolled out in blocking mode, tactics, tactics, and procedures, and then hardening uh your active uh stance and the infrastructure in order to block those attackers and those attack campaigns. What's interesting is um I'd say that was highly effective uh and really still is effective, except for this idea of the supply chain hack in the NPM world, um, where not everybody is following the same speed of attack, or as we say, speed of threat. Um and that's where you can have a supply chain uh vendor inside of your perimeter, or it could be your GitHub or repo become compromised that you're drawing from. So this speed of attack has been very successful, this way of thinking about threat informed defense shifting left and the speed of threat rolling out custom detections. Now we have to think about it in a much faster way. I think we saw the remote access trojan of Axia um access in 90 seconds after download. Uh, we saw the attackers uh really focus now on mass precision, which is something we haven't seen before. And uh in the last uh Clud attack, we really saw six minutes in a lot of the NPM repositories being compromised and then injected with uh malcode. So um from that standpoint, things are going great. Uh CISOs uh continue to uh be needed and have uh wonderful jobs. Um from the attacker standpoint, they've really shifted now from either standard credential harvesting now into long tail attacks, uh, to obfuscation attacks and to injection attacks, uh, where they're really um credential seeking, but also you know, looking um not so much to sell those credentials, but what can they uh further use those credentials for in subsequent attacks?

Robby Peralta 4:58

I know that you're no longer in the crypto industry, but I just I forgot where I heard this, but people, somebody was saying that there's no like safe place to hold your crypto anymore because everybody just no matter where you put it, it's gonna it's gonna get taken. Do you I know you can't say that you agree with that, but can you maybe explain where they're coming from and how true that is?

Jeff Schiemann 5:19

Where they're coming from is in in the fact that uh out of all the crypto losses in 2025, there were about 3.4 billion losses due to either exchange or private key losses. And there had been this theory that if it's not your keys, it's not your crypto. Um, and until 2025, that was really pretty much the the holder's uh uh axiom was unless you hold it privately, privately held secrets, that it's really you're holding in some infrastructure in um in uh escrow in some way. Well, out of the $3.4 uh billion dollars hacked last year direct from uh exchanges, um all of the exchange money was returned. So the exchanges are keeping people whole in terms of losses. This goes to the Bybit 1.5 billion loss. Um the exchange was able to cover those losses over their 10-year profitability. Um, however, there was one key loss that was about a half a billion, 500 million loss in private key. And that private key wasn't covered in terms of loss. So, what we've seen in crypto, and this is happening everywhere uh in the financial sector, is we've seen basically regulators assume that in these new financial products, the uh company will keep those customers whole. They will be remunerated in full value, not partial value, not some sort of insurance 10%, 20%. But if it's the exchange that had the loss, the exchange is required now or in many jurisdictions to cover full loss or will be by the end of 2026. There's regulation pending. And now the risk comes if you lose your private key, you have to cover your own loss, which of course nobody can, right? So there is a big shift uh between not your keys, not your crypto to now holding things on uh crypto exchanges to make sure that those exchanges will cover the loss. And the Bybit was an exemplary uh supply chain attack, right? Uh so even though what we saw in 2025 uh was new and large for the crypto world, it has really become the standard uh post-September 2025 and the first MPM hacks that we're continuing to see here of compromise third-party suppliers, visibility over the entire attack chain uh by the attacker, and then code injection at the most important part of a crypto transaction, uh, compromise of the contract, taking over a poisoning of that contract, and then executing uh with substantial risk to the company all of that money out of their um out of their exchange within a few minutes, uh, a few exchanges of the of the Ethereum contract. Um, so what I've seen before in crypto is exactly where we're headed, uh, where we predicted in 2024 the tools that we would need are helping us greatly in 2026. Um, and now we're seeing that attack space shift uh into a faster uh ODA loop, as you said. You know, observe, organize, decide, and act are still our key metrics. And what are the tools that get us to see the ODA part of that, the observe, organize, decide act loop that we need to work within?

Robby Peralta 8:15

A lot to unpack there but if I understood you correctly, I I actually was completely wrong. Like I was under the impression that don't give your crypto to somebody else because it's gonna get stolen. And you're saying that actually, yeah, even if that happens, you get your money back. But if it gets stolen by you, you're not ever gonna get it back, which definitely isn't

Jeff Schiemann 8:31

exactly until 2025. That was the axiom, not your keys, not your crypto. People held it privately and believed privately private security was the best security. Uh, even keeping things in cold storage, uh, and you know, keeping things off offline or you know, double cold as they would call it, not just one key or signing key. But the real shift was um, you know, in crypto in 2025, this large theft, uh, and the regulators you now are coming out with increasing rules that, you know, losses on exchanges or crypto losses need to be fully covered uh by the institutions, which of course raised the liquidity and capital requirements of those institutions. It's pretty amazing that a company could cover 1.5 billion in losses through past profitability, right? Um, but there there you go as uh financial institutions and risk.

Robby Peralta 9:17

I'm not sure if you've seen the heist by the BBC where they're going through these really complex, fantastic attacks. That's what you were defending against, right? That was where

Jeff Schiemann 9:27

defending against APT 20, 28 and 38 uh North Korean nation-state attackers who were focused on um, you know, uh monetizing their attack chain very quickly, which to some extent until February this year was a was pretty much the MO, pretty much the campaign from data extortion to ransom to monetization, crypto theft. But we saw the striker hack, right? The striker hack was 100,000 deleted uh devices using Intune with no ransom. It was pure destruction, right? So we're starting to sort of see a shift between, you know, people uh who are working at the nation state, uh obviously this is due to the kinetic activity in the Middle East currently, but shifting from, you know, having to protect for just monetary loss, uh, whether that be directly for crypto or data extortion or ransom, to actually have now having to protect infrastructure for infrastructure loss or data destruction loss, as we saw with the Stryker Medical Company.

Robby Peralta 10:24

Can you just tell me what it was like to be inside of the company at that time when all these attacks were happening? Like how was it to be a CISO as your last the last two years?

Jeff Schiemann 10:32

Yeah, so boards don't want to know all the things in the past. What boards are trying to do is predict the future. What boards are trying to do is say, what is our key risk indicators? So one of the things I set out was a key risk indicator that just said, you know, okay, I believe we're every nine months going to have some sort of compromise or report. And against that compromise and report uh stake in the ground, you know, every three quarters having something that we would need to declare, um, you know, how do we measure against that? You know, what are the type of threats we're seeing? What is the type of development infrastructure we need to put off new threats evolving? Um, and then measure and report against that in a KRI to the board. So a key risk indicator. And the KRIs are different than KPIs. KPIs are backward looking, KRIs are forward-looking. They want to know what the risk landscape looks like, forward looking, where are you prepared for that risk landscape today? Um, and that's really important for CISOs. Most CISOs are either too technical, uh, so they're they tend to come from a technical background of a head of operations or security, um, and they like to talk about the things that blink and spin, as I call it. And or some CISOs are very much compliance rated, so they're very much performance-rated, compliance checklist rated. When you're dealing with new and emerging threats, you need to be able to create your own KRIs. You need to have a board and executive that trusts you to create those KRIs, and like you said, then be able to go out to market and look. And particularly in this area where I was looking for this reduction of the observable um organization decidable act loop, the OTA loop, going to market in 2024 was an interesting experience. Um, going through all the vendors and saying, I need this, I want to do this with it, and I want to implement it by X date. Very few companies and very few teams were able to do that out there. Um, and really uh Nick's team, uh, the genesis of Nick's team is really that team that was able to provide it for us uh in the crypto world. And it came down to uh the ability to operationalize what is a forensic artifact into cyber threat intelligence and then deploy it into detectionist code as soon as possible. The equation that birthed that was um a successful attack by North Korean actor in 2024, and uh, when we saw that time was of essence and time was going to be the thing that was gonna push us all. So the reduction from weeks to days to hours. I would say now we're still in the days, you know, we're under 100 hours, we're at about three days worth of research and uh CTI creation to do threat hunting campaigns, but that's far better than where we were uh two years ago. And now you can uh subscribe to services out there where if you're not the first attacked, you know, if you're not the first one they move against, you will have the ability to deploy uh reasonable detection technology and threat hunting um scripts in order to see your vulnerabilities and hopefully harden them before the attacker gets to you uh on the list.

Robby Peralta 13:33

Key risk indicator. I've never heard that term before. Uh is there anything else you want to say around that?

Jeff Schiemann 13:40

Yeah, so key risk indicators are very important for boards and for executives. And and what they're gonna tell them is the landscape, uh the radar, what's ahead of them. Think of it like a regulatory uh change radar, right? You your illegal department, regulatory department, compliance department are looking at laws, regulations, things that are out six to nine months, 12 months. They may be in comment uh out for public comment, they may be in private comment with their legislators, they may be in the implementation stage, like the CRA, the Cyber Risk Resiliency Act, or the AI Act in the EU. So they all have to go on a radar. And the same with a KRI. KRI looks at uh what is the company's current capabilities? You know, what are you defending against? What are the threats in that threat profile? And then most importantly, looks at what's the development of the attacker's profile. You know, what is the attacker doing? You know, when when um APT30, the Lazarus group, uh stole 1.5 billion from Bybit, there's an X assumption that 900 million or a billion or 1.2 billion is going back into RD, right? Like they're just not taking that money out and buying Ferraris with it, right? They are putting it back into their infrastructure to monetize and accelerate the attack surface and the infection vectors for all of their next campaigns. So, really creating the KRI is explaining uh to the board what it is as a perimeter that we can defend against, how successful that is, because if you think you're going to have one attack every nine months and then it becomes 18 months, you've had non-reportable attacks because your um your controls have worked, right? Your detections have worked. You're proving that you're defendable at that point, that uh you have a good defense, a good dynamic defense, and you're able to improve and keep up with the attacker. In the instance where you don't keep up with the attacker, well, that's also a learnable event. That's also looking to see a teachable event, what is going on in the space where the attackers are moving to. In early 2025, we saw some of this in um code created by an attacker, bespoke code created within about three days of their initial penetration and infection. So we are seeing code already move from two, three, four weeks for bespoke code to people running code now within, you know, like I say, a hundred hours within about three days. And we provided some of that intelligence to law enforcement and obviously generated uh CTI over that, and also gave that then back uh to the financial services community by by looking for sort of how to accelerate the CTI into detectable code. So all this exists. KRA needs to be forward-looking, it needs to be like a radar, it needs to compare your internal um readiness and capabilities with what is developing on the horizon from the attacker. What is typically used at the board, which has very little value, and I've heard lots of CISOs say it, is you know, attacks are continuing to increase in complexity. Well, what does that tell the board, right? It doesn't tell them anything, right? Telling them the past number of attacks or the past complexity is is a KPI. It's a performance indicator. You know, our past attack level was 52 in the last year, and be the previous year was 36, and and the complexity was across maybe identity, right? As a surface. What you want to put in the KRI is we see three or four, you know, emerging new attackers. We see that their techniques, tactics, and procedures are changing, and we see them moving away from identity and forecasting the next six months or one year that they're going to move into software or supply chain. So, yeah, it's about forecasting, it's about understanding the type of changes and communicating that readiness for the board to make a risk decision. Um, risks don't say static, and neither should your projections, the KRIs, they should be future laden, future looking, uh, like a risk radar, and then you know, different from the KPIs, the performance indicators, which are more backward looking.

Robby Peralta 17:38

So, two of the things I want to pick up on um, you know, understanding the threat, what is the attacker doing? You said as long as you're not the first, you have a chance to defend against that. Uh, what does that look like in your world from uh from where you're sitting?

Nick Maeckelberghe 17:52

So it's a lot of indeed uh understanding the attacker and just in CTI, knowing uh exactly uh what is going on there. But before I answer, let me just uh take a step back to to pick on what Jeff initially said. So uh in the beginning you said, yeah, Jeff exactly knows what he wants, uh, and and I will never forget a statement from him saying, you know, buy what you need and not what the market is offering you. Yeah. So look, this is what I need, yeah. I need you to move to help me defend at the speed of which attackers are also attacking. I need you to help me get to the speed of trade, the speed of defending, because you know, he's in a crypto world, so different kind of of risk perspective, but nevertheless, a very good um, I would say, motto to live by. And so it it all starts there. It all starts, okay. Look, what is the threat landscape, which is important for my client, you know, which attackers are moving there, how are they operating, study them, ingest the intelligence, uh and then operationalize it. So then, you know, change the cyberton intelligence into a certain attacks, continuously launch these attacks in the environment, and then at some point, for every attack that doesn't get detected, you give them something which they can immediately operationalize in their uh in their environment, right? Something else that he said back in the days, which is reports are good, but reports are are over. Yeah, I don't want you to do point-in-time testing, I want you to do continuous testing because the attackers and the threat landscape moves, and I want you to move together with the threat landscape to make sure I also move my defenders to the threat landscape. So a report is very good uh to print off at the printer, and then uh the more papers, the bigger uh the stack weighs, the harder you can slam it on the board when presenting it, and the more budget you can hopefully get. And that's about the only thing which for for me reports are good for. It's to to show you what you've done, and the the bigger uh and heavier the paper stack, probably the more budget you can get. But for the rest, but clients are really looking for also in the speed of thread is this uh this remediation. How do I move from a 60-page report with 10,000 words to something which I can operationalize in my environment? It lands typically at the sock team's desk, and the sock team then, as we all know, they're all a bit uh overworked. And most of them they ask, Great, which burning platform do you want me to you know kick further in the back and where do you want me to fit in to translate the report into now operational defense?

Jeff Schiemann 20:02

And and Nick raises Nick raises great points here that he's a bit modest of. And this is Nick's greatest question, now what, right? You have a report, you have 500 pages things to fix, right? And what Crimson 7 really does is it provides the now what, right? It's not just the criticality, which you can see, you know, 500 things reported, 132 is the most critical thing to do in the next, you know, today or tomorrow timeline or next minute timeline, but now how to do it, you know, where does the detections come in? What's the starting point for that? And I think if you look across the industry and you look at almost every tool set, you'll have great reporting tools, you'll have good CTI out there, you'll have something, you know, part of this mix is being supplied in the industry, not the now what, not the fix it for me, like not the now where do we go type of starting thing. Most of these tools simply end in a report. And uh, and that drives CISOs crazy. Like the one thing CISOs don't need is another report, right? What we need is the now what answered.

Robby Peralta 21:03

So back in the day, last time we talked, I'm just imagining how you would have done uh Crimson 7 or done this back in the day. They would observe an report, they'd break it down, uh, they'd maybe run the attack in your environment, see what's triggered, and then they'd write the code if if it was not triggered. That was a loop that used to be seven to ten days, now you're down to three days, if I understood you correctly. But along comes new technology, LLMs. What does that look like for you now, Nick? How are you using or is that like how has that process changed back then and how is it looking today?

Nick Maeckelberghe 21:35

That's exactly I think one of the main reasons why it goes down from seven to three days. Yeah, seventy-two hours, is like you say, then at some point ambitious, of course, to do it like a same business day, yeah. So a couple of things we see is uh most of our clients they already have intelligence uh in in some uh shape or form. And then also my question is always yeah, great. And also that how do you operationalize that in your environment? Nine out of ten times when you really dig Down uh under the hood of what they're doing with it, it's uh ticket enrichment. Yeah, it's not really that they are doing uh practice things about it, doing threat-ending about it, or or creating certain campaigns. It's really a ticket enrichment we're doing. You also then ask the question, yeah, great. So, how much of the uh hits do you get on your tickets for it? Oftentimes it's anywhere between 10 and 30 percent, not more. So it means 70% that you get is about noise, right? It's a big pipeline of CTI, millions of I0Cs, it's mainly noise. So, what we have built is uh we work together with a couple of partners who give us procedural intelligence, which is uh very important because you have to imagine a defender has a budget, but also the attacker has a budget. Yeah, anything that costs us a bit more of time, we we typically move on and move to the next thing, yeah, which is highly likely to succeed. So the most easy things for attackers to change is let's say compromise domains or IPs or hashes or the tools, yeah. This is very uh easy for them to change. Uh what is more difficult is the behavior, how they move from initial access until data exfiltration. They typically have a script or a procedure, they might use different tools or different techniques on the way, but more or less their modus operandi across that uh chain stays the same. And so if you can understand that as a defender and also as a company, and you change the way you provide your tools and your services to focus on attacking behavior, or if we also like to call it procedural intelligence, this is where the key is. Yeah, this is what clients can defend against them with the proper uh attacks, I mean the proper uh defenses.

Jeff Schiemann 23:32

Yeah, Nick raises a good point also about red teaming, uh continuous purple teaming, threat-led penetration testing. I know um vendors, I know customers, I know companies in the market who spend a whole year trying to figure out what the intelligence is for their threat-led actor uh penetration test, defining that intelligence, defining the campaigns, defining the step-ups, literally nine, 10, 11 months before they get the report back. Like that is not moving at the speed of threat. You may find out what happened in last year's campaign, and we're now changing campaigns so quickly, easily monthly, if not quarterly, the attackers are changing the vector and their the amount of exploits that they're using, what they're using to attack after they harvest their credentials, what they're doing follow-on. Um, so one thing that's happening is in this continuous purple teaming is not looking for a 12 to 20 week window with three of your corporate engineers to then go do your TLPT with your vendor and to design the intelligence and to build the reports. It's saying, now let's drip in every month. Let's look at the intelligence every month that's being generated. And I think that's going to go even to bi-weekly rather than just monthly. But you know, let's look in the last 30 days. What has been the attacker's pattern? What have they developed? What is our new uh detection improvement that comes through that? And that's that purple teaming drip process. So you go away from having a big bang approach, one report to thump on the table every 12 months, you're behind the attacker's campaign. You then have to re-establish your intelligence and your threat intelligence every time you profile a new attacker. Nicks team has created a tool that allows you to select attacker, profile that intelligence, the latest intelligence very quickly. Uh, we're talking days, not weeks and not months, uh, and then look at the attack chains and then profile the detections you already should have, put those in place even before you start the testing, and then run the latest test and figure out where the step-ups would be or what type of detection response you would have. And that's usually what I've seen when companies have gotten hurt, is they've they said, Well, we had a security report done last year. Okay, the attacker moved on, right? His TTPs have improved, they've moved in a different direction and since the last report. And if you only get a yearly report and you're fixing things only then on a six-month or yearly timescale, you'll be 18 months behind the attacker's latest tools. I think what we've really seen in 2025 and continue to see from to there today is the attackers are moving now quarterly, monthly. They're really exploit um very fast, uh, as faster than we can, of course, fix things, patch things, but we now to keep up with them uh at that speed to really learn the intelligence at that speed and deploy what we can uh in terms of tech detection engineering. Um, Nick also mentioned the idea of a budget. The attacker has a budget across the MITRE attack. Now, there are 15 columns in the in the latest MITRE version 19.1 that they've moved away defense evasion into a stealth column divided in half because it's the most propagatory uh prolific column now. Um, by doing that, the attacker has to decide where he's going to spend his time, his budget. He wants to stay under the level of detection, but he needs to laterally move, he needs to credential access, he needs to remote uh credential authority, he needs to exfiltrate, he needs to download beaconing. All those things have a cost to the attacker. Every time you roll out a new detection, you're listening more with higher fidelity to the signals the attacker has in your environment. And if you can stop them at any one point, assuming breach, assuming he's in at some credential level, but let's assume he can't get off uh device. Well, he might be on a non-critical machine where you now you have him trapped, you have the signals, you can go in and clear that up, uh, clear that device of the attacker before you have a critical event or reportable event uh that damages somewhere your protective controls. So I it all comes down to how fast uh we can think about these things. Traditionally, TLTP and red teaming has been done, you know, increasingly too slow. And this continuous deployment of threat intelligence, continuous detection deployment is really what's needed to defend against the worst actors.

Robby Peralta 27:42

Um Nick, we talked about the concept of headless security services. And I was at a conference the other day with my colleague, and I said, uh, you know, what's mnemonic gonna look like in 10 years? And it was a joke, I think, but it also kind of scared me. He said, We're just gonna be a skill in CLOD. And I was like, Oh shit, that's a very that's a very scary thought. And headless security services, for anybody that hasn't heard of it, I'm not sure if I've understood the concept, but it's basically that your product or your service can be put into an AI and it will do the job that the product or service would have done, but using Claude or something like that. How do you look at the the future of security services and everything we just talked about, given the fact that maybe the the employee of the feature is maybe a you know, maybe it is Claude and not a person?

Nick Maeckelberghe 28:30

Interesting question. So I think that what you're saying right now about headless security, we also call it prompting, uh internally the company, the prompting file. It's this is going to distinguish good cybersecurity vendors versus average ones, because and a very simple question to ask to any technology vendor trying to sell you a solution with AI is great, can I hook up my own AI and how much will or will not the results or the performing state the same, right? If the answer is no, you have to use uh claw or copilot or whatever because we use it, I'm very cautious of my wording because this is still very new, but it's very safe to say that they rely heavily on the open LLMs. So really companies who are able to tell to you, look, here's our solution, and you can hook up your own AI. It means this concept of headless security, it probably means that they have somewhere a secret source file, as I as I like to call it, where they have dumped all of their intelligence in it. If you look at the detection engineering, they've dumped all the intelligence in it saying, okay, here is how a detection rule should work and should should be written. Here are the syntaxes, here are the the tables which come, for example, from Sentinel. Now, just to name one, here's how it should be, like, and here's how it should be with technology vendor X, Y, Z. Really a prompting file, because if you have this experience and you know how it should be, and and it will essentially redirect your product first to that file, and then it will go, of course, also to the AI for the for the heavy bulk and the massive work. So I think that is a concept which is very important. It will distinguish a lot of cybersecurity vendors versus uh, let's call it the vibe coders who just saw an opportunity in the market and said, let's build a product out of it, or people uh who are experts in their fields and they have decided because of their expertise, let's build a tool, an automation, because I see the the markets needing it. And and this concept of headless security is embedded in the environment because it's uh it you know it shows the quality and it will uh reduce the hallucination. So I think it's a very uh interesting thing, and I think that companies should should, if they're not busy with it, they should really invest in it. And it's the same for for clients, yeah. If you have, like I said, AI is an amplificator. So if you use AI, whether a vibe coding tool not in your environment, work on this concept of headless security, dump all of the knowledge of your you know analysts in there, saying how it should do, how your environment works, because it's tailored to your environment. So then you can the AI goes first, let's say, there before it goes out of the let's say the uh the big AI uh model to get all of the other things which you might or might not have missed.

Jeff Schiemann 31:02

I would add to that just the the feature-gazing part of this question. Like we currently know today, roughly half of all internet traffic is coming from a bot or AI agent, right? So human human, human to bot, about 50-50. In in a year, humans will become a rounding error, right? We can see the projection of how many agents will be out there, agent-agent, agent communication. So we're gonna see that explode. We're also gonna see the development of what we're now calling neo cloud. And this neo cloud is the releasing of GPUs uh for AI models by cloud vendors. So, you know, um Facebook is probably meta is probably gonna be the first one to do this and look at releasing cloud space, as we've seen before with cloud technologies, but particularly for AI model hosting. And as soon as you get into this releasing of the AI model hosting area of the neo cloud, you have all the initial questions like we had with digital cloud. Is it infrastructure as a service? Is it platform as a service? Is it software as a service? So then you get people into the security of the cloud infrastructure, which is what Nick's talking about, how to secure the special sauce. That gets into the weights and the profiles of all the LLMs. And I think that's where people can quickly make a managerial decision today whether or not they should adopt an LLM or not adopt the LLM, is how much control over the weight do they have? If not, you're training a system that you have no control over your information going into, and you have no control over the information coming out of, or at least clearer transparency of the information coming out. And then you really have to wonder what's the value going to be in that system in 10 years or in five years, or you know, could be in six months. But um I really think those are the things that we're facing. We're facing more and more um AI agent to agent uh through MCP communication. We're facing the new neo cloud that's going to come where we're gonna have the same issues as at infrastructure, as a platform, or security, and we're really going to need to know about the waiting of the agent. Um, there are reports already, of course, of AI poisoning, uh people coming in and redoing the AI agency. So, you know, AI weights, I should say. And then that gets into this whole question where what is the version of truth put forward to your security analyst from the AI versus your security manager? Is it the same version of truth? Does your AI manager not see the blinking light in his report that the AI engineer or the security engineer sees? And we're gonna rely on a lot of AI to generate that version of truth. I really think we're gonna move in the next uh year, if we don't already see it now, with the credential harvesting and thefts from the NPM packets. We're moving from just identity as the attack surface to trust as the attack surface. Now the question is who do you trust? Where did that packet come from? Where does that dependency come from? How do we know it's not malicious or how do we know it is malicious? I really think we're gonna move very quickly into a trust surface, um, not just an identity surface area.

Nick Maeckelberghe 34:08

So I think the world of SaaS as we know it, with which is you know, uh clients logging into fancy dashboards and telling them what to do, or showing them all kinds of interesting menus and great databases. I think that world is slowly but surely coming to an end. Yeah, I think as as clients are adopting already companies adopting AI, developing their own MCP servers, they want to be literally talking to your MCP server, no more dashboards log in. Um, you know, one dashboard for uh your SIEM, one dashboard for your bus tool, one dashboard for your threat-tenting tool. They all need to be uh working with each other, connecting with each other, API skins need to get uh maybe traded off. Why? Just all MCP to MCP, you connect all of your technology vendors, their uh MCP uh server to yours. And then when you ingest their data, which you need, you chose them for the quality of their data, not their fancy dashboards, although sometimes you also have you know user experience is also things, uh I guess. But you give the context to your uh MCP server, this is why it stands for. You ingest all of the MCP data of your chosen technology providers, and before it, while it gets in, you still already have maybe some automated process. If you ingest this kind of attack playbook, here's how you should behave in my environment, here's what you should do. So I think this is really where we are going, where uh this AI to AI uh agent that will um uh keep talking to each other. Yep.

Jeff Schiemann 35:24

I tell CISOs to think of this world. You access uh your resource every day, uh, you go in and you interrogate a baseload of questions, and it gives you the portal for those questions, right? It gives you the data you need for the perspective that you have. No longer do you build that perspective you have based on the amount of data in the system. The system now encompasses all the data that's possible in your environment through MCP uh server hosting and connection hosting. And now when you ask it a question, it's able to formalize that portal for you upon your question, upon your perspective. Um, that becomes challenging for CISOs because we've never had the questions sort of be unlimited. We've always said, well, this is my identity system, so I'm gonna ask it identity questions. This is my CTEM system, I'm gonna ask it exposure management questions. This is my antivirus system, this is so-and-so, right? The system of systems. Well, now we're coming to a system of decisions, and that's a much uh higher order of thing to start to think about. It's gonna have a lot of benefit uh for those who can think a system of decisions versus a system of systems, but it's also gonna have a learning curve. And you know, what we see is the attackers are adopting it much quicker um than the defenders are, as has been the history with cybersecurity. We're as close as we've ever been to catching them, we're also as close as we've ever been to losing them, right? It's it's uh it's always a speed race.

Robby Peralta 36:51

Ah, I want to keep going. I have so many more questions now that I had going in. But uh any final words, gentlemen.

Jeff Schiemann 36:57

Yeah, so uh the speed of threat is continuous, it's uh always improving, it's always increasing. We're just trying to keep up with it. In some cases, we have a chance to get ahead of it, but we have to start thinking uh today about what we're gonna do tomorrow. And like uh I told Nick several years ago, you know, don't buy what the market has, buy what you need. Most CISOs are looking at the market and saying, you know, how does that solution set work for me? And they're not coming to market first. What is my problem set, and come to the market with a problem set and then deal with a vendor who can provide a solution set. Those are really some uh fundamental things that CISOs need to get over uh and figure out how to do in their own environments and their own business. But um, in order to move at this speed of threat, and we see it increasing, and we've always seen it increasing in terms of complexity and in terms of uh number of threats, um, we really need to have a longer-term version uh and vision of what it is we want to deploy. And you can only get there by deciding how your organization is going to be structured, you know, two years from now. What is the threat you're trying to defend against six months from now? How do you get between that six months from now and the two-year uh vision of that?

Nick Maeckelberghe 38:09

So speed of truck is not a product, it's a mindset. It's much like zero trust, right? It's a framework, it's a methodology you you apply it. It's gonna be for every company a bit the same, as long as you indeed adopt it, that you you know have the ambition to defend at the speed which attackers are moving. Um, in in terms of budget and like uh buy or or build local AI, uh greater AI models, but use it to empower the skills you already have. If you don't have the skills, buy it, MSSP, any technology provided, but use AI to help contextualize what you need from it, how it needs to be delivered, how it should look like. And and lastly, also what Jeff uh mentioned uh about the mitra. I saw companies during uh my tenure in cyber spending thousands and thousands or millions of euros on the left side on the minor the initial access and your security awareness trainings. That's great. Not to be gonna say stop doing it, but you know, like Jeff said, uh it's okay if you can stomp the attacker at any point in the chain. It doesn't always need to be the first step, that is great. But we saw clients with uh a great security posture at you know the first two or three, but once we got in, and it was super easy to do later movement, so spend the budget, spread it over across the entire uh chain, because it's okay if you stop them at any part in the chain. The sooner the better, of course, but any part is okay. So spend the budget, buy what you uh don't have, and build internally with AI what you have.

Robby Peralta 39:37

Thank you so much for sharing all your expertise. And uh I'm already looking forward to our next episode, gentlemen. Thank you so much for your time today.

Nick Maeckelberghe 39:44

Thank you, Robby.

Jeff Schiemann 39:45

Thank you.

Robby Peralta 39:45

Thank you. Bye. Well, that's all for today, folks. Thank you for tuning in to the mnemonic security podcast. If you have any concepts or ideas that you'd like us to discuss on future episodes, please feel free to hit me up on LinkedIn or to send us a mail to podcast @mnemonic.no. Thank you for listening. We'll see you next time.