mnemonic security podcast

When security hits the fan

November 16, 2020 mnemonic
mnemonic security podcast
When security hits the fan
Show Notes Transcript

Morten and Robby recorded this session as part of their virtual presentation at the CERT-IS conference in Iceland last month. The episode is also available in video: https://youtu.be/Izfb7-wA_0I 

For this episode, Robby welcomes Morten Weea from mnemonic’s Threat Intelligence team. Morten is a PhD candidate researching decision-making in incident response and an experienced Incident Handler that often works with advanced persistent threats (APTs).

Robby picks his brain about what actually goes down when a customer calls after realizing what shouldn’t have happened, has happened.
Or even more importantly, what his advice is for organizations before a serious incident occurs. They also discuss when it’s appropriate to trigger a full-scale incident response, and what sort of incidents shouldn’t. 

Technical level: 2/5

Host: Robby Peralta

Producer: Paul Jæger

mnemonic:

From our headquarters in Oslo, Norway, and on behalf of our host Robby Peralta. Welcome to the mnemonic security podcast.

Robby Peralta:

When the proverbial sh*t its the fan, who you gonna call? Back to a serious note, sometimes that call is in vain, as there is nothing even the best of Ghostbusters can do to save the day. But if you could ask a Ghostbuster for advice before a cyber drama occurred, what would you ask? Hopefully some of my questions will overlap yours because I brought in one of the mnemonics very own Ghostbusters to pick his brain about what happens when the proverbial security hits the fan. Morten Weea, welcome to the podcast.

Morten Weea:

Thank you.

Robby Peralta:

We are in Iceland right now, say hi.

Morten Weea:

Hello Iceland

Robby Peralta:

But Morten Weea, who are you and what do you work

Morten Weea:

My name is Morten and I work with threat with? intelligence in mnemonic. But th t's not where I started. I st rted some eight years ago in th information security bu iness, and I've been working th full cycle of incident re ponse since then. I started my climb in the security op ration center, detecting and as essing incidents and alerts wh re I worked for one and a ha f year. As I said, I analyzed th events that we got on the sc een, real time 24/7

Robby Peralta:

Good ol SOC work.

Morten Weea:

Yes, the steady work horse of every business. And I also came up with some recommendations for how to fix he problems. So that was how t all began. And then I move on to governance risk and comp iance

Robby Peralta:

Out of the SOC into a suit.

Morten Weea:

Yeah, more or less. I was not focusing on the technical things. And the more I was doing more management, the work in helping your management understand why information security is important. Basically tried to make them prepared for any instance that could happen with having sufficient processes and procedures and good information security foundation within the organization. So after some years there, I ended up in threat intelligence, where it's awesome, what I'm doing now. So I work on the sharp incidents. But though I only work on the incidents that weren't to the full blown IRT, so not like everyday operational incidents, but where the shit related has hit the fan.

Robby Peralta:

APTs

Morten Weea:

Yes, I'm gonna have to move out and fix the problems. Yeah, like superheroes. '

Robby Peralta:

Sexy, very sexy.

Morten Weea:

Indeed. So what I mainly do is incident management, making sure the real heroes analyst gets what they need, when they need it. And they also work as a middleman between client management and technical expertise, making sure the technicians are shielded from the ties and translate the nerd speak to business speak and vice versa. Hmm. I kind of like doing this because I have a background both in management and technical background, with the master thesis in, in our master's degrees in informatics, and also one in management.

Robby Peralta:

Wow, so you like school,

Morten Weea:

I like school,

Robby Peralta:

You must really like school because now you're a PhD candidate.

Morten Weea:

Yes, I'm a PhD candidate researching decision making in incident response, or more precisely, how to make sure decisions taken in incident response, are the right ones in the right time. So basically, make sure that you do the right things as fast as possible.

Robby Peralta:

Let me just stop you there, that that's really important, because in a incident response situation, you know, more? Yeah, more than anybody else, how decisions get made just very hastily, not very seldomly the right decision to make if you're doing it really quickly?

Morten Weea:

Well, if you're basing it on a gut feeling, it's a good chance it's the wrong decision you're taking. I'm trying to prevent people

Robby Peralta:

from trusting their gut.

Morten Weea:

Yeah, well, not necessarily. Because sometimes it's good to trust your gut. But you know, if you have someone on your team that you give more face value, for instance, and then you end up trusting him when his information isn't that good, necessarily, and then trust him over someone else with better information. And that makes your incident response not as good as it could be. So I'm trying to figure out how to not discriminate based on, you know, friends,

Robby Peralta:

and organization matters. That

Morten Weea:

shouldn't really matter. And that's no easy task.

Robby Peralta:

Right? So, incident response, explain your definition of it.

Morten Weea:

Well as in response, could be could be viewed as an organized approach to addressing and managing the aftermath of security breach or cyber attack. You know, That's not necessarily a good definition of instant response. Because there are so many attacks, that shouldn't be a problem for people, you know, like having scan or, or whatever going on on your network, you have to be able to identify them, because it could be the first step of often the time. So you need to know that the scan is ongoing. And then you need to, you know, have this logged somewhere so that when when they escalate their attack, later than you, you're able to connect, connect the dots, what scan was the one got coming, coming before the attack in this case, you know, but that that shouldn't necessarily warrant a full blown IoT response, and should be maybe call more like this operational incident, but the soccer whoever is working on first line is able to deal with cope with you should have some playbooks or, or whatever that defines these kinds of things. And maybe a little, surprisingly, ransomware should be one of those as well. DDoS should be one of those as well.

Robby Peralta:

I'd like the ones that you wouldn't define as incident

Morten Weea:

response. No, it shouldn't really, you know, trigger the full blown incident response where where, you know, you have this incident manager, Adler, information manager, analyst, communications team, No, you shouldn't have all these things in place, when you're hit by ransomware. Because ransomware, you just shouldn't shouldn't pay, you should just restore from backup. And if you don't have backup, then you have a huge problem. You know, so when, when you first hit with ransomware, then it's not much we can do. It's not like we're going to break the encryption. It's not like we're going to fix something that's going to be very costly. And it's a lot cheaper just to have, you know, good backup, good working backup.

Robby Peralta:

So somebody calls us right now and says, Hey, we got hit by ransomware. We don't have backup, what do we say? Tough luck.

Morten Weea:

Really? Yeah, more or less, because nothing we really can do. You should just get the ransomware out and plug the hole. start all over again.

Robby Peralta:

If I can ask you then what do we you know, what are people calling us about? You guys are so secretive in wires, you never tell me anything? Tell me good thing?

Morten Weea:

Well, our clients are calling us about mostly anything that they don't have the competency of, or skills, or no resources to fix themselves. Or they they have some resources, sometimes some skills, and they want to fix a big part of it does come do you know, a system in incident response? Some helping hands? Yeah, helping hands on depending on our availability, and, you know, the seriousness. And then of course, we could come help them. Because this is what we do. We're professional incident responders. So we fix these kind of things. But But mostly when when, you know, the Power Rangers or whatever, yeah, we're going to call ourselves really need to work is when we are dealing with the advanced persistent threats like nation states or very advanced crime syndicates. So

Robby Peralta:

and is that because you don't want to kick them out, basically knew they want, you know, are they?

Morten Weea:

Well, eventually, the goal is to kick them out. But it's not necessarily that we are able to kick them out, or that we should kick them out just immediately, because we don't know who we're dealing with. We don't know what they want to get out to our clients say say if we, we have found them in one segment of the network, doing some kind of recon or whatever, and then we just indicate that we know that they're there, but we haven't necessarily seen them in this part of the network, then they could just go under the radar hide, and just let that part be kicked out. And then they could keep on operating in the part of network where we haven't identified them yet.

Robby Peralta:

What sort of actions would we do that would let them know were there? First of

Morten Weea:

all, we could, you know, start plugging the holes that they are exploiting? Yeah, you know, out of the blue, just shutting down there, what we have observed the way in so we could also do like, stupid things like patching the vulnerability in front of them or, you know, start cleaning up after them, you know, leave traces because it's not like you've seen the movies where they sit in a basement in the in the hooded sweater without lights on, by being like very fast on the network, or the keyboard, you know, just exploiting in shouting, I'm in No password breach felt that. Yeah, I mean, and that's not exactly how, how it works. Because these are regular human beings, they are at work, they work nine to five, and they have holidays off, you know, we see on Chinese New Year or whatever. And then we see the activities on the decline. And then they come back, rejuvenated and help them feeling well,

Robby Peralta:

like the second of January.

Morten Weea:

Yeah. But you know, they have this moving holiday. So it could be like in the middle of March. Yeah. And then we just see less activity. And the same with some, some crime syndicates, they have this summer off coding, where they just take everything offline, go to the beach. Yeah, no, not necessarily. They they fix the program. They patch it, they make it better, they improve it, and then they deployed when they're done. So it's not like they're having a vacation necessarily, but they they're organized, and they're doing things in, you know, during the summertime,

Robby Peralta:

it's because they know that we're not at work, and they have less opportunities window

Morten Weea:

or window opportunities. I won't necessarily speculate in why. But we see there are some clear patterns of when the advanced actors are, you know, not at work. And we see that they have they're sleeping, they're not 24. Seven, necessarily. They have different teams, you know, they they have designated tasks have to do. So when this breach team has come in, and they're leaving all the information to the the other team that's going through, you know, exploit whatever, they

Robby Peralta:

keep it going. Yeah,

Morten Weea:

yeah. Hmm.

Robby Peralta:

Speaking of phases, yes. What what are the phases of incident response? Like, how does it start? besides us getting a phone call?

Morten Weea:

Well, it starts before this. Okay. Yeah. Because it starts with a planning and preparation. Yeah. That's the governance risk compliance part of it, where you have to prepare, you have to, you know, make some what ifs? flowcharts, what to do when this happens, who to escalate to? What's the escalating trigger point? Who has the mandate to do something about this?

Robby Peralta:

And what percentage of organizations actually have that in place? Because I've been hearing this for forever, that apparently, it's chilla

Morten Weea:

mentioned in their minds, everyone. But when, when things get serious, that's when they see that whatever they have in place is not sufficient, or it produces bottlenecks or,

Robby Peralta:

or whatever. That's the point of like, tabletop exercises, right?

Morten Weea:

Yes, that's great. And other exercises as well. Because then you see, it's, it's a lot better to figure out where things go wrong. When you have this safe environment, where I'm the bad guy, and not the actual bad guys, the bad guy, because I'm not going to leak your data punish you. Yeah. Planning. Yeah, yeah. And then the next one is testing and reporting. It's important to have visibility in your network, if you have, you know, if you have a lock on your door, but you don't know, you can see what's happening inside the house, or you can see when the door has been opened, who opened the door, whatever, all you know, when if you leave the house and come back, is the status when you left, understand this when you come. And then if things are as they were when you left, and you just assume that no one was there. But you will know, for something is missing. And they know that someone has been there, but you don't know who you don't know, when you don't know what else they did. You know, so detection and reporting is important to have visibility. So the next phase in incident response is assessment and decision, which is the face where you have to assess what, what has happened, you know, the doors are maybe opened in my house, I got an alert, not nothing's missing, I'm assessing the situation, is something gone, there's something broken, what has happened, if I assess it to be not that serious, then I wouldn't necessarily call the police to come to my house, they'll file it, you know, for some that it's some insurance claim, yeah, something legal happen. So, so I need to file it somewhere. And then the response, and their response should be proportional to do my assessment. So say if I figured out that someone came into my house to call my jewelry to call my computers to call my money, and I would need the police to come and investigate. Maybe they broke something as well. And now we have this huge case. So we should figure out who was it and how can we you know, return to normalization. So the last face after the response is the lessons learned and lessons learn. means what did I do wrong to invite them These people in what could I have done differently? How could I, you know, prevent this from happening again. And if it happens, again, someone reaches the door, maybe I could hide the computer, lock the jewelry somewhere else, you know, make sure they don't get it, so they can steal it from the consequences of the breach will be no less severe. So those are the five phases of incident response, as I identified previously, the planning and preparation is for the governance risk compliance department. Detection and reporting is for the sock. And also the assessment and decision is also sock material. And then the Incident Response Team or dirty comes in in response, handling it, and then they deliver lessons learned, which, you know, kindness, back to governance, risk compliance. And on the side of all this, we have the technical, the technical solution where you, you know, we do penetration testing of yourself to identify where it could be some possible vectors to attack you, etc. You have operations to, you know, install appliances, fix things, you know, be prepared for when the shit hits the fan. Hmm. So what happens when the shit hits the fan? Hmm.

Robby Peralta:

Now, what does happen? When should its event?

Morten Weea:

Yeah, people call us and then, you know, I would like to say we push a big red button and it starts blinking. And, you know, we have these tubes to jump into and just on the way down to our Batmobile we get suited for for the incident. But that's not how it happens. Unfortunately, we have to do some kind of boring tasks before we can start, you know, the mercantile part of it. Commercial stuff. Yes. Do we have an agreement with the customer? Do we have resources to fix this? Yeah, you know, should we do this 24? Seven? Is Christmas coming up? Do we need to force people to be available for us to handle the incident? You know, these are people. So they have to, they have people considerations, and we have to make them as well. And if we don't have the resources, then we couldn't, we can't really help. The ones calling. It could be that we have already deployed all our resources on other assignments, or we just don't have, what resources they need. Because if they have this very specialized problem going on, we wouldn't necessarily need to have the specialized expertise. And if we have a lot of available personnel, but no one with that expertise, then

Robby Peralta:

there's no point. Yeah,

Morten Weea:

no. expertise. And also, that's, that's the the boring part. Yeah, yeah. And that's something you should consider at least a little bit before you decide to outsource this competence to someone else. Because if you don't have this agreement in place, before aronsohn hits you, then it's not guaranteed that you get the response you need when you need it. And that's, that's also a risk that you need to consider. In the preparation phase, it's completely fair to say we don't want this in house, because it takes a lot of practice resources to maintain, you know, the competencies. And we are, I think, 200 or so people that can handle incidents, and not all 200 have the same background or same, you know, area of interest that they want to focus on. So say we have a couple of reversers a couple of login two sets, you know, and they do different stuff. They, they could all be a part of the same incident but doing different things. Yeah, different things. And having having a reverser in house is like that's good that way. Yeah. But it's it's doable, but it's a if you're

Robby Peralta:

DNV are big. Yeah, yeah.

Morten Weea:

Because it costs a lot than that person or if you have more than one, you have this community where they could, you know, evolve. But if you have that thing, the incidents involved with reversing tasks, and they have a huge problem. First place So your goal should be not to have the nine for all these sources. So that's that's our niche because we go from customer to customer or client client and fix this so we can get the experience. We know how the the deputies evolve and see, you know, they're doing something said this this year, and then they're doing these things the other year, and then we have enough assignments to just have the people constantly updated. educated on?

Robby Peralta:

Is that a threat intelligence team and emoticons incident response? Yes,

Morten Weea:

yeah. Awesome. But part of the reason because we we have to know what's happening, and respond accordingly. And we are fortunate enough to be big enough and have enough assignments so that people could stay on top of the game. And that benefits us. And that also benefits. It's the clients that need our resources. Because we are, we're updated, or less.

Robby Peralta:

That's a stupid question. How long does this process usually take from, you know, customer calls us wakes us up says that we need to help. And then we say, Okay, cool. Tell us what's happening. They explain it, we call them back and said, Okay, we have some guys for you. They get on their PCs start working,

Morten Weea:

how long? Are they going to be there for others? Depends? Yeah, it really depends on the

Robby Peralta:

Do we have an average that we've calculated

Morten Weea:

well, as most of our clients, they just want to, you know, have our stamp of approval or whatever. So we could just move in, do the assessment and deliver the report. That's, that doesn't take long. But then on the other hand, we have the more serious cases where where we have, you know, an advanced persistent threat on the other side. So we will need to have more people to take more time. So but that's the the other end of the scale. So that could take years.

Robby Peralta:

And that does. We do exist organizations out there that are being attacked like that, right?

Morten Weea:

Yes. And that's an ongoing process. And that's more like a cat and mouse, know where we have. It's a nation state on the other side, they want to get some kind of information, and then we should just prevent them from from getting that information or that it shouldn't come out. Because that's the, that's a consequence.

Robby Peralta:

But if you're there for many years, what are you doing, like operating honey pots? And just like just confusing them throwing them off?

Morten Weea:

Yeah, cool,

Robby Peralta:

Cool, awesome. I don't even know what that entails.

Morten Weea:

yeah, it's, that's a good question. And we have to give the attacker something, but not something that's valuable. I'm not something that compromises the clients of the client, or whatever, you know, they should have some not important information. So but that, you know, we have to balance a little bit, is this is this compromising? Could this have a legal consequences for the client? And, you know, are we indicating that we know that they're there for the attacker? So, you know, we have to give them some but not all. And it's a it's a game of cat and mouse? Sounds? Yeah.

Robby Peralta:

That sounds li e so much fun, when you're no the person that's being atta ked? But by the way the communication part, that s an art of its own right? I fe l like honestly is the best pol cy, and it's weird because if I see a company thats like, yeah, we're getting hacked or under cy erattack. I look at them, lik, Cool. Thank you for tellin us like I automatically ave like, I sympathize for them That's probably because I ork with cybersecurity. Other p ople may not like that, like, h, that's a bad company. What d es mnemonic say companies shou d do? Is honestly the best pol cy, or is there a strategy

Morten Weea:

It depends. It's, we're not the communications expert. So I'll just have to, you know, wrap my mind around this myself. So but honesty goes a long way. So if you're owning up to whatever's happening, and then you're sharing enough information for people to know what's going on, then that's good. But if you're sharing too much information, then you're obviously alerting whoever is attacking that you're on to, onto though. And also, even, you know, the communication part is one way because you're not getting communication back. So if you're just telling everyone what's going on, then obviously the perpetrator is also getting the information that you're sharing. Yeah. So that's not good, necessarily. And the other part of this, this problem is, say it's the 12th time this year that you have been hacked, you know, should you just go out there so well, were hacked to get too bad

Robby Peralta:

Good morning everybody, it's Monday and we're...

Morten Weea:

yeah, you know. So if, if you're constantly communicating that you're being hacked, and you're, then you come across as somewhat incompetent, maybe learning from your previous mistakes, though, I think sharing the right information, the right amount of information in at the right time, crucial to do some specific assessments, hmm,

Robby Peralta:

Is there any circumstances where covering it up to be actually may be the best thing to do?

Morten Weea:

Well, you know, with all the new GDPR legislation and things like that, you're obligated to give information to authorities and the public and whatever within it, or whatever they call it. But they also have this exception where you don't have to disclose the information. If it's an ongoing investigation,

Robby Peralta:

Why doesn't everybody just hide behind that then?

Morten Weea:

Well, because you can't have an ongoing investigation for a year 234. You know, so if it's a serious thing, then obviously, you should hide behind the ongoing investigation part. For as long as you need it. But, but it's also good to share some information on what's happening. Know, if you, you're swift with the response, and just fix things and get them out. And that's your goal. And you can respond and then disclose, you know, because we have already fixed it, and then people are okay, okay, but it's fixed. So let's move on. But if you're if you're too early out there in the game and say, well, we have been hacked, and the advanced persistent threat now has access to all the pipelines. In the case of ATM, they could poison, you know, your water supply. But Have a good day. Yeah, we're fixing it, but we don't know who is doing this, or what they really want. You know, that's, that's not kind of good information. So we have to be a little careful with what you share and, and how, and consider the consequences of, of your information. Obviously.

Robby Peralta:

I see why it's a it's a fun job what you're working with here.

Morten Weea:

It's a it's interesting, huh?

Robby Peralta:

Do you have any, like, last words of advice? Do you have any, you know, with all this, you're studying this you're working with in real life. Somebody had as another podcast, but he said, you know, the whole point of security is like, actually, you know, the whole point of his SOC actually, was just when something happens just to clean it up, you know, incident response. So your whole security scheme should be giving you everything you need to respond to an incident. And that was an interesting way of saying and I agree,

Morten Weea:

I agree. Because I usually when I explain the steps in all the preparation, the, you know, detection, assessment, response, etc. I used to draw circles, you know, indicating each step in the circle. And then I used to know, this regular size to the preparation part. And then I come to the response, and then it's a huge blob. Because if you have a small preparation phase, then you have to do most of the Yeah, most of the job in the prep or the response phase. But if you're good at preparation, in know what's going on can hit you or how you should respond when it hits. And you have to have a lot less in response. So the goal is to have a perfect balance between these two phases where you have put enough money into the preparation phase. And then you wouldn't have to know open your wallet when a response comes out when the response comes in. And that's there are some things you could do to just start. And that's considered the different threat actors. Now we have the basic script kiddies on one side of the hacktivist in the middle on the new evolved crimes in the kids, more up to them, the nation states at the end, and they're like us, they're the horror, they could be seen as some kind of stare where the first step is this rickety. It doesn't have a lot of skills, it doesn't have a lot of motivation. The goals are, whatever comes along. And then the persistence is like once they're in, they're in the moving on. So if you get them out there, they're out. But that being said, you shouldn't you shouldn't let them in, you know, because these are not persistent at all they are, if the things they have seen online doesn't work, they go on them. Yeah, they wouldn't know how to, you know, escalate or evolve their attack. So they move just on to another target. That's why the preparation should at least, you know, indicate at what level do we want to keep the bad guys out before we need to focus or put any money into responding to this. So I would say the good good advice is to have enough in the beginning, just to get all these nuisances away, so you don't have to deal with them. You don't want to be owned by a 14 year old that just found some cool tools, and then exposed all of your data, you know, because you don't know who can pick up That data and say, if you have the pipeline's a script doesn't know what he's dealing with, he just puts it online to see it, look what I found. And then some bad guys could just pick it up from there, they don't even have to tack you don't end it, that kind of information. So so that's not, that's unfortunate. So you should just move up on the stairs until you're, you know, confident with not having to handle this, and you're prepared to handle, you know, the more advanced just

Robby Peralta:

to be a little more concrete here, what are these smaller steps like ransomware? For example, that's like one of those things that you shouldn't have to deal with. Right?

Morten Weea:

That's an attack that's on on the lower part. But I was thinking threat actors as well, because the threat actors, just foreign things are they they want to do some some harm against you. And then they have, they have different skill set the motivation, etc. So they are just evolving. So crime, crime, or crime syndicates, they they want to do, you know, they have this kind of goal, they want to make money on you, or whatever, no selling information that you have to other parts. And then the nation states, they have this one goal that they really want. So if you're a target of a nation state, then well is you're going to be on some way or another because they, they may, they will get it now. Yeah, but they make zero days, and they exploit zero days, and they get in and get information. And then you need the visibility to see them and know what's going on. And maybe learn from that attack to later. So we don't have to build enough preparation to withstand the nation state because you can can't do it. More or less. But you could pick off a lot of the less advanced attackers. But the DDoS is mitigated fairly easy. ransomware Well, it's not he's not necessarily prevented. But it's the consequences can be prevented that you just Backup and Restore. And, you know, time before the ransomware existed in network. So yeah, you have to do that kind of preparation. And also you should practice, practice a lot, tabletop exercises, role playing, red teaming, if you if you're advanced, play around, have fun. It's when when people come together and pretend they're the bad guys, you can see the imagination of your employees, at least, usually when I was going around and doing and doing it. revisions audits are my final question to my oddities where if you were the bad guy, and you had to do an attack right now, what would you do, and the competence and inside knowledge that most of these employees had led them to, you know, make up this extreme path of exploits that could just ruin the entire company in minutes, because they knew where all the holes were and where the vulnerabilities were. And, and having that competency within your own organization and letting them partake in a in a, you know, exercise could give you valuable information. And then it should just work with whatever information you get from the practice, and prepare, either digitally,

Robby Peralta:

I've done that tabletop exercise, when I was 22. I was at my one of my first, it was at ISF and was one of the first security conferences that I was at. And the guy just looked at me. His name is Hans Peter. He was like that was his job. His main thing was to do these tabletop exercises really good at it. He looked at me and he was like, you're the CEO. I almost cried that day. Because the whole room people watching you and making bad decisions is really intense. So it's, it was fun to look back and I learned a lot.

Morten Weea:

So yeah, it's a safe environment of messing up and doing the wrong things. As long as you're not you don't have a culture of exposing those kind of people punishing them, but rather learn from what they have not done the right way. Then you could improve the process or whatever, to make them do it right the next time. Because if there's something someone that makes a mistake, yeah. Rarely does is making the same mistake again. So if they already did it in a safe environment, and they're probably not going to do it. Hmm. When the when it gets serious, huh. Awesome.

Robby Peralta:

Well, Morten, thank you. Thank you for sharing your knowledge. I we had like some sort of a script but I don't know where it went Well, that's all for today, folks. Thank you for tuning in to the mnemonic security podcast. If you have any concepts or ideas that you would like us to discuss on future episodes, please feel free to send us an email to podcast@mnemonic.no. Thank you for listening, and we'll see you next time.