Kentik - Network Observability
More episodes
Telemetry Now  |  Season 1 - Episode 11  |  April 4, 2023

Data-Driven Defense: Exploring Global Cybersecurity and the Human Factor

Play now

Today's cybersecurity landscape is about less about knowing all about hashes and encryption, and more about understanding data, politics, and how adversaries operate in the real world. In this episode, TJ Sayers, Manager of the MS and EI-ISAC’s Cyber Threat Intelligence team at the Center for Internet Security, joins us to talk about data-driven defense and how the human factor plays a much bigger role in cybersecurity defense than we realize.


TJ Sayers
TJ Sayers
Manager of the MS and EI-ISAC’s Cyber Threat Intelligence team at the Center for Internet Security

TJ Sayers is the Manager of the MS- and EI-ISAC’s Cyber Threat Intelligence (CTI) team. He completed his graduate studies in 2018 and holds several industry certifications. He also serves in the U.S. Navy Reserve.

Transcript

Phil Gervasi : Security incidents, malicious cyber attacks, breaches, they often manifest themselves in some sort of performance degradation. So maybe a slow application or maybe a slow network. Sometimes to the point that services become completely unusable. But sometimes an attack isn't so obvious, especially with today's more sophisticated methods. Sometimes we don't know when we're under attack. For days, weeks, or even months. So how can we ever really truly understand what's going on today in terms of security? Especially when our infrastructure is distributed across campus, across a remote workforce, private and public cloud, microservices architectures and intertwined with the public internet itself. Joining me today is TJ Sayers, manager of Cyber Threat Intelligence at the Center for Internet Security. A nonprofit group that helps private and public organizations secure their infrastructure, data assets and implement secure best practices into their own operations. No small feat, no easy task, and today we're going to answer that question. I'm Phillip Gervasi and you're listening to Telemetry Now. TJ, thanks so much for joining today. Looking forward to talking to you about this for a long time now. So thank you for joining. Before we get into it though, I do want for our audience's sake to get a quick explanation about your background in the field and your experience in technology and cybersecurity in general.

TJ Sayers: Yeah. Perfect. Thanks for having me on, Phil. I really appreciate the opportunity and looking forward to a profitable discussion here on the visibility side and in the security side. So my background has been rather unconventional getting into the space. So I actually started off in political science and public policy for my collegiate studies. After I finished a bachelor's degree, I was like, " What am I going to do with my life?" And the military seemed very appealing at the time. So I ended up enlisting in the Navy as an intelligence specialist. And did a brief stint on active duty and hopped into the reserves. And I had a same kind of epiphany when I got done with the military training going into the reserves as, " What do I want to do now? I did the military thing, I'll be in the reserves. What do I want to do civilian side?" So there was a graduate program that was of interest to me. I got into the grad program and one of the requirements was an internship. And I should note too, even my grad program was public administration- focused. It had some sub- disciplines in homeland security and cybersecurity. And because of those two things, I ended up taking a course in cyber and I struck up a good professional friendship with my instructor who brought me on as an intern at the Center for Internet Security. And I was in the internship for a while and I was hired on as a full- time employee. I've been at the company for about six years. I'm still full- time in the Navy as an officer as well. And pretty much all of my cyber experience since that point has come from on- the- job training and certifications. And things like incident handling, computer forensics, malware analysis, and reverse engineering in variety of different roles. So liaison capacity down with DHS. I was the elections' analyst for a time at the organization when the election security component was first being stood up. And I spent, probably the majority of my time, just looking at cyber threats across what we call the SLTT landscape. So the state, local, tribal, and territorial US government landscape.

Phil Gervasi : I have to say, we've had many guests on the podcast and other podcasts that I've done in the past. And it is such a common theme that we come to this field, whether it be security or traditional network engineering or whatever, to have an unconventional path to get there. Right?

TJ Sayers: Absolutely. Yep.

Phil Gervasi : My background, TJ. Yeah, you know my background, I started off as a high school English teacher for the first four, five, six years of my life. Before getting a help desk job. And then for whatever reason, getting into network engineering instead of whatever other avenues I could have taken. So it's really interesting to hear that. And I think it gives us, you and me and others... Not to say that people that don't have an unconventional path, that have a very computer science degree and they go right into an internship, that they're lacking. But I do feel that we have a unique ability to see things a little bit differently. Like, for my sake being able to communicate in customer presentations and do that sort of thing. Again, not to say that people that don't have that background can't do that well. Of course I work with them every day, but I do appreciate that and I think it's a benefit probably in a security field as well. I have to assume, with your background in public administration and that sort of thing, right?

TJ Sayers: Yeah, absolutely. In fact, that's one of the things that when we go around and speak to schools and things at the company, we try to encourage the students that it's not necessarily a computer science, very technical degree program that you get into this industry. There's a lot of different pathways into it. And I think one of the biggest value adds that we can derive from that unconventional pipeline is that we bring in people who can public speak. We bring in people who can take very technical content and put it in terms that a broader community would be able to understand. And that's a challenge that I think, coming from a purely technical background, people encounter is that they're very versed in the technical material. All they talk about is technical content. They talk about it with peers who are extremely technical. And then going and trying to communicate that to people who are not in the field. It's hard to digest. And it's also interesting too, I've mulled this over recently. Cyber and networking, they're technical, but there's a lot of other industries out there that are also technical. Like, I think of Navy. I'm in the Navy. Monitoring and doing things with radio frequencies, looking at sonar for submarines. There's entire disciplines that are gauged around that stuff. And they're extremely technical. Even legal disciplines, healthcare, they all have their technical jargon and they're technical in different ways. So I think the real focus is critical thinking. It's something I communicate to applicants coming onto the team and the organization, is critical thinking is huge. Are you able to communicate your thoughts effectively verbally and in written form? And a lot of the technical skills you can learn on the job.

Phil Gervasi : Yeah, for sure. And you mentioned a few of the tasks that you've done over the years earlier, and they sounded pretty technical to me. The types of forensics and investigations that you did, so. And that actually leads me to my first question. I'm familiar with some of the security attacks and the things that are going on simply because our company does have a threat fee that we ingest. And that we disseminate to our customers and we use as part of our database. But what would you say in your experience, are the most prolific threats that are actually going on today that we should be worried about and thinking about?

TJ Sayers: Yeah, so I'd probably bin this in two categories. It's a great question. The first would be financially motivated threat activity. So this is everything from theft of intellectual property. It's goes all the way towards ransomware incidents. Where they're deploying a ransomware payload on your network, and then they're trying to extort you to get your data back to pay that ransom demand. And we've even seen offshoots of that where we call double extortion, where they're then exfiltrating the data off the network after they've encrypted, or prior to them encrypting it. And then they're saying, " We're going to post this to the public if you don't pay us." And now you have extra added pressure for you to pay. So I would say that's the first one, is financially motivated stuff. It's extremely prolific. Everything's interconnected now, and there's just endless ways for bad actors to make money by attacking organizations in the cyber world. And then I would say that the second is the more persistent nation state affiliated threat out there. This is much more slow and steady and low. Trying to float below the radar, gain access into government networks. Even organizations like yours or mine, private sector industry, healthcare, pretty much every industry you could think of, there's valuable data there. There's value by just being able to get access to the network and gain some level of backdoor access or persistence. And, or countries like China, Russia are known for doing this, where you will see, " Hey, we're seeing some bad activity on the network." And then you go and do a forensic workup to investigate and you realize they've been on there for two plus years. So that's definitely a huge concern, but it's not as much of a halting your operation and disruptive concern...

Phil Gervasi : Right.

TJ Sayers: On the nation state side. The financially motivated side of things will certainly bring an organization to a grinding halt.

Phil Gervasi : So is there a validity to this narrative that's going around that a lot of these bad actors, these bad guys, it's almost like a company unto themselves. Where they have practically an HR department and their own staff and they take off on the weekends and holidays. And they're there literally just to make money. But their method, their business plan, so to speak, is to attack you and to use ransomware, whatever other means, to get money from you. But in any case, means aside, business plan aside, they kind of operate like a business. Is that true?

TJ Sayers: Yeah, they do. We have seen the typical criminal syndicate model move over to the cyber world. Then that's been one of the biggest changes over the last few years and decade. Is that, it's gone from one actor having some techy skills and able to conduct breaches and steal information and do bad things to full- blown organizations that are operating to make a profit in the criminal underworld. So this is especially true when we look at countries like Russia. Russia pretty much has this policy that as long as you don't attack us, you can attack anybody else and make money off of it. So we do see organizations, everything from people who are doing the malware development, people who are coding and making the delivery aspect of it, right through compromising network devices to make a botnet. You see it all the way down to money laundering, partnering with certain people, insiders at banks or using cryptocurrency, using mules. So it is a very elaborate organization in a lot of instances when you're dealing with some of these sophisticated cyber criminals.

Phil Gervasi : And so a moment ago you mentioned when you see something going on in the network and it maybe fires off some alert and then you go to investigate it, what did you mean by that? What are you looking at out there in the world? Obviously it's different for different contexts I'm sure, but what are you actually looking at to tell that there is something bad going on?

TJ Sayers: So, I could speak confidently from my organization's perspective, a lot of other organizations have somewhat of a different model. But I think typically it's some type of defense in- depth layered approach. Where you have a firewall at the perimeter or some form of network IDS or IPS that's monitoring for traffic at the network level. We at CIS have what we call Albert. Albert's a network IDS that runs off of Suricata. And basically we capture NetFlow and anything that matches the pattern for the signature. We'll then send an alert to our SOC for them to analyze. So that's a network piece. So we're looking particularly for anything that's abnormal, right? Are we seeing connections from services to obscure ports that they would not normally connect to? Are we seeing significant bit transfers across things that we've not observed before? That's just at that level. And then you'd go down to Suricata saying, " We know this particular domain or this URL pattern is associated with this particular malware." If we observe any of that and it matches the signature, that'll send an alert to us to investigate further. And usually we'll work with the member to help identify, did it go any further? Was it blocked? We also run into some issues too across the industry of, where exactly is the IDS or IPS placed, right? Is it before the firewall? Is it after the firewall? Is it in line? Is it out of line? So that also complicates collection there a little bit at the network level. So the other things there would be host level detection. So things that we would look for on the host, which there's two different forms. There's XDR, which is extended the detection and response. And then, there's industry term EDR, endpoint detection and response. And these are basically just more heuristic- based host level antivirus solutions that organizations can deploy below the network level. So this is going to monitor things like spawned processes, registry changes, new files getting created, files being created by things that should not be creating files, escalation of privileges. So that's a normal user, and all of a sudden the user is becoming an admin user. The user accessing resources they haven't accessed in the last six months, maybe accessing certain resources that should never be accessed by that user. So just monitoring things at the host level to give some more detailed host level insight into whether or not this is normal user behavior. Or if this is more indicative of some type of intrusion or malware infection.

Phil Gervasi : So this certainly sounds like a defense in- depth, like you said. Because you mentioned DPI, you talked about packets, you talk about flows, you're talking about host level information. And by host level, I have to assume that you're talking about endpoint, but also the host as in the resources themselves. So if I'm some organization with remote offices and a primary and secondary data center, hundreds of bare metal servers and thousands and thousands of VMs, a public cloud footprint, SaaS providers, I mean those are all hosts. And those are all different metrics and logs that you want to collect, right?

TJ Sayers: Yeah, absolutely. And that's one of the challenges, in fact, that we have is the sprawl of infrastructure in some of the organizations that we seek to provide our services for. Oftentimes we find in investigations when we do identify an intrusion, that the adversary has a better network map and mapping of assets than the organization did. Just because of the size and enormity of them.

Phil Gervasi : That's interesting. Yeah, interesting and sad coming from a network engineer background. Because we're always complaining about not having good network documentation, and we're always trying to figure out where traffic patterns are or where traffic is going to. So that is a very interesting point that you make. So you're collecting a tremendous amount of data. So you're ingesting who knows what logs. You're collecting flow information packets, all this stuff. There's got to be a data science workflow on the backend that you're able to figure this out. Otherwise, you're literally just trying to find signal and the noise. And especially if you're trying to investigate a real- time security threat, that's got to be really difficult. Especially if you're talking about multiple customers and doing this as a service and not just for your own small network in whatever city you happen to be in localized.

TJ Sayers: Yeah, absolutely. And historically, we used to keep everything on prem and it quickly became an issue of scale in the enormity of the information that we were ingesting. And we're upwards of hundreds of petabytes a month now in what we're collecting and pulling for analysis. So we've moved over to a cloud solution and that's where most of it's stored. And we're able to search retroactively for many, many months in the past to look for things. Because oftentimes it's a problem with the security incident, is you may not notice it immediately. There's very easy ways to get in through a compromised credential and that looks like a legitimate login because that's a legitimate user. And so you're able to get in that way. And it's not till 3, 4, 5 months later that we're like, " Hey, that login that happened there is actually a bad actor. That person doesn't even work here anymore." Or, " That person was on vacation when they logged in." And next thing we're observing is malicious activity on the actual network. So that's one of the storage issues we've run into and so industry problem is having enough data set, having enough data to look back retroactively once you realize bad's happened. So the ways that we do that is cloud storage, we're able to search it really efficiently. And then, for my team in particular, we have what's called the threat intelligence platform. So all of this data, we have hundreds of threat feeds flowing into it. We're able to ingest internal data from member reports and our collections into that platform. And it really tees it up for the individual analysts to look at. So they're able to build out actor profiles. They're able to look at specific indicators like, " This IP has been observed in this traffic. And this IP is tied to these five other incidents that have happened over the course of the last two months." And you can start building a threat picture of what we think's going on here. So that's really CTI specific. But more organizationally, we have in operations, we're bringing in a new one now, but we have an existing SIEM where our SOC analysts live and they're able to identify alerts coming through other network telemetry. And kind of identify where this is obscure or this is probably normal activity and we'll send out alerts to the member. But I would honestly, Phil, I would say by and large, the way that we collect information and the relationship that we have with our membership is most key. So we observe something bad, we'll escalate that to the member. But the most important interaction in that is continuance of discussion with the member to identify and investigate on their end. So that's usually where one Suricata fire or one obscurity in network traffic leads to, " Hey, we think something's going on here, can you please look for these things? Or look for these areas?" And that's where CTI would come in and say, " We've seen that pattern before. And usually after we've observed that it's one of these five things that we see next. Have you seen one of those five things? Okay, you've seen one of those five things. This looks like it's going to be a full- blown incident. Let's bring an incident response." And then we start helping the customer troubleshoot. We call them members, help the members troubleshoot through the incident and try to identify exactly what the extent is. And like I mentioned earlier, usually the cases, the actor either makes a mistake, they get a little too noisy, and it's identified that way. And then we realize they had access into the network for several months up to two years or more, for the more sophisticated actors. But the storage aspect's definitely been something, and the SIEM and the threat intel platform help with the analytics side. And then we do have data analysts at the organization. So some of them specialize in just warehousing. How to store the data so that it's easily searchable. We have people who use our studio and write their own code to use statistical models to look at that stuff. And we've recently started using Power BI and things like that to create visuals, and to be able to see a picture. It's one thing to have all these ones and zeros and this flag and that flag. And these indicators from source and destination IP. It's another thing entirely when you're able to visualize that information and to start say, " Oh, wow, look, we have clustering over here. And then it's reaching out over to here, and that's not normal. It shouldn't be reaching out in that capacity." So that really blends a lot of insight for us. So we're really big into the visualization side. Not just for identifying things, but also for communicating effectively up to leadership and out to the membership that, " Hey, here's what we observed, here's what we think it's doing, here's what you need to do to first close things off and then begin to remediate. And hopefully get back up on your operational feet, as it were."

Phil Gervasi : Yeah, that's so interesting. Because we at my company use so many of the same workflows and tools are similar, I'm going to say similar. But for somewhat different purposes. But how do you ingest a tremendous amount of data? Because if you start to add up all the different types, it's a lot. It's a very diverse data that we can collect from the way we do networking today, the way that we deliver our services over the public internet and all that. So there's a lot there. So just volume. But also the variety. I mean, we're talking about very, very different formats. So one of the things that we have to solve is, how do you plug these things into mathematical algorithms? Whether you're using machine learning or more basic statistical analysis, and then compare values. When one value is millions of packets per second, and over here you have a value that's a percentage. And over here you have a value that's just a security tag, it's a random tag, it's just an identifier. So it doesn't even represent an actual value. So that's where you get into the really interesting stuff like machine learning, pre- processing, normalization, scaling, all that kind of stuff in order to then take all that, plug it into whatever models you're using. And then identify hopefully, strong correlation in causal relationships. Now, one of the struggles that I have seen in the industry, not so much on the security side because I'm not as plugged in. Which is why I wanted you to talk to you today, is the rate of false positives. So for example, on the network visibility side and now network observability side, having this correlation pop up and then the system says that these two things are related and they're really not. And all of a sudden you have a false positive and you're wasting your network operation cycles trying to figure that out. And being able to then also add the subjective component of the engineer. Because sometimes it is a strong correlation, but it's not that important. So for example, I had this weird DNS thing happen over here, and now my hundred gig link, that's typically one meg, is now chugging out along at 10 megs. Who cares? A 10 meg uptick on a hundred gig link doesn't affect anything, nobody cares. Maybe I will if it keeps trending. So that's been really hard too, is adding the subjective component. Which I have to assume is very difficult for you. Or at least your organization. Does that make sense?

TJ Sayers: Yeah, absolutely, it does. That's certainly a problem we deal with as well. It's one of the reasons why I highlighted earlier in our discussion how important the interaction is with our membership. And being able to say, " For your organization, we're seeing this. Do you even have that process? Or do you even have that application in your network? Or is this just a false positive that we're getting?" So I would say by and large, probably 85% of the things that are coming in...

Phil Gervasi : Oh really, that many?

TJ Sayers: Are ruled closer to the false positive side of things. And that's why we still have some tier one analysis that's done after the initial sweep. And then we're trying to only send out high confidence alerts to members, because everybody experiences it in many industries, but alert fatigue is huge in security. You're just constantly inundated with this fire over here, this weird thing happening on this system. Is it something that's can be easily fixed? Is this something that's legitimate? So we really try to provide just things that are high confidence out to members.

Phil Gervasi : So how do you defeat alert fatigue, especially in the context of cybersecurity? Is it a manual process of just having more bodies going through and looking at the alerts, and seeing if they're legitimate or not? Or are you able to do that in some kind of a programmatic manner?

TJ Sayers: Yeah, so that's really where the SIEM comes in. So security information event manager. So that's like a platform, it's a UI even that the analyst can live in. And that does a lot of machine learning and baselining for us. So it helps us kind of automate some of that. So it doesn't have to be a manual review on every single alert. And then the other big component is understanding just the threat landscape. Oftentimes we'll see old signatures or old activities still designated in open sources as being a bad. And so you'll still get a false positive because something will be matching the signature. But we're able to say based on more extensive research that, " Hey. That particular threat actor group is not using that infrastructure anymore. So even though it was bad in the past, it's no longer bad." So you can whitelist that signature. I'll hammer again, relationship with the member and being able to just outright ask them, " Hey, we're getting a ton of alerts for this specific thing, can you investigate further?" And oftentimes they'll say, " That's traffic going to our cast register in the lunch line." Or something like that. And it's not even internet connected and it's just a local traversal, and it's truly a false positive. You could just shut that off. And then we reduce the amount of alert fatigue that's coming in. So those would probably be the biggest ones. And then I would say the more proactive we can be as an organization to get indicators that we know are active and malicious out to them, so that they can block them proactively will reduce the amount of alerts that we're getting. Because it's not going to traverse the sensor if they're just blocking it right at the perimeter before they're able to even get in.

Phil Gervasi : Yeah. So this is a data science problem. I mean, ultimately what you're doing is just ingesting a tremendous amount of information and trying to mine through it. I'm assuming, your members are your customers, right? That's synonymous, correct? Actually, that's a question I want to ask you is, what is the mission of your organization and how are you structuring this? Because you mentioned members, so is it different than just having customers? You're also a nonprofit, how does that work?

TJ Sayers: Yeah, so the Center for Internet Security is non- for- profit. It's the organization that's focused on like the CIS controls, many people know them as the SANS top 20 historically. But they're now the CIS controls and there's 18 of them. So CIS focuses on basically industrywide best practices, hardened images, benchmarks to gauge where you are at with security. And the mission of that organization is confidence in the connected world, essentially. Where I work and where we're more operational dealing with the threats on a day- to- day basis is in what we call the ISACs. So Information Sharing and Analysis Centers. There's two of them under the CIS umbrella. One of them is called Multi- State or MS- ISAC. And the other one is elections infrastructure or EI- ISAC. And basically those are funded through the Department of Homeland Security. Specifically CISA, the Cybersecurity and Infrastructure Security Agency. To provide cybersecurity incident response services, threat intelligence, support out to state, local, tribal, territorial, and US election offices. And I know that SLTT term can be a little confusing. So I think the easiest way to think about what is our membership base is any taxpayer- funded organization below the federal level. So that's everything from a local library, schools...

Phil Gervasi : That's inaudible.

TJ Sayers: All the way up to some tribal casinos, major US government infrastructure in the SLTT space. Pretty much covers every critical infrastructure sector you could think of, right? Water and waste management, transportation infrastructure, energy infrastructure, healthcare. You name it, telecommunications, that all falls under the SLTT umbrella in the US. And our membership is open to all SLTTs or election offices, as long as you're taxpayer- funded below the federal level.

Phil Gervasi : And you're working with these organizations not only to monitor, but also to work through some sort of maybe audits if they're a regulated industry. But also for doing the forensics analysis in a postmortem or in real time trying to stop an attack. Is that right?

TJ Sayers: Yep, yep. On the ISAC side, we do everything from initial monitoring, sending the alert out to... If we do observe bad thing happened, we'll do a full incident response and forensic workup. And then we also have the proactive side of things where we're looking at one event, that maybe let's say happened in Texas. What happened in Texas, is that unique to Texas? Is that a threat actor specifically interested in one organization? Or is this indicative of TTPs that are more TTP tactics, techniques and procedures that are more broad across the larger industries? So we're often able to proactively identify a incident in one state and get that information out to the rest of our membership. Which is all US states, territories saying, " Hey, we observed this here. We believe that this is something that may also impact you down the road. Here's the things to proactively defend yourself." So we do both the reactive and the proactive side of things in addition to network and host level monitoring.

Phil Gervasi : So then we talked about how you have to, and you do, ingest this tremendous amount of telemetry from networking resources and hosts and all of that stuff. Is there a difference though, when you're doing some sort of proactive investigation or remediation against a security threat? Is it then still just flows and logs or are you looking at some other kind of data, some other kind of telemetry?

TJ Sayers: It's going to be a collection of all the things that we talked about previously. And then I would say the big thing with the proactive component is going to be understanding the motives and more of the human side of some of these threat actors. So threat actors take breaks too. We talked about that at the... You mentioned that actually in passing at the beginning, is the ebbs and flows of data. We also observe threat actors take holidays off. They're interested in anniversaries. So right now is near the anniversary of the invasion into Ukraine with Russia. And we're on heightened alert because we realize that these anniversary dates are important to some of these actor groups. So we're at a heightened posture monitoring for things that may occur, and we're able to proactively get that information out to members saying, " Hey, this date's coming up." Or, " Hey, this thing's happening. And we always see some type of action there." It's also understanding kind of broader US policy and how other foreign governments and criminal groups respond to that. So Iran is a great example. They're very tit- for- tat. If we do something, they're going to respond. A lot of times that's in a cyber capacity. So we could monitor like, US is sanctioning this or we're having diplomacy talks that go awry. We then expect Iran to lash out. So it's kind of the human element in the proactive space. And that's really what my team specializes in, is the proactive component. Is trying to understand the threat actor, everything from the criminal organization all the way over towards your state affiliate, or your state- sponsored organization. And what drives them, what's their motives, what are they after, what are the historical trends and patterns that they're doing? And then we couple that with the technical collections and what members are feeding to us, and marry those two things together. And then we're able to give them a little bit more of what we call anticipatory analysis. So it's not, bad thing already happened, right? It's left of BOOM, as we like to say. So before BOOM happens, we're able to tell members,'Hey, we think this is going to occur. Here's what you do to protect yourself." And hopefully if they implement the guidance and we have our assessment correct, they're able to prevent an incident from happening at their organization.

Phil Gervasi : So where is the investment today, and where is the focus today to improve the cybersecurity landscape in methods and in your practices?

TJ Sayers: Yeah. So, there's a lot of things happening in the industry, and I think most of the focus now is on partnerships. So building relationships in the community between state, local, tribal, territorial US governments and federal resources. So there's things like the state and local cybersecurity grant program, which gives federal resources to state and local cyber programs to help them expand their capabilities. Some instances the designation of infrastructure as critical infrastructure. So actually having that federal designation gives the federal government the ability to fund and provide resources to some of these under- resourced communities. And it's usually an issue in the SLTT space, is that they're heavily underfunded. And any funding they do have is most likely not being directed towards cybersecurity- related endeavors. It's designed to make sure the infrastructure is available and people are able to connect. And security is unfortunately an afterthought in most cases. Because security is very expensive. So that's a huge component too. And then I would say probably the biggest thing on this level is growth of public and private sector collaboration. So basically partnering between federal government entities, state government entities, and the private sector. We've seen things even in the recent National Defense Authorization Act. Which is more of a defense and military- oriented budgetary document, that has included language like allowing United States cyber command to partner with private sector organizations for more of response, or hack back, or offensive measures against organizations, criminal enterprises or foreign governments that are attacking US infrastructure. So partnerships is huge. And then if I was to talk about technology, to be honest with you, I'm a little jaded in the technology space. I have seen so much and I have observed the adversary in so many different ways. And I'm near convinced that if there's enough resources and determination that there's always a way into a network, that a compromise is almost impossible to avoid. And I think it really comes down to what technologies, what methods of detection can we come up with to stay ahead of the curve. But really to address what I would call the, as an industry kind of thing, is called the pyramid of pain. So addressing more of the behaviors, the tactics, techniques and procedures of what threat actors are doing. Instead of just addressing trivial things like hash values or domain names, IP addresses that threat actors can very easily change. And I think we're still making that shift in the industry right now, moving away from just blasting out IP addresses, domains and hash values. And having people block those things. We're getting into the world now, where EDR, more advanced network monitoring stuff is looking for patterns of behavior that are known to be bad things. Some of the things we talked about earlier. Identifying those things and being able to proactively block once that behavior is captured and not just, " Hey, we observed this bad IP address, let's block the connection."

Phil Gervasi : Yeah. So there's no silver bullet from a technology perspective that will solve all our problems. It sounds much more process and best practice oriented.

TJ Sayers: Yes. Yeah, unfortunately. Unfortunately. There's not. It's a whole layered approach, reducing the likelihood that you're a low hanging fruit. And increasing the amount of pain as it were that it would take the adversary to attack your organization. We tend to think of criminal organizations and foreign adversaries as being, like having unlimited resources. They're just as resource constrained as we are at local organizations and private companies. We only have so much money where there's only so much we can do. So if we can increase the cost of them being able to carry out attacks, that's going to reduce the amount of attacks that we see. And I think one interesting area I've really been trying to push towards is having a more efficient and cheaper way of backing up data. Data storage in and of itself is enormously expensive, as we discussed. Analyzing that data in all the different types is hard and can be expensive. But backups are pretty much by and far, especially with ransomware being out there, especially with having your whole organization brought to a grinding halt because everything's encrypted. The single best solution is having a proper backup policy in your organization. Are you actually able to back up all of your data? Are you able to restore from that backup if something happens, I think is key? And one of the biggest bars that afflict every organization is that backups are enormously expensive. To back up all of your infrastructure and all of your data is not only complicated, it's expensive. You have to test it regularly to make sure it actually works. But finding a solution, finding some technology to make that more efficient and cheaper, I think would put us way ahead in some respects, on being able to go through some of the challenges we see in the security space.

Phil Gervasi : Yeah, handful of snapshots and a couple of cold spare switches on the shelf is not sufficient. So what are the punitive measures that we can take? Or that, I don't know, you take or the government that you interact with take when you catch the bad guys?

TJ Sayers: Yeah, so this is a federal law enforcement thing. And I would say in some spaces a US military thing. So there's unfortunately not much that we can do from an private company and our membership- based, SLTT- based sector to levy punitive actions on actors attacking us. But what often happens is the FBI gets involved, other federal law enforcement agencies get involved, and they're able to trace back what these attacks are. Who may have levied them and find out the people behind the scenes responsible. So we see things all the time, like warrants being posted by the FBI, sanctions being posted on certain businesses and banning travel by certain individuals. And sometimes we see arrests where people are caught up in vacation and the FBI is able to swoop in, or another federal law enforcement agency is able to swoop in and address them. Because we have a relationship with that foreign government to do that. We can arrest and extradite out of the country. But by and large, I would say a lot of the threats that we see, they are traced back. Eventually, we find out who's responsible. But they live in a country that there's no extradition on the books. So we can't do anything really outside of post their face all over the internet. That they're responsible for this thing and ban them from traveling to any countries. Or putting sanctions on their assets if they have foreign assets. So that's kind of the law enforcement side. And then I would say more like, if you think of cyber command, US military, NSA, etc. They have kind of offensive defense, if that makes sense. So taking more of an offensive approach to defense. So before the adversary attacks us, we're able to dismantle their network, we're able to shut things down before they're able to attack. We saw this a lot with the elections over the last several years. We've seen this particularly with some very established cyber criminal groups. One of them in particular is Emotet. Emotet was a very prolific modular banking Trojan, that eventually morphed into a downloader and propagator of other malware. Particularly ransomware. And they were a very elaborate organization, similar to what we talked about earlier. And cyber command and the federal government were able to take down some of their infrastructure in the lead up to some of the elections that were happening. Because there was an indication they were looking to attack. So there's that aspect of the punitive side. But unfortunately, most of these actions are being taken by people outside the country, even though they're using US infrastructure. So it looks like it's US IP attacking another US destination, but it's all by proxy. They're operating by proxy to do stuff like that.

Phil Gervasi : I mean, there's a few different stories out there about the threats to our public utility grid, and you hear about this stuff going on. I'm thinking about Russia and Ukraine right now, for example. Is that one of those things that keep you up at night?

TJ Sayers: No, honestly, Phil, nothing really keeps me up at night in the threat landscape. There are certain things that give me reflux when I think about them. Some of those things are power grid. There's been an enormous amount of interest from criminal groups and state- affiliated groups in that sector. Because one, the financially motivated actors realize, " Hey, if we bring down this pipeline or we bring down the grid, there's no better way to have pressure on the organization to pay us than people calling them en masse that, 'It's the middle of the winter and the power's out and I'm freezing.'" And there's enormous pressure if you were able to attack something like that to get money for it. The problem with that is that, that's a surefire way to gain federal attention. So if a criminal group was to attack electric infrastructure or gas infrastructure, just the power grid more generally, that's going to elicit a pretty strong response for some of the punitive measures that we've talked about previously. The other ones are just data aggregation at certain companies. So healthcare, there's a lot of information collected on people's healthcare that are segregated to a very small amount of large healthcare organizations. Same thing with marketing. There's a handful of very large marketing organizations out there that have most of the information on the US population. So being able to target one of those organizations, you kind of have a treasure trove of information on US persons. Which then you could use for social engineering. You could use it for follow on attacks and impersonation, and just the ability to conduct identity theft and things like that. It's really endless possibilities with having that much on somebody. And then the last one, which I am going to say last, but it's probably the paramount focus of my job over the last couple of years. And truly what disturbs me most is the espionage and strategic influence operations coming out of the PRC, China. So they have a lot of policies that are outright, you can go look them up, that are specifically designed to exert their influence across the world. And to gain a massive competitive advantage against the US. Everything from stealing intellectual property, gaining economic advantages, and usurping our industries. So companies like yours, Phil, visibility, other companies that are working on proprietary technology, doing things better in the technology space. China's interested in all of that. And they're going to replicate it and try to do it better once they steal it. So that's a huge thing that concerns me. And I'll just name a couple of things maybe for the audience. If you really want to go down the rabbit hole on some of their policies, you could look up like the Thousand Talents Plan. So that plan is basically designed for them to send out Chinese citizens abroad, get them trained up in foreign universities, even get jobs in key industries and sectors. And then hire them back into the country and leverage all of the stuff that they've learned to kind of advance China in some of those key sectors. So space, technology, healthcare, even agriculture are some of the ones that are wrapped up in that. And then you have the Belt and Road initiative, is the second big one. So basically, they're fielding out technology to countries in Africa and South America and they're building infrastructure. They're building what they call a belt and road to increase economic commerce and to expand and try to bring some of these countries up economically. At least that's what they say. But all of the infrastructure is Chinese infrastructure and by and large it's backdoored. They're able to monitor everything going on in those initiatives and they're exerting enormous pressure on these countries when they partner on this initiative to have a very close buddy- buddy relationship with China. Made in China 2025 is a whole third one, another rabbit hole there, you could go look into to that. And basically it's moving away from China being viewed as the country that makes all the trinkets. You go to the store and made in China, everything seems to be made in China. They want to move away from that notion over towards being a innovative technology leader in some very key spaces. And the way they're going about doing that is through massive collection and stealing and gathering of intellectual property of primarily US companies. Through hacking it and stealing it, or through coming to your organization and saying, " Hey, you're a startup. I really love what you're doing. How about half a million dollars? We'll fund you if you'll share with us and work with us." So they're spending enormous amounts of money to gain inroads to American technology companies, to try to get access to information that they wouldn't normally have. And the end result and the end purpose really for them is to replicate that technology in China and continue to gain again, a competitive advantage. And then the last one is debt traps. This is a big one we see too, kind of coupled with the Belt and Road initiative. But this is basically a way for them to go to an underdeveloped country. Say, " Hey, we really think it would be great if you had a port here. So we're going to fund it, we're going to build it, and we'll finance it. And it's going to get you this much economic billions, trillions of dollars a year you'll have in economic transactions because you now have a port. And you can just pay us back for paying for the port." What ends up happening is that it's a debt trap, right? The country that got into the deal, can't pay it. China then says, " Well, now we own your port." Or, " Now we own your infrastructure." And they're basically preying on victims in that capacity. So all of these things kind of go into the larger picture of the great power competition is shifting. We're seeing it in the cyber world. We're seeing it with China and Taiwan, and there's a very heavy emphasis on trying to dethrone the US on the world stage. And that's going to have massive technological, economic and strategic ramifications for us down the line.

Phil Gervasi : Yes, TJ, this was a really interesting discussion for sure. I wasn't expecting the direction we were going to go in there toward the end, but certainly a reminder of just how complex the problem with cybersecurity really is. From a technological standpoint, a geopolitical standpoint, just people standpoint. Defense in- depth actually isn't a term that I've heard in a while. You mentioned it today. And based on the distributed nature of how we deliver services today, it makes more sense to me now more than it ever has. So really great topic. So TJ, we're going to end here. If anyone in the audience would like to reach out with a question, comment, to learn more, how can they do that?

TJ Sayers: So it's kind of funny, I work in security. I don't have any social media presence. I try to stay pretty private. So I guess the easiest way would be reaching out through you and the podcast. And I'd be happy to chat with anybody more fully if they're interested in any of these topics.

Phil Gervasi : Great, thanks. And you can find me on Twitter, @ network_Phil. Still active there. And you can search for me on LinkedIn as well. Now, if you'd like to be a guest on Telemetry Now or if you have an idea for an episode that you'd like to share, please feel free to reach out to us at telemetrynow@ kentik. com. You can also follow us on Twitter and LinkedIn as well. Thanks for listening, and until next time, bye- bye.

About Telemetry Now

Do you dread forgetting to use the “add” command on a trunk port? Do you grit your teeth when the coffee maker isn't working, and everyone says, “It’s the network’s fault?” Do you like to blame DNS for everything because you know deep down, in the bottom of your heart, it probably is DNS?

Well, you're in the right place! Telemetry Now is the podcast for you!

Tune in and let the packets wash over you as host Phil Gervasi and his expert guests talk networking, network engineering and related careers, emerging technologies, and more.

We use cookies to deliver our services.
By using our website, you agree to the use of cookies as described in our Privacy Policy.