Unlocking Backdoor AI Poisoning with Dmitrijs Trizna

Nic Fillingham: Since 2005, BlueHat has been where the security research community and Microsoft come together as peers.

Wendy Zenone: To debate and discuss, share and challenge, celebrate and learn.

Nic Fillingham: On the BlueHat Podcast, join me, Nic Fillingham.

Wendy Zenone: And me, Wendy Zenone, for conversations with researchers, responders, and industry leaders both inside and outside of Microsoft.

Nic Fillingham: Working to secure the planet's technology and create a safer world for all.

Wendy Zenone: And now on with the BlueHat Podcast. [ Music ]

Nic Fillingham: Welcome to the BlueHat Podcast, Dmitrijs Trizna.

Dmitrijs Trizna: Hey, hello.

Nic Fillingham: Thank you so much for joining us. Thank you so much for staying up late. I understand it's 9:00 PM where you are. Could you introduce yourself to the audience? Who are you and why is it 9:00 PM where you are?

Dmitrijs Trizna: Yeah, I'm actually originally from Latvia. It's like really small country on the northeast of Europe, but right now residing in Prague, it's in Central Europe. So part of Microsoft, we have a good, let's say, development center here doing our stuff. And yeah, I'm momentarily part of M365, the security here, building, let's say, non-trivial heuristics on top of the system logs to identify adversaries in M365 and actually in Azure networks. So yeah, been on both sides of the security landscape, blue team and red team. And then that's kind of both parts for some time and lately switched to more like machine learning slash AI focused analytics that kind of brings a novel flavor to all these things. So yeah, that's briefly it.

Nic Fillingham: Thank you again for joining us. You are joining us in part because you were one of the presenters at BlueHat India, which occurred about a month ago. We were in Hyderabad, India. We were there together. We met each other in person, which was fantastic. I got to see you present. I want to quickly come back. I think you're our first guest. You're definitely our first guest from both Latvia and, you know, anyone sort of based in Prague. Tell me more about Latvia. It's a country I am- embarrassingly don't know much about of. What should we know about Latvia?

Dmitrijs Trizna: No one knows anything about Latvia usually outside of the kind of radius of, I don't know, 200 kilometers and people usually mix it with Lithuania, which actually resides right next to it. But yeah, we are nice small country and a lot of forest and really nature based landmarks. It's pretty flat though, but you can have a really good hiking places around, some wildlife, and yeah, in this sense, but still little but proud country trying to grow. Yeah, doing a lot of efforts to kind of be noticeable on a world map, right? But actually post-Soviet, in '90s, we kind of gained independent. Since then, doing okay-ish, I'd say growing economy and yeah, I don't know, that's briefly it, and if this makes sense.

Nic Fillingham: It does. I'll definitely be looking up the Wikipedia article for Latvia following this to learn a little bit more. You're going to talk about your session from BlueHat, which was titled The Impact of Backdoor Poisoning Vulnerabilities on AI-based Threat Detectors. Before we get into that, though, love to learn a little bit more about you, learn about your path to Microsoft, your path into the security research space, your specialty, your focus on AI and the intersection of threat detection. Does your story start- I assume your story starts in Latvia, did you? Is that where you grew up and went to school?

Dmitrijs Trizna: Yeah, yeah. Basically like a regular education, nothing special. But yeah, at some point in the career, you get the- like we have a kind of dangerous neighbor and securities and safeties is paramount there. So I joined at some point the energy provider in Latvia, who kind of is an industrial setting is one of the targets that would be targeted in case of any military escalations. So security is pretty serious there. So I was a security engineer in Latvenergo, so-called, for four years. Basically a blue team member building mostly focused on network security. Back in the day, network security was a big deal still. Right now, it's more of a like solved domain given the proper implementation and the problems reside mostly on identity level and maybe host level, but like host, I mean endpoint level. But yeah, started on network security, then switched to, still in Latvia, on the red teaming side. Did some pen tests and red team engagements there. But yeah, cannot hit the, let's say a ceiling, what you can get in Riga, our capital, in a market there. So that's why I decided to kind of go and look for more promising opportunities. And Microsoft, actually, one of them where I got this like pretty mature setup that would benefit from AI-based logic because a lot of kind of you have to have already mature security posture, right, to get this step. Like usually people really lack on basic things and especially in mid to small business. You have to have this enterprise level maturity. But yeah, I've got two educations, actually two master degrees. Second is in Helsinki, that's interesting part of Europe, you can like pivot around- and Helsinki is in Finland, but I studied there remotely during the COVID times. And yeah, that's interesting. Actually doing right now a doctoral thesis in Italia, which is another part of the Europe. So really kind of like bringing it all together and different people, different backgrounds. Yeah.

Nic Fillingham: Awesome. And your current role at Microsoft. I'm literally reading your LinkedIn here. So this is not for memory, but it says you're a security researcher and you're the tech lead for AI-based cyberthreat detection solutions for Kubernetes and Linux platforms in the Microsoft 365 backend. That's a pretty good description. Do you want to sort of elaborate a bit more on sort of what you do in that space?

Dmitrijs Trizna: Yeah, sure. Actually it's expanded a bit. We started with M365 and now we go towards the like Azure kind of wider scope that they request our expertise there and we implement some heuristics for them as well. But basically, right, every cloud, there is a joke, every cloud is a server, my son. If you've seen this comic somewhere in the internet because for like customers, cloud is this abstracted thing, but for cloud providers like Microsoft or Amazon or Google, the cloud is basically a set of servers, right? And you have to- it's a somewhat conventional infrastructure. And most of the cloud runs on nowadays on Kubernetes and in a more kind of dynamic way. And that's basically where all those SaaS services live, and that's where we implement our threat detection heuristic. So if someone breaks through the SaaS service and some of those, for example, Exchange or OneDrive or Azure Kubernetes itself, we have to identify if the parameter where the kind of customer is allowed to operate is breached and the detection like starts to, I don't know, threat actors starts to operate Microsoft resources jumping from customer to customer operate laterally. So we have to protect this part of infrastructure too. So that's where we collect telemetry to identify what happens there. And yeah, basically my domain is Linuxes and Kubernetes. So that's where we get data. And there is the simple logic, right? You detect stuff that we know that is bad signatures or maybe anomalies, but you want to build more like non usual behavioral models on top of the systems, like how system behaves or maybe how set of systems together operate. And you kind of want to identify some non-common patterns in these behaviors. We built our own kind of attack bots that represent maybe threat actors that we are able to catch and as well we kind of built a modeling of like how normal behavior looks and maybe try to identify out of baseline activities there. So, I don't know, when you go too deep into this, it becomes too cryptic. And if I can elaborate in some aspect, please ask the more precise questions here, but yeah.

Nic Fillingham: That's great. No, that was a wonderful description, thank you. And so, you know, you talked about the two types of threat detections. You have the signature-based detections or detecting things that we know exist and we know what to go look for. And then you have looking for the unknowns, the unknown unknowns, and this is where AI is coming in to play here. So when obviously AI, the buzzword that's everywhere, how should folks listening to this episode, when you use the word AI or the acronym AI, in this context, what is it? What actually is it behind the scenes? Is it an LLM? Is it something else?

Dmitrijs Trizna: Yeah, it's really good question because there is lots of ambiguity around this, right? Especially like nowadays, AI almost equals to LLM, but AI is generally kind of pretty wide umbrella that incorporates set of techniques like ethics. But yeah, one of the conventional sets is the machine learning, right? And you don't always go towards LLM direction to get valuable component out of the AI. While LLMs are awesome, it's actually like, I believe everyone's mind was blown out in the last two years, they have a limited capability. So let's say in security operations, LLM has a huge potential during the triage stage when basically they help security analysts to understand what this alert means maybe, some steps how to investigate. That's a really solid spot for LLM or maybe for some like co-piloting pentester or threat hunter, that's as well great spot. But when you actually want to build detections, LLMs are no go. First of all, detections are machine to machine operations. It's like happens tens of thousands times within a day. LLMs are too big to be such that fast. They're too slow and you have to build more targeted models, like more narrow models that focus on specific subset of techniques. For example, let's detect only lateral moment or let's detect only reverse shell connections. That's what actually my model that we talked on BlueHat India is about that it detects specific unique Linux reverse shell detection that are not capable or being detected by signatures. So you have to widen the scope. Yeah, for this purposes, you use more conventional machine learning methodologies where you define your own processing of this data. And this data is basically usual system logs, which is semi-structured data format. It's not fully structured. So you have some fields that has ambiguous values like command lines, but it has some structure that you can use in the pre-processing to empower those models. So yeah, this is interesting mix and it's actually not widely kind of known in public literature how to properly approach it. So it's pure R&D, it's kind of you invent your own stuff in applied way having a lot of domain knowledge, it's so cool in this sense, yeah.

Nic Fillingham: Let's now talk about BlueHat India. So again, the title of your presentation was the Impact of Backdoor Poisoning Vulnerabilities on AI-based Threat Detectors. So I'd love to just sort of pull apart that title a little bit just so that the listeners can sort of understand what's going on. So poisoning, backdoor poisoning vulnerabilities. So you know, the AI-based threat detectors is the focus area here and the type of attack or the type of attack you're saying is backdoor poisoning, is that correct?

Dmitrijs Trizna: Yeah. Let me, maybe, let's start, right, there are two components maybe in the title, right? The first part is the backdoor poisoning vulnerability in specific set of vulnerabilities, but vulnerabilities of what? And vulnerabilities of AI-based threat detectors. So we built threat detectors, right? We want to detect threats in our environment. When adversary operates, we want to see it. The conventional way to do this is signatures. So yeah, we implement stuff and detect like Mimikatz if it's seen in common line, we want to see. But yeah, as I said, we want to build behaviors of the systems to kind of cover this unknown unknowns. So that's why we build those AI-based logic on top. But when we implement this logic, like it's nice, it might see something previously unseen, but by bringing it to the table, we actually introduce new set of vulnerabilities in a whole system that didn't exist before by bringing AI on the table. Literature shows that AI has a weaknesses itself. And one of the set of the weaknesses is the backdoor poisoning vulnerabilities. And basically in this talk, I show how an adversary can misuse these set of vulnerabilities and what impact they have. It's actually pretty significant, they're great in the detective performance if the adversary performs a set of manipulation that targets this specific AI component.

Nic Fillingham: Got it. So while the AI-based systems for threat detection are powerful and important, they themselves are potentially vulnerable or are vulnerable. And so in this talk, you talked about how backdoor poisoning vulnerabilities are-

Dmitrijs Trizna: Misused let's say by-

Nic Fillingham: - misused. Misused.

Dmitrijs Trizna: Yeah.

Nic Fillingham: Thank you.

Dmitrijs Trizna: It's basically a discussion within AI Red Teaming flavor. If I heard this like hot topic, right? The people right now are actively testing AI itself from a red teaming perspective. They go and like try to break it. And usually, it goes in a direction of LLMs, right? People go prompt injections, jailbreaks, they want to misuse how LLMs answers, they try to do it like some nasty stuff and prove that they are kind of like people, like, I mean, good red teamers and do awesome thing. It does not limit to LLM. So you can red team an AI system, like a defensive AI system, right? And you can red team it in a way so you degrade its quality at all. And yeah, that's what I show here. >. Nic Fillingham: So obviously this is an audio podcast. It's not video. So we don't have the slides or the presentation for folks to follow along as they're listening. However, hopefully by the time this episode comes out, the video recording of your session will be available to watch on YouTube and we'll make sure that the link is in the show notes. But with that said, can you, Dmitrijs, walk us through the session, walk us through your hypothesis and how you conducted this research and what you found. Yeah. The thing is, right, this model, what it takes a data, basically what happens on a system to identify whether there's something suspicious happening there, right? The model I showcased takes the basically process creation arguments or known widely as a command line, basically representing what command was spawn on the system, what process was created on the system. That's one of the examples. I'd say those set of vulnerabilities in AI detectors are applicable to wider set of models, but this is the exact showcase that we will probably better stick to to make some clarity around the concept. So this model takes the command line and tries to split it in a smaller tokens that kind of are unique elements for AI model to identify something bad happens there or not. And attacker, by assuming that there is this sophisticated logic, can do some additional manipulations to degrade its performance. So without assuming that there is model attacker, right? Executes, let's say, a technique, let's say a reverse shell on a compromised Linux machine that connects to him back and the model detects him because it's trained on vast variety of commands that can be used to invoke the server shell and then another set of behavior that is normal in a system. So it distinguishes pretty well. ML actually does it kind of easily with even simplest models. But if attacker assumes that there is a more sophisticated logic, before actually going to exit, like before jumping to execute the actual TTP, technique, tactic and procedure, patient like state funded, they can wait for several weeks or months before actually activating the end goal of the attacker performing the set of TTPs. During this time, they can deploy a legitimately looking backdoor in a system. So they don't do malicious logic, they compromise the system, but don't do a post exploitation actions. They deploy basically a legitimate set of gibberish, but this gibberish is kind of customly crafted. So it distinguishes well from all the malicious logic you have and usual activity that happens in a system. So it's some, for example, Python printing command that print some unique set of token. And this unique set of tokens, which is basically legit and does not trigger a sock, is the actual backdoor. So by this deploying of periodic legitimate activity, the threat actor associates this set of unique tokens, which are not seen any other in a system with the legitimacy. And after, for example, two weeks when the model sees this data, the threat actor deploys actual malicious logic but includes with the actual malicious logic this set of unique tokens. And yeah, as we see, we've done tests on a huge set of model architectures like neural networks, like more conventional algorithms, even like transformer model, which powers the GPT, we see that this manipulation of injecting patiently like even the small number of carefully drafted tokens degrades the performance of such detectors, like, basically to a level of random guess. For example, if we inject just 14 times 30 backdoor tokens just over, like let's say, every day over a set of two weeks, the performance of like classical neural network drops from 99% to 40%. So it's basically a random guess. So yeah, but by this, like by being more clever, right? The landscape changes. We have AI now everywhere. And the red team operations potentially will change with it as well. And like 10 years before, we haven't thought about a AV evasion at all. Right now, threat actors or like EDR evasion. Right now, threat actors build their software with intent in mind to evade something like CrowdStrike or Defender or to kind of not trigger them by executing the payload potentially later, like in the next recent future, the threat actors might think about, yeah, attacking an AI component of the defenses by deploying something like backdoor to poison the training set of the AI component in the defensive side. And by this basically diminishing the quality of AI detectors that way. I don't know, I hope by listening to this, it makes sense because it's, yeah, it's non-trivial concept to explain in just words, it's usually well supported by kind of step-by-step, maybe visual support or, yeah, it's definitely a separate topic in the universities, right? We right now speak about a branch of science called trustworthy AI, where like serious dudes doing a research for 20 years discuss how to break an AI and it's basically an open question. It has no solution. So yeah, like covering it in 10 minutes in just voice is hard. So yeah, if there are some clear like gaps in the description, Nic, please ask questions that might kind of enlighten those.

Nic Fillingham: I don't think this is a perfect analogy, but I want to start here because as you were explaining this process, what came to mind was the analogy, again, I don't think it's perfect, but of the boy who cried wolf. Are you familiar with that sort of idiom?

Dmitrijs Trizna: Yeah, of course.

Nic Fillingham: Right. So in this case, though, instead of perhaps maybe crying wolf, the APT is sort of crying something sort of benign. Maybe they're crying sheep, right? They're saying sheep, they're doing it, as you said, 14 times or some sort of multiple number of times. And the AI here is just going, I don't need to worry about that, I don't need to worry about that, I don't need to worry about that. And eventually, the threat actor says, sheep, but then they also maybe under their breath say, reverse shell, and then sheep again. And so the AI is like, hang on, I saw sheep. I know sheep is benign, or I think it's benign, but there was something else there, but it was also surrounded by sheep. So I don't know.

Dmitrijs Trizna: Let me stop you. It's awesome. It's awesome analogy actually. Yeah, that's great. But let me modify it a bit.

Nic Fillingham: Please.

Dmitrijs Trizna: Let's assume wolf is the bad word that he wants to scream and the sheep is something that happens in environment a lot. And those both cases are something that we don't want to scream because they're kind of prevalent a lot. With this targeted backdoor poisoning, the threat actor actually tries to select something that is not representative of either. So because if you scream sheep, the AI-based threat detector does not change a lot. It already knows that sheep is good, but if you scream something like completely unrelated-

Nic Fillingham: Like gibberish.

Dmitrijs Trizna: - gibberish or yeah, like spaceship.

Nic Fillingham: Spaceship. Love it.

Dmitrijs Trizna: The boy screams spaceship, right? No one expects to see spaceship in this setting. And the models learns to associate that spaceship, it's something normal because the boy screams it a lot. And at the end basically the actual goal of this logic is to cry wolf at the end, right? We want to do wolf thing, like we want to execute this malicious technique. But yeah, we cry at the end like spaceship wolf and basically the model is, oh, it's still spaceship. So it's still good. And yeah.

Nic Fillingham: You talked about how over a short period of time, the confidence of that model can go from 99% to 40 because it's seeing, you know, it's being prepped to look for or to see spaceship and to think of spaceship as something sort of benign. And then all of a sudden, wolf gets sort of associated with it and it's confused now, doesn't know what to do and it's like, ah, maybe.

Dmitrijs Trizna: Right. Your highlighted points, one is that frequency and quantity plays a role. The more you scream spaceship, the more you have time in a target environment, the more degraded the defensive model will be, in a sense like the more patient you are, the better chances. It's still probabilistic logic, right? The AI is generally non-deterministic. Everything that happens in the AI world basically has some sense of stochasticity and it's random. But you have a higher chance by sitting longer and screaming Starship more. Another point here is that why we scream Starship is because it's something unusual for model. It's called, when we speak about it, it's in more technical terms, we can call it that Starship is located in a sparse region of the models latent space. So model learns those kind of representations from bad stuff and the good stuff in this, let's say, models mind space and it's occupied by some examples. And you want to target something that is sparse, that doesn't have a lot of samples because by placing your backdoor there, you can influence model's decision boundary much easier there. And decision boundary is basically kind of a line in this mind space that segregates malicious from benign. So if you cross the line, you are not anymore malicious. And by placing this kind of somewhere in a sparse region, this backdoor, you can easily affect this decision boundary there. And then you place your malicious logic close to the sparse region by placing this backdoor too. So yeah, it's already kind of too technical, but that's why the logic here is a Starship. That's why it's not a sheep or wolf. It's something unique. And maybe for people that are more familiar with the topic or that are listening us, this might kind of unveil some interesting tweak here.

Nic Fillingham: And hopefully again, by the time that people listening to this episode, the video recording from your session will be up there and they can go and watch it after this. So what is the takeaway or not the takeaway, but how -- Okay, so we know that this boy who cried spaceship concept now exists, how do we protect against it? Or how do we fortify AI systems so that they don't drop in their confidence levels either so severely or at all?

Dmitrijs Trizna: It's the whole point,right, of this discussion is to outline this question. So I'm glad you came to it. And basically one of the things that this kind of research publication I hope could achieve is that people start thinking of security of AI as a part of their threat perimeter, right? Or their posture. Because by bringing new stuff on a table, you increase your perimeter, you increase vulnerabilities that your system have and attack surface basically goes wide and south. So like by bringing these things up, we actually bring really new techniques that we have to at least be aware of. And backdoor poisoning attacks are just a subset of attacks that are known to degrade AI quality. There are more easy to understand evasion attacks or so-called adversarial example. So it's basically the same, where for example, in computer vision domain, the like researchers perturb to the pixel slightly, so the image didn't change, right? You as a human see it's still the same, but it's completely different image for an AI model. And for example, you can change a stop sign with a sticker or a set of stickers to a like no limit sign or like autonomous self-driving car. So this is an example of evasion attack. But there are known ways to minimize impact of the evasion attacks. One of the most prominent right now is the adversarial training is when you, by training model, you include those bad examples in a training set. And this can significantly reduce the quality of some of the attacks, even those that are not present during this kind of adversarial training. So there are another set of attacks or maybe techniques to like tamper with the model from the privacy space like model inversion or membership inference. Those are basically trying to ensure any sensitive components around the training of model. For example, in biomedical research, they try to implement AI models as well, but given an access to this model, you try to infer whether there's some specific person was a part of this model's training set and maybe get some sensitive information, I don't know, on your severe illness or something. So these are the privacy attacks that are still some sets of techniques to kind of make models more robust. In this domain, the robustness is the term to use to define kind of model's ability to defend those attacks. And coming to last to the poisoning attacks, actually it's pretty tricky. With poisoning, I can say it's harder to defend against. There is ongoing research in various directions, for example, how to maybe do apply some logic on top of the training set and take a look whether it contains some of the backdoors or some of the poisoning samples because they can be clearly seen if they are from, example sparse region, this should alert and should be kind of easy to detect that there is some peculiar input to the model that we don't want to get there. But another outcome from this research, I've tested multiple models to this specific attack and practically all of them were vulnerable except one algorithm, which is gradient boosted decision trees. And so for this case, the gradient boosted decision trees, even after pretty huge set of the injections and size of the backdoor were still capable to detect like 95% of the attack samples, maybe 90%. So it's not perfect, but it's still acceptable. So one of the maybe outcomes here is when you guys implement AI models for themselves or for yourself, try to build a threat model, try to maybe ask your red team or hire red team to do some engagement and maybe identify for this specific deployment scenario, what is the most robust, like set of the configurations. And for this case, we actually have a robust set of pre-processor and architecture that is kind of non-vulnerable to this model, but basically all of them, all of the rest are severely vulnerable. So by making this analysis, before deploying this solution to prod, right, we just could skip this kind of red teaming thinking and deploy the best model to prod. But by bringing this like logic to the table, we clearly see what is the best solution here and what we can choose. So yeah, regarding the poisoning, some of the options might be robust out-of-the-box and yeah, exploring this. And I feel that field will mature. I will paraphrase this and I speculate that the security field will mature with more and more threat emulation frameworks appearing in this space. Right now, we have like Cobalt Strike and Metasploit and huge set of C2, the whole red teams doing these things, emulating the actual adversaries so we can defend against them. We don't have really good kind of out-of-the-box solutions for something like this. It still requires a huge set of kind of custom knowledge to build attack models. But I believe there will be eventually appearing solutions that people just can take and use on their set of yeah, like pipeline and see like out of the set of like attacks, out of the 20 attacks that all focus on privacy, on poisoning, on evasion basically, and see what are the best options among them all for something like this.

Nic Fillingham: You talked about one particular, I'm not clear on the taxonomy here. You said gradient based decision trees, is that right?

Dmitrijs Trizna: Yeah. Yeah. Gradient boosted decision trees.

Nic Fillingham: Decision trees. So you said that one was somewhat robust and you said, you know, the confidence only dropped slightly. I guess my question is, when we think about a model that has been degraded or attacked or poisoned so that the confidence level drops, is there a threshold that, you know, is a warning sign? So you know, 99 to 40%, I think everyone would agree, like, whoa, that's a huge jump, but is 99% to 95%, is that something to sort of be worried about? This may not be the right question to ask, but I'm just sort of wondering in terms of these confidence rating changes, is there a threshold, sort of a tolerance that is sort of acceptable?

Dmitrijs Trizna: It'll be the most political answer at all, but it'll be, it depends right?

Nic Fillingham: It depends.

Dmitrijs Trizna: It depends on the application. And like if we speak about security applications and threat detectors for instance, then we don't want to, yeah, like the narrowest resource here is the false positive rate or basically human analyst attention. We don't want to flood them with a set of false alerts, if like human can analyze maybe seven, $8 per day. And it's a question whether this poisoning attack degrades quality of model so that it increases false positive rate or it just increases the false negative rate. And false negatives are, just to clarify, is the samples that kind of are malicious but model misses and in this case, the degradation to be like 95% for instance, it won't increase the false positives, the analyst won't see increased rate, but there's some, like out of the, I don't know, 1000 executed attacks, you might miss like 50 of them or something like that, right? And I would say this is acceptable risk in this stuff, but it depends, right? Again, the severity of model, the severity of deployment, but it's still, like security never depends on a single point of failure, right? We built a multiple heuristics to cover each other and having like this 50, like 5% false negative rate on one of the detectors and 5% false negative rate on the second consecutive detector that may be detects the next set of technique that adversary operates, this together degrades the kind of ability to evade them significantly. And when we speak about like this random chance about 50 to 40%, yeah, chaining them together is still severe problem. But when we can speak about this few percent degradation, we can accept, I'd say generally, this set of false negative rates. And none of the detectors, it's actually, you cannot really valuably measure false negatives because you'll never scope the unknown unknowns pretty enough, but you can measure false positive rates. It's complete set. Like, we can go in the direction of how you measure quality of security detectors, but it's, yeah, it's never ending story and a lot of spears are broken here. So again, answer is it depends

Nic Fillingham: So do you love this space? Does this get you excited? Does this get you jazzed? Do you love this?

Dmitrijs Trizna: Yeah, I'd say what fascinates me in this space is this custom R&D, right? When I take a new project, I cannot open, I don't know, Stack Overflow or Google and Google my answer. There are none. Basically the whole domain is just outlining itself. And it might that these solutions might exist somewhere, but due to the specifics of security industry, they are mostly hidden or proprietary, right? No one says like, hey, we use this technique to build defenses because like everyone still tries to capture some security of obscurity and definitely some in-house solutions present. But at this level of maturity, there are a few of them, I'd say, in the world. And yeah, it's really hard to influence by any public literature. Maybe scientific papers or academia can share something openly, but those are definitely too raw to be implemented right away. They are not going in mind with the production deployment, with the false positive rate, with the quality and quantity of data. So basically anything you do is custom made and really high quality custom made. It requires knowledge from huge set of domains like AI, security engineering, red teaming, like three distinct domains collapse together to build something qualitative here. And that's what is awesome here. And that's what's interesting. When you go towards more still important steps, but more mundane ones, like how well it'll be performing over set of time, like basically maintenance questions, then this becomes like harder to say that it's awesome, but it's still part of the work, right? It's like you can draw prototypes like on the left, on the right, but while they're not in production, they are not valuable at all. So they have to come to this kind of, you have to bring the metric question on the table. So this, yeah, it's a complex question, but the most thrilling part is that it's custom. And yeah, that's why I'm trying to share as much as possible in the space like NDA approved, let's say, generally applicable knowledge in a blog post. Actually, I have some writings in Medium and in the conferences like this because it's, yeah, the domain is too raw to be kind of saturated by knowledge.

Nic Fillingham: Got it. Well, I know that the audience at BlueHat India loved your talk. You know, we saw some great comments online, we saw some great comments from attendees. Just briefly, what was your experience at presenting at BlueHat India? It was obviously the first time that BlueHat India had existed, so there was nothing to compare it to in terms of a BlueHat conference in India. But yeah, I'd love to hear your thoughts on presenting.

Dmitrijs Trizna: Yeah. Like I know for me, it was one of the most hospital week. For me, I'd say the people are so welcoming. I don't know whether you feel this, Nic, but you felt special. I'd say, I don't know, the treatment, the willingness to help to make your existence at that moment as like problemless as possible, right? And like, just everything, like to make your experience the best. And I felt this from basically any person I met there. Another interesting distinction from maybe most common Western cultures is that people are really like genuinely willing to learn a lot. Maybe we see it here as well, but it's so real and so kind of wild willingness to learn from them that they are basically asking anything, they then so much questions, so much, like cannot -- yeah, it's hard to express, but really deep interest. And that's what I see even from the people that are not maybe directly relate to the field, they're still deeply interested in the outcomes and the kind of preconditions and to everything. And yeah, that is interesting, right? On such a conference, you definitely will have a broad audience that work with a web application, that work with, I don't know, reverse engineering, that work with, they don't know anything about AI or they don't know anything about that conventional set of infrastructure security. Like still practically everyone was super interested and willing to go deep. And yeah, that's something distinct to India, I'd say. Yeah. And generally, conference was awesomely organized. I don't know whether you were part of the organizing team, but if you were, then thank you to you and all of you. Awesome work done. Really great.

Nic Fillingham: It was a great conference. Yeah, we had a wonderful time there. And I echo everything that you just said in terms of the wonderful hospitality and generosity of the people there. And yeah, just the willingness to inquire and be curious and ask questions was fantastic to be a part of. We're coming up on time. This has been an amazing conversation and I think we could keep going, but I need to let you go. I want a couple more questions before we do that. What's coming up? What are you working on that you can talk about or are you presenting at any conferences coming up that you'd like to discuss?

Dmitrijs Trizna: There are several directions that I'm trying to do this, let's say, publicly viable research. Of course, some of the work is really actually proprietary that is hard to generalize and share. And but some of these ideas is basically building some correlational logic in a form of a graph of activity and try to model the graphs instead of in a separate instances. And this is actually promising direction and I'd say that's where security is heading, the defensive security, defensive analytics, trying to segregate all the never-ending stream of data into a separate chunks of interconnected relations and like analyzing those relations independently. And it's an open question, how to build those, how to slice this data, how to make it efficient, that's super cool and interesting. And yeah, probably at some point, I'd like to share some ideas about this, maybe in a scope of Kubernetes security, right, which is an open ended question. And again, too little is known publicly. So I'll try my best to generalize some of the working ideas here and share with them. Another branch of active work is in relation actually to my PhD group that I'm kind of working with Italian university. They're awesome people doing really great research in direction of like how to attack models. And my domain of inquiry mostly relates to the malware detectors and attacks on malware detectors, basically AI-based detectors. And we actually bringing in a course to the BlackHat this summer in Las Vegas to the States about basically building ML, machine learning based malware detectors and attacking them and drafting perturbing the malware samples so that they can evade those models. So yeah, it'll be a two-day course. And yeah, if any one of you planning to come and have a budget somewhere, feel free to kind of join. If not, please reach me directly if you're interested in topic, I'm always happy to share this, all the information without any cost at all to anyone willingness to learn, I'd say. And that's kind of two directions. I'm kind of occupying my head right now, like I hope at some point to have more time. Yeah, but it's like so tight resource, right?

Nic Fillingham: Absolutely. And where should people reach out if they want to get in contact with you or follow along? I think you mentioned you have a Medium perhaps, and is there anywhere else you'd like people to go to either follow along or get in contact?

Dmitrijs Trizna: The Medium is basically a blogging platform. It's not maybe best for chatting, but any kind of mainstream social media, Twitter or LinkedIn suits. So please feel free to write me. I might not answer for a week or so, but definitely at some point, I will now have this logistical part of my activity where I answer all the stuff definitely because people write me on some of these questions and I write to some people. And it's one of the best ways to find like-minded folks, right, that work in a similar sub-domain of fields which are becoming wider and wider. So yeah.

Nic Fillingham: Awesome. Dmitrijs Trizna, thank you so much for presenting at the first BlueHat India and for joining us on the BlueHat Podcast. It's been a great conversation and I look forward to chatting with you on another episode at some point in the future.

Dmitrijs Trizna: Cool, Nic, I really like your questions, really well guided, and in this sense, thank you for offering me to join you. And yeah, awesome discussion. I agree.

Nic Fillingham: Anytime. And thank you for staying up late. We appreciate you doing that. So thanks for joining us and we will talk to you on another episode of the BlueHat Podcast.

Wendy Zenone: Thank you for joining us for the BlueHat Podcast.

Nic Fillingham: If you have feedback, topic requests or questions about this episode.

Wendy Zenone: Please email us at bluehat@microsoft.com or message us on Twitter @MSFTBlueHat.

Nic Fillingham: Be sure to subscribe for more conversations and insights from security researchers and responders across the industry.

Wendy Zenone: By visiting bluehatpodcast.com or wherever you get your favorite podcasts. [ Music ]