Allie Howe Hey everyone, welcome to today's episode of the Insecure Agents podcast. I'm your host, Allie Howe, and today we have Dor Sarig here to talk to us about the new SAIL AI framework from Pillar Security as well as Pillars AI Security life cycle platform. And today we are on the backstage of Black Hat here in Las Vegas at the beautiful Bellagio Hotel. So super excited to talk to Dor about what's going on here at Black Hat and what he's up to at Pillar Security. Dor, would you like to introduce yourself?

Dor Sarig Yes, absolutely. Thank you very much for inviting me. Great to be here Dor Sarig almost 20 years in cybersecurity, wrapping up my second decade. Started very young, got into hacking. That was my passion, breaking stuff. I saw it almost like trying to solve a puzzle. I was very good at that. Almost got into trouble at some point, but eventually that's what kick started my professional career in this space. I spent close to a decade working with the Israeli government, mostly offensive cybersecurity and threat intelligence and alongside that LED product roles where I had like the technical hands on experience on one hand and also the building and working with with customers on on on the other hand.

I was in Simulate the product manager and then at Perimeter 81 before the acquisition to Checkpoint. Yeah, very happy to be here today.

Allie Howe Awesome. Yeah, Thank you. So excited for you to be here with us and spend some time today. How has Black Hat been for you so far?

Dor Sarig Yeah, I think it's my fifth or sixth year, but every time it's always great to be here. I think it's a it's amazing opportunity when you think about it, when you have so many tons of people from the industry you've been and can be great to, to meet and talk with, connect, learn from the great sessions that you have here. So, yeah, always great to be here during this week. Like so many things to to do. So always like, not enough time, Yeah.

Allie Howe Yeah, it's great to see familiar faces from RSA and be able to see people at regular cadences at these conferences. So super cool to reconnect with folks. What is Pillar Security been up to lately?

Dor Sarig Yes. So Pillar Security in a nutshell is a company that helps teams enable AI into organization in a, in a safe and secure way. And, and the way that I like to approach it or the pillar of philosophy is really looking into this problem for a first principle perspective, right? Like what has changed in the world that requires to have new security for AI And, and when we look at that, we always like to look at what we already know, which is how do we secure software today, traditional software and we have the depth like security life cycle with the different stages of how you build software and how you secure it.

But when it comes to AI, I'm sure that we all know by this point that this is a new kind of software and the way you build it is also new. So the way we see it, it Pillar is it almost creates an entirely new development life cycle, which now needs to be secured end to end. So the project we're taking here is really process driven. How can we infuse security across each of the stages of this AI life cycle, from development, testing, experimentation to deployment and, and usage. And it's this is what we've been doing over the last year and a half at Pillar.

Allie Howe Amazing. Yeah. And just for our audiences context, I think because I personally didn't know this when I had to learn about it, but you know, what makes the AI life cycle different than maybe DevSecOps pipelines of the past? And for me, it was really interesting to learn that, you know, machine learning scientists or data scientists, they work in Databricks and these platforms that are not typically caught or scanned and the typical DevSecOps pipeline. So we need tools like pillar security that have eyes on what's going on there. So making sure that we're scanning different models before we run them just so we don't have like model serialization attacks happen.

Making sure that we're looking at models that you know, what data it was trained on, looking at AI SBOMs, for example. I mean, there's so much that goes on the specific to AI and why you need a specialized platform for this. Do you want to expand on that at? All

Dor Sarig Yes, absolutely. Yeah, exactly. So we have yeah, the other AI and data platforms from Databricks, Snowflake and others where we know that AI and machine learning activities is happening. It can also create like lots of risks. But there's also, I would say like beyond the CICD where, you know, now AI is happening on the endpoint workstation level, right. We have employees in the organization building their own agentic systems or utilizing MCPs on their machines. The providing the tokens and the permissions outside of the security life cycle and awareness. So like this is like where it's like extends and it needs to be governed in a way.

Yeah. And I would just like add to also what you mentioned, just like we have, you know, tools like Snyk that eventually scanner code for vulnerabilities and maybe packages. Now we have other kind of assets that we need to evaluate. So we have models for models deserialization attacks or any other vectors or poisoned instructions. We recently published a code research where we were able to inject malicious instructions into a GGUF file that is being used as a template for models, right. So we like to say Pillar that inherently data is executable and no matter in what form it comes in touch with those models, it can serve as an instruction that can be also malicious.

So just like Snyk scanning code and packages, we should also scan model files, but also the tools themself and what can they do and the instructions given to the models and so on. That's what we also do at Pillar, yeah.

Allie Howe Yeah, I saw your research too about the Cursor rules files that you were all able to scan and find Unicode, hidden Unicode characters in them and warn people of those. Because, I mean, people are just sharing these role files everywhere and they're not even aware that they need to scan these for different attacks. So it's really amazing to see you help educate the community and that way. Does Pillar have any sort of like resources for like, I don't know, like a checklist or something like all these different places that you need to go and scan and think about, make sure you're covering like maybe MCP tools or your Cursor rules.

I mean, there's, there's so much out there now, so many assets like you talked about that we need to be worried about.

Dor Sarig Yeah. Great question. We actually published a book, A blog post recently about it. We call it the AI sprawl in a way. And we mapped I think over 30 different AI assets. So you have the model files, you have the MCP, you have the configuration files for coding agents that we just mentioned, like rule files, for example, and you have them, the instructions themselves, you have the frameworks notebook on its own, it's an asset inside it. It has some inactivity credentials. Tokens themselves that are used for, you know, utilizing the different models are as well an asset that needs to be secured. So we're really looking at that again from a first principle point of view.

What are the crown jewels that we need to secure here in each of the stages of the life cycle here?

Allie Howe At Pillar Security, what's something surprising that you've seen?

Dor Sarig Yeah, that's a good question. I can tell you one, one story without getting to like, you know, customer names or, or, or verticals. But you know, when we speak with security teams and and ask them like, what's your AI footprint in the organization, where and how you're using it? Some would told us at the beginning at least, like, well, we're not using AI, we're not there yet. Well, it is that's what they thought because after we, we did our, our comprehensive discovery exercise scanning, it was like 3000 code repositories, which wasn't their entire code repository, but we surfaced over 30 different AI projects that we're not aware of at all.

So you know, like security team seeing that and like there's like who allowed that and, and what's the risk here? And and how can we take it out right now some of the world in production as well, right. So I think it also come to again, this is entirely new development life cycle with new assets. This is like almost a new language that you need to now build just like you build with security and R&D for the last decade or so now needs to happen with data and AI with security.

Allie Howe And then one other thing I wanted to touch on too that I was excited about was the new SAIL framework that you all came up with. I was able to contribute to that as well, which was super exciting. It was really cool that you partnered with the community and invited a bunch of individual contributors to come on and help influence that framework. This is really awesome. I think when the security community is able to come together and bring our unique ideas and perspectives and what we're seeing and work together to create a helpful resource for teams. What I personally really like about the SAIL framework is I feel like it's very practical.

It breaks down the different stages of the AI life cycle and will list the threats that you should be worried about per each part of the life cycle. So I think it's a really easy way for someone that maybe doesn't have an AI security program today to be able to look at that and approach and think, OK, where should I start? Which I think is really difficult for teams because one thing that I'm hearing when I talk to people today, which is very different from when I talk to people, I say, like, you know, in the beginning of the year is there's now so many frameworks.

There's too many, there's too much noise. They don't know which ones to take on. Whereas in the beginning of the others, like there's like no guidance and I work a lot with them. OWASP, help them create some of their documentation also. So now it's like we have all of these resources, but I think the ones that are the most practical and hands on instead of like theoretical and most actionable, those are the ones that are really going to be helpful. And I think that's what SAIL is. And I was really excited to have a conversation with Steve Vandenberg as AI security architect at Coactivity on a previous Insecure Agents podcast.

And he was talking about he works at a healthcare or health tech company and he did HITRUST for compliance with the NIST AI RMF section tacked onto it. But that wasn't enough for his security program and not his his eyes. So we also used SAIL and some of the controls from SAIL and I do compliance for AI startups as well. And that's very similar to how I run my program where we'll take something like SOC 2 and add on custom AI security controls. We'll put those in the trust that are making sure that we're advertising that we're taking AI security seriously because like some of these frameworks today don't innately offer AI security control.

So I think that's a great way for someone to take a framework like SAIL, add it to their program and be able to go to market and say, OK, here's all the controls we have for AI security. And, you know, we're using Pillar, for example, to influence and prove that we're, you know, actually implementing these controls. For example, I think a lot of the controls in the AI frameworks today are have something around filtering AI inputs, outputs, having some sort of AI runtime security in place. And that's something that Pillar offers.

Dor Sarig Yeah, yeah, yeah, yeah. So I'm happy to tell more about like sale or maybe like the motivation behind it. It's exactly the use case that you mentioned that happened with, with Steve, that also happened with almost every of our customers and partners be worked with over the last year and a half. It's that there was like great frameworks out there, but when it comes to how do you really take them and and deploy them while you build those, you know, AI initiatives and project and like there's like some gap that we that we realize like, OK, so there's an indirect prompt injection or prompt injection.

This is the risk, but we're just like this risk. Meet me in the development process. For example, you know, indirect injection can happen inside an MCP configuration as well as like the like the prompt itself for all the instructions or red teaming. We know that we need to do red teaming, but red teaming. But what's the right methodology here? So I would say that like SAIL is kind of like, you know, in a way first, like trying to point where you should look at first. Like this is the area where security needs to be, you know, checked. And of course, with the different AI assets that we mapped here.

So it goes way beyond from security in this case, you need to secure each one of the assets in this AI ecosystem that now has created. And yeah, the fact that we were privileged to work with leaders in this industry, including you and and and Steve and others, it was like great to to learn from you, from your from your experiences, because we really believe that the practitioners needs to lead this this effort. Yeah, I would say that we we made SAIL to really be you know like this go to manual practical tool that you can apply no matter which stage you are in this like a security maturity because you know different companies in different stages, no matter how big the company is to apply, help you apply security to the level that you you know are comfortable with based on your risk appetite.

Allie Howe Yes, absolutely. Yeah, I think that's a great call out too, is that it really differs where people are in their AI security journey. And I think sale can really help no matter where you are in it. And with a lot of the different AI startups that I work with, they typically will get these security questionnaires. Even if they've already done Sock 2, they'll still get a bunch of security questionnaires around AI security specifically because I know, like Sock 2 doesn't really cover that innately unless you, you know, add in your own custom controls. Maybe you've pulled something from the sale framework out of that end, but if not, a lot of these companies start getting these security questionnaires and all of these different potential customers that they want to take on Have questions around how are you going to handle my data securely, especially if you're using AI?

And how can I feel safe as a customer handing my data over to you to handle? Is that typically an inflection point for someone to come to Pillar Security and ask for your help?

Dor Sarig It is. It is. And I would say this is where you realize that security is not only just for security, it's really a sales enabler in a way. So the security teams we work with are eventually the enablers and not the blockers. And you know, like one, one example was that, yeah, like one of the security folks we're working with just told us like, hey, we got those bunch of questions from most of our customers, very big ones that are not willing to, to use our product or those with, with AI until we are not addressing their their needs around data. What are we doing with the data?

How can you make sure this data is stayed safe, not being used for like, you know, training and other stuff. So yeah, definitely security is going to be one of the thing like key, like business enablers in the area.

Allie Howe When you talk to, you know, potential prospects, customers, how far along are they typically in their AI security journey?

Dor Sarig Good question. Question. Yeah. And you know, it's always changing as we know in this era. So, yeah, I mean, we, we have, we're working with customers that have, you know, AI application in productions that are in volume of like millions of like prompts and interactions daily. So it's like very massive. And we have others that I guess like more bigger on that regard. But they're just like now building their AI security road map. This is their main task. It's a group of like AI center of excellence and, and sort of that and, and, you know, really seeing like as we are their AI security partner in this case, like helping them holding their hand from the beginning.

Because in that case, before jumping into red teaming of guardrails, first we need to define what is the AI policy in the organization and how we can, we're going to enforce it. So this is where we can also start and help with, you know, discovery and mapping of your AI assets in the organization.

Allie Howe Yes, that's such a great point. It's a great place to start. Before you can start to secure and manage things, you need to know what exists right before us and discover those. I know you had mentioned something about what you saw Black Hat that piqued your interest on the around taint analysis.

Dor Sarig Oh yeah.

Allie Howe Do you want to explain like what that is and why that's could be potentially helpful for AI agents?

Dor Sarig Yes, absolutely. So, you know, when we look at the agentic risks again, from the first principal point of view, what is an agent right under the hood when we are unwrap everything, get to the to the atoms here we have at the end of the day, foundation models, right? That are the kind of like CPU in this case. But on top of that, you have the instructions you give the agents and the tools they can use right, to interact with different software and services. And this is the attack surface that we are looking at. So when we look at agent or, or a specific task that agent can do, such as let's say read messages from Slack, a read only, right?

So on it's own, it has this risk, but it's kind of like contaminated in this era. However, when we are looking at other tools that this agentic system can can trigger, we can see, let's say there's a, a GitLab integration with write access, right? You can write code and push code. So when you think about it, it is the combination between those two that when are integrated, this will create the, the, the toxic combination. So let's say there's an indirect prompt injection that is hidden inside of one of the Slack channels and the user is just like prompting, hey, like summarize my Slack messages for me.

And by doing so, the indirect prompt injection is triggering the agent to go and call a different tool, the GitHub tool in this case with a pull request like a commit with malicious package to be deployed in production, right? So this flow would have been possible without Slack and and and GitHub combined. So when we look at taint analysis or a way to really identify those like we can utilize taint analysis as one of those like great frameworks where, you know, we have the idea of like tainted data, right? Every data that is not secure, insecure. In this case, everything that come touch with models, we should treat as insecure like 0 trust and then see how this like data like travel or what's the flow of this data and where it gets to those like sinks where it's like can 'cause like sensitive data leakage or or other kind of risks.

So it's really like fascinating to kind of like try to utilize the taint analysis in order to discover those different branches that right, we're not aware of them in agentic workflows.

Allie Howe Yes, that's super exciting. I feel like that's going to be super helpful for figuring out how data is able to pass through agentic workflows and get exposed potentially that way like you mentioned. I think it'll be also interesting to see if people start using this technology taint analysis, which I know have been used for like SAST tools in the past for figuring out how identity management is supposed to work with agents. Just to see an agent, you know, maybe assume a human identity to create a task or complete a task and then maybe it's a multi agent system. It's going to pass that human identity down the chain.

It's like, how far should that identity go? Where does it lead to when you know, that identity starts to be connected to different systems like you mentioned, maybe it's connected to Slack, maybe it's connected to Google Drive and who knows, you know what that identity should be Short lived, long lived. I've met a lot of different startups recently. I feel like I've had a lot of conversations with people that are trying to solve OAuth for agents. So really excited to see how taint analysis might be used in the future.

Dor Sarig Yeah, yeah, I think taint analysis is, is one of the ways to really like to understand like what's happening out in the hood. Like you can't debug those system. That's the most like it's like a problem here. Like you can't like, you know, put like a point in like trying to understand like what caused this to to happen. Yeah. I mean, you know, it's like, I think what we're going to see at the end or, you know, like. Within 12 or 18 months from from today, like when you look at where like those capabilities are heading with more autonomous software, who do you think will pose the biggest cybersecurity risk on organization?

It's going to be the employees or those autonomous agents that are going to be an extension of those employees and now going to use your e-mail to do stuff right. So they can fall into phishing just like you can or browse the Internet to do their tasks. And again, you need a secure gateway to secure their interactions or then downloading files or executing code or running code, right. So all the security controls we've built so far, we're focusing on securing human beings and and their flows would now need to adjust to those identic systems. That's gonna mimic the same behavior. Yeah.

Allie Howe Yeah, it's amazing. I feel like how much we don't have control over in agent workflows, like of course that the LLM itself is very non deterministic, but then we're connecting into different MCP servers which have tool descriptions that, you know, we didn't write, someone else wrote and rug pulls happen, tool poisoning happens. I mean, there's just so much we have to have eyes on and monitor now that we just, I mean, the easiest thing is just to stick everything into a private subnet in your cloud. Everything's on Prem, everything's contained, you have control over it. That's great. That's a perfect world. That's just not how agentic systems work today.

And especially if you want to be innovative and use the latest technology, like that's just not how your setup's going to look. So I, I personally think having runtime security and A and a tool like pillar security in place to keep an eye out of, like discover what AI you're using and then monitor it and making sure that it's behaving as it's supposed to but remains aligned to your use case over time, I think that's going to be something that every team's going to have to have, yeah.

Dor Sarig I I totally agree with you on that. I think that, you know, it's, it's not only enough to really identify the different like risk, but also how we can like mitigate them in this case. So I would say like one of the core capabilities of pillars like on one hand is like continuously identified those risks during red teaming that we are doing, which is really application centric, right. We're not testing the model with how it's integrated into MCPs or other AI assets. This is where the attack surface is happening. But on the other hand, it's like how can you utilize these insights from those assessments and optimize the guardrails or make them more adaptive the application.

So this is really where we're heading. Eventually, like every application, every agent there have will have their own set of of guardrails which will be again optimized to their risk profile and and permission. The scope which will also constantly will be changing and and updating.

Allie Howe That's amazing. I think we just kind of touched on this, but what's next for Pillar Security? I always really enjoy watching your research that comes out. I read all of those blogs. Super interesting. Anything that you want to foreshadow or highlight for us?

Dor Sarig Yeah, sure. A lot of things are happening. It's pillar. So, you know, when we look at identic systems and you know, like software is in, in general, when we're talking about this space, it mostly focusing on one main modality, which is text, right? Because it's like much more than that as we know. I think it was like Andrej Karpathy or like has this like, you know, comparison where like take LLM and compare it to a CPU and then you have the different connection integration just like you have with CPU today to the Internet and to audio and video and so on. So each one of them will have and have their own attack surface.

So like how, you know, we're able to identify it, evaluate the risk and secure those modalities as well. This is something that we are very excited to work on with some of our partners. And I'm just like saying, you know, when we're looking maybe a little bit more ahead. So we talked only about software, but about about hardware, right? So if the LLM is the brain here, when what happened when you connect to this brain, hands and and legs and you give it the ability to move in the world and interact with the physical world, then how you know what will happen, what, what can go wrong if this can be hijacked or, or abused?

So this is also areas we're looking at.

Allie Howe Amazing. Yes. Well, I've always enjoyed following your research and we'll definitely continue to do so. I'm super excited about those new updates that will be coming out and where you're headed with Pillar or so much you've done already with the entire AI life cycle platform, which is a lot to secure. So super excited to continue to follow your work. Is there anything else you'd like to talk to us about today?

Dor Sarig No, it was very fun to be here. If you want to learn more about Pillar, just, you know, go to a website LinkedIn, search for the SAIL framework. Everything is a website. It's a, you know, free open source, community driven effort that we are constantly improving. So if anyone wants to contribute to the SAIL, we are more than welcome to invited to contribute and yeah, looking forward to next time.

Allie Howe Awesome. Yeah. Thank you so much. Thanks for spending the time with us today. We really appreciate it.

Dor Sarig Thank you.

Allie Howe All right, that's it for today's episode of Insecure Agents with Dor Sarig, CEO of Pillar Security. See you next time.