Allie Howe Welcome to today's episode of the Insecure Agents podcast. I'm your host, Allie Howe, and today we have Kikimora Morozova, an engineer at Trail of Bits with us today. Kikimora, would you like to introduce yourself?

Kikimora Morozova Absolutely. Thanks for having me on. My name is Kikimora Morozova. I'm an AI Security engineer at Trail of Bits and also a computer science undergraduate at MIT.

Allie Howe Amazing. I'm super excited to discuss the research that you published with Trail of Bits, but before we jump into that, I was just wondering if you could tell our audience more about Trail of Bits, just in case someone isn't familiar with the work that you all do.

Kikimora Morozova Absolutely. So Trail of Bits is a premier cybersecurity consulting firm that specializes in helping engineering teams build secure software. We focus exclusively on software security, discluding Trisha, IT, network security or enterprise security. Within that we have teams for OPSEC, research, engineering, cryptography, AIML security and others. I personally have been very lucky to work on AI security related problems for both the OPSEC and the Research and Engineering branches.

Allie Howe One of the blogs that caught my attention from you all recently was this one about image scaling attacks. I think we're seeing more teams develop multimodal AI. Could you tell us more about what that research entailed and what an image scaling attack is?

Kikimora Morozova Yeah, So just for some definitions, image scaling attacks are attacks that exploit the image pre processing layer in AI systems and use that to cause unintended behavior, whether that's prompt injection or back doors or poisoning of some kind. And these have been known in the literature for quite some time since about 2019 when they were mostly used for kind of backdooring models. But Trail of Bits' main research contribution here was my colleague and I, Suha Sabi Hussain, worked on a project weaponizing this against production AI systems with the kind of prompt injection angle. So basically saying, OK, AI image systems don't see the same way that we do, where if you show us a high resolution image, that looks basically the same as a low resolution image most of the time.

But AI systems, if you show them a high resolution image and you don't scale that down, that's a lot of compute, that's a lot of processing power. So companies don't want to do that. So as a result, we scale images down and hypothetically they're supposed to look roughly the same to the AI system like they would to a human. But the algorithms that are used for this are vulnerable. And so if the algorithm that's used for this is too aggressive or it uses some kind of like prioritization of various pixels in an image or for others, you can exploit that once that's for backdooring or now for prompt injection.

Allie Howe Interesting. You feel like we see sort of maybe similar things with text when it comes to indirect prompt injection, where people will hide A prompt within a file or they'll use hidden characters. So like our eyes can see it but the LM can see it. Is that the similar concept to the how these image scaling attacks work where a hidden prompt gets hidden in the image somehow?

Kikimora Morozova Somewhat so it is the same core idea of, you know, covertly injecting data key innovation here as compared to, you know, white text on APDF, which we've all seen and and does still work on production enough times to be concerning to the human eye. Even if you were very trained and perceptive and we're looking for these things, you really couldn't catch it as compared to APDF. You know, if you're going through and text highlighting and you see at the bottom, hey, there's this like hypothetically empty block of text that's been highlighted, something's probably wrong. Or, you know, you have these automated scanners for like Unicode characters, so you can make sure that there's not like a little prompt injection tagging along an invisible character when you're pasting content into your LLM.

But this doesn't exist for images, and there's really no way to tell that your model has been injected unless it's already doing malicious behavior.

Allie Howe Wow, interesting. How does someone hijack the downscaling algorithms that are used with images?

Kikimora Morozova Yeah. So there's kind of a couple steps here. We don't know how the LLM sees for most of these black box models, because of course those algorithms are generally speaking not public. So first you have to fingerprint things. This is something that Trail of Bits built an internal test harness for. And something that we do is we say, OK, what kind of processing steps is this model doing? Are we converting file types, maybe PNG to JPEG? If we're downscaling, we try to fingerprint what that algorithm is. And not only what is the algorithm, but what is the library and what are the parameters being used?

Because that's the most effective way to say, OK, how can we reverse engineer this? How can we make this attack as tailored as possible? On one of the examples in the blog post that you brought up, we were doing this on Gemini, right? And so we were like, OK, it seems like this is using a technique called bicubic downscaling, which is known in the literature, but there isn't essentially no downscaling algorithm is invertible inherently due to the nature of what you're doing, which is crushing data. So there's not a current injection suite for this. So we say, OK, what pixels within an image is this bicubic algorithm focusing on or prioritizing as high importance?

And then we say, OK, let's focus on those. Let's see what kind of perturbations we can embed. And in the specific technique that we did, which is not necessarily generalizable to all downscaling algorithms, but most downscaling algorithms have a technique that will work. We said in dark areas of the image, we're going to perturb the red channel because this is the least visible to human eyes, but very visible to the downscaling algorithm, as in we will amplify that red change over the image. Result of this was, I think it's visible in some of the images on the blog post. It's basically saying in these black areas of an image, it looks black when at highest resolution, but then the background becomes this kind of red color and the text stands out as black on that red color.

And then at that point, the model looks only at that downscaled version and can see OK, there's text in there that I, as the model, should be acting on.

Allie Howe Wow, that's super cool. I feel like this is similar to the concept I was learning about where people were trying to hijack audio models as well, where they were just like adding these perturbations or whatever you call them to audio files which are invisible to the human ear. But the LLM can understand it and cause it to take a different behavior. When you all are figuring out how to do this, whether it's an audio model or you know, the image model, how do you begin sort of attacking these different models? Like is it a lot of just like trial and error, like figuring out like what works?

Like if the human error can't hear it then like how do you know where to start I guess?

Kikimora Morozova Yeah. So actually good examples from recent audit that we did. So obviously can't say the client or the target, but we were working on a AICLI development tool. So we look at, OK, what kinds of inputs does this system accept? So for the audio example that you brought up, not applicable here, but for example, we say, OK, what is the range of audio that can be encoded in the kind of audio file that this accepts? Does the system then limit, you know, cut out maybe the top amount of however much frequency it is that the human can't hear? If the system doesn't do that, well, we know that that's immediately differential.

That's something that the system sees that a human auditor or human user does not see. And this is always a dangerous concept, especially with an AI, right? Because if an intelligent system could see context that is being provided by a human and therefore trusted, and the human can't see the context that they are providing, this is a problem. And so we'll go systematically through each of these different kinds of files and say, OK, where are these differentials present? And then we say, are any of these exploitable? And sometimes they're not, right? If the top part of the range is so small that you can't really embed anything interesting in there, or maybe the system just deprioritizes the top part of that range so much compared to the bottom part that it's essentially non workable, then we move on to the next thing.

But often they are, and that's why this approach works so well is because it can systematically say, OK, here's all the places that there could be differentials. They can go to a client and say here's the places that you should be securing now. But also, if you add additional functionality, look for these kinds of differentials in your new file formats.

Allie Howe So many follow up questions to that. I guess first, when the client goes to secure themselves from these kinds of attacks, like what are the typical of mitigations that you all suggest?

Kikimora Morozova So there's a couple. The biggest thing, at least in the kind of OPSEC AIML view is design patterns. The end of the day prompt injection is a flaw of the architecture rather than a true vulnerability. So it's always going to exist in so far as we are using LMS in this way. LLMS are important at this point to the way that a lot of products function. And so we can't say just cut out the LLM, but we can limit the LLMS privileges. It was a very interesting research paper that came out a few years ago basically for like design patterns and securing AI agents and they presented these six new design patterns.

Some of these are very valuable, especially to multi agent systems, but some also work on just any multimodal system. For instance, the big pattern that they suggested was the action selector pattern where we say, OK, this model can take in this natural language data, whether that's from an image or otherwise, and decide which of the kind of predefined actions it's allowed to take. And then it picks one or more of those and just does them and doesn't get feedback from those actions about what it should do next. This is pretty secure in that say you have asked your LLM to go make an HTTP request to some external location and this is a GET request.

Whatever comes back from this GET request can't then you know, prompt inject the the context window. There's also some problems here because there's functionality that you need feedback from whatever you're doing to be able to implement. And so there's a couple of other design patterns that we can recommend or modifications those design patterns that we can recommend to clients to say, OK, how do we balance usability and security? And how do we basically have models that can decide before they can be poisoned by some external thing that they might find through a request of some kind what they're going to do. And that way they can't kind of escalate their own privileges based on some external piece of data.

Allie Howe Yes, I think I saw that same paper as well. I think I saw Simon Willison talk about it as well, like with the dual LLM pattern, which is like another one of those six mitigation strategies. I feel like we're waking up to this as an industry where how you architect your agent, like you said, really opens up yourself to different like risks. Prompt injection is just a part of AI. It's not going away. It's not like a true vulnerability. We have to assume breach and assume that a prompt injection is going to happen. And so making these architectural changes like you're suggesting, I think are a great example of adding in hard boundaries that can actually stop the agent from taking further actions or acting on untrusted data and getting into a lethal trifecta type situation.

So I think that's really, really great advice, especially when I'm hearing more and more people talk about how, you know, LM guardrails, for example, are a soft boundary. They help, but they're not like a true solution to those hard boundaries and those architectural changes. I think those are definitely great ideas for sure. You said most of this research was done on Gemini. Is Gemini more susceptible to image scaling attacks or are other models as well?

Kikimora Morozova Not necessarily. So important thing to recognize here is it's not a specific downscaling algorithm. It's not even a specific signals processing algorithm. This attack is based on the Nyquist Shannon sampling theorem, which is something that we bring up a little bit in the blog post. But essentially boils down to if you're going to be processing data in a way that's lossy, in a way that's especially severely lossy, like a lot of these systems are. Because you know, if you have a really big image, you still want to be able to process it, but you have to squash it down by quite a lot in the dimensions axis.

Then that loss inherently allows for some amount of embedding of malicious data that can be revealed because you can't accurately get a sample of everything in that image because of how much you have to scale it down. Gemini just happened to basically be the target that we chose. And the attack works on Gemini. We wanted to have a kind of production proof of concept to kind of make it clear to people that this was not a theoretical attack. This was not something that's just in academia because this was very well described in academia, but something that is a real legitimate threat that people have to be thinking about when designing multimodal systems.

And as multimodal systems get more kind of prevalent, you are, I'm sure this could be done on other AI systems, whether that's, you know, a ChatGPT wrapper of some kind. So like not actually the model and the model provider, but rather some kind of like third party product wrapping it or on even the provider itself, like how Gemini is a provider. The important thing to note here is that whether you're building on top of an existing system or whether you are the existing system, you have to be thinking about what is actually happening to multimodal inputs and is the user seeing the same thing that the model is seeing.

At the end of the day, if you had more transparency of, you know, this is hard to do on a CLI, I get why this would be difficult to implement technically. But if the user can see the same, you know, downscaled image that the model is seeing, then they can identify, oh, look, that doesn't look like what I uploaded. That's clearly prompt injected. Until that happens, we can continue seeing these kinds of vulnerabilities.

Allie Howe To clarify, are you saying that model providers like Gemini and Google for example, should be changing their CLI's to help you understand if there's a prompt injection visible in the image or not? Or is this responsibility to prevent image scaling attacks more on the user and using some sort of tooling on there and to make sure whatever image they're uploading does not have an image scaling attack?

Kikimora Morozova As lovely as it would be for every user to have the time to kind of check their images, unfortunately this isn't reasonable. We did come out with kind of like a test suite tool called Anamorpher that could be used for this, but objectively, no one really has time for this. It is on kind of developers AI applications, whether that's the provider or third party service to help protect their users. Again, at the same time, these things are technically difficult, especially on the CLI, especially for developers who might get, you know, decision fatigue and just say Yolo, you know, allow everything. I don't really care if this is an untrusted repo and protecting against that is a very difficult problem and ends up being a lot of a social engineering problem and a lot of a UIUX problem as opposed to even a like strict AI security one.

But I do think it is still an important problem to think about and protect against, especially in kind of high importance applications.

Allie Howe Yeah, I wonder if like the future will look like uploading an image to one of these models and then before it actually gets sent to the LLM, some sort of like code is run that reverses maybe the downscaling or whatever you call it, and then shows you what the image actually looks like. And then maybe you could see the untrusted content in it. And so it would ask you, OK, are we sure you want to continue to upload this? So like maybe that would work. But again, I hear your point too around like, you know, maybe people just get decision fatigue. I think there's just so many trade-offs with AI security and usability, which I mean, that's the tale as old as time when it comes to security and cybersecurity in general.

I think as we learn more about these risks and how to prevent them, users might have to come to play and understand that it might take more of their time. But yeah, it'll be interesting to see, I guess, what the future looks like in terms of trustworthy AI, trustworthy LLMS that don't involve the user being responsible for checking everything or they're going to run into decision fatigue and eventually, you know, the humans not going to make the right decision. Super excited to watch that space evolve. I don't know if you've any thoughts on where you think the future is going when it comes to that.

Kikimora Morozova Yeah, I'm also very excited about where that space is going. I think some of the most kind of pressing problems to solve, an example related to processing attacks in general would be if you have a 40 page PDF and you show a user preview, are they going to read a 40 page PDF twice? Absolutely not. The whole reason that they uploaded this, so they don't have to read the 40 page PDF. And so you have problems like this where you have to figure out, OK, how do we convince the user to do the bare minimum amount required for like human intervention kind of side security without turning it into a relying entirely on the human for security or just tiring out the user because at that point, what's the point?

I think a lot of the time these kind of design patterns can mitigate this sort of risk where kind of just the consequences are not quite as bad. And so if a user does get decision fatigue and does make the wrong choice, it still ends up being OK. But I think ultimately there's just a lot of kind of new research coming out on this right now that I'm very excited to watch.

Allie Howe Can you expand on that? Where a user makes a decision, I can still end up like being OK. I'm trying to think of like a concrete example that I could like tell our listeners.

Kikimora Morozova So for example, say we have some kind of code developing assistant and there is an image that is just kind of in the repo. And so it's within the context window of the LLM. And somehow the LLM starts processing this image and it's a malicious poisoned image. And so the user has now introduced poison content into the window and hasn't looked at it properly and so doesn't realize that it's poisoned. Well, if the LLM happens to be a kind of LLM that DE prioritizes images as compared to text and the text has nothing to do with, you know, whatever the image has to do, then it might still end up being all right.

Or if the image tells the LLM to go call out and phone home to some random URL and like exfiltrate user data, but the LLM has used an action selector pattern or planned and execute pattern and hasn't allocated that. It will warn the user and say, hey, like, why is something in this context asking me to go call out to this endpoint when I was just supposed to use that like, like read image kind of tool? So these are cases where other guardrails can make it all right. But again, these are never guarantees, right? Because the injection can be much sneakier. It can be much more covert and can bypass these things.

Or it can be for something that doesn't require exfil, but it's still in some way dangerous, like maybe poisoning a local context file. So at the end of the day, it is still very much important to prevent these signal processing attacks. But if we don't make as an industry design pattern, recommendations and mitigations, other attacks that are prompt, injections will come up and will remain dangerous even if the signal processing issue was somehow magically resolved.

Allie Howe I see, that makes sense. It kind of reminds me of Johann Rehberger's research that he put out around some of his exploits with Claude code where he got Claude code to essentially ping his website and append like a secret to the sub domain part of the URL that he was using for his website. And he was able to do that through like a text, like textual, like prompt injection. And that was because I think he said that ping was one of the commands that Claude allows like by default, you don't have to like give it permission to run that command like even without like YOLO mode, it'll just do it by itself without you saying, yes, I'm it's OK to run like a ping command.

So I guess like if an image was uploaded to Claude code, which I, I do that sometimes for debugging. Well, I'll just take a screenshot of whatever I'm looking at, but it's and then just like give that to Claude from like extra context and that image could have an image scaling attack in it per SE. But if an image was uploaded to Claude code and then for images, you wouldn't be able to use the ping command. Maybe instead of like it being allowed with text, maybe it's not allowed with images. And so like using that architectural change there that could potentially mitigate that image scaling attack.

But again, like you said, like it could still like maybe change a different file or take a different route. So like it's so important to protect against these image scaling attacks, but there are certain architectural changes that we can do to make the consequences of them like less.

Kikimora Morozova Absolutely. Ultimately like of course, I can't speak on any given to one particular because Gemini's the only one that I really tested in production on this and that the research team was focusing on. But one thing that we did identify is that at the end of the day, the real solutions here are both like industry wide design pattern changes and trying to look from our prompt injection vectors. So this happens to be one that's a particularly sneaky. And so we as research team think that it's interesting to look at, but there's so many out there, many of which have yet to be discovered and which are probably even more covert than this one, at which point prompt injection needs to be considered as kind of a priority to architecturally prevent since it's an architectural problem.

Allie Howe Are we seeing image scaling attacks in the wild? Or because they're so covert, maybe they're happening and we're just not aware of how often they do happen.

Kikimora Morozova So I can't speak in any given client or audit, of course, but kind of some of the inspiration for this did come from something that we saw in the wild where there was kind of a signals processing issue on an audit that we identified and said, hey, this is kind of strange that a piece of multimodal data is being processed in a way that might be lossy. And that's kind of where we come up with the idea for this attack. So there's definitely in the wild providers that are vulnerable to it. I'm not familiar with any threat actors that are doing this, but also I'm not a threat actor researcher so I'm not sure.

Allie Howe When a typical client comes to you all for an audit, is their motivation around like, OK, I think this might be happening. Can you confirm if it's happening or you know, I'm, I'm aware that this could happen. I don't, I want to make sure it doesn't or you know, I'm pursuing some sort of compliance or AI framework and I want to comply. Like can you help me add technical control such that I can comply? Like what? I'm just curious like what's their motivation for wanting to prevent this?

Kikimora Morozova Yeah. So we as a company don't really do compliance just in general because that's not within it's a scope. But for other things, sometimes a client will come to us in a very specific concerns, often based on our research on our blog posts, including this one. And so a client might say, hey, we want to make sure that we're as projected as we can be against something like prompt injection, especially against covert vectors. And that's the point at which we go into the framework that I mentioned earlier in terms of just kind of systematically saying, OK, what are the vectors that we can see that are multimodal in nature, that are maybe multi agent system in nature or something like that.

And then we can make recommendations from there. The motivation for clients prevent this is a essentially these tools are going out to their clients and in the same way as any companies motivated to make their products as secure as they can be. This is very much one of those instances, especially now that this research is published. It is now in the common knowledge kind of domain of cybersecurity, which means that if there were actors that weren't aware of it before, they might be now. And so because of that, companies are even more incentivized to protect against this, although I'm sure that, you know, there are people that have come up with this idea before.

Allie Howe Yeah. No, that makes sense. And that's kind of one of the questions I had around Anamorfer, the git repo that you all put out to help teams understand more about what an image scaling attack is like, how it's done and test against that. I guess. Can you tell us more about what Anamorfer does and why you all created it?

Kikimora Morozova Yeah. So Anamorpher is an open source tool that is basically a harness for things like image scaling attacks, although a major release for that is coming up. Stay tuned on that. That should give it more functionality across kind of different prompt injection, multimodal prompt injection spheres. The point of that essentially to say, OK, we're a blue team or a red team and we're testing a product and making sure that it's as secure as it can be to these multimodal attacks. We as a team, this is a lot for internal, but also for white box external auditors who know, for example, what the downscaling algorithm is or can otherwise fingerprint it.

And then say, OK, can we craft a payload that if the system is vulnerable to this and the particular downscaling algorithm that's being used or other image processing algorithm that's being used as vulnerable. And a morpher lets you just kind of automatically see what the results of a downscale would be from a kind of like injection harness for a lot of these basic downscaling attacks. Sorry, downscaling algorithms. So for instance, if you have bicubic bilinear, these are some of the kind of names for the most common image downscaling algorithms. And a morpher comes built in with a script that will make an injected image for you from an image that you upload and from another image that contains the text that you want to upload or that you want to inject with.

And then it also has a kind of reveal side that will show you the output of that. The point of this is basically to make it as easy as possible for auditors who team and red team alike, to test whether or not their product is vulnerable to this and incentivize people to pay attention to this. Because if it's something that's easy to test for and pretty high impact, a lot more people and companies are going to be willing to test against this and make their product as secure to it as they can be. The philosophical motivation here is pretty similar to any other kind of red teaming open source tool where ultimately someone's going to come up with this idea at some point.

And if it's not well known, and if it's not something that a lot of security professionals are aware of, such if there's not really a tool to test for it, then it's a lot harder to actually protect systems against it prophylactically or before some kind of 0 day were to happen. As opposed to, you know, if this is something that a team is aware of or we tell individual clients but we don't make public knowledge, it makes the industry security lower.

Allie Howe Yeah, I agree with that. That's really cool take because that's what I was wondering kind of as well as like well, anamorphic definitely empower security teams to test and red team their own application, but it maybe could also help attackers also create like image downscaling attacks. But it's way better to be like educated as an industry around like what, you know, these attacks are, how do we secure against them? Like I think a, you know, rising tide, so to say. So yeah, that totally makes sense to me. And super excited to see future releases for Anamorpher.

Kikimora Morozova Thank you. Yeah, that's kind of a big goal of Trail of Bits. We have a lot of open source tooling that we do, including something that we presented AIxCC, so Buttercup for kind of AIML testing suite. These tools are important because while they do empower attackers, I think our industry is built on knowledge sharing and we all do better when both attackers and red teamers and blue teamers are kind of aware of the same attack surfaces and so can more effectively prevent against them.

Allie Howe When you publish this research, did Gemini have any like comments? Like were they interested to like fix this or anything?

Kikimora Morozova So it was a little bit of press coverage around this. We did not disclose this to Google beforehand. So because of how kind of important we felt the vulnerability was and how it could impact a lot of systems kind of quickly. And so a extended vulnerability disclosure process gives threat actors more time to exploit this in the wild. Essentially. They had spoken with kind of a media outlet that had reported on this, both from talking to Trail of Bits, so me and Suha, and then also from talking to Google, it seems that they were receptive to fixing this on Gemini specifically. This does rely upon an external MCP connection, which they recognized, but also did say that this was a problem and something that would be kind of worth paying attention to.

They did not classify it as a vulnerability. And I think that's just kind of a difference in terms of terminology and like what we consider prompt injection to be whether that's an architectural flaw or vulnerability.

Allie Howe Yeah. And then back to our earlier conversation, I wonder if they added those mitigations to, you know, show the user, hey, like this is what the image looks like when it's, you know, upscaled and there's the reverse of whatever the downscaling is. And you can see the untrusted content that may or may not be in the image. Do you still want to use it kind of a thing? Like, I don't know what we call that, like product injection, like protection or verifiers maybe. I'm not sure. But I guess like if they're not calling it a vulnerability, like there's still like an action that could happen from LM providers where they could provide that functionality.

Kikimora Morozova Yes, I mean, I do think user, first of all, user transparency is always good. And I do think that there are solutions to this. Whether or not LM providers choose to implement this is ultimately a question of how they're balancing security versus usability. And if they're transparent with their users about what their calculus is on security versus usability, that's kind of the most important part. That way users can make an informed decision about what provider they want to use or developers can make an informed decision about, you know, who's API they want to be calling to at the end of the day. And Google wasn't at all hostile on this, to be clear.

They just have different internal terminology for this than we do.

Allie Howe Yeah. It'll be interesting too, to see how the different LM providers like may use security as a business enabler or a differentiator between themselves and others. And I wonder too if like if more of these LM providers maybe, you know, add this sort of functionality or start talking about this, then perhaps they all will. It just maybe takes one or two to kind of just like start the trend.

Kikimora Morozova Yeah, I think that there's very much future potential to do this. And depending on what the threat landscape looks like if we do start seeing this be exploited in the wild, that might motivate more providers to look at it. I think it really just depends on what the future for this attack looks like. And I'm of the belief that, you know, signal processing attacks have a lot of variance and a lot of kind of future possibilities for them where transparency is 1 mitigation. But also these providers or even, you know, developers on top of them that like like hook into these providers to do various tasks can use design patterns as well as in mitigation, even if you know, the core provider doesn't do some kind of pre process or doesn't show the result of some kind of pre processing on an image.

Allie Howe If a team is starting to work with and build multi modal AI systems, where is the best place for them to go to learn about these design patterns or mitigations?

Kikimora Morozova I would say that I've learned a lot like reading the Trail of Bits blog and other blogs like it. We have a couple of blogs focusing on this, whether that's Pajamas which is kind of about multi agent systems, to things like this that are focusing on image scaling attacks. Also learning about vulnerabilities like this requires monitoring kind of the open source community. A lot of open source tools come out about issues related to signal processing, and I'm sure there's kind of a lot of talks on this as well at various conferences. I think that at the end of the day, a lot of kind of boutique security companies like Trail of Bits will post about issues like this.

As they come up to following any of those, whether that's Twitter blogs or kind of more formalized blogs like what Trail of Bits has can be really helpful.

Allie Howe Yeah, super excited to follow more from Trail of Bits yourself and everyone that's working on these image scaling attacks, other AI security research, I guess in a similar vein, like what is next for Trail of Bits?

Kikimora Morozova Yeah. So right now we're working a lot on AI assurance, which is something that we're kind of prioritizing within our practice. There's a few positions open within that right now actually. And in terms of clients, how we are very much prioritizing the biggest AI companies so we can make as much of an impact on kind of ecosystem wide issues like image scaling attacks. And we're also contributing to audits on the kind of open source side, like the like PyTorch audits that we did somewhat recently to kind of understand well, if people are in the ML space at all, whether that's, you know, home brewing models or using local LLMs or kind of relying on these bigger providers to kind of prioritize security within that.

And then on the auditing side, we have been working on continuing to enhance Buttercup and use that as kind of an internal development tool. Also hook in, you know, MCP connections to all of our internal auditing tools, kind of make sure that we as a company use AI as much as we can to speed up the workflow while still relying on the expertise of our engineers. So basically we want to focus a lot on AI as it continues to grow, whether that's on the kind of internal research side, the external research side or the side.

Allie Howe Amazing. Wow. Thank you so much for your expertise and sharing that with us. I personally learned a lot. I'm sure all of our listeners did too. And yeah, just to recap, there's lots of different ways to create these image scaling attacks and the consequences can be pretty severe. All the different kinds of problems, actions leading to, who knows, whether it's remote code execution or, you know, unexpected actions from the LLM data exposure, who knows? And I'm super excited that Trail of Bits is on the leading edge of AI security research. And we'll continue to learn more from your blog and your work. Kiki Morozova. Can people connect with you if they want to join in on the conversation?

Kikimora Morozova Yeah. So you can find me on Twitter at Kiki, under score, Morozova. Also, I'll be talking about this at BSides Berlin and also at Black Hat EU. So if anybody is coming to those conferences, you can absolutely flag me down there and I'd love to talk about this.

Allie Howe Amazing. Thank you so much. Thanks everyone and we'll see you next time on Insecure Agents.