Allie Howe All right. Welcome to today's episode of the Insecure Agents podcast. We are at AI Engineer Code Today. The focus of this conference is talking about coding agents as it applies to how AI engineers use them and develop them. These developers have had some recent troubles with MCP. MCP is the model context protocol that Anthropic developed and published last November in 2024. So after this entire year, we've seen some skeptics come out for MCP as well as some defenders. So today we have Dex Horthy and Ian Livingstone here with us today to challenge MCP and defend. So Dex will be our challenger and Ian is our defender.

So without further ado, let's get into it.

Ian Livingstone Amazing. So happy to be here with Dex. This is very exciting.

Dex Horthy Yeah, I'm really glad this came together. Me too, I did. I did not know I was going to be the MCP Challenger, but I'm happy to do it. I'm ready.

Ian Livingstone I got, I got all my talking points loaded up and looking down there and Dex and I decided and Dex and I decided that we're going to be friends after this still, so it's all going to be good, all right.

Allie Howe Let's do it. Let's do it. Let's. Do it MCP has been out for about a year now, and Anthropic's 23 Four blog post announcing MCP. They said MCP's aim is to help frontier models produce better, more relevant responses. How is that going today?

Ian Livingstone I mean, if you think about where we were in 2024, right, You had ChatGPT early versions of Claude. Claude code wasn't, if I remember correctly, fully out at the time.

Dex Horthy It was, it was like 6 months before, six months before, right?

Ian Livingstone Exactly. So there wasn't a way to bring any context from any other system into the model. So the experience of using ChatGPT or any of these LLM apps were like, either you went and built like a complex rag store and you had to pull all the data into the rag store for the vector store. But there was no way to do it like as a user, just bring some data from my Google Drive or anything else into these apps in a way that was standardized uniform and worked cross agent, right. And so from that perspective, today I can build, you know, as a, if I'm a SAS app or a developer, I can build an MCP server and expose a bunch of context from my specific thing into any of many of these agents and it's available, right.

So I think from that perspective, big leap improvement. So I think pretty successful at the base delta from where we were to where we are today.

Dex Horthy Yeah, I think you bring up something about you're talking about like, OK, before MCP I kind of had to custom code all these tools. Basically, if I'm not an engineer, I can't add tools to my AI unless I'm going to pick up like you have to like write code and get LangGraph or CrewAI or any of these frameworks, plug them in, do the agent loop. Like you can code, you're OK. But even then, I think one of the biggest mistakes people made in the early days of MCP, and they're probably still making it but most people have figured this out, is like if you are building an agent, you should probably not use MCP.

Like people thought MCP replaced SDKs and you're just like, OK, instead of importing the Vercel client into my TypeScript app, I just use their MCP server instead. And I think that's like you can do that. I don't think it gives you any benefit. I think the real benefit is like if your users are non-technical, it'll anyone who's like technical enough to like write a little JSON file or paste in an MCP server URL. You can build your software as an MCP client and then you can enable non-technical people to extend the functionality of your app without you having a custom implement all these integrations. And so I think that's really powerful.

And I think like the more people plug into that and stop trying to use MCP and their custom built agents, that was I think a big unlock for people that it took a while for that to people to pick up on like. That's that that's the right way to do it.

Ian Livingstone But yeah, and I think I think part of the challenge is like if you think about tool calling in general, like computer use tools like where why did we start with? Why did Anthropic invest money here? Like if we take a whole step back in the veil, what was going on at the same time OpenAI was working on browsers, the OpenAI operator demo and from a strategic positioning perspective, they had to bring something to the table that allowed them to have a relevant story about computer use tools. You know, I think that the other aspect of this is if you you sit back and say, OK, well, why is it had been so hyped?

Like why are people talking about it? Because it's the first time where anything that wasn't Anthropic OpenAI, Google, Gemini, Microsoft, you know, white labeling OpenAI stuff had a story where they can say, Hey, we have a relevance relevant we have we can bring data and actionability into than this new paradigm and create relevancy for our products and also unlock value for our end users. And those end users could have been developers or they could have been anyone, right? Like, is that why would I want to go and artisanally create some magentic loop if all I want to do is bring some data from like a Google sheet into ChatGPT and Cursor, right?

Like, I don't want to do it. And more importantly, if you're like a builder of one of these, like, you know, API layers, one of these apps, you don't want to like artisanally create some API thing, you want to actually like implement some standard, the agents will understand. But there's a whole gap between like intention and what has happened and some of the downsides, which I'm sure Dex is going to be more than happy to sit and talk about.

Allie Howe Yeah. It's straight into my next question, which is essentially is like who is using MCP, who's getting value from it? YC is famous for saying build something people want and when it comes to MCP, like who really wants MCP, who's getting value from it? Is IT enterprises trying to distribute their product to different LLMs or is it end users that are using MCP clients?

Dex Horthy Yeah. I mean, I've talked to a lot of enterprises that are like, hey, we're building a chat bot. And so again, this, I mean, this is the same point again. They're like, OK, so then we need to build this as an MCP server. If one team's building an AI agent, then all of our internal teams who are going to expose data to this AI agent are going to need to build their things as MCP servers. And they were a Java shop and there's no MCP like libraries in Java. So they ended up like reimplementing the MCP SSE protocol in Java just to do a thing where I'm like, well, if you own both sides of the picture, I, I don't think you need MCP.

I don't think it's where I mean, unless you're like 100,000 person enterprise where you basically function as 50 separate companies. I I still think it's like overkill in a lot of cases. I don't know if that answers your question.

Allie Howe I think it does and you also answered it earlier and you said like non-technical users actually are getting value at MCP because they can connect tools that way.

Dex Horthy Exactly. Yeah, I think people, people who can pull a bunch of MCP's in. I think there's a lot to be said for like quality. I forget who was some. I read the post on Twitter today. I think it might have been Dax, but it was basically like the marketplaces are really exciting, but they have to be curated. And so you end up with like MCP marketplaces that are like 50,000 servers and seven of them are good and the other 49,000 whatever are really bad. And then eventually the people who want to guarantee product like quality, take it off the marketplace and just serve it in house instead.

And then the marketplace is useless. So I'm not anti MCP marketplace but I would love to see better curation because my experience in the first six months of MCP was like oh I need like I want to try Gmail thing. I want to try Google Docs or spreadsheet or whatever it is. And every single thing that you would try would just be like this barely freaking works. I don't know who made this. I really don't want to give them my OAuth keys. Like how do I make this better?

Ian Livingstone Yeah, I think these are all really, really fair points. I think I'll, I'll, you know, address sort of linearly here. I think. I mean, first, first and foremost, if you're an enterprise, you're not just thinking about like agents I artisanally build, you know, you're thinking about the things I buy and adopt as well, right? So, you know, a lot of like for Keycard, a lot of our customers, the reason they're building MCP's, they're building them at scale. It, it tends to be because, you know, they actually have bought like a Claude code or a Cursor or they bought a ChatGPT or, you know, some agent off the street that supports MCP.

And they both want, they want an interoperability story where we build once get lots of value and, and MCP gives them the ability to advertise and expose functionality to these work flows in a way that's different than traditional APIs. And also his promise around things like federation support, discoverability, not entirely there yet, but that's that's. Sort of the reason. So if you if you talk to them, that's, that's the justification.

Dex Horthy And I think there's a lock in side of that too, right. If both sides of the equation are interchangeable, the tools and the actual like agentic software, you can swap any of them out and you're not locked into these like coupled systems.

Ian Livingstone Bingo. So I think that's actually one of the like, if you, if you think with the enterprise, when they think about buying software or deploying software, they're really looking for like how, how do we give our ourselves the most optionality and how we bet on things that enable us holistically across the thing. Like they really want the punch once standardization so that they have one place where they can deal with security challenges. They have one place where they can deal with governance challenges and auditing challenges. But also one thing that they can sort is be like, hey developers, you want to like build an agent or you know, like expose something to an agent.

Here's the answer. It's not perfect, but it's a better answer than everyone doing 1000 different things in none of it working together. And I think that's true. In the enterprise, and it's also true between companies and across companies in the enterprise.

Dex Horthy So your answer might be there's no way to prevent this in the enterprise. But how do you keep this from becoming like a spring web sphere situation where you have this like monolithic middle layer that does everything and way too much and it actually like becomes calcified in the enterprise and it makes things slower over time,

Ian Livingstone Yeah. It's a really good question. So the first part of what you originally said, you know why I think the second part you talked about was really about discoverability and quality, right. OK. Like, like there is, there's this, there is a fundamental issue which is we have this in open source in general and it on the web, right? Because it's like we want a world where anyone can create anything and put it out there. But also like it's really hard, especially with MCP's, especially with MCP's started to be like, is this thing good or is it bad? Is it secure or is it not? Like in truth, because of where like Anthropic started and what they started with.

Like there is what they started with wasn't very good and it was designed for something very different than what people actually are adopting and using it for.

Dex Horthy Well, yeah, it was like a land grab, right? You had like prompts and responses and all these things and like. The only thing people really needed was tools, but I think Anthropic was like, OK, cool. We might as well just take as much as we can. And then if, if it whatever catches on, we get. And it's like you have a bunch of weird extra protocol bits sitting on a thing where it's like, OK, there's the it's not loss leader, but it's like this thing drags the rest of the protocol along with it. And some clients are implementing certain things and some are others. Like would you be an advocate for like an MCP 2 point O that dropped a lot of capabilities or do you think those need to be in?

Ian Livingstone I, I think we in a way kind of have that with the current versioning scheme. Like you could have a versioning scheme like the next release could be like, and we're dropping support for local MCP, right? Like maybe they decide to do that. It might actually be better for everyone if they did drop local support. And I'm, that's the question we should talk about. Or maybe they want to drop a drop like SSE, right? Like, so I think there's things that they want that they will, we have a mechanism with the versioning standard where they could do that. And we already have protocol negotiation and versioning negotiation built to the standard.

So we have the capacity to do that. And I mean, I think a lot of like what we're talking about with MCP reminds me of like OAuth 1.0 versus OAuth 2.0. Like for those that don't know OAuth 1.0 was a security nightmare and an absolute travesty. It was really hard to implement it. Was really hard to have this. HMAC signing and yeah, it was wild and like not to say that. OAuth 2 is like this amazing protocol it's it's not but it is certainly like leagues better and more secure if you include the right extensions and it's also enable user federation across apps on the web for the last decade and that's finally changed like been a better outcome for everybody.

And so you know I with protocols, it's sort of like it's never going to be great, but but ideally it's not awful. And I think like OAuth for, like better or worse is I think we're better off having it than not. And I think MCP at the end of the day and we'll we'll get into it. But I think at the end of the day, like it's an original intention. And what's turned into I think is a lot like a lot of the original vision and tension is still there. But it's going to take time to evolve the protocol into a thing that actually makes sense for everybody as we better understand what agentic workflows even look like, right?

Allie Howe Yeah, that's a great point for sure. And Speaking of how many times this protocol has changed, there's been about like 3 different auth mechanisms added over the course of the last year and multiple different versions of the auth spec. So now they're supporting dynamic client registration, pre registration and a new one called CIMD, which is OAuth client ID and metadata documents. Have we finally figured out auth for MCP? Like have we fixed it?

Ian Livingstone Oh, I get, I get this answer. We've made it much better, right? Like I think thinking about OAuth in general, like dynamic client registration has a huge amount of problems with it. Like it's basically unauthorized, right? It's UN authenticated. And the fact that the only reason it works is because there's a delegation involved where I, oh, at least I know it's Dex or Ian that is trying to catch my OAuth server. But like in terms of the client, it's kind of like who cares, right? Like there's no proof point, you know? Client ID metadata solves a huge amount of this problem.

Dex Horthy Yeah, so this is the thing where they put like the URL to the server is the client ID.

Ian Livingstone Exactly.

Dex Horthy OK, tell me why that's so much better.

Ian Livingstone Well, because it gives you an ability to to like basic create an identity. And the identity is both secured by a like the OIDC metadata document that's available. So you have a bunch of information. It also gets rid of client ID secrets, right? So no, you don't have to copy around and store a bunch of client ID secrets because they're actually moving from a symmetric key base like you know, like here's a hash, approve the hash they match to a PKI based system where in that OIDC metadata document you can just publish the keys. You can just put a JWT in your thing. It's a data in case the server has your public key, then you can I mean, OK, so here's so that's a broadly for everybody.

That is a big security improvement. And it's then back to TLS, which is why the URL matters, right? Because we can build on top of the things that made the web Federated, like the Federated secure web possible owning, owning T like domain. If you own the domain, you trust the DNS, you trust the, you know, the certificate, primary certificate issuer, then you kind of you're building on some things that already work. It's it, it solves a lot of the problems associated with agents that can be like addressed by HTTP, but it doesn't solve agents that aren't available on over like HTTP or, or the web in general.

Dex Horthy What mechanisms are in place today? Because obviously we have the same problem with like Authorization headers. In HTTP, what is in place to prevent the like chaos that sometimes emerges when you have stringly type where literally the entire structure is just a string? Anyone can put anything they want in there. Like. How do you avoid the like proliferation of a million different ways to do the same thing?

Ian Livingstone I would love an answer to that. Right, like. I think ultimately this is where. Standards begin to fall. Apart more than more than anything they're they tend to be directional and it's like back to just like ignoring client ID metadata. But like OAuth is a good example where people sort of all follow the spec. But if you, if you built any OAuth server at scale that's interoping with other OAuth servers, you know, oh, actually everyone has done this weirdly slightly difference. There tend to be common patterns, but there there's not perfect implementations. And that is something agents actually can help us with. Unfortunately, Unfortunately, yeah, yeah.

It would be great if like there was JSON schema for everything and strict validation. But on on the flip side, I think, and I've had a lot of conversations with my Co founder Jared about this. Unfortunately, with strict validation, when you go to evolve the clients and the servers, you actually create a lot of interoperability problems. And there's this exactly. So there's this trade off between like, do we just remove the strictness and just like try and evolve the standard in a way that we understand that like not going to break it is not is like it's always backwards in battle because we just drop all the strict typing and just like kind of assume the clients figure out.

Or is it the inversion where we include like a lot of versioning inside the standard and a lot to to solve all these different use cases? I think the truth of it is because of how long it takes the IETF and others to like get like a like a spec basically to something that's like accepted is the limiting factor here.

Allie Howe And it's true, while we wait on the spec to have updates and maybe we fix the protocol, there's things that people are doing today to potentially secure MCP. Cuz there's a lot of security vulnerabilities out there today that are related to MCP, including rug pulls, tool poisoning, cross server tool shadowing. One of these came to light recently in a Trail of Bits article they published where a tool poisoning a bad tool description essentially came into the agent and asked the agent to export the entire chat history and the name of SOC 2 compliance. And because the model knows enough about security to know what SOC 2 is, but not enough to know exactly what SOC 2 is, decides to export the entire chat history too.

Ian Livingstone And just upload it to some attackers URL or something.

Allie Howe Yeah, exactly. So definitely a lot that needs to go on to secure MCP and unfortunately it's in the hands of developers to mitigate this. So is MCP just a huge security liability or does MCP potentially offer us a location to insert auth checks at the network layer? So maybe it's a security advantage.

Dex Horthy Interesting. Yeah. I mean, have you read Simon's post on like the lethal trifecta, right? Of like the things that make it MCP unsafe is what it's like access to like private data, yeah. Like feeding in untrusted input like MCP's off the Internet or like fetch results and untrusted output too.

Ian Livingstone And also you can technically give me untrusted output, right? Because like the thing that the MCP's talking to can actually talk to something that's untrusted because MCP's basically a middle man in the middle. Interesting, right? And like that's where you get sort of that's a variation of tool poisoning attack where it's like the output that comes back from the tool actually then goes to the context and poisons the context window and then changes the behavior of the agent.

Dex Horthy Yeah. So, I mean, I, I spent a lot of time building human loop systems because my take was always like, if you want to get agents to do anything useful, you should probably like give them access to tools that do things that matter, like. But it when it comes to like modifying data or sending emails on my behalf or querying large parts of the database, like I want to approve it. I mean, there's a whole async thing of like, if you're not sitting there staring in the chat window, how do you actually mechanically do that over the network? But the other side for me was like, if I have a browser agent or something running out in the world somewhere and it's going to, I mean, browser agents exist because most websites don't have APIs and the good ones even have like OAuth.

But essentially like any agent that wants to do something like swipe a card on my behalf, I mean, you can think of what I think of as state-of-the-art in which authorization basically. So we don't have that yet. We have fine grain auth in something like GitHub where you have tons and tons of granular permissions. But even with kind of what I would consider close to state-of-the-art, all you get is like, you can merge, you can only merge PRs. That's the only thing you can do. And you can merge it on only this repo. And for me, that's like nowhere near tight enough. Like I want when the agent wants to do something that matters, I want to get a notification on my computer or my phone that says, and I basically sign a key that says you can do merge this pull request on this repo for the next 30 seconds.

Otherwise, you have to ask me again. And I'm like waiting to see something like that in MCP. I think that would make it a lot more useful to actually let people be. Comfortable having out of band outer loop agents have access to do meaningful things and access to meaningful data.

Ian Livingstone I think that's spot on. And I think that's true not just for MCP, but it's true for the web in general, right? And, and like agents, you know, like we didn't have to do this previously because it was always a human. And so as long as you could authenticate that the person doing the thing was a human. So there was a button, clicking the button, or a human wrote the code right, That a human was involved and we're just going to trust their judgement. And now we have this thing that likes to hallucinate or do lots of crazy stuff once in a while.

Dex Horthy Or trust weird instructions that someone sent you in a support. Ticket or something?

Ian Livingstone Exactly. We're moving from this world of like super deterministic human driven workflows to this world of systems of action they're taking making decisions and reasoning on our behalf and our identity and access systems are totally broke because everything about the Dex just said is 100% correct. It, it is an issue that manifests in MCP because of how we think about MCP and where it sits in our, our mind in terms of like enabling agents to do actions. But it's not, it actually an MCP issue. It is an OAuth issue to the extent that like OAuth doesn't support a proper standard for like how would you present this type of workflow today?

Dex Horthy Isn't there like the rich authorization paper that is like per per action request?

Ian Livingstone Yeah, there's some. There's some, but it's not standardized and no one, there's very, very limited support for those types of concepts. But I mean, certainly that's the world we need to get to, yeah, to to actually enable like any form of like human in the loop, autonomous, like agents that are equivalent to like a self driving car.

Allie Howe So you're saying this isn't an even an MCP issue. This is an LLMs issue because now we're working with non deterministic systems. It's driven by LLMs.

Ian Livingstone Yeah, LLM independent, whether it's MCP or not. Like if we wanted to realize the promise of what these things can do and how they change, like the way we interact with software, it doesn't matter. It's MCP, we're using fucking OpenAPI documents or we're using SOAP, right? Like we're still going to have this fundamental, like as a user, I want to like explicitly be able to write a policy that's a subsection on a per task basis of what I'm allowed to do. And I only want to like assign that to this identity and I want it like cryptographically secured. So the downstream authorizer, you know like the GitHub MCP, GitHub MCP are gonna be API like oh, so this.

Agent is agent XI can validate that it comes from user Yi can validate that and only once this authorization policy applied yeah, cool. I can do that and I have no way to do it and I have user wise public key and I can see that they signed the request.

Dex Horthy I mean, I think a lot about like this is similar to how like Apple Pay works, right, Where you get these like short lived cards that are only good for, I mean, not Apple Pay, but a lot of like, yeah, you get these ad hoc credit card numbers that only good for 5 minutes and they're only good up to $100 and things like this.

Ian Livingstone Yeah, exactly. And, and I think there's some, there's some beginnings like you can see in, in commerce where comps trying to, they're trying to begin solving this problem, right? Where they're like, if you go look, a lot of commerce standards, it's like, how do I get like a, like a, like a fake card ID effectively that's like a, like a representative, like a credit card that's like they can delegate to some identity. And it has like a Max spending limit of 50 bucks, right? And, and that's our first initial take it like how do we do this? But there is not broad good understanding of how it is going to work at all.

But there's a lot of good thinking. You know, there's things like biscuits and verifiable credentials. There's a bridge off stuff you mentioned. So it's, it's coming. It's just that that is unfortunately very far out on, on the curve of stuff today. If we think about like what people are trying to do today, it's sort of I want to connect some data into my agent and I want the agent to be relatively accurate and maybe it can take action on my behalf. And it also doesn't like, take my PII data and shove it to the browser and give it to Google or something, right? Like that's like, unfortunately that's a layer of like security challenges we have to deal with just to get like, people to step one of the process, yeah.

Allie Howe That makes sense, so at least sounds like there's some hope. But there's definitely some issues with MCP today that users are reporting, especially developers. MCP servers can come with hundreds of different tools. Not all of them are necessarily relevant. And when you have more tools, it makes the agent harder to decide which tool is correct to use. There's also this problem where tool responses can fill up the context window. So lots of context engineering is needed on the developers perhaps to make MCP really work properly. Also, they have to manage things like we talked about before, like implementing auth that's optional. The spec doesn't even require that.

They also have to deal with all of these MCP security vulnerabilities that we just talked about. So while MCP might be great and the best protocol we have today per SE, there's still a lot of work that the end user, the developer, has to do to make it work properly. Is that fair? And given that, can we still call it a good protocol?

Dex Horthy I mean, the classic example, this one's not even like fun anymore, but this idea of like the GitHub MCP, if you don't filter the tools, it's 60,000 tokens or something, when you could just use the GitHub CLI, which is already in the training set, right? And so like if you're using any coding based agent, which like you're talking to GitHub, you're probably building software with that. But you look at the GitHub, like MCP, it's actually designed that you only opt into the set of tools that you want. Like it has a bunch of flags, but nobody reads the docs. So it's like as tool designers, as MCP designers, I think it's really important to be very like careful with, like this is a problem we can solve with good design.

I think like it's a little bit on the developer to be aware of, hey, check your context window and see how many MCP, how many, how much of your window is being taken by? And I'm talking about this a lot tomorrow of like the smart zone versus the dumb zone we like to call it, which is like you're over 40% context used for most tasks. You're going to get a severe degradation in performance past that point unless you like have a perfect trajectory and you're like, really like manicuring every single step of the workflow. So it's it's a little bit on developers to make sure that you're maximizing your time in the, the smart zone of the coding model of the coding running agent.

But you look at a server like I love context 7. Context 7 by Upstash is not just like the responses come back in markdown, which is very succinct. There's no JSON. It's literally just plain text that comes back. But also they let the model request how many tokens. And so there's like a model on the back end that says, goes and reads all the docs and then based on the size of the response, it gives back. And so you have even progressive disclosure where the model can say, OK, cool. Like that was 5000 tokens. That wasn't enough. Give me, give me the next 5000 tokens or give me the 10,000 token version because I don't have what I want.

And so it's like, I don't know, this is like this is an MCP problem, but it's also just because MCP is so easy to use and you could just install a bunch of local stuff. I think it ends up you've give, we've given a lot of people like a massive foot gun. And I'm I'm curious how like you think we can solve that?

Ian Livingstone I think this is like the most relevant like honest, like reason to be a disenfranchised with the current MCP ecosystem. And I, I actually don't put it on end users at all and or even agent builders to a certain extent. I think we're missing a middleware layer, right, If you think about it. I mean, we started with a spec that basically said just where what people did is they took OpenAPI and they just ripped it into a shit ton of tools, right? And there is a reason that like MCP, actually like beginning was, well, we actually, it was RPC based. We need to invert like agents can't think in terms of taking a bunch of APIs together, training to workflow and then running it sequentially in a way that like it's deterministic that you'd expect from a user asking an agent to take an action.

And so we ended up with this like thing with RPC based so that you could be like, well, actually it's an inversion of control from the way we used to design APIs. We actually want RPC. So instead of being like, add this thing, add this thing, add this thing, add this thing, check out. It's like, hey, I'm buying this thing, check out, right? And that's a one call that can be done in one loop request response instead of like asynchronous workloads that or data that's built over time. Now that's an API design problem. Now the thing we're talking about is a tool bloat problem, which is, well, really, if you think about what in what nature, let's say take a Claude code is in the context under that you're managing.

And Dex knows way much more about context management engineering than I do. But you know, why would that in that context, why would that agent ever need the all to all of those tools? But that doesn't make MCP bad. What it does make certainly is there's a missing layer, which is there's tools, Then there's the tool discovery negotiation layer, which says based on the tasks that we've asked for this thing, yeah, what tool should this thing actually know about? Or do we call right. And that like so that there needs to be a management, it's a part of the harness. There needs to be a management layer between endless tool bloat, right?

Like what is the like, basically at the point where it's like, hey, context window have the entire universe of possible things you could do in every call, Like obviously that's not a good idea, you know, based on the way that any of these work. So we need we need something else. And I think the the criticism, I think that's maybe appropriate is to say we're missing a best practice from Anthropic or from the model context protocol ecosystem. They have started to present one with the way that they the idea of like using code to generate code tools as a is a sort of a middle ground there.

Dex Horthy Well, and that's also cool, right? Because you don't have to pass the responses through the context exactly. Model could take the response of this and send it to this. And I don't want to see it, I just want to like execute this pipeline.

Ian Livingstone Well, why exactly? So why would you want any of that data or any of that stuff? And also like, why would you ever want to expose any like whether or not you're you're doing like dynamic ephemeral credentials, like what Keycard does or you're, you're actually just using API keys. Like why in the world would you ever want to expose an API key or any type of sensitive data directly into the context window? Right? There's there's definitely like a harness and you want some of that heart you and you want that, that harness and and tool calling is differentiate from which content, what's what's in the context window, what's end up there, what ends up there?

And then there's the layers around all of that. And I think if we were to step back and kind of think of this as as a multi agent system, one of the multi agent systems underneath all of the sub agents is there is the tool calling layer and how and then tool discovery and then selection of those things and management of it for sure. And I think we're just missing it.

Dex Horthy I'm going to say this, Ian, because I love you. Please don't ever say multi agent system again.

Ian Livingstone OK, you got. It

Dex Horthy no, I'm, I'm, I'm mostly joking, but there are a lot of times I hear people say multi agent system and I immediately assume you have no idea what you're talking about. And so there's like there's a nuance there and like, what do you when you say multi agent system, what do you, what do you mean? Like what are you talking about under the hood?

Ian Livingstone Yeah, I mean, like, if you think under the hood that tool calling layer just talk, there's going to be a level of non determinism there, obviously, right. And I and like, and so we step back and say, well, you have there's there's multiple layers of LLMs being applied to solve this problem. Let's think about like the boxes here, each box with an LLM, you could call it an agent, right? Like.

Dex Horthy So this is like treating, treating an agent basically as a tool and it's just like anything else. So whether your MCP server or your other type of tool has an LLM behind it or not, from the main agents per like point of view and from the design of the system, yeah, it's just agents and tools.

Ian Livingstone Yeah, exactly. OK, Yeah. You need like, yeah, exactly. And there's been discussions like even before MCP, there is unfortunately a Gorilla actually Gorilla GPT. There were like ideas of models that just specialized on a tool discovery and tool calling that focus on solving this this problem right of like, well, how do we figure out what tools to use?

Dex Horthy What do you think about these systems that like do like a vector search against 1000 tools and just pick one and give it back to the model?

Ian Livingstone I mean, it's it's. Or pick 5. I mean, let's assume they were, if they're good, right? Like, and I think that's the problem like this comes down to how good are they for the task at hand in the same way like, and how in distribution is whatever it is that we're doing is in the distribution of those of those tooling models, right? And, and I think like in truth, like if let's say there's a world where there's a lot of that data, then they could be very good. Now, do we have a ton of that data today? I don't know of a massive data set that has that data today.

And that, that's part of the challenge here.

Allie Howe Sure. Yeah, I feel like with the tool calls, the more tool calls we have, there's a potential where the LLMs can get trained on that. So they might get better at it. So we might not need things like code execution. So do you think that will notably improve MCP if we start to train on that?

Dex Horthy I don't know. I don't, I don't like using the training data for knowledge ever. I, I like to think of LLMs as like, think of a person who has lived 10,000 years, they have seen and read every single thing in existence. But unless you activate those memories, they're going to have a really hard time learning about that stuff. And so I'm always biased towards like, if you want really good results, just take everything about your problem and make sure it's in the context window versus like waiting for the models to have enough information to be able to one shot. I mean, maybe it'll happen someday, but like waiting for the models to good good enough to one shot things without additional context is feels like it's good.

It might be a while.

Ian Livingstone I'm I'm actually curious Dex, I would love your like your perspective on this. David Cramer at Sentry has built I knew Sentry agent MCP Yep. And he's, you know, he's a, he's a thought leader in our industry and it's. Been really fun watching his journey on Twitter about like the first thing they tried and the next thing they tried and then what was the final answer was basically like we actually built an MCP that has an agent behind it and if the the only input is the main agent calls it with natural language, right yeah, OK and and what's. Your. View on that as a.

Solution to some of these some of these problems we just talked about. Like when we say multi agent, you could, I mean, there is an agent behind that, that Sentry. And if you're using Claude code and use that, that's a, I'm not picking on your, your dislike of the word multi agent, but like that is a system of interaction with multiple agents involved. Yeah. And like, so I'm curious, like what is, what do you think of that approach? And do you think that helps improve some of the problems we talked about? Because when I look at it, I say it's not perfect, but it's certainly better than what we had.

And it also begins to allow Sentry present a level of intelligence that's unique to Sentry.

Dex Horthy Yeah, I mean, I'm, I'm not actually against the idea of multi agent systems, but exactly what you just said is like a much better thing. Just it's like, oh cool, there's a main agent and it calls tools and the tools take natural languages, their inputs. And in the back end there's an LLM and it's connected to all the Sentry APIs. Like, yeah, I know it's more to say, but if you can say multi agent system, but you can't say that second thing, then it's like, OK, you need to like be more clear about what you're describing because it's, I'm going to talk about this tomorrow.

We have a lot in the AI, worse than any other industry before of this thing that like Martin Fowler came up with this in like 2006. He calls it semantic diffusion, which is basically like you have a term that has a good meaning and it's useful and then it gets out in the world and everyone starts using it. And suddenly the the word itself is completely useless and it doesn't mean anything because it means 100 things to 100 different people.

Ian Livingstone I mean zero trust insecurity is the same thing. Where does your trust mean? It means nothing now. It used to mean something very, very specific, Yeah. Now it means absolutely nothing because every vendor. And same with AI, right? Like, what does AI mean? Nobody actually knows what is an agent?

Allie Howe I don't know, yeah. Yeah, MCP is not the only protocol out there today. There's actually lots of them that have come up between 2024 and now. There's a particle an article on arXiv called Different AI Agent Protocols and it was published in June. And basically they break down these protocols into inter agent protocols and then context oriented protocols. There's actually a lot in the inter agent protocol category like A2A and ANP, but then the context oriented category, there's really only two that they list, it's MCP and agents.json. The difference between the two is agents.json is stateless but MCP is stateful. What does that mean for how unique MCP really is?

And is the stateful component to MCP something that sets MCP apart from all the other protocols?

Dex Horthy I mean, I'm curious to hear you work with more people doing MCP in the enterprise. Like I have had experiences with MCP clients that don't really support the stateful part of it where they're relaunching the server every time they want to make a call. And I'm curious what you've seen in terms of like, are people actually using the stateful parts of MCP? Because I know it has a lot of weight on the client side as far as like life, cycling these things, right?

Ian Livingstone Yeah, I think I I so that I think we should like talk about the I have many opinions. So let me break this down in a structured way that at least can get all the thoughts out of my head. So I think one thing I will say is what what I'm seeing and A to a is interesting and certainly like the original thing that Google like pushed around A to a. And then what they have today is like whatever was very close to like a Google thing, right? Like when A2A first came to look like Google designed this, This looks like everything I'd ever seen from Google.

Very clean code, very clean code. But it looks like Borg, not Kubernetes, right. If you were to like make make or it looks really a lot like TensorFlow, not PyTorch, right. PyTorch obviously became much more popular than TensorFlow because it's easier to same thing with Kubernetes versus Borg. So A2A has come down and what A2A looks like a lot more like today is a competitive protocol to MCP in terms of discovery of agents. And if you kind of look at what MCP is doing from it's like agent, it's cards, a lot of like the metadata they're adding to MCP. It, it feels like what's actually happening is you're seeing this convergence between MCP started as tool calling A2A and, and friends started just like, how do I discover agents?

Actually, I think what we're seeing in the Sentry MCP we just talked about is like an example of these two things are converging and there probably isn't a massive difference now from like my company's perspective, like we'll support both it, it doesn't matter. We're like kind of agnostic to this whole this whole specific debate. But I think at the end of the day on the stateful over stateful stateless side, agents.json and and there are cohort of architects like people, people who have not spent a lot of time doing what Dex does every day. Who are just like, well, isn't this all just like OpenAPI or like, you know, SOAP or some other?

The LLM is turning natural language into JSON over and over and over again. Why does it matter what the like transport? Looks like it, it, it doesn't right. And so the, the problem with some of these, these experiences like stateful or stateless, I think there's a, there's a tug in a pole in the ecosystem between people who very much understand and realize the value of the future of, of agents, which is going to be. And the unique thing being what that agent can do is completely dependent upon the prompt and the context and data and action abilities access to. And that's different. You change any of that and it can do new tasks it couldn't have done before, right?

You have, and that's where the statefulness is important, right, is it's state. And content engineering is inherently A stateful, a process is a stateful. There's state associated with that's at the core of how all of this works. And so when we think about agents.json versus MCP and I, I think about like the A to a versus MCP dynamic, I'm sort of sitting here being like, look, agents.json is an rich version OpenAPI that is a way to advertise things with some extra metadata that a thing could consume and then turn into an agentic operation. I think if all you want to do is advertise a bunch of static APIs, that may be RPCs, maybe OpenAPI, an agents.json or some version of that is sufficient.

But if you think about the future where tools and agents, it becomes disintermediate, like what is an agent? Whereas a tool is almost like, you know, it's almost like relativity in the sense that like my perspective of what's the tool versus my perspective, what's the agent is completely dependent upon the use. It's like I'm using another thing. Well, that's a tool to me. But that other thing may actually be an agent because there's an LLM everywhere. And if we step back, we think about like the transition that software's going through. We're going from a world where like there is no like almost no non determinism in our software to a world where like non determinism is everywhere.

And it's like the core feature and the thing that makes all of this useful and awesome. And so at the end of the day, like, I think all these differentiations don't actually matter. And we're gonna end up in a position where the statefulness is what drives the adoption. It's what drives the unlock. And it's also what results in an experience on the web that is inherently agentic versus inherently driven for humans, right? Like and we're, we're at the very beginnings of that transition period.

Dex Horthy Yeah, and there's this big explosion when like REST APIs became very en vogue, whereas every single website needs a REST API. Most of them. Got it. But like REST APIs are for developers. Yeah, there's 47 million developers in the world.

Ian Livingstone How many or was it 47 million? I don't know the number. It's on that order. It's large now. Yeah.

Dex Horthy Yeah. But there's gonna be billions of agents. And so we may actually see more MCP in A to a than we see REST APIs being adopted because there's a bigger market for people to consume these things for.

Ian Livingstone Sure, like every person could have like 10 agents or 100 agents or whatever.

Dex Horthy Yeah. I mean, as long as you don't call it a multi agent system, I'm yeah, yeah, yeah. The other thing I wanted to add is like the coolest thing about a 2A for me when I was comparing A2A and MCP because they both all work on the same underlying. Like an A2A client is the same as an MCP client where you you go discover what's available and then you tell the LLM, here's the tools you have available right at the core in the model layer. It's doesn't change what was really cool to me about a 2A, especially because we were working a lot on human in the loop systems.

And when an agent wants to call a tool and then you want to go like page someone on Slack and then if 5 minutes later they don't respond, you page someone else on Slack, get the approval back in. Well, you can't really build a production grade system if that state of that request is sitting on the call stack somewhere because your server crashes come back up. And so like A to a did this really cool thing and like the most common use case I heard people using A to a for was actually using humans on the other side of the A2A protocol and basically having an A2A client call that agent.

But it was actually going to a person and then using the asynchronous nature, which is something that MCP is like trying to figure out now. But it's like, how can a MCP client fire off a request and then get notified later that it was done and keep doing other work in the background? A2A had that built in from day one, which is one of the things like OK this is really well thought out along certain dimensions that MCP is going to take a long time to get to.

Ian Livingstone Totally. I, I think that's completely, it's a really important observation. And I think actually, if you look at the web talk about REST APIs, a lot of people know how to like, know how to like, what is a good REST API look like? Because you maybe have used Stripe, right? Like kind of like at the idea. I like their primitives. It's kind of flat. I can post and I create. I like don't have super high nested hierarchies. And you know, we use like some type of open API to like document the thing. And now you have like a REST API that most developers can consume.

The the problem is, is like most apps in the world are actually static. You know, they're basically CRUD, right? They're not asynchronous. They're like one call is done, request response, request response, right.

Dex Horthy The client is maintaining like that was the whole thing of REST is like the client maintains the state.

Ian Livingstone Yeah, it's not totally true, right? Because you go look up the session and session on the back end and things like this. But, well, I mean, the other thing is like in computing, there's always stages a question of who owns it, right? Like, and in REST, it's like, or in modern web design, it's like, oh, OK, well, the, the state lives in the client or the browser or in the, you know, the thing that's doing it Orange being stored. So there's always states someplace. The other other component of it is that most apps are never asynchronous, right? So you go and think about what are what are design patterns for building asynchronous workflow APIs, right that are stateful where like what you have is a piece of data that represents actually, I asked you to go do something that's going to take a long time to it could be take 10 hours, 70 hours, 100 hours, 250 hours, ten months from now to happen.

Like there's not there's not good convergence, unfortunately, on what those look like. There's some ideas like agent gateway out there, but there's not wide adoption of this stuff and agents because of things like human the loop and because of the the length of time it can take to think and plan and do deep research are inherently asynchronous. And importantly, the like the transition in UI/UX are going through right is removing from a world of like synchronous human human computer, human computer, human computer to hey, computer. Here's my intent. Go interpret it. Come up with a plan. I trust you.

Dex Horthy Don't send the e-mail until I approve it, though.

Ian Livingstone Yeah, but don't send the. But maybe send it because maybe I trust you enough that you send it. Right. And that's that's a trust equation. And that evolves over time as the agents get better. But to your point, maybe I maybe don't send it until I've approved it because I don't trust you. And maybe that's the right decision for you and that agent in the context of what you asked it to do.

Dex Horthy OK. Yeah, I think that's right. Yeah.

Ian Livingstone So I, I coming back to the stateful VS stateful, it's like apps are changing, work flows are changing. Asynchronous work flows are at the core of the future of computing and how we interact with computers. And we need a good mechanism for how we manage that. And it should be standardized so that it's easy for agents and developers to build those kinds of applications. And also for us to have a prolific web and a prolific computing experience that can deal with all the different long tail tasks we have out there.

Dex Horthy We've been chasing it for a long time. We have one protocol to rule them all that makes everybody's life easier. Once every 10 years we get something that happened with cloud a little bit. It happened with Kubernetes a lot. Hopefully we get it in MCP someday.

Ian Livingstone Maybe, maybe it's MCP. The one thing I will say is I do think there's a lot of reason for MCP to find success and there's a lot of reason why it has so much hype and people still talk about it is because it actually is solving some itch someplace.

Allie Howe That's true, yeah. It'll be interesting to see where MCP is at in a year from now. So maybe we'll have to do this again in a year. Will MCP still be around or not? I'm pretty sure it is thanks to this conversation. But yeah, if there's nothing else either of you would add, I think that's that's a wrap. Thank you so much for this.

Ian Livingstone Was fun. Are we still friends?

Dex Horthy Yeah. I think we're better friends now.