Containerize your agents!

I'm the co-founder of uh dagger. we're a programmable build engine and like we're going to see in a minute we can run all sorts of things other than builds and before that I was the co-founder of Docker so I want to talk about containerizing your agents I know there's a theme uh and why you would do that and how so tool calling very cool um fantastic Innovation now not only can the AI talk back at us and make things up it can now take over our life and run things for us and replace us awesome uh very powerful primitive everyone's building software on top of it right this is uh a very accurate diagram of how it works uh you know you might have read it anthropic had this great blog post about building agents uh that's the theory this is the practice you have a crazy person inside your code just calling tools messing with your environment right um You got to manage that now um but not to worry multi-agent will fix right we're going to narrow it down have more agents doing more specialized things this is what's going to happen now you have 50 crazy people inside your code and they're interacting with each other they're interacting with your environment and with you in increasingly insane ways and uh the complexity is just going to get worse as our uh agents get more uh complicated more ambitious we've seen this movie before this is how every web application uh used to look like 15 years ago if you think it's bad now if you think web apps are bad now those of you who were around 15 years ago was really bad these big crazy monoliths but then we split them up we we cleaned up the mess kind of right right we came up with this thing and then a whole ecosystem around it I know it is a theme sorry I'm a little obsessed with this um but this is important because what we figured out and we by we I mean the whole Dev Community is how to SP break down the monolith into manageable pieces right you take one piece of the app and then you put it in a box and you have some repeatability and certainty about what that box is going to do how it's going to work you can ship it conversion it you can uh run it several times and you have some repeatability in what's going to happen right and then you can assemble that like Lego now you can start building um more scalable robust systems and you get some sanity back right so I propose that we're going to have to do the same thing for agents right we're going to have to take those you know insane brains and then put them in a jar and then have some control over what we connect the jar to right inste of just letting them run crazy all over the place that way we can build cool robots so instead of this giant monolithic robot just messing around with your environment picture these cute adorable look how adorable they are these cute adorable robots that you're building that's your software it's the little pieces of your software and you're plugging brains into them and you're making them do what you want them to do and you control the constraints you got these weird legs you got the Wheels Etc you see what I'm saying so that's what we're going to have to do uh we just need to develop the tech to do it so that's where I try to demo doing that this is where I need my professional M holder thank you oops all right so um you ready for a demo yes okay wow I'm doing not too bad on time so what we're going to try and do we're going to try and build Melvin Melvin is like the toy open source version of Devon you know Devon we're not going to actually build all of Devon we're going to try and apply this principle of building little components and that are very simple almost comically simple and then build up from there and see how far we can go basically so I'm not going to try and blow you away with an actual replica of Devon right just want to give you an idea of what's possible with this pattern and then encourage you to maybe come uh see me afterwards and talk about how we could build it together that's the idea oh yeah yeah yeah is it better bigger oh good okay w we got a pretty young audience then I can't read this okay um right so this is a dagger module um but first I got to explain what dagger is okay so dagger is an open source engine it's a tool you install it on your computer I Haven it installed here you install it on your server uh and then it acts as a runtime on them for what for your builds so you hit its API uh you got sdks and go typescript python a bunch of languages uh or you can uh call it from the command line have a little interpreter and then through that API you dynamically uh construct a dck a dag of what a dag of immutable artifacts and the functions transforming them what's an artifact container container State uh so container snapshot directory snapshot file secret um reference to to a network endpoint things like that um and then you just transform and transform and transform and you're building this dag and that has great properties like it's very cachable uh it's very repeatable and we use that um with our open source Community devops Engineers use it to make their builds very repeatable very scalable and then actually before I go to the end then let me show you what it looks like so this is the little interpreter I was talking about uh dagger shell it's a little interpreter but under the hood it's calling the dagger API so what I'm going to do in call is call functions like container is this big enough okay bigger bigger bigger like this okay so I'm typing a command that looks kind of like a shell pipeline container pipe this pipe that right but it's actually hitting a completely typed uh discoverable API right uh so I'm calling a function I get a container now I'm calling another function downloads an image I'm going to call another function uh you know add a file with new file hi oh no wait actually yay um someone dared me earlier to say that my favorite language is PHP so there you go are you happy said it so I'm building a container with this these these um container or a directory or whatever and I can just build up and build up and build up right and so here I'll just say give me a terminal so I'll just run that it's going to run it got some caching boom I got my container I'm in a container with that little yay. txt file that I built right uh I exit it it's gone it's in the cache and I can iterate again okay let me add this let me add that right that's how people use dagger think a picture like a Docker build with an API some sdks on top and then um then you could keep building from there so we got some very large teams automating their builds with this um but then what happens when you start automating your builds with us they're containerized right it's all running in containers so it runs the same in c local machine that's kind of the benefit um then you realize hey wait a minute I can run anything with this because it's programmable as long as it's shaped like a build like it's a dag of immutable states with functions transforming them and then we have an open source Community going out there and looking for things that look like that oh I can dagger eyesee my tests I can dagger IE my deployments dagger eyesee my um data pipelines and then one day someone said hey I can dagger an agent because guess what llms work like buildt llms work in the pure world of math right they have an immutable State it's their context then you call a function you add data to the context you know a prompt a reply pool call that's all immutable States transformed by functions so it turns out works great with a dagger API so what we did this is a Dev version of dagger this is the part where it might blow up is we added a par called llm so now I can just call llm this gives me an immutable empty llm State here it's a GPT 4 we just plug it on you know any model basically uh and then I'm going to chain operations so obviously I can add a prompt like that's easy you know but I'll show that after I guess should I show it show it um with prompt you know name is Bob and then I can I can get the history so far right you know by the way this is all traced with open Telemetry and we have a cloud service that shows you the traces it's great you know it's talking back whatever okay so far so good okay okay so the cool part obviously you don't need like this whole thing to prompt that's easy cool thing is the tool calling right so to call tools what do you need you need functions with uh types arguments you need a schema right well we have that the whole dagger API is made of objects with functions and uh typed API for these functions and then also we have the state right we know the state every step of the way because we have the whole dag so what if you could take any object say for example this container that I built before and then I call LM and I say with you can say it's go completing here not a cache volume no container yes what it what dagger did is it detected all the types available in the API and then I can give it as an argument any object of that type for example this can you see it oh yeah remove terminal better okay so I gave it the container now what it's going to do is dagger automatically is going to detect all the function available in that container object then it's going to plug them into the model and say this is your environment now right it's connecting the robot body to uh the brain call that the BBI the body brain interface it sounds cool and then I can keep chaining so now I can prompt it I can say um I'm going to try and prove here what's going on you have access to a container by the way just keep in mind everything I'm doing in the terminal here you can in code right so picture yourself writing something that looks exactly like that but you're chaining function calls in python or in go or in typescript we actually we're going to get to that in a second you have access to a container um read the file yay.

txt at path we haven't solved prompt engineering at Pro exclamation point um it mentions the programming language install on the container what is needed to develop in that language see what I'm saying like Okay we're trying to have a create some sort of a toy agentic Loop here I'm not saying the language that I only promised I would name it once so I'm not going to say it again um okay and then I'm going to look at the history after it does that H go okay so okay so it's calling oh oh oh too okay wait it's going too fast wait let me okay I'm going to go to the browser look at this Trace in the browser H okay so this is my tra Trace you know all open Telemetry can go back and say what did it do okay um I prompted it here oh yeah thank you at the reminders thank you okay that's my prompt right uh where is it uh you know what actually I got the history back it's done let me just do it here uh sorry okay go back up ah I got to have to make it smaller now sorry uh okay wait is this still readable no I'm trying to see the whole thing in one shot is this readable okay so at the top you see my prompt you have access to container this is just rendered on the terminal from the from the trace so the trace you can look at in the terminal save it for later whatever so you get the prompt then here's the magic um we plugged that container object it has a function called file it's in the dagger API and it's all you know documented so the model said okay I got to read a file okay file cool it got it returned an ID okay let me look up the contents of that you can see here the API returned my my favorite language is beep uh and then it said okay I'm going to do what was asked and tried to execute that's also dagger API let's let's APK updates ah error because it's an Alpine container I think it's the what I give it I forget yeah Alpine so error it gets the air AP oh second shot okay okay let's try APK uh it tried okay madees another Dr it it worked so now it's going okay I'm going to install PHP blah blah got an error uh it's like am I doing this right what is drro is this you know so this is the agent figuring things out and looks there is results yeah okay it's Alpine so what's wrong you know um I don't know what was wrong oh it figured out okay just one of these is doesn't exist I'll just install PHP good enough okay I'm done okay okay I did it basically it's saying I did it I did it okay did you do it how do I know well I can get the object back right so I'm going to query back from the state of the L give me the container back that you modified now I'm back in what I was showing in the beginning right it's just another it's a function that returns a container and then I can say right let's get a terminal in there so oh yeah and let's bump it up so good and then we're back in the container we got y. txt but normally PHP is installed there you go the agent installed PHP on its own okay so okay so that's the Primitive I'm over time I'm going to try and finish real quick um sorry okay so how do we use this here so here I'm writing a module and a module is just a little bit of code this is you know that extends the dagger API with my types and my uh functions and then any other client like someone else using the Shell can call that right and so this is my Melvin module it's got a new type Melvin and I I filled in the magic right this magic goes here it's going to be a function we're going to write a function that takes an assignment like this is a program I want and it's going to return a container with a go program ready to be built you know right what why is it called Melvin oh it's called Melvin because it's you know it's a it's a joke on Devon Devon Melvin yeah open source clone of Devon yeah have to name it something okay so um what we're going to do is we're going to write we're take this little modular approach we're going to write the smallest possible object that Melvin would need to get work done right I'm not going to give it a full container object I could but it's going to get lost because there just so much to do uh I'm going to just constrain it this is the magic of agent development right you want to be in control of the constraints sometimes you want to give it a lot of tools sometimes very little tools it should be very easy to kind of decide and change and split right um that's what we're looking for like a rapid itation Loop so here I'm going to I'm going to look for a module that someone else wrote uh because there's a whole Community out there making these diagram modules already um I'm going to look for one that uh has actually wrote it spoiler but the demo um three functions what you need what's the base berum for work space you need a container you know in the state then you need a read a read file function a WR file function and then you need a build function and I'm going to really make it simple the build function doesn't even take arguments you just build so I'm going to hard code this to be a go environment right to make it simple so I'm going to give the model an an object with three buttons basically like read file right file and then does it build and that's it and and constraint it to the max so I'm just going to show you the code for that I you know wrote it um it's called toy workspace and I told called it toy workspace to emphasize that I wrote this it's 49 lines of code you can write this this is go could be python could be typescript and if you see here you recognize container. from dot okay Mount aach volume change the work deer it's basically Docker file but in code right um this is my type my toy workspace it's a type with just a container in the state and then read is two lines oh no it's one line you know take the path and I I get that container in my state I get the file I return the contents you know right the opposite you know I I I take the container I add a file and notice it's you can see the immutable nature of it it's getting me a new immutable container and then I'm setting it in my state and then I'm returning new state so you can see the chaining and then build is like I'm going to execute go build in this container that's it just that's it okay so I'm going to use it we're going to use that now um let's go back to my little magic goes here I can't read it hell where is it main.

go I oh there thank you okay so I I pre-wrote the the the version that actually works it's down here [Music] Tada okay I'm just gonna Boop cooking show go program that actually works yeah I pre-cooked this go program okay here's the magic you ready Tada it's eight lines wait seven lines okay so here's what we're doing uh I installed that o that other toy workspace module right you do something like this uh dagger install to workspace you can do this over GitHub also like you can install some random person's module and then magically you have bindings so now I can have I can instant shate a Tory workspace so this is my workspace before the agent right just give me a workspace then I instantiate llm I give it my toy workspace right notice notice that it can magically recognize that type um a little bit of prompting here nothing fancy show you the prompt really basic I'm not a pro AI engineer at all this is my prompt you know you're a programmer you have a workspace please use it please don't use this thing you know can't handle it um here's the assignment and I just have a tiny little variable system I just take my string here I put it here prompt bar and then boom I can put it here this is the basic prompting okay that's the loop it happens here uh this I just give it the file right and then I get the workspace back same thing I did with the container give it the object give it the prompt Loop get the object back and then that's after right if my IDE by the way a completes all of that right can say okay that's of Type Toy workspace and here if I wanted I could say container with not we working oh with new file see what I'm saying it's the same type system I can just keep chaining stuff but I won't I'm going to keep it simple get the container out of the workspace return it boom okay so does it run let's see let's find out um I got to reload this so I'm going to relearn the shell it's going to load the current module so now we have this eight line function calling this 49 line object as an environment and plugging into llm can you make it slightly bigger one more time yes yes sir and now in addition to all the other stuff I called before I can just call go program and I can say write me girl clone we're going to keep it simple here okay go okay so now has only these three functions it tries to read main. go see that read that's my function that I wrote and it fails because there's nothing there then it writes a file oh it's writing a file it's calling build and build work worked no I don't believe that oh no no wait I'm going to go see that in the browser let's go find out what happened here some cool emojis maybe oh yeah that one's going to be cool okay wait this is our tracing Central this is our team you can see every everyone's traces from the dagger team just like running stuff pretty cool this is our CI on the left um give you a tour later if you want okay so this is okay this is what happened prompt tried to read see that like big there making it bigger Tred to read main. go okay no main.

go I'm going to write main. go this is what it wrote you know code that we know knows how to do that and then it called build my build function that I wrote that I showed you right and what did it say did it work oh no see it said ah you forgot the config file you forgot something and then it said okay oh right I forgot the go. mod and so it added a go.