Introducing Agency Swarm Realtime Voice Interface

imagine if you could have an eii agent that could run your entire life for you what if you could just say hey Sky what's my schedule as like for today your schedule for today 5:30 a. m. to 6:30 a.

m. meditation and journaling 6:30 a. m.

to 12:00 p. m. deep work 11:30 a.

m. to 12:30 p. m.

meeting with Helen Elizabeth Wayne arseny and Merlin Ravens 2:00 p. m. to 3:30 p.

m. workout okay awesome can you check all of the emails I have unread in my mailbox you have 10 unread emails the latest ones include need more details or want to address any of these now can you please open that website that the client mentioned in one of the emails the website has been open okay cool and can you tell me what's on this website on the website you see an e-commerce store with a variety of Kitchen products is there anything else you'd like to do yes can you please tell the project scoper agent to create a project scope for this potential client the project scoper agent has created a detailed project scope for the e-commerce client specializing in Kitchen products it covers objectives key deliverables requirements timeline collaboration and success metrics would you like to review any specific section in more detail uh no can you just save this project scope to a local file please the project scope has been saved to a local file named projector scope Ecommerce kitchen products. txt so that's how it works I've been saying for months now that the way we interact with technology is going to change soon and today we are introducing agency form voice in interface which allows you to interact with your agents in a more natural way but the best part about this project is how simple it is to customize yourself so all it takes to add your tools and agents is literally just dropping them into either the tools or the agency's directory and that's it and for those who do not know how to create tools or agents themselves we of course also have the cursor rules file which will allow you to create all of the tools and agents for you so let's Dive In okay so first huge shout out to indev Dan this project was fored from his PC python realtime API assistant repo however with this repo there were a few significant challenges specifically it made it incredibly hard to add tools especially if you're a beginner all of the tools in the repo are inside the tools.

P directory and in order for you to add a tool you would have to like create and modify code in like three different places because at first you would have to like create a function then you would have to create the schema then you would also have to upload it to your agent so that's definitely challenging for not just beginners but even for pros which is why in our repo you can do this with a consistent B tool class interface that you can extend with your own capabilities and then all you need to do is basically just drop it into the tools that trory and this assistant will instantly have access to that tool so as you can see the code is much cleaner if you know how to use my framework you know how convenient it is to use penic for tools so the same principles apply here additionally it made it incredibly hard to add agents in fact agents in this repo are not really agents they are just basically tools so as you can see here it has like a tool that calls another lolm and it has a specific prom structure and then just Returns the response from this lolm but in ouro again you can just simply drop your agents into the agency's folder and then instantly this voice interface will allow you to interact with these agents and it canot only interact you know with the CEO of your agency so for example if you drop like an agency here with the COO agent on top it can still interact with other sub agents in this agency and lastly you can't use assistance API here while of course in Agency for all of the agents are built with assistant so it means that you have code interpreter and file search out of the box so that's basically what we've done here and now let me show you how to actually use it how to add your own agents and tools if you already have them and then later we'll also do a step-by-step tutorial on how you can create new tools and agents with cursor all right let's dive in so the first step would be to clone the repo if you run into any challenges here just ask Char gbt and then open it in cursor after that the next step would be to install the requirements so for this project to run you need to install UV which is a modern package manager I really like this choice to be honest and you can install it with just a single command so you can either use Mac OS or Linux command so just run this in your terminal and this will install UV after that you would need to copy the n. sample file so just copy this n. sample file and here add your own open I key and also rename it into justn so you need to copy this and rename it into n and then add your API key then you can also modify personalization and this actually another thing that we modified from the PC repo in the PC repo it was hard to modify the instructions for your agent while here you have access to the assistant instructions so you can easily know add your own information about know what you do about yourself and the tasks that you want this agent to perform and how Okay after that install Port audio so just simply run this command Brew install Port audio in your terminal and finally just run the UV sync command which is like pep install basically in UV like this you can already use this default assistant with some of the tools that don't require authentication however if you want to use tools that use Google cloud apis like for example the tool to check your calendar or the tool to check your emails and of course you can also build more tools on top of this with other Google apis you would have to get the Google Cloud credentials so in order to do that we first need to create a Google Cloud project so go to console cloud.

google. com and here just select your project if you don't have a project just create one it's not that hard and after that you would need to enable two API so first of all you would search for Gmail API in the search bar here on top and then initially here you will see an activate button or something so just click on that button and then wait until the API is enabled after that do the same for the Google Calendar API then click manage and here under o off consent screen you would need to configure your app so just name it as the name of your project like for example voice interface and then when creating this app so I already have this app created but I think it will go through the same process here so just click save and continue here on add or remove Scopes you would need to add the Scopes from the the readme file so here just copy all of these links right here and then add them under manually add Scopes just add it like this and then click add to the table so do this for all of the Scopes I've already done this before so I'm not going to do all of that and then just click save and continue again in your test users just add email addresses of everyone who's going to be using this application and then click save and continue again okay now what we need to do is just get the credentials file so go to credentials right here and then just click create credentials here on top and select all off client ID for the application type select desktop app name it again as the name of your project and click create awesome so after that click download Json and this will download the credentials. Json file to your local machine and you would just need to rename it into simply credentials.

Json so like I have right here initially it's going to have like a very long name but you just need to name it credentials. Json and place it into the top level directory of voice interface repo so yeah that's basically it for setup this is literally all it takes now you can use it with the default tools so we've added some example tools for you that you can run out of the box and to do that just simply run the final command which is UV run Main hey Sky how are you hello how can I help you today awesome so sky is online and yeah feel free to play with this even like this it can be really helpful I'm actually going to be using it as a COO you can use it for a lot of different purposes actually just by modifying The Prompt but if you want to take it a step further what you can do is of course also create your own tools and agencies so if you want to add agencies as I said it's very simple you will just like drop them into the directory so let me show you for example let me take the agency that I developed on the previous video which is the content creation agency so let me copy that and then go back here and just paste it into the agency's file so now as you can see the content creation agency is here but what you need to do is also just change the UTS so you can just select all the code and say change to relative Imports and yeah that's the smartest way to use AI ever as you can see I added three thoughts this is why open I chain this model for $100 million so now we can run this again and this voice assistant will have access to this agency so I actually got an error because the T package is not installed so I just forgot to install the packages from the previous repot so just make sure to copy the requirements. txt file also put it somewhere in the root directory and then just run UV people install dasr requirements.