How To Build Scalable AI Agent Teams | AI Agent Architecture 101

okay so today we're going to talk about AI agent architecture and we're going to kind of take a bird's eye view on the AI agent space in general right now all of the players not just the AI agent like Builders but you know some other players as well and then I'm going to present like my idea about you know what's the right way to do this especially the right way to build it in a way where it scales over time and then some practical steps on how to like get started right so we're going to

look at take a look at the space we're going to go over like where we think you know it's actually going and then we are going to um talk about what what action items you can do like right away right so you know I'm just going to get into it there are tons of AI agent uh you know you know companies right like they're they're building AI agents that you can pretty much buy either off the shelf or like quickly create right so like this one AI agents power up productivity you know enter in a

couple of things and then all of a sudden you created an agent cool agent. this is done by like HubSpot right and D over there is kind of leading the way you can see he's he's the main contributor uh here right so off the shelf let me get an agent right Lindy build AI automations in minutes so a little bit different than an agent but you know they're going for the same thing CRA they're like personifying AI agents essentially all your AI employees in one you know digital assistants you know Cassie uh SEO me I

guess buddy right personifying agents Lena autonomous agent for it HR Finance tickets right create and manage GPT agents that your users love right you can do the whole call thing that vapy does it's like man there's like so many here like right so many options if you're interested in agents and you just Google agents like there's just like a ridiculous amount of options now so many people are seeing the opportunity but I think that they're they're T they're going about it the wrong way open AI is posting blog articles about you know know companies building

agents using their model Salesforce finally released their Einstein SDR and sales coach built into their platform so now if you're an agent Builder you're like oh man you know these big platforms are building agents into their own platforms right like where why why would I build agents right is what you should be thinking frankly um beam customer service agent right so we get requests to build customer service agents but you could technically just go buy one off the shelf it looks like so why wouldn't you do that right so you got all these options to

essentially have an agent that's either built into your current platform or just buy one off the shelf that someone's created right and then you got these platforms and this is only a couple but you know flow wise a lot of people ask me about flow wise yeah they're great I mean you can build your own agents build your own tool build your own agents looks some similar to uh n8n right stack AI for Enterprise in particular but you know these guys are actually taking a really good approach um especially on the data side of things

these guys get it I talked to the founder from there not too long ago um and you know I think they kind of get the space more than others but yeah they're working on it you can build agents here you know then you have kind of these like open source some hacky uh looking sites agent GPT autogen you know crew AI even I have up here a little bit more I mean I I think these are this one is less code heavy um but you know you can build your own agents relevance AI super popular

one you know people are building entire businesses on top of these platforms as like we will build your AI agents and Implement them right and and a lot of people think that that's what we're doing too right like oh you guys are like the inad in guys who build agents and it's like we love inad in we think it's like the best platform for it but we're not the inad in guys necessarily right um but anyway great I mean relevance is a great platform for it they have a really good interface here um and then

but you can also get an out of the box agent it's not necessarily like fully out of the box I don't think uh like I think they do some level implementation but same idea I'm sure they'll come out with more like look inbound outbound yeah mind Studio this is more like AI automations but you can pretty much build the same thing right have some kind of form which we're going to add to our website but have some kind of form people fill out the form and then all of a sudden an agent on the back

end take some actions right provides more value than you would if it was just booking a calendar event or entering your email or whatever right you can do that with mind Studio you can do a lot of things but you know and then crew AI obviously so you got places that are offering like off-the-shelf agents or basically really quick to spin up agents and then you got your platforms that are you know build your own Agents from scratch right integrate your Tech stack and build your own Agents from scratch right so what does this mean

an explosion in startups and businesses and and frankly like Hustlers right entrepreneurs building agents and then offering and selling selling them an explosion in platforms to go build agents right it's extremely easy so the barrier to entry is not super super high um it's a little high but it's not crazy right it's not like you have to learn a programming language so what does this mean it means that most people are playing the short game and there's a lot of value and money that's going to be created and extracted during the short game like if

you bought a customer service agent or you bought like a receptionist like phone call Voice agent off the shelf and it works for you that's the immediate value that you're seeing right so it's not like it's nothing right it's not like they're this is this is the wrong approach period right so there value will be created but there's also the long game which is and I'm going to get into detail on it but which is a complete operating system shift at least on the business Ops level a complete shift ground up right and that's what

we're focused on and obviously there's a lot more value um and have money backs here money backs here but it's more about the value right the value that this is going to provide overall so how can I play the long game Let's explain so everyone right now is focused on this layer right so if we think about traditional SAS it's essentially a database with a graphical interface on top of it that allows your employees and yourself to do stuff right like either save data and extract it in an easier way or execute some kind of

action on the back end with data right but it allows you to kind of interact with it and press buttons and drag and drop and move things around it's built for humans right and the people who are mostly interacting in these platforms are kind of like your lower lower level entry level employee right and right now everyone is focused on and ourselves included at least at the start is focus on okay we have SAS that's cool but how do we replace the humans that are interacting with the platforms right and so everyone's focused on creating

agents to replace that that layer right but the truth is and I think I have this out of order but let me see yeah so the truth is if we think about it from a groundup perspective we have the a database but the data right now is on all your different platforms it's on Gmail it's on your calendar it's on your CRM or your ATS it's it's spread out across your various different platforms and if you're a more mature business you probably have a data warehouse where all of that is coming into but what it

means is when you're building agents they don't have full context you know they have just the amount of contacts that you need to give it to execute that micro workflow that you're having it do but essentially let's try to go one one for one pairs here a master contextual database is what we want to build right all the data that's coming into your business all the data points that are being created all the data about your business goes into it gets embedded into a vector database and then you can build Foundation agents on top of

it that can manipulate and do whatever needs to be done with that data take actions right and then you have workflow specific agents that use those found foundational agents to do like more custom workflows depending on your business right and I'm going to get into more detail on but what that's essentially doing is like okay there SAS not necessarily a data warehouse but SAS has a database SAS here is like graphical interface and then lower level employees are like interacting with the SAS right probably need to make this better but So eventually what's going to

happen if it's like all agents right is you have your master contextual database as the foundation then you have your foundation agents as kind of that first layer of interacting with right now what would be your Tech stack and the data and then you have workflow specific agents right that are just handling a workflow like an SDR agent or a customer support agent all of those off-the-shelf agents that I showed you then later we'll get management agents Chief of Staff agents custom executive agents like you know as the as the models get better and the

reasoning ability and the ability to plan for longtime Horizon um strategies right um you know then we'll get some higher level agents right but for now now it's like workflow specific agents and and Foundation agents and what you start to realize is like okay well if SAS is built for humans and humans are the ones who are primarily interacting with the platform and the database and moving the data and the information back and forth from one place to another whatever and we're replacing that activity with agents in general what you realize is like okay well

SAS is probably going to die right if we're thinking about agents like as a full infrastructure for a business it becomes an entirely new operating system and that's the key it's not just about SAS Foundation agents will replace SAS right and I say this is not just about SAS because all of these providers of agents are trying to build SAS you know they're trying to build a software company and it makes sense because it's the best company to build right now because you get the valuations that you want you get the exit that you want

right in some ways it's easier to build in some ways and it's more of like a product that people pay monthly for it's insane but it's really just software as a service the service component used to be done by people and now that it's just replaced by software eventually that software will become agents and agents will do that service and so SAS as we know it today will not survive I think it'll I think it'll die in like five years three to five most likely 10 and they'll still be some platforms and you know we'll

still have the browser and but for business it's going to be too costly it might change so anyway I'm I'm jumping back and forth but anyway I have some ideas about where this will go and I'll get into that in a second but traditional sass as we think about it today will not survive assuming the agents get better in the models get better which I think they will so how could this look in real life well it could look like where you have a complete in-house team of Agents where everything that you do on your

Tech stack right now all the platforms that you need to get in order to operate your business is actually just all in-house and you just have agents doing all that stuff for you right you have a database so it's got all the data that they need and then you have the tools and the Agents to go and take all the actions that you're doing right now on various different platforms but you're doing it with just your internal team of Agents right or what could happen is you have like a smaller internal team of agents and

everything is gated basically so like gated access to domain specific teams of Agents so let's say you take instantly AI right now instantly is a cold email platform it allows you to run cold email campaigns right so it's probably the best one out there and right now if you want to do cold email you sign up for instantly you put all your email domains your email accounts there you warm them up you write your campaigns there you know you can even get your leads there now and you launch it well in the future let's say

that you're not actually physically logging into instantly physically writing the campaigns physically importing all of the leads there right physically putting in the variables that need to be put in pressing all the buttons that need to be pressed right you're not physically inside of instantly instead instantly has their layer of agents that are making sure that they get the data that they need and then using that data to go and execute the cold email but they're interacting with your layer of Agents right who understand your business in the context or whatever and so when you

tell them hey we need to set up a new uh cemo campaign for this new offer that we have Boom the your agents have context they know who the offer is going to go out to they know the writing style of the email that you probably we want right and so they just send all that information to instantly team of Agents instantly team of Agents does whatever they need to do with that data to go and execute that cold emo campaign right set it up and execute and but in this case you would be paying

instantly still monthly for access to their system essentially right instead of it all being in house and your team is doing it you're paying instantly and so it's probably going to be a combination of both here and it probably won't be as Extreme as it is today day where we have to have like 20 different platforms operate our businesses right and pay subscriptions to do it it's probably not going to be that extreme because you'll be able to do a lot inhouse but there are certain things that you know these companies have they have a

moat around or a technical you know there's there's some kind of like technical thing that it would be harder to do for you inh house right it would just be cheaper to just or more efficient to just go and pay for access to their platform and have your agents access their platform right so it's probably going to be a combo of both if we're looking at domain specific agents it's kind of like how I explained right instantly got their cold email agents HubSpot has the crn management agents Gmail calendar Assa they all got their agents

right and your internal database and agent teams are talking to those agents to go and execute the actions that need to be executed instead of it being like your team or your vas or whatever hopping on these platforms and doing it manually the agents will go and do that stuff right and you pay for access to like their systems right but the actual value here is the workflows the strategies the IP the patents the copyright that's the real value ultimately it's not in the tech because the agents will be able to do the tech it

will be able to handle the platforms and do the tooling like you know you you come up with a micros SAS or a SAS platform that's got a better UI than you know the the industry leader and so you you gain a little bit of market share that's probably not the game we're playing anymore the game we're playing is how do we develop really really Elite workflows that businesses can download and embed into their existing systems so if you take like deoe or McKenzie or any of the consulting firms who've worked with thousands of businesses

in a very specific Niche right and so they've kind of developed Sops and strategies and processes around you know developing the best sales team and operation and workflows right they have all this information about how to build the best sales team possible and how to operate that sales team in a way where it's like the best in the world right and so they have all of this information they could technically I mean in the future package up that information I mean they could do it now but they could package up that information and you can

basically go buy it right like you have your team of Agents you buy that workflow right the best sales operations team ever best sales operations workflow ever you buy it embed it into your team into your team and now you have the best sales operations workflow ever right so the value wasn't in any kind of platform or you you had the the best phone system you used open phone instead of twilio or something right like that's not the value the value is how like how are you piecing the tools together how are you using the

tools right it's the workflows the strategies the IP the patent the contents or the copywriting right it's the content it's the ideas that's the real value and so at some point like web 3 or blockchain or you know whatever will come into the fold and we'll be able to assign ownership to these you know to information right so things aren't like pirated as much as they are now um and I know information wants to be free but when people realize that that like the value is that it's not in the tech it's not in the

execution necessarily um you know then I think the prices for these things go up I think the the gates around them start to go up right and then we a whole Marketplace starts to explode right so cool so what do I do now how do I actually what does this mean for me like practically right now well start start with the basics the number one thing is the the data so start building your contextual database that Vector database start capturing all of the data that's going on in your business build foundational agents that are connected

to your techstack right now SAS isn't going away immediately so we got to work with it and then build custom agents that are specific to your workflows right sounds simple start building your house Master database essentially a centralized database with all the company data it's going to be embedded and you're put into a vector database you can get pine cone quadrant and then extract all your data that lives on every platform that you use put it to the database like add it in there right next you want to set up data capture so every data

point needs to be collected and added to that Master database there's likely going to have to be some Innovations here on uh efficiency over time but for now capture every data point add it to that database so review your text stack identify where the data points are being created email being sent calendar event being created new project new support ticket Etc then set up the automations that will collect that data and add it to the master database in real time next you want to build your foundation agents so these are the agents that will sit

on top of your entire text stack right so review your text stack identify the API endpoints that are available list out all of the input data needed to execute that endpoint so if it's send a message for email you need a recipient email address their name the subject line and the message right those are required then there's some options where you can do like you can CC somebody BCC somebody change the sender name add an attachment whatever list out all of those parameters all of those possible endpoints and then use those to build the tools

for each endpoint so send a message is an endpoint create a calendar event is an endpoint creating an engagement in HubSpot is an endpoint right and there's par that need to go in to those endpoints so it could go do it right so like make sure you're collecting all that stuff and then attach those tools that you've built to an agent and write a general prompt right you are the Trello agent right you can take any action on Trello here are all the end points right put in the Define the input schema right set it

up to be a general agent for that platform Foundation agent and once you do that you can start to build custom agents and this is this is the part where everyone has skipped to they're like oh customer support agent SDR agent you know HR agent okay cool but if you skip those early Parts in a year or two it's not going to matter like it's really not going to matter you're you're still going to be stuck on some platform that's got your agents on it that you've worked with you know because you bought them out

of the box which you can get value from now you absolutely can if you have the paino now do it but just realize like there's a groundup approach to this that is going to happen and by the time you get to the custom agent point you will be so much better off if you had done the steps prior um and it will make it scalable which is the idea right you buy off the shelf agents it's not really scalable um so anyway agents that are specific to your business so these are custom workflow agents let's

say like any workflow that an AI could do create an agent to handle those give that agent foundational agents as tools and then use your imagination creativity for how to get things done so let's say you you typically have um you know a recruiting coordinator their job is primarily to uh you know kind of like post the job post on the actual platform feel the incoming uh applications um coordinate the meeting right make sure that the candidate is moving through the process they're staying on top of it make sure the hiring manager opens up their

calendar for the interviews right a lot of communication a lot of coordinating well they need access to the ATS they need access to email they need obviously understanding contextual understanding about what they need to do right but access to the email just means give them an email agent access to the ATS means give them that ATS agent right and so then build an agent on top of it where it says okay and you also need a calendar agent you okay you know email back and forth with this um candidate find a time on the calendar

that works for them and the hiring manager and then log what you just did in the ATS it's like okay you have all the agents that have access to those platforms cool now let's build a custom agent that all they need are those other agents as tools to just go and do the workflow that I just mentioned right that's that's what your employee is doing currently right now the foundation agents think of them as SAS think of them as the platform itself the platform itself right now we have to kind of say okay we're doing

API endpoints and we're connecting it to to the Tex stack and that's how we're doing it right now eventually that's not going to be the case like I said but anyway this is the way to build a system that is scalable over time with AI agents despite the evolving and shifting uh AI space right despite any of the updates that come out from open Ai and others this is what's like in my opinion this is what you you're going to have to do this is what's going this is what's coming right because it's not about

like how can humans better interact in a digital space which is essentially just how how how can humans better speak back and forth with computers right which is like why we even have SAS to begin with you know and we have like graphical interfaces and people get paid Mill millions of dollars to make buttons look really good it's not about the human interaction with the computer anymore we have that solved it's it's natural language it's not even about the human reasoning to some extent we have that solv to some extent so if you don't need

that anymore what does that mean it means that everything has been paired down become way more efficient ient and you know how's it going to look is it going to look like everything's gate kept and you know you're a couple of Agents have to you got to pay for access to these really you know customized workflow agents that are are really really good really really robust and you're you're paying for them and you know it's it's it's essentially SAS but there's no reason for you to log in and sign up and and interface with it

but you're essentially paying SAS like that's probably where the industry is going to go they're going to try to or is it going to be all in house are you going to build everything you're going to build your own database and your own actions on top of it build your own Sask because now it's easier to do that right it is easier you could build agents that build other agents and we're working on that right now so this is just food for thought something that I've been thinking about for a while and if you've talked

to me at all in the recent weeks you know that I've been ranting about this um you know and it kind of is similar to the concept of uh you know Foundation models but then people are talking about llms like models that are specific to a niche and it's like you know why why does that matter so much um maybe it does maybe it doesn't but the agents will be specific to use cases workflows platforms right that's the future I know I do a lot out the idea is still it's clear for me but I

probably need to do a little bit more writing on it to make it very very clear you know um and who knows I could be wrong but the approach that we're taking is a long game which is a bottomup change in how we operate businesses and taking an agent first approach to it as opposed to oh I got all these people I got this current text stack in database how do I just like replace the people right how do I how do I replace it on a workflow level as opposed to a bottom up approach

right now I'm repeating myself but think you get the idea if you have any questions let me know um I I did start a Discord kind of snuck it in there in the in the uh description feel free to join it but we are going to be Shifting the community itself to a different um place we're going to kind of like own it right U but I'll let you guys know when uh when we do that so yeah that's all for today