Azure AI Studio vs Copilot Studio

if you're wondering about the difference between Azure AI studio and co-pilot Studio or in fact Azure open AI studio and the bot framework and all of these pieces that allow us to create co-pilots I'm going to take you through an explanation and a demo here to help you get this sorted out so Azure AI studio and co-pilot Studio are actually two quite different things and it's not necessarily one or the other they do work together so to start with what I'm going to do is give you a bit of a demo walk through of azure

AI studio and then we're going to spend most of the time having a look at the co-pilot Studio experience so that you can get that real sense and understanding of what these tools do and how you can choose which one to use and when so to help you understand this I'm going to frame this up using three c words cost complexity and another phrase called conversational orchestration now I reckon as we go through this video I'm going to come up with more words don't worry we're not going anywhere near anything that we shouldn't put on

YouTube let me know how many you find as I go along here co-pilot is also going to be another one of them let's start here with Azure AI studio now this is a place that you start if you want to do something custom you want a customized AI solution you want a lot of control over what you're doing the concept here is that you are starting with your AI model and having a lot of control over what you can do with it but that's the starting point and you can do so much here this is

not a full tutorial on as your AI Studio but this should be enough to give you a sense of what it is so this is my Opening screen you've got all of these things that you can do here if I click on explore you'll see the first thing that happens is that you are given the option to find the right model to build your custom AI solution so this is the starting point these are all sorts of models in here at the time of recording here we've got 1624 of them let's go in and click

on the filters and you'll see that we've got things here that are models that are coming from Microsoft as well as meta as your open AI hugging face all of these different things you've actually got the ability to come in here and choose to Benchmark your models so let's say I want to uh and I'm just going to pick it random because honestly for me this is not my world I don't know all of these things but let's sort of pick out a couple of things here just to sort of put them into this comparison

chart and you'll see you're getting things that are telling you about the model accuracy the model coherence there are a lot of metrics in here and numbers now if you know what these things are this is your world if you don't this is probably not going to be for you and when we get to co-pilot Studio you'll see how that actually allows you to start from a different place where all of this is taken care of for you but if you want to be able to say I want to use the GPT 35 turbo model

or this one or the the Llama model or whatever it is or I'm looking for models that will generate images or use Vision things like that then you can start here and have full control over all of those things you are also responsible here if you're doing it this way for building deploying everything so you have that greater control you have that ability to do those customized things but it's also going to mean that the time to deployment is much greater the cost of building it is going to be much greater plus you are actually

paying for whatever Services you choose here so that cost is a really important factor so you'll end up with something like this where you have a project you've got all of these different Tools in here where you can fine-tune the model work with the playground as I said not so much my area but lots of things in here that allow you to tweak and play with all of this stuff importantly you've also got control over the search the index the data sources so this is something where we're using the broader Azure tools here I've got

the search in here so we could come in here and say I want to work with my data and you've actually got all of these different kinds of things so you could be working with your data Insider SQL database file storage Cosmos blob storage this could be at scale and then you've also got control here of doing things with um semantic ranking Vector indexing all of these things around the search now I said I'm not going to go deep diving into all of that but but essentially Azure open AI is sorry I'll come back to

what Azure open AI is in a moment Azure AI is is a place where you can work with all of those models in a very custom way and you're building a custom AI solution with control over searching indexing all of those kinds of things but the complexity is quite High AZ your open AI Studio which I accidentally mentioned before very similar concept but this is just the open AI models so this is where you're working with those models that are used underneath chat GPT whereas the Azure AI studio is actually giving you if we go

back to our explore here models across a whole lot of other providers so those are the tools that are available to you in there now the difference between these things and working with co-pilot studio just as a starting point if I go back to my home here look at all these things we're starting to talk about custom AI assistance as your AI on your data multimodality SDK code is another factor in here but it's not as simple as just proc code versus low code but there another CW word for you this is the starting point

for co-pilot Studio completely different experience now this is a SAS product so the difference here is that if you're working with Azure AI Studio you are building deploying maintaining doing all of those things yourself from start and all of the maintenance as you go this is basically ready to go for you providing a bunch of stuff out of the box I don't have to in fact I can't choose my model as a starting point here but importantly this is a tool that is allowing you to do some other things so you can build your own

custom co-pilot that's the first option there which is basically building out an experience that can contain both generative Ai and conversational orchestration we'll come back to that in a moment you can also extend the Microsoft co-pilots that are already provided so Microsoft 365 copilot is the main one here that's the one that sits inside teams and word and Outlook and all of those things so if you want to take that experience and extend it to plug into your data sources but run the whole thing through that Microsoft 365 chat experience you can do that here

but to be clear this product comes with its own Standalone license it's us $200 per tenant per month depending on how much you're using you can add capacity in there you don't have to have co-pilot from Microsoft 365 to use this but if you do have this this is the tool that you can use to extend that I'm going to focus on the create a custom co-pilot here for the rest of this video though because I don't want to confuse those things with the first party co-pilots even though that's the extensibility experience in here so

what this allows you to do is to create your own custom co-pilot like I said this is an outof the-box solution think of it as um as a nice analogy of if you uh if you're old enough to remember having to hire videos or hire d DDS you have to maintain the machine that you use and you go out and get them and you've got all this Hardware this is like Netflix you've signed up and it's just there and you just consume what you want so this ability to you don't have to maintain you don't

have to deploy anything it's ready to go so what happens here when you come in to create a custom co-pilot is you have options to do two quite different things more than two but the two that I'm going to focus on here so if we go to say create a custom co-pilot the first thing that happens is it says do you want to give it some knowledge with generative AI by entering your website you don't have to just use a website here you can use this to put in documents from SharePoint you can upload documents

of course I've got one I've prepared earlier my favorite one here is the better healthbot which is actually connected to a website that is called The Better Health Channel this is a public service website in the part of the world where I work that helps people find health information it's very information rich and a really good example that I like to use here so I've basically pointed my co-pilot at that experience so let's talk about the generative AI experience first and then the conversational orchestration experience which is a really important part of what you get

if you work with copilot Studio that's very different from Azure open aai so Azure open AI I've chosen my model I've worked with all these complex data sources picked up my indexing all of those kinds of things this one all I've done is paste that URL in and that's it so I can just come in here and say what should I do if I need to call an ambulance and what that's going to do is just take my question refer to that data source and it's using the services that are built in here to generate

an answer for me including a link back to that website now I didn't do anything here other than point to that URL what this is doing under the hood it's using a GPT 3.5 5 turbo model at the time of recording which is April 2024 that's actually hosted on the Azure open AI service underneath but I didn't do anything about that I don't need to know that I'm telling you that cuz it's in the documentation but this is this idea of a SAS product I can just come in and that's all set up for me

so the difference here is that there's no complexity the cost is much more controlled but I don't have the amount of control and customization that I had when I was looking at the Azure open AI service or the Azure AI service the the idea here though is that the time to Value the time to being able to use it is super quick I could just plug in my data source and it's ready to go so you really have to think about what kind of solution you're looking for do you really need something custom really these

large language models and the large language model you've got access to here is highly sophisticated if you're going to go in and start customizing large language models or building your own models that is something that needs a team of data scientists it needs a lot of expertise and awesome if you've got that in your use case genuinely needs it and you'll pay the costs associated with it so you need to weigh up that cost complexity customization but do consider this this is actually something that with some of the twixs and I've got a whole video

here on how to get the most out of this generative answers capability you can actually go quite far with this so just really weigh up the costs of those things and what you actually need the other really important part the other the third C how many am I up to so far and by the way don't forget to give this video a like it helps get it out to other people and comment how many C's you found uh in here so the other thing here is this conversational orchestration what we mean by that is that

if your whole solution is just generative answers which is what I've done here I can just plug that in or plug in my SharePoint data sources or whatever it is and off it goes and that's it but that's not always the case so we've got an example here for instance where we've got watch should I do if I need to call an ambulance so we might want for this particular question instead of just directing people to that website we might want to give them some more information so one of the things that happens in Australia

here is that there are other services other than an ambulance some people will call an ambulance for things that aren't emergencies and block up the system so what we can do here is direct them in a controlled way this is this idea of conversational orchestration I want to actually control the experience experience so that for certain topics the conversation is forced down a predetermined path with things like variables and answers and specific questions or I could plug into other systems so I want control over some of those things and then this generative answers is my

fallback and this is quite common if you think about your use case if you're just starting with a model and that's all you want your whole solution is just generative AI with Azure open AI that's awesome but if you want this control over the topic then this is something you can do here the Azure bot framework which has now got a different name that I'm just going to check on the side here because they do keep changing things so this is also called the Azure AI bot Services now you can do a similar thing there

that is a high code solution though you are actually starting to customize things you'll have more control over the interface of what's going on not going to demo that one here but that is a sort of a high code way of doing it and these things do work together as well well so let's have a look at this concept of a topic I'm going to come in here and I'm going to create a new topic and this is where it gets a little bit uh a little bit meta so we're going to go I'm going

to create a topic from I'm going to use co-pilot to build me a topic for my co-pilot are you following this is essentially the ability to say let's call this one um calling an ambulance or whatever it is I can say create a topic too so if you saw the old Power Virtual agents or you worked with creating bots in the past you have to give it sort of examples of what are the prompts what are the things that the user is going to type in that will trigger this conversational path you don't even have

to come up with the phrases yourself now this is actually really clever so I'm just going to copy paste something I've done before rather than watching me type help a user to understand when they should call an ambulance and when they should go to a doctor instead so this is now going to create a topic for me and I can edit it so we'll we that's come up faster than I can explain it we're going to bring this along here so it's come up with the trigger phrases here call an ambulance emergency medical assistance need

immediate help so these are the kinds of things that the person might type and this is actually going to then redirect when people do that into this topic it's actually come up with a message generated for me now I can edit all of these things I will show you the editing in a moment so here we go call an ambulance if visit a doctor if this is pretty good so what I can do now let's just go over here and grab this one again what should I do if I need to call an ambulance and

you'll see what happens now is that it's gone into this topic so we can see the little ticks on the side I'm working this by the way just all in the test studio in the end this co-pilot actually gets published somewhere externally on a website or if it's an internal use case into teams there's a heap of channels and things but I'm just showing you the mechanic behind the system here does become not just in the test canvas but you'll see it's actually gone into this ambulance topic and instead of the first time round it's

used generative answers as that fallback to find the answer from the data source I gave it now it's coming in and giving me the controlled path let's do something a little bit more sophisticated here we're going to create another topic from a description again we'll call this one ambulance and I'm going to give it a little bit more so the first sentence is the same as what we had before but now I'm saying the bot will be used in Australia where the emergency number is triple 0 ask if the emergency is life-threatening if the emergency

is not life-threatening they should go to a priority Primary Care Center try saying that 10 times fast now hopefully someone genuinely using an emergency is making a phone call but anyway we're helping them out so you'll see what happens here we'll just close this down is that it's come up with something different so we've got those trigger phrases again you can edit the trigger phrases so you'll see here the 911 for an ambulance is actually something that isn't quite right for my context so I can actually go in and change those things as well and

then what we've got is the it's actually created a whole topic here that says is the emergency life-threatening it's looking for a bullan bullan answer of yes or no and then if it is we're going to go to this message that says please call Triple 0 immediately and if it's not then it's going to do that and then we could actually add add in a link to those things you could use this to escalate to an agent you could use this to uh post to another system or get information from another system so you can

do a whole lot here with the topics and this is this idea of conversational orchestration so co-pilot Studio actually let let's demonstrate this before we go any further so the same question that I had before what should I do if I need an ambulance and we'll watch it go through here so it's coming up with this question is the emergency lifethreatening no and now it's going to go down that path so co-pilot Studio here is actually doing two different things it's doing that generative AI stuff plug into your data and just answer those questions but

it's also able to do that conversational orchestration thing so that you have control over the things that you want to have control over how many C's did you find in the video by the way I think there's at least six in there so the concept here Azure AI Studio you're going to use that where you want control over your model customization over your model if you're primarily about building out a complex AI solution whereas if you're in a world of you want to be able to create this custom co-pilot you want to be able to

not have to deal with all of the maintenance of all of those things and you're just ready to go and you want to do this conversational orchestration that's all good as well the thing where we do have the crossover of being able to use both is that you can actually use this generative answers to pull in a model that you've made in as your open AI so if you do need both of these things you want this lovely outof thebox conversational orchestration experience and the complexity of a model you can do that check out my

video here for how you can do that and thank you for watching