Azure AI Engineer Associate Certification (AI-102) – Full Course to PASS the Exam

hey this is Andrew Brown your favorite Cloud instructor from exam Pro bringing you another free Cloud certification course and this time it's the Azure AI engineer associate also known as the ai102 published here for free on free camp and the way we're going to achieve technical certification is by doing the lecture content Hands-On labs in our Azure accounts and as always I provide you a free practice exam so you can go sit that exam get that certification put in your resume or LinkedIn to go get that promotion or new job you've been looking to get

if you love free Cloud certification courses just like this one the best way to support the production of more is to purchase the optional paid study materials that includes things like practice exams flashcards quiz lits um downable cheat sheets and more which you can find over uh at exampro and this specific one is exampro doai h-102 if you do not know who I am I've worked in the industry for I probably 20 years now and I've taught courses on adus Azure gcp kubernetes Oracle terraform Linux and more so you're in really really really good hands

but let's just jump into it [Music] ciao hey this is Andrew Brown and we at the start of Journey asking the most important questions first which is what is the Azure AI engineer associate so this certification also known as the AI 102 is an AI certification as the name implies focusing on two things the managed AI services and working with large language models okay so I just want to warn you that if you're taking a Microsoft Azure exam they are very code and script driven heavy you'll probably see code in your exam so uh developer

knowledge is a must but you're in good hands because my background is a developer and we open as much code as we can to make sure that you are prepared uh consider this certification if you want to uh directly learn how to uh work with open AI large language models now there are other models available in the Azure AI uh studio and we actually do launch them just for fun but they're not on the exam and these are the ones they want you to know if you want to have deep knowledge on implementing aures managed

offerings uh or you want to know how to work with rags and Vector databases or if you're a cloud engineer uh upskilling to take on AI engineering responsibilities probably more focused on gen because the time of this video that's what people are interested in so let's take a look at our Azure road map and so this is kind of the recommended path now this is not all of the Azure search because I cannot fit them all on the screen but these are the ones that are in your uh perview so you should start with the

a900 take that AI 900 and then move on to the AI 102 now the AI 900 covers both the certifications in lighter lighter formats whereas the AI engineer is focusing more on those managed AI Services luckily if you've taken my AI 900 you'll be very well prepared for the Azure AI engineer because I did so much more and Beyond programmatically here basically almost helping you pass the AI engineer at this level but I've gone way more deeper in the AI engineer um with the manage services and a lot on uh gen because that is what

people are really looking for um let's talk about how long it will take to pass this exam well if you're a beginner about 24 hours that means that you've never taken Azure exam before you never worked with llms or a Services you don't have experience with python maybe you have Cloud experience but not programmatically and uh not with Azure exams or LM AI Services probably longer than 24 hours could take you up to 40 hours to be honest but it just really depends on you for the experienced person they have taken the AI 900 they

have worked with python code uh they have basic familiarity with LMS and AI man Services it'll be 12 hours if you took my AI 900 course you're probably looking at six hours because I really helped you out um but anyway you're looking about 14 14 hours average study time 50% lecture in Labs 50% practice exams I recommend spreading the the workload out across two weeks one to two hours a day take your time do not try to rush it in a couple days you'll not remember it um but that's that uh uh what does it

take to pass exam watch the lecture videos do the Hands-On Labs okay uh it's very very important because especially with Azure um documentation does not often match the um the apis and the apis do not match the console um Azure likes to move quick and break things and so if you really want to understand what's going on you have to do those Hands-On Labs even my labs might break even though I've done them four five six seven times the um Azure is weird where it's like you might think you have something perfectly built but it

just decides not to work so um you know just be patient there and don't get stressed uh do paid on online practice exams to simulate the real exam and so you can go over to exam pro. a102 to get those additional practice exams we give you one for free on the platform but you know buy those additional ones it's going to help you on the exam it's going to help me to make more certification courses for free for everybody uh let's take a look at the content outline so there are the following domains we have

plan and manage an a solution Implement content moderation Solutions Implement computer vision Solutions Implement natural language processing Solutions Implement knowledge Mining and document intelligence Solutions Implement generative AI Solutions and remember that each domain has its own waiting this is determined with how many questions of domain notice that Azure has ranges for their freaking sections it's very frustrating you never know how many questions you're going to get on the exam I don't know why they do this I do not like that as a testing practice but that's just how Azure rolls uh where you can take

this exam in an in-person test center or from online from the convenience of your own home Microsoft delivers exams via Pearson view online or the PE view network uh of test centers so there might be a school or private private um place that has a computer lab they might have a test center partner with Pearson view you go there you give them your ID and you use their facilities to the exam if I had to choose I would always do it in a test center because it's so much less stressful and less things will go

wrong these exams are proctored meaning someone is watching you as you do the exam to make sure you do not cheat so no business when you're doing these exams and uh this is not the correct graphic here I guess it's just kind of got filled in wrong but that's okay the passing grade here is 700 out of a th000 points so you need around 70% to pass aure uses scale scoring so just be aware of that um that even if you get 70% you could still fail so aim for 85% and that's what's going to

help you out there are about 40 to 60 Questions on this exam that means you get 12 to 18 questions wrong there are no Penal for wrong questions you're looking at multiple choice multiple answer drag and drop yes and no and more so probably case studies and Azure just had so many different kinds of of question types um so just be prepared for that the duration of the exam is 100 minutes you get one minute per question no a bit more than that I would say like 1.5 minutes something like that so the exam time

is 100 minutes the C time is 130 minutes C time refers to the amount of time you should allocate for the the exam this includes the time to review the instructions Show online Pro uh Show online Proctor your workspace read and accept the NDA complete the exam provide feedback at the end uh this exam this certification is valid for 12 months so that means it has a one year CER uh onee period before you have to recertify that sounds intense except um Azure lets you do renewals for free okay so hopefully that gives you an

idea of what's going on here we are going to look at the exam guide much closer in the next video um and you know I just want to point out that uh the way I I make my content it is I love to show you uh everything that includes mistakes so notice here that there was minor mistakes and I'm correcting them as we go uh this going to be the same thing when we do our lab content uh the reason why I do that is that um especially when working with Azure things never go as

expected I could do the same lab three four five times and every single time I'm going to get a different result so it's very important that I show you the troubleshooting so that you get stuck you can do it too okay but um yeah I'll see you in the next video [Music] ciao hey this is angrew brown and we are taking a look at the exam guide for the ai102 and so what I want to do is go through it and tell you where I stuck true to the exam guide and where I deviated as

not all these services are worth your time but some of them are and there's some additional things that I added that probably will be in future uh exams but are really essential if you want to be an AI engineer so just understand that there going to be some differences I'm going to get my head out of the way so we can look at the PDF in all its Glory or sorry HTML page as we haven't had PDFs in many many years let's scroll on down here and so one thing they're going to note is uh

need skills with python and C now we use Python uh for the majority of this course and we do a really good job with it could you see C examples on the exam yes but if you know how to do in Python you can trans the skill's over so it's not going to be super hard if you do not know c um D below says you should know what rest is and we actually have to use the rest API in some cases where the sdks are in beta or they just don't work as expected um

so we will use rest API and we use sdks and with me we always do as much programmatic stuff as we can because that's going to give you the best chance of being able to do it in the real world but also be able to remember it for the exam so I made sure that we do things programmatically happy quite a bit in this course let's take a look at at a glance so here we have plan and manage Azure AI Solutions Implement content moderation Solutions Implement computer vision Solutions Implement NLP Solutions Implement knowledge mining

document intelligence Solutions and Implement generative AI Solutions so let's take a look at the first one here which is plan and manage an AI solution so here they're just saying just know what all the Azure AI services are avail available okay and I'm just actually going to go pull up just for a moment here portal. azure.com and we're going to go over to Azure AI services and on the left hand side these are all the services that are available now not all these are used um so like down below here these ones have been reworked

some of these just won't appear like personalizer Health insights we're not going to see that immersive reader we're not going to see that but there are ones like AI search computer vision face face face API custom Vision speech search language translator document intelligence also known as form recognizer but they just love renaming things here um so yeah we are going to cover the majority of them um and so you know I don't have a comparative section but we go through all of these services so by the end of the course you should know what to

choose implicitly based on what we've learned um again get my head out of the way here plan create deploy Azure AI services so responsible AI so I did some updates to responsible AI not much has changed in the last few years they have a couple extra documents but again not super hard to know what to do for that um here they're talking about Azure a services so how to deploy it and how would it work with a cicd pipeline implementing with containers so we show you both of those here we have diagnostic logging they talking

about like managing keys and things like that this stuff is pretty straightforward so there's not a whole lot to really say about it um you know so we don't really cover these Part Parts in much but again it's just like most Services that's pretty part of the course I'm not sure why they have that text in there then we have Azure AI content safety now there is another API in point called um content moderation I can't remember if we use it in the course we definitely use content safety and content safety has content moderation in

it but uh we might end up coming both apis but we definitely make sure we fully cover Azure AI content safety we have an images which is under computer vision resource um it has its own SDK so here um we talk about generating image tags Azure AI Vision which is just Azure a AI Vision Studio we programmatically work with it quite a bit then down below we we there's for analyzing videos now we're on to NLP so natural language processing so we have Azure AI language uh so extract key phrases entities sentiment of text language

used in text identify person personable identifiable information we do this quite a bit uh and one thing I want to point out is that we actually end up using in the course to set up our environment um Azure ml uh Studios um uh compute because this was the easiest place to do it so understand that we will be using Azure ml Studio not necessarily for deploying ml pipelines but at least for the environments and so we programmatically work through all these examples very thoroughly then we have Azure a speech these kind kind of kind of

melt into each other because they're very similar services but we do do this I'm not sure we do ssml um yeah I think we I mean I remember doing it but the question did I do in the a 900 and did I carry that content over I'm not sure um but uh that one's not too hard if it is missing then we have the translate language it's a bit confusing because some of these Services can do the same thing or they might be using utilizing the same SDK so I recall us translating but the question

is did we specifically use the translate language resource I'm not sure then we have Implement and manage a language understanding model so this actually used to be called Lewis uh Luis but for some weird reason they decided to um get rid of that service it's still in in here down below here language understanding but now it's under the language service here but the experience is exactly the same okay the the difference is that they've added aure AI search so that you can bring in your knowledge base from there and um you can use open AI

to help you generate things like utterances and so I just want to be very upfront with you that I do not go through and make labs for it again so I have a video that will show you how to get to it I have older videos on how to uh build language understanding projects when it was called Lewis but the thing is it's just a name change and a shuffle and I just can't stand using the service because it is so clunky all right we have we have slides I'm not even sure if we shot

the videos on it but we definitely have slides in the PDF you can download that will just show you screenshots of the updated experience but I'm going to tell you the old lecture content totally fine it will give you the contextual information that you want it's just a name change Lewis to language understanding and it's now found in a different location the same thing with this this is talking about Q&A this used to be called Q&A maker classic now it's just called custom Q&A moved under um the language service and I make it very clear

how to find that in a lab that uh that's before the lecture and then the older lab content okay we have document intelligence um so this I believe is form recognizer uh so they gave it a different name which is down below here uh so this is something that we did before and I believe I expanded on it in uh this course here by adding additional examples for Azure AI document intelligence but you will see that it is just form recognizer renamed we have Azure AI search which um before was called M cognitive search and

it really was just a fulltech search engine but now it it is a vector database store and is very important when we're working with llms so I gave this one extra attention because it is such a useful service not if not if if you don't really like Azure search it's not that important but if you want to work with LMS professionally there's just a lot of stuff that you need to know and you're going to get full knowledge uh with the amount of work that I put into here then we have open AI service this

one is about launching um open AI so if we go over to here I'm just going to quickly show you actually it's here on the left hand side we have Azure open Ai and so you launch this resource this allows you to then deploy large language models um but it does get a bit confusing because there's a thing called Azure open AI studio and then there's Azure AI studio and they look really similar um and they do have some overlap but they are a bit different and I I make sure to spend a good amount

of time making it clear uh the differen the studios and the services down below we have some generative stuff like J using Dolly uh using open AI models all sorts of stuff now something that is not in this exam guide that I definitely gave you because you absolutely should have it is promp flow so promp flow is a way of um uh you could say multimodal or you could say it's a way of orchestrating coordinating multiple actions with your agents and other llms and things like that but I thought it was super super important and

even if it does not appear in your exam today there's no way they're not going to add this in the future because it is such a powerful service I think the reason why it's not in the exam guide right now is that it's in preview but it's so so thoroughly developed uh there's no way they're clawing this service back and so uh you're going to get that uh as an addition and you really should uh study it um even if it's not on your exam okay but uh yeah so there you go um so the

only thing again is I was a little bit lazy with Lewis language understanding and Q&A but you have the resources there from prior uh you'll just have to make that translation yourself okay I wanted to spend a lot of time with the llms and they don't really detail it uh very descriptively here but I I think there was fine tuning in here as well I didn't see it fine tuning yeah they kind of hide these as like on liners that was another thing that I really did a good job uh job was explaining fine tuning

and showing you how to do fine tuning um but it's weird that they just have it as like a single line item even though it's so darn important um but yeah there you go okay [Music] ciao hey this is Andrew Brown I want to get you set up programmatically uh for how we're going to work with specifically AI services so when we're working with AI Services I like to work in jupyter notebooks because I think that has the most Synergy uh for development and so uh technically vs code can run notebooks and so we could

do it locally we could do in GitHub code spaces but where I'm going to do it is I'm going to do it uh in a notebook within Azure machine Learning Studio now Azure AI Studio which right now is still in preview has the ability to launch um notebooks however it's just utilizing Azure machine learning uh notebooks underneath and it actually makes it more confusing using a studio because at this point it's just not uh well-developed service um so I'd rather just save you the pain and we're going to go ahead over to Azure machine learning

um and what we'll do is spin up one here and from there we will then um have our compute there all examples are going to up in the Azure examples repo um I just started this so depending on where you come in uh there could be lots here or little um but at this point this is my starting point here I'm going to go ahead here and create a new work space so we'll go ahead and do that um I'm going to go ahead here just say um AZ or just say AI workspace could be

AI or ml workspace doesn't really matter so I'm just going to call this my AI workspace uh it could matter where we launch this I'm going to launch this in um maybe West us the reason why is that uh when we're dealing with um machine learning Services some types of compute or gpus are only available in particular areas uh I know like when you're doing fine-tuning uh uh at the time of this video they only appear in West us and um Sweden so I'm going to stick with West us it's not normally where I would

put things but that's where we going to put things here today it's going to create a storage account it's going to create a key Vault it's going to create application insights uh which I'm not sure I guess we do need it we can't say no um and container registry which I'm just going to leave blank for networking options we have public private uh private with approved outbound pretty standard uh for um Azure services to provide three levels of network isolation but this UI looks a little bit different than some other places but we'll go ahead

and hit review and create and so we will go and create um this Azure machine learning workspace okay so I'll just pause here we be back in just a moment all right so that was just the review stage we'll go ahead and now create I thought it was creating sometimes I think that it's creating when it's not um but it has to do with that combination there so now it's deploying uh and we will wait for that to uh complete its deployment okay all right so that resource is completed let's go ahead and um launch

this here so I'm going to now go ahead and launch studio now launching Studio doesn't cost anything so right now we aren't really worried about any spend with the exception of storage accounts and anything else that's collected there apparently there's a lot of preview features uh that's cool wow there is a lot of preview features um and uh that is very interesting looks like there's even stuff for Visual Studio code but again I want to keep it simple and I want to work um with an environment that uh is going to be easy so over

here we have environments and we have compute I'm trying to remember which one it is um no these are predefined environments so this is not exactly what I'm looking for but you know I want to just point out that when we start working with Azure AI services or open AI Services things might appear here as well so um just always double check this environment uh because it can get overlooked but I think what we're looking for down below here is compute and so here it says get started with Azure machine learning notebooks and R scripts

by creating your compute instance and so that's what I want to do um is I want to create a compute over here so what is the cost for compute here I do not remember but they do have servus instances which is really interesting if we go over here man servess instances utilized by prompt flow compute session um um right now I'm not super confident with that I'd rather just stick with regular compute instances because that stuff is very clear um but I can to see if I can look up pricing very quickly here so we

understand what we're getting into now we actually might you know what I'm going to do I'm just going to start clicking here because it might just tell us the cost oh there we go okay you know sorry I'm just used to using things like AWS where they do not tell you um so since the last time I use this this is different which is fine but notice we have some Associated cost so we have uh 8 16 cents an hour 28 cents an hour 37 cents an hour so it really depends on what you're doing

so if you are um you know working with something you might have to have something and then spin it down so you'll have to uh be mindful of how long you have this running for but if we're doing AI it's very hard to avoid any kind of spend not at the top we have CPU and GPU if we're using the Azure AI Services API we aren't doing the compute the who's doing the compute is going to be um it will actually be Azure itself right so we might come back here and spin up something that

has computer attached to it but for now all I really want to do is I just want to uh work with CPU for general purpose the lowest possible cost here it looks like this is really low looks like there's also some other options if we can go cheaper I would love to do that because again we're going to just use apis for now there we go and so we wouldn't obviously want to do that uh for anything else but when we're using apis but this is what I'm going to spin up here today we'll go

ahead and hit next um oh so it can shut down over time which is good I'm going to just put in 30 minutes here just in case so that's really nice um do we want any other options here so configure security sets such as SSH virtual Network manage identity for compute really depends on what we're doing right now again I'm just trying to show you that experience of spinning up uh one of these compute instances um but I don't think that we need to do anything here create a computer instance on behalf of another user

well we don't need it on behalf of another user it's going to be us do we need to assign any identities we might want to do that but for now I'm going to go ahead and just hit next um I don't need any startup scripts and here we can add environments so I guess if we create an environment ahead of time we could attach one I don't need any of these so I'm just going to ignore them so go ahead and hit next next we'll leave tags alone and we'll just review what we have so

we've chosen the cheapest possible option um the compute name is called Andrew 2 not the best name I don't like that name I'm going to go ahead and just say AI compute since this resource can shut down it makes me think that uh we can shut it down when we're not using it which is great um we'll just say AI compute uh AI AP uh managed API compute okay and uh that way we know exactly what this is for don't really need single sign on but that is totally fine we'll go next um I'm not

going to add environment because I'm not sure what to do for that right now and we're going to go ahead and hit create and so that is going to um create our compute if you have a larger type of computer with gpus attached then it might take longer to spin up this should not take to too long so I'm going to pause pause here I'll be back here in just a moment okay all right so now our uh compute resource is running and we have a few options the vs code web is um the newer

option um but I'm going to stick with Jupiter lab uh I'm going to assume Jupiter is just Jupiter notebook but Jupiter lab is the more modern version of Jupiter notebook so we'll go ahead and open this up here again this whole video is just to um spin up a resource so here it says jupyter lab is suggested and so it's just is telling us what is uh being included in here uh and so what we're going to need is we're going to need some type of Kernel um and so usually kernel appears up here but

I don't see it so I'm trying to remember how we specify our kernel kernel usage is missing H so it's possible that we should have uh included one initially here but we do have some notebooks uh like azure uh ML and things like that um so let's try 3.10 SDK version 2 and so I'm hoping if if I do this it's going to add a kernel for us let just say print hello world and we'll run this here and it runs it so yeah now we're getting kernel usage which is great um and so that's

that's what we wanted here okay so now that we have that we are in good shape and um I mean what we should really do is just make sure that we have API access um the way we can do that is just by um testing some kind of API so just give me a moment to think of some kind of like test case that we can do here okay all right so I think what I want to do is I want to go ahead and again use some kind of AI service again we'll probably do

this at some other point but I just want to have some kind of use case that we can test here um I think that working with text is always really easy so a translator is not a bad idea so I'm going to go ahead and just create a translator very quickly here there are many ways that you can do this one is making Azure AI services and utilizing it that way but for now I'm just going to go ahead and use the translator separately here and I'm going to place this into my AI workloads managed

workloads I going keep this around because I might end up creating a bunch of these I'm going to stick with the same region that the other one's in so this one's going to be West us and do we have see West can I type that no West us there it is I was getting confused because we had these ones at the top here but I'm going to stick with West us this is going to be my um translator translator test and all I really want to do here is I just just want to make sure

well I might put some numbers here on the end we just want to make sure that uh this works before we move on and actually do real things because there could be permissions issues and I just want to get through that here and just make sure that it works but since we're using the API that shouldn't be an issue um unless we need some other thing that we need to download and actually I do believe there is something we need to download so originally when I did this um this was the exam Pro C free

a AI 900 this content is now for both the AI 900 and ai102 but um for any of these I think pretty much the first step was setting up um some kind of install like this right so I just want to make sure that works before we do it um so I'm going to go back over to here and see if this is ready says it is we'll go over to here sometimes they will provide instructions on how to implement this if they don't it's not a big deal we can uh try stuff here but

I want to go over sample code no that's not what I want let's go over to python um and I suppose this is one way that we could do it but this is not using any kind of SDK okay so not exactly how I would want to implement it I want to make sure that we can um uh do this probably the way you would implement it here so we do have Cog cognitive services and so I don't believe that this name is changed because it'd be very hard for you us to change that sorry

Azure I'm going to go over here and just type in Azure cognitive services and we're going to type in maybe GitHub and so there is a joined SDK is there a separate one for the translator could be this one here no that's just an example so we have cognitive Services speech SDK cognitive Services spe s DEC so this is not what I'm looking for I'm looking for uh the translator right and this one's Azure samples which is fine but something that we might want to take a look here is let's just take a look at

how they're loading this in here over to Python and wow they are just not providing the same information as they used to okay so give me a second I go look and see if I can find an SDK for this so I can see that there is an Azure text translation SDK I'm actually surprised that it's in preview that might explain when we went over to here it's showing us raw python code um so you know again not exactly how I would like to do this but if it works I guess we'll go ahead and

do that um so because this one's in preview and I don't really want to show you something that might not necessarily um be completed you know what we're going to do it I think I think they're probably going to end up using it so we'll go ahead over here I'm just going to say dismiss here we're going to paste this in here and what I want to find out is if we're going to run into any issues that's what I want to find out can we programmatically work with an API I'm not sure why this

is not copy and pasting copy there we go right click paste edit paste what is going on paste paste paste hello copy paste paste paste yes right click copy paste there we go um and generally when you install stuff you're going to put a percentage in the front here that tells the notebook uh to use a system command which is PIP so let's go ahead and see if that works so see how there's a little Aster that means it's thinking it's downloading and it is working so we're able to install this again just trying to

make sure that we have everything that we need for this particular kernel which is working with apis um and so here it says install it and then down below we need to include a couple of imports so here we have credentials and text translations so we'll go over here and give this a go and we can run that um says invalid syntax well I'm not sure how that could be invalid when we literally copied it straight from here so let's go back over to here and by the way we can name these notebooks so this

one could be named something as um translator so we have from Azure core credentials import text translation client from Azure AI translation text import text translation client huh okay so maybe they've changed the name of it and this is the part where it's kind of risky utilizing um uh preview stuff right because if it's in preview they might change the naming on us and that's probably what's happened here so what I'm going to do is go ahead and go to GitHub here when we have this kind of issue what we can do is just take

a look and see if anything has been changed so it says Azure SDK for python text translation so I guess they're all kind of grouped here at um together and they're not really telling us what the import is here so clearly there is the credential here right and this kind of looks useful so I'm going to go ahead and bring this on over I wasn't trying to make this super hard but apparently uh it's not as straightforward for the translator so it's just my poor choice of choosing that service and so we'll just put that

up there we can change this over to be markdown so that way when we click uh if we click that we can um resources here up the top we can click through that and get back to that very quickly so what I'm looking for here is the import statements and they don't show but they do have sample so let's go ahead and take a look here and what's interesting is like obviously there's the docs but you can't always trust um Azure docs um sometimes the code examples are better or you just have to experience it

for yourself so we have sample text translation languages so that sounds really good because I just want something really simple and here they're just saying import sample text translation client so it seems like they've abstracted out the code and here it says Azure AI translation. text so if we go back over to this one so it's Azure AI translation text and so it looks like maybe the syntax syntax is supposed to be uh this period syntax so let's go ahead and grab that I have a very strong background in programming so for me this is

um uh not too hard right but for you if I wasn't here I'm not sure I'm not sure how you'd handle it okay so we'll go ahead and paste it in here and I'm going to just fix the ination I don't think the order matters but I'd rather have the core on top here let's see if we run this is this going to fix our problem run that so now we don't have a problem so yeah the documentation is not 100% correct so here they are showing I mean well this one has the period syntax

but this one doesn't but then this one down below here is correct so I think maybe what happened is that they just uh possibly made a mistake maybe that's our issue here okay so if that's the case I'd rather rely on maybe U this one over here and so here they're saying uh once you do that get an API key because you're going to need one then create the client um using the API key here so I'm just going to copy some of this in here and we'll need to set ourselves an API key those

to say API equals this and there's probably like a safer way to pass environment variables I'm not very good at this I there's like a way to set n bars but I always fail so I just paste them in here um and I do apologize for that just be careful when saving your notebooks and that you do not expose that information we'll try our best to try to pass it through but um we might have to just hardcode them in here so the idea is that we get back our credential object we are passing credentials

here region to region we aren't specifying the region but everything is uh Us West so um it'd be nice if we had an example of what that looks like because sometimes they want a title Cas sometimes they don't but anyway so that's the text translator we go down below here this is just telling us what supported languages there are I just want to translate so it looks like the way we translate is we are going to have this here actually I'm just going to grab this whole code block because it looks pretty good to me

and I'm going to bring this down here and paste it in um I don't like that it's for indented I guess this is defaulted to four but um normally when you're working with something like python two space indentation is what you should utilize but this might be set to uh whatever that is and I'm not exactly sure where to change that maybe we type in tabs here or [Music] spacing so yeah I'm not 100% sure where that is but I'll have to just live with for for uh for uh space indent it's not a big

deal but over here we have our Target languages so we have CS es uh de so I'm not sure what CS is that is Spanish that is I believe German so here I have this is a test I'm actually going to switch this out and make this JP for Japanese assuming that's what it is so the idea is that it will take in this uh text so our input text elements just going to write our um in input text I just don't like the naming of it and we'll just say Langs and this will be

LS and by the way this is too small another thing I can do is try to bump up the font I know what I originally recorded this people were saying hey Andrew this stuff was too small so I'm going to go ahead and increase um the tech editor font size if it would increase here and this is probably the reason why I didn't do that because it just doesn't work so I'll just bump it up manually like this there we go and hopefully that is a lot more readable for you I do apologize and so

we'll scroll on down here um and so we have this in here I'm just carefully looking here so this looks pretty good so all we're missing here is the API key so I'm going to go um back over here and I now need that API key so on the left hand side I would assume it's going to be under something like called apis so let me just carefully look for that here keys and end points is probably that so we'll click into this and so we have ourself some keys um so I'm going to go

ahead oh so it's says West us so that's probably what we want so I said earlier I'm not sure what we need to set for the um region so I'm going to go ahead and type in region here and do this and then we'll need our API key here so I'm going to go back over to here I'm going to look at the first key just copy it of course do not share your keys do not save this stuff um into repos be very very careful about that my repo is public so I really don't

want to do that and so the key is here I just want to make sure that it works and then we'll we'll test and see if we can get the environment variable to work okay so what I want to do here is I want to um run this so this is where we are now this could have been uh one step lower here not sure why I did it in this order but these both Ran So it's still the order that we wanted to be and let's see if this works no problem so far and

this is um the big question here is is this going to work so run this and it's taking its time using 80% CPU which is a lot for not doing much and so here it's saying missing one required positional argument content um and so this could just be a change with the API so instead of body it might want content instead it doesn't tell us underneath what it's utilizing here so let's go back over to here and let's go back to overview and look at the code sample because that's going to give us an indicator

of the parameters it wants CU here it says body right um but then down below here we have content so we get the request body and we are turning into application Json we're turning into a jifi string and then we're passing on his content so the question is would this require this to be Json I don't think so so I'm going to go back over to our environment and what I'm going to do is I'm going to take a guess sometimes you got to do this and going change this over to content and we'll see

what happens here and it's saying keyword argument only two missing required keyword so does that mean that this is now two like this we try this so here the requests is not authorized because the credentials are missing or invalid so we're getting closer right again I just guessed at what those inputs were based on my programmatic knowledge um but it is a little bit hard to uh figure out from here and also from the documentation it's not super clear but we can scroll on down and take a look and see what they're doing and they're

just pointing to the sample repository so they don't even really show you um how to do it they kind of leave the whole thing out which is interesting but now that we've gone down this route I really do want to solve it um and so my major concern is like do we have a permissions issue what is wrong here um so just give me a second okay you know so one thought I had here looking at the keys is that we have the web API and containers right so it's possible that these only work with

these two methods I would expect that um what the SDK is using is the web API or the same underlying API however when we go over to here you're seeing that it's using cognitive Services um to list out uh keys so it makes me think that we need to use cognitive services to generate out a key cognitive Services have been renamed to Azure AI Services um and so what I'm going to do is I'm just going to back out here for a second we're going to go to Azure AI Services which is actually cognitive services

and notice that we have this one with this logo and this one with that logo I believe it's the same thing I'm going to double check and make sure they're the same Azure AI services that is the original logo yeah and so this is what was originally here it says Cog services and apply services are now aurei services so this is the old way of doing it um and then when you go over to here where you're looking at this is the actual Azure AI Studios icon but they've just moved everything over here for whatever

reason and you can actually see that I have an Azure AI Services hub from uh from earlier um but the thing is is that we want to have an API key and so we have this is that going to create me a hub if I go over here it is get access to open AI speech Vision translator Azure services with a single API key quickly connect to services together to achieve more insight so so this is one way to do it but the question is like why can't I just use the standard standard key so

again I have a lack of confidence here because I think this is actually going to spin up Azure AI studio so if we type in Azure AI Studio I'm just going to make sure if that's separate here no okay so this is azure a services and Azure a Services is just a promatic way to work with a bunch of these apis with a single API and when you spin up azri Studio it's going to do that as well just create this indirectly but I would really like to just with this specific API so just give

me a moment I'm going to play around okay all right so I haven't really found a solution yet but I'm going back to this example here because maybe if we can uh utilize this one and then we can rule out that you know our key is fine but maybe it's our code so if we go here it says insert my keys and region into the sample code meaning that it should update down below here I'm going to go ahead and copy the sample code I'm going to assume that on copy it's going to show me

that information information right and we'll go back over to here and I'm just going to make a new one a new notebook here this will be um translator yeah we'll stick with 3.8 Azure ml which is fine um and we'll rename this as I don't even think we needed Azure ml we could have just done that with uh we've just done that with um the standard one here but we'll say trans lator simple okay we'll go ahead and just paste that contents in here and so if we scroll on up it didn't uh it didn't

do the thing that it said it was going to do so I'm going to drag this one up here and I guess I'll just have to manually get these not sure why it's not uh working but that's fine we'll go over to here to Keys we'll copy this and we'll paste this in here so we have those two parts ahead I know this is like a big detour but you know sometimes this is just what you have to do to get things to work so we going to run this and then this and then it

says your location this is um West us so that is fine I'm going to go ahead and just bring some of these things up here like the location here right and we'll do that um I'm just take these two lines out we don't need that and so the rest should just work let's find out if that works from English to um um I'm not sure why that's a [Music] problem but I suppose what we could do is just put it on one line and maybe that will resolve the issue my primary language is Ruby not

python but I know python good enough when I need to figure it out so we'll go ahead and try that and maybe it's just expecting commas here it really makes me think that um Azure does not test their products so we are getting translations back so this is working so we have ruled out here um that it's not our uh API key it's it's something that we're doing over here now notice that this has from and to so I'm wondering if that's also something else we have to specify here it's also specifying API version here

it has the content type which is application Json it has body with text in the interior here and then passing along like that so here it's not very clear what it wants and so we obviously know this is in preview so I think that that's okay if we can't solve it but you know I you need to know this is what it's like working with Azure products the sdks um if you do not have a programmatic background you can find yourself being very frustrated even if you do have a pratic background you can have yourself

very frustrated and I just want to show you I'm going to go over to Twitter for two seconds because I want to show you before I started doing these uh these recordings here I had to say my positive affirmations which is I will record Azure AI Labs a day I will success successfully Rec uh make the labs I'll be PID patient and I will not doubt myself remember sometimes Azure just doesn't work even if you know what you're doing so do not get stressed um and understand that it's it's challenging for me as an expert

in Azure um uh to to work with this stuff okay but uh yeah so the last thing I'd like to do here is I just want to see if we could load in uh this environment variable um the the API the environment variable because we really don't want to embed things this way okay and so I'm going to do is I'm going to uh try to do that um I know it's like OS and Bron so how to load environment variables python Jupiter Jupiter notebook because there's a specific way to do this um and I

just need an example and [Music] so I'm thinking here that uh the challenge is if we get into the environment so the idea here is that we can open up a terminal say new terminal and then this terminal should be connected to our kernel but that's not necessarily true because if we go over here we could end up with uh multipl here so there's a way that we could open this I just want to make sure like we have a guarantee that we're opening this environment in the terminal and so that's what not 100% sure

about just give me two seconds you know what I'm not 100% certain but I'm just going to try and we'll do a test here and see what happens sometimes that's the best way to find out so I'm going to import um the OS like this you know I I know this seems a bit excessive but I'm going to make another notebook here we'll just call this EnV test here so we'll say I just need um SDK version 2 even though we're not doing anything with the SDK and I'm going to go ahead and just rename

this and we're going to call this EnV test and so we're going to import our OS we're going to grab this value here we'll go back over to here add that there we'll say my test uh bar all right and so what I want to do is go over to terminal here again there's is there some way we can open this in a terminal hm no okay so we'll go over here and we'll bring a terminal open and so what I want to know is if I set an environment variable here and this one's called

my test VAR and it's actually set we don't need to or we could do export like this export and then we'll do an equals it we'll say hello world okay so the idea is that we have that now if we type in EnV my test bar didn't do exactly what I wanted but that's okay we can do Echo dollar sign my test VAR there we go so that is echoing out there notice it says Azure ml python 38 so that makes me think that the kernel that it's using is not necessarily um this environment test

one but we'll find out here in just a moment so I'm going to run this I'm going run that and notice that it's saying does not know what this is my test far my test far so I would really like this to carry through because that way we don't have to worry about accidentally putting this in our stuff and committing it so what I'm going to do is going to try to restart this kernel restart the kernel and see if that makes any difference we can also clear out our inputs here so just uh clear

all inputs here restart y I'll try this again then this doesn't know what it is so another thing we can do is we can change the kernel so I'm going to change this and let's see if we can make it the same as this one here right and so what I'm hoping for by changing this kernel that maybe we've set it here and it will know about it because it's telling us Azure ml python 38 so do this we do that and that doesn't work so I would really like to know how to get this

to work so just give me a moment I'm going to do some research okay so I didn't find an exact solution here but um another thing we could do is we could try to just run we can try to set the environment variable using that magic that magic command with the percentage um there's also suggestion there's a kernel Json file I don't know where that is I don't want to dig into that so let's try this solution because this might work okay um so we have where it says percentage environment or set environment and so

we could do this so I'm going to go ahead and grab this here I'm going to go back over to here and I'm going to go ahead and just say um my test VAR test testing and I'm going to grab this I'm going to make this above here on this line here does it need an equals in between it I mean that one has it for sure I mean this one has it as well where it's environment whatever whatever or with the equals I'm more comfortable with the equals um I suppose you could do it

either way but I'm going to go ahead and I'm say uh hello world and we're going to run this so now I want to see if I run this will that work oh it didn't complain so we'll go ahead here just say print API key which is not what we're actually doing here but that's fine it's just a test far and so here we can see our um environment variable now here's the question if I reset this kernel restart is it going to uh retain that so we'll go ahead and do this okay and that's

fine so we'll go here I'm going to skip this line I go here it doesn't know what it is so just understand if you reset the kernel you're going to have to set that again um and so that's pretty much the best solution I have here uh so what we'll do I'm going to go ahead and just um go back over here so it's really up to you like whether you want just remember to clear this out or to go through all that process to have a hassle there but I'm just going to clear this

and I'm going to bring this over here because this is our failed attempt but I still think the code is worth hanging hanging around so um this attempt failed because the API is in preview and we could not find a working example um uh we were getting authentication issues uh despite using the correct API key so instead uh with translator it's recommended to use the web API um code example okay so that's our thing there um I really want to have this in my repo and so probably the last thing I want to show you

is just like how you can bring in um a uh like a environment or workspace to work in um but I need to bring these into um into here right so I'm in GitHub and if I press period on my keyboard what that's going to do is open up no compute behind it so I don't have to worry about any costs and the idea is that I can just work with files here and so I'm going to go ahead and just download these two files I say download and download I might just rename this one

before I do that so renamed failed and we'll download that one as well and so I'm going to go back over to here and then I want to bring them on in okay so we're going to go to our uh downloads here and I'm just going to go and make a new folder here and we'll call this translator and then I'm just dragging in these ipn PS now one thing we really have to watch out for is um this part right now I'm going to end up tearing down the service so it's not a big

deal for me but I don't want uh this to be here so I'm just going to um so replace me noce it's printing it down below there so that's not good either so I just want to whoops take this out of here uh and I want to clear out all the inputs so I'm not sure we can do a terminal now no oh yeah that's what I thought you can create a terminal Contin environment okay that's fine so I just want to clear out these inputs I'm not exactly sure how to do that um clear

inputs from I'm not sure I'm not sure so what I'm going to do I'm just going to delete these and I'm going to delete my local files that I just downloaded and I'm going to go back over to here and I'm just going to clear um all outputs okay so we'll clear that one and I just want to take this out of here okay I don't want this in here I don't want to print out the key I'll go back over to here and and and I want to make sure that's cleared I'll just clear

this one out as well clear output restart there we go good I'm going to go ahead and download both of these again and all I really want to use this for is to facilitate copying this over so it copied it downloaded one of them but not the uh allow multiples that's fine and so now I have both of them downloaded it's off screen just so you know and so I'm going to select those two here I'm going to drag them on into here I lost my folder so I'll make the folder again translator and then

we will bring these whoops we'll bring these into here as such move and I'm just going to go commit these so just say um Azure API translator and I mean I suppose we did that translation I never really paid attention to what the output was so I'm actually just kind of curious so I'm G to go back here and just undo this so I was just curious did it actually work I think it did right so just say run run run and we do have it yeah so we have these two here um maybe this

is not the Japanese oh we don't have it in that's why I wanted to see Japanese I'm just going to put it back in here and that is not a valid target language maybe it can't do Japanese I don't know I thought it could do that well that's totally okay um but anyway we have that there so this is now committed so now the next question is like imagine I want to work with that um uh that there so normally on the left hand side there's a way that you could bring in um our you

could load in a repository but I'm not seeing it here h so yeah I'm not sure but normally you'd have some kind of Version Control here on the left hand side and I'm thinking that this one's so simple that it just doesn't have it and that's totally fine like if all we want to do is um bring them in manually we can do that but what says over here Jupiter lab development team is excited to have robust third party extensions let's go ahead and enable that maybe that it's an extension that that we need to

enable I'm not sure if we're even to enable these here I'm clicking it oh I had to be patient maybe so I'm just going to click that I'm just curious if we can get uh Source control in here if we can't doesn't really matter but we'll find out here in just a moment uh as I clicked it a lot okay all right well I mean I enabled this but it never really uh reappeared that's totally fine so we'll just say that we achieved what we wanted to achieve here I'm going to shut down all uh

all kernels I'm going to shut down all terminals um and so we are done here for now just to keep this a bit organized I'm going to make a new folder I'm going to call this one translator um and the thing is I'm just going to keep um bringing files over into here and so I'll just keep backing them up into uh into the um this repo here the Azure examples repo on exampro github.com and so we consider this done I just want to make sure that we go stop this Works place so I'm going

to go ahead and say shut down and so we'll shut down this Jupiter lab now that doesn't necessarily mean that it's shutting down this but we'll just double check and make sure we'll refresh this is it shutting down no so I'm going to go ahead and click this I'm going to say stop and we're going to stop that and so hopefully next time I spin it up it works I might just start this up one more time just to make sure that it still works I'm going to stop it here we'll wait um and then

I'm going to start it up and again just make sure that it comes back to its original state okay all right so that is now stopped I'm going to go ahead and spin this back up here so we'll start it up again um oh I I tried clicking here but we have to hit start up here and so we'll wait for that to start up and I I just want to make sure that um I can open up that jupyter notebook again or Jupiter lab and then we'll just stop it one more time and then

we'll call this video quits all right so that's running again I'm going to go ahead and open up this in Jupiter lab I just see what do we have do we still have our files do we have anything so we'll just go ahead and say your build that's totally fine so yeah our stuff is still here so we are in good shape um I just might make a another subfolder here and call this Ai Ai and drag this on over to here just so it's a little bit more organized and so we can call this

done um I don't think we have to stop the workspace from here uh like in Jupiter Labs we can just go ahead and shut it down here or stop it sorry the idea is that we now have this provision we have a working environment and this is what we're going to use for managed uh API compute if we need um any kind of specialized ones we'll do that separately we'll consider this done and I'll see you in the next one okay [Music] ciao hey this is Angie Brown and this follow along we're going to take

a look at Azure uh AI Studio or Azure open AI Studio they're very similar uh but they're also uh different and so I want to make that clear distinction um so far most of the videos that we've been doing programmatically we just spin up Azure services or in some cases open AI services and we haven't really gone into that studio but the studio is something that we should really uh know um and so I I want to point out that we do still have our um AI Services we made from a previous video I just

go over here and so we can technically enter the studio uh through here or if we created open thei we can go here and we can get some of the functionalities of the Azure AI Studio I recommend that if you're going to work with the studio you go and create uh a studio resource first um because that's going to create all the stuff that we need um to scope our projects and things like that but I'm going to do it both for open Ai and also Azure AI Studio U so that we can distinctly distinguish

or understand the difference between those two because they look very very similar and they are very similar um but you know how Azure is they like to do things 10 different ways and uh it's always hard to know which is the most modern or best way so uh let's first type in Azure AI Studio or sorry open AI studio and see if we get that so we have um Azure open AI which is fine and yeah I guess that's the only way we're going to be getting into opening I studio is by creating open AI

resource so I'm going to go ahead and do that right now so I'm going to create this one just say open AI studio and we'll just choose East us which is totally fine actually I'm going to go west because some models are only available in West and I don't want to have a hard time uh if if that's the case we'll say um open AI Studio here and then the price and tier is going to be standard I'm going to go and oh doesn't like the name so I'll put some numbers here on the end

and we'll go ahead and hit next next next and we'll give it a moment here and hit create so that one is creating and now what I want to do is go over to um Azure open AI Studio okay so we type in AI studio so notice it pops up as Azure AI studio so slight difference uh we're creating the open AI resource and then we'll launch the studio and for this one it's showing that it has um the Azure AI service but if you create from here it's not just going to create just this

resource it's going to also create um Azure AI Hub so I'm going to go ahead and do that I'm going to create a new group here I'm going to call this a um AI Hub right and this will be for the studio I'll just put studio in there as well so we know what this is for I guess we don't need random numbers I'm just thinking that this is the name of the resource I'm going to stick to West because I know that um there are some again some models that we cannot use in other

places we ran into an issue where there was a model we wanted to use that wasn't available in West so that was very interesting but I'm going to go ahead here and just type in um as aiub and put some numbers here so it doesn't complain um and that looks fine so no it says connect AI Services included in U open Ai and here it will actually just make a new one for us so when we actually do this it's going to create both AI services and open AI uh so we're going to have some

redundancies there and it's not a big deal uh just depends on what we spin up on the interior of those things but they're not going to cost anything for us to spin those up as such and so we'll wait for that to spin up here okay well and that was just the review stage so we'll go ahead and hit create they get me all the time like that but we'll wait for um that one to be ready okay all right so both are spun up and so remember we have um both open AI spawn up

and Azure open AI Studio but the Hub so if we click to this resource here we now just have to wait a moment and notice it says Azure AI Hub we'll go ahead and open that up there and then over here we'll go to the resource for open Ai and from here we can open up open AI Studio okay and so why do I say that I recommend that you create the hub first is that it just adds a a layer of um organization so you can go into let go back here for a moment

we can go over to um AI Services right here and we can click into here and we can open this in the scope of azure AI Studio but it's not going to be in a hub it's also not going to have um open AI connected to it so sometimes in Azure there is an easier way to do things where it's going to set up a bunch of stuff some people like to set them all up individually and connect them um but yeah many times that doesn't work so we're just going to do it the way

that I know that will absolutely work for you and you're going to notice that these look very similar this is open AI Studio this is azure AI Studio Azure AI studio is newer technically it's in preview right now the idea is that it's supposed to replace Open aai studio or they're both going to be round in some in some regards but the key difference is the model catalog so if you go to the model C well that's one of the key differences if you go to model catalog for open AI the only stuff that you're

going to see see is the um models that are from open AI company if you go to Azure studio and you go to the model catalog you're going to have a much larger selection of um models here okay so we have a lot more going on here now I just want to point out that if you want to actually use these models especially in Azure AI uh Studio you may have to request access so it's not showing me that here right now but um when I first used it I had to to request access um

and so it would say request access here you'd fill out a form and it would take 48 Hours it took me more than 48 hours um and they didn't tell me when they were available they came back and I was able to use it if you do not have access to those you're not going to be able to do much at all so make sure that you do that and you wait um for open AI I think when I used it they were readily available but again you know if you have that issue you might

have to uh do that um so yeah so they're pretty similar I think that Azure AI Studio has more but um you know we're going to do as much as we can in the opening eye studio and if we have to go over here we can do that but yeah just flipping between the two you can see you know there are um some additional things that appear not to be there like prompt catalog and all prompt catalog catalog is is a way of collecting your your prompts so uh it's just a place to save them

so if I click into here it just shows an example of a save prompt that we could use again okay so um yeah hopefully that is clear and so that is the initial setup and then we will go in and start us utilizing this stuff [Music] okay all right let's compare AI versus generative AI first starting with AI so what is AI it is computer systems that perform tasks typically requiring human intelligence this includes problem solving decision making understanding natural language recogniz ing speech and images um and the ai's goal is to interpret analyze and

respond to human actions to simulate human intelligence in machines and I create a large emphasis on the word simulate because it's not emulating so simulate is where we are mimicking aspects resembling the behaviors of humans or other things emulation is when we're actually replicating exact processes in machines it's the real virtualization of the the human mind and you could say that's pretty much AGI so the artificial general intelligence but um the point is is that AI is a simulation not an emulation AI applications are vast includes areas such as expert systems natural language processing speech

recognition Robotics and more um for Industries it's across the board if you're were talking about B Toc everyone's probably experienced a customer service chatot I hate them but that is uh probably the number one use for uh gen or AI uh we have e-commerce so recommendation systems so think like using Amazon and maybe it's using AI there and you're not aware of it automous driving Vehicles U Medical Diagnostic so lots of verticals there for different Industries let's talk about generative AI so generative AI is a subset of AI focusing on creating new content or data

that is novel and realistic it can interpret or analyze data but also generates new data itself it often involves Advanced machine learning techniques so we got our Gans our vaes our gpts like the transformer models and we're going to be talking about Transformers not uh Autobots and Decepticons but actual uh architecture for llms I really should throw some uh some of those if they weren't copyright I would have them in there uh generi has multiple modalities and if you've never heard the word modalities think of it like senses you know you have Vision you have

you have taste things like that so for J we have things for vision for text for audio right uh and there are some odd ones like molecular so something that generative ey can do is it can do a a drug Discovery via genomic data hopefully I'm saying that correctly um and a lot of people associate generative AI with large language models which generate out human like text and is a subset of of gen but it's often conflated with being AI due to it being the most popular and developed and it has strong correlation to the

text modality because they usually do that but um uh large language of models can be multimodal uh meaning that they can work across multiple modalities but it's mostly text so let's just sum this up and make sure we know the distinction between Ai and gen so AI is focused on understanding decision- making gen is about creating new and original outputs that doesn't mean that J can't do the the former but it has that added benefit of generation for data handling it AI analy analyzes and make decisions based on existing data J uses existing data to

generate new data and unseen outputs for applications um AI is generally more applicable because you know it just is whereas gen is very focused on Creative uh uh Innovative generation of synthetic stuff um and that M threw me off there it's not supposed to be there but I think you understand the distinction between the two and there you [Music] go hey this is angrew brown and we're taking a look at what is a foundational model so a foundational model is a general purpose model that is trained on vast amounts of data we say that a

foundation model is pre-trained because it can be fine-tuned for specific tasks so just remember we are training the model and we're going to say that it's a pre-trained model but imagine text images video structure data all sorts of data massive massive amounts of data that is going to produce your foundational model and from that foundational model it could do all sorts of things prediction classification text generation it might be limited to very specific things that it can do but the point is that it can do a lot of things um and the reason I want

to bring up foundational model is because you hear it a lot when we're talking about llms and it becomes a bit confusing how to distinguish it between llms and a foundational model but llms are a specialized subset of foundational models they are foundational models that uses Transformer architecture so if you remember that that llms are foundational models but they're specifically using the Transformer architecture then that will help make a whole lot of sense okay [Music] let's talk about large language models this is going to be short even though there's a lot you can say about

it I just want you to uh remember a key thing about large language models so a large language model is a foundational model that implements the Transformer architecture and we're going to spend a bit of time learning about the Transformer architecture in upcoming videos but uh the idea is that um you have natural language I can get my pen tool out here so we have natural language as our input it goes to large language model it predicts uh output for words and as it produces each word it feeds it back in and continues um to

produce until it is done so during the training phase the model learns semantics or patterns of language such as grammar word usage sentence structure style and tone that's what makes it so good at at uh interpreting uh uh language and giving things that sound with uh language understanding because it has that ability to um understand the semantics of language it would be simple to say that llms just predicts predict the next sequence of words because as you use the model it outputs a word on the end of it and keeps feeding in and in and

in and in until it's done but the honest truth is researchers do not know how LMS generate their outputs because there are so many layers um and there's so much going on there that at this point right now the level of complexity makes it very diff ult to truly understand how it is reasoning its output um but it looks like it's just doing word for word but there is a bit more to it okay but there you [Music] go the Transformer architecture was developed by researchers at Google that is effective at natural language processing due

to multi-head attention and positional encoding and here is that architecture it comes from that white paper attention is all you need because that is the special mechanism that it is utilizing to pull off the Feats that it is doing um I try to remember what came before it it was like cnns and RNN so convolutional neural networks and uh recurrence neural networks and recurrence neural networks could kind of do what Transformers do um but they just had an issue with scaling and being able to remember everything that they were looking at and so this architecture

found a way to do that and that was with positional encoding and multi head attention how important is it to know this architecture um it's good it's nice to know so you get a bit of an experience in terms of what's going on there but to be honest working with um llms constantly it's just like you kind of forget about this and so it doesn't really inform any of your workflows or decisions I guess it's just more like by looking at this uh you have more confidence reading white papers right and and looking at some

of the stuff of these architectures so that's why we're looking at it but Transformer architectures are made up of two components or two parts we have an encoder and that's you get my pen tool out here so it's very clear what we're looking at but it's this thing here right so that is uh our encoder I'm just going to erase that there and so you can get the idea that the one on the right is going to be our decoder let's read about what the encoder is so reads and understands the input text it's like

a smart system that goes through everything it's been taught and picks up on the meanings of words and how they're used in different context so that's the high level and then the decoder based on what the encoder has learned this part generates New pieces of text it's like a skilled writer that makes up sentences that flow well and make sense and uh as far as I understand that once you're uh you put your data in here it comes through here right uh and it has to be already embedded and then once it goes through here

it's it's going to Output uh that um that stuff and it's going to go into here and then each word as it iterates through it's going to go through here each word is going to go here it's going to produce your sentence with the next word and it's going to go all the way down here and then add the next word and then feed all of this back in and again and again and again so it's just going to keep looping until it runs out of um ability to write or it decides to stop um

and there are very specific components that we're going to look at the multi head attention in the positional coding so we didn't really describe them here but there they are and you'll see them up close here in just a moment [Music] okay so tokenization is the process of breaking data input and in most cases text into smaller parts so here on the right hand side imagine you have a string and you're going to break it up into its parts uh which we represent as an array here and then we're going to give it a unique

ID to the models vocabulary so when we're working with llms you have to tokenize inputs and depending on what llm you're using it's going to use a different tokenization algorithm so for example if you're using GP uh gp3 you'd be using bite pair encoding if you're using Bert you'd be using word piece if you're using Google T5 or or or GPT 3.5 you use sentences piece you won't really notice this when working with llms especially if you are uh utilizing something like Ama or manage service because um these apis are taking care of this um

algorithm for you so you just input it and it works there but when you're working with um llms the input text must be converted or tokenized into sequence of tokens that match the model's internal vocabulary what are we talking about when we say internal vocabulary well when an llm is trained it's creating an internal vocabulary of tokens of all the stuff that it knows right because if you consume uh the the world's knowledge uh you want to take all that knowledge that text break it down into all its unique components tokens and then assign a

value to it and so these large models could have between 30 to 100,000 tokens it could even be more than this or less depending on your model but tokenization is very important so that it understands what's going on here there are some things that we could talk about like what happens when it uh uh encounters a token it doesn't know but for the most part um this is tokenization that you need to know [Music] okay let's talk about tokens and capacity because it really matters um about how much you can produce so when using Transformers

the decoder continuously feeds the sequence of tokens back in as the output to help predict the next word in the input so what are we talking about here so here imagine we have our input as the quick and so we feed into the encoder the encoder is going to produce um semantic context so that the decoder knows what to do with that text and then the decoder is going to Output the next word so this is the quick brown and what it does is it feeds that sequence of tokens back into the decoder and produces

the next word and again and again and so the question is is what is the capacity required to run this and so there are two components that we care about memory and compute so for memory each token in a sequence requires memory so as the token count increases the memory increases the memory usage eventually becomes exhausted and you cannot produce anymore okay so now for compute models uh a model performs more operations for each additional token the longer the sequence uh is is then the more compute is required so a lot of AI services that

offer models of service will often have a limit a combined input and output because it really has to do with the uh the length of of the sequence so if you have a huge input then you're not going to be able to generate a lot of words because you're going to hit that uh sequence token limit um a lot quicker so hopefully that makes it very clear about how memory and compute are uh inter time with tokens um the way cost gets down is you know they have to figure out a way of um reducing

or making the model more efficient so that's helping to reduce the memory compute um there's other things you can do so if you have a conversation it gets too long what you can do is summarize the conversation um and feed it back into there so it it doesn't exactly use all of the context of what it had before but it can do something similar and help that conversation along [Music] okay so what are embeddings well before we can answer that we need to answer what is a vector so a vector is an arrow with a

length and Direction um that is the simplest explanation if you're talking to a mathematician they're going to have a more fancier explanation but the reason why this matters is that a vector needs to exist in a vector space um and and so what is a vector space model it represents text documents or other types of data as vectors in a high dimensional space so right now we're uh only looking at a a 2d axis but in reality this would be in at least a 3D AIS but the idea is that we have these documents and

these documents represent some form of data and they are plotted onto our into our Vector space uh with distances between them okay and the thing is the distance between these other documents are going to correlate the relationship with them so maybe these documents up here I'm going to get my pen out here maybe all these things have to do something with um let's say vegetables and these ones all have something to do with um uh let's say meat and this is dairy products over here so the way these things are organized on the um in

the vector Space is really dependent on the type of embedding you use what are embeddings these are vectors of data used by ml models to find relationships between data and you'll find that often you're going to be using a machine learning model to create embeddings and there's specialized um uh machine learning models just for embedding so you'll see something like coh here which is a company that produces um or creates their own ml models they'll have like command R but it'll be like command R embeddings and what it does is it takes an input and

it outputs in embeddings to be placed into a vector store so different embedding algorithms capture different kinds of relationships and so it could be the relationship could be uh similarity in words in terms of the way they are spelled or it could be the length of a word uh or the the relationship could be contextual which is like um you know is is the context uh related to a specific industry or vertical so the embedding is going to change uh the relationship that is going to be um projected into that Vector space and these um

ml models that produce embeddings are looking at not just like a single relationship like let's say length of word but multiple relationships and correlating that to put it into Vector space you can think of embeddings as external memory for performing a task for machine learning models embeddings can be shared across models which uh would give us a multi model pattern to help coordinate a task between models but um yeah there you [Music] go positional encoding is a technique used to preserve order of words when processing natural language Transformers need positional encoders because they do not

process data sequentially and would lose order of understanding when analyzing large bodies of text the pre recursor to Transformers is um RNN so uh recurrence neural networks they operated in uh sequential order so they could retain the order of words however uh it made it hard to scale and to uh remember a large amount of words uh to a point so positional coding is a way to fix that in the architectural diagram for Transformers you'll see positional en coding right after embeddings in this architectural diagram here we have positional codings up here so the idea

is we have our input it gets tokenized um turned into tokens then embedded into embeddings it's going to go to position positional encoding where it inserts those uh points and then we're on to our Transformer here okay but let's take a look at the input a bit closer so imagine you have each of those words or those tokens you're going to give them a positional vector and that's how it's going to keep track of words as it's getting mangled uh and interpreted through the whole architectural uh diagram [Music] okay let's take a look at attention

so attention figures out how each word or token in a sequence is important to other words within that sequence by assigning them word weights or token weights or attention weights if you will um so I want to talk about three types of attention we have self attention cross cross attention and multi-head attention and some of these are combined you'll see that in a moment but let's talk about the first one self attention computes attention weights within the same input sequence where each element attends to all other elements and when you see this it basically means

that as attention happens it keeps feeding uh itself right back into itself the same sequence so use in Transformers to model relationships and sequences so words in a sequence you have cross attention computes ATT tension weights between two different sequences allowing one sequence to attend to another sequence this is used in task like translation where the output sequence decoder needs to focus on the input sequence encoder we have multi-ad attention so combine multiple self attention or cross attention heads in parallel each focusing on different aspects of the input um so using Transformers to improve performance

and capture various dependencies simultaneously so how can we look at this in a practical way for our uh architecture for the Transformers so here you can see that in blue where it says multi-headed self attention it's multi-headed because it's receiving multiple inputs you see VK uh q and um I believe that the Q is like for query key is for key and V is for Value has something to do with like how search engines think kind of like if you were if you were to let's say use YouTube and you were to type in a

query there it would match to Keys which would then return you back a value so that is the best description I can give for it it's self attention because it feeds back uh its own sequence it's going to be the same back and forth um there on the other side we have multiheaded uh cross attention so it's multi-headed because it's receiving multiple inputs so we have the vkq but it's cross attention because it feeds sequences uh sequence inputs from two different sources remember it says that um cross detentions two different sequences well we have V

and K coming from the encoder and then we have q which is actually coming from the decoder so it's cut off here but the idea is that the decoder is feeding itself right back into itself and it goes through here into this uh one here and then we get the que and it goes right there okay so again it's not super important to remember this stuff it's just to get you a bit of exposure to looking at these architectural diagrams and to see that there is a way to understand them uh but they can get

very involved and it might be very hard to retain that information unless you are um actually very invested in understanding and building these things [Music] okay let's compare supervised unsupervised and reinforcement learning starting at the top we got supervised learning this is where the data has been labeled for training and it's cons consider test driven because you are trying to make a prediction get a value back so when the labels are known and you want a precise outcome when you need a specific value returned and so you're going to be using classification and regression in

these cases for unsupervised learning this is where data that has not been labeled uh the ml model needs to do its own labeling this is considered data driven it's trying to recognize a structure or a pattern and so this is when the labels are not known and the outcome does not need to be precise when you're trying to make sense of data so you have clustering dimensionality reduction and Association if you've never heard this term before the idea is it's trying to reduce the amount of Dimensions to make it easier to work with the data

so make sense of the data right uh we have reinforcement learning so this is where there is no data there's an environment and an ml model generates data uh and and makes many attempts to reach a goal so this is considered uh decisions driven and so this is for game AI learning tasks robot navigation when you've seen someone code a video game that can play itself that's what this is if you're wondering this is not all the types of machine learning uh and these in specific unsupervised and supervised is considered classical machine learning because they

he heavily rely on statistics and math to produce the outcome uh but there you [Music] go so what is a neural network well it's often described as mimicking the brain it's a neuron or node that represents an algorithm so data is inputed into a neuron and based on the output the data will be passed to one of many connected neurals the connection between neurons is weighted I really should have highlighted that one that's very important uh the network is organized into layers there will be an input layer one to many hidden layers in an output

layer so here's an example of a very simple neural network notice the NN a lot of times you'll see this an ml as an abbreviation for neural networks and sometimes neural networks are just called neural Nets so just understand that's the same term here what is deep learning this is a neural network that has three or more hidden layers it's considered deep learning because at this point it's uh it's not human readable to understand what's going on with it within those layers what is forward feed so neural networks uh where they have connections between nodes

that do not form a cycle they always move forward so that just describes uh a a forward pass through the network you'll see fnn which stands for Ford feed neural network I just describe that type of network uh then there's B back propagation which are in Ford feed uh networks this is where we move backwards through the neural net adjusting the weights to improve the outcome on next iteration this is how a neural net look learns the way the back propagation knows to do this is that there's a loss function so a function that compares

the ground truth to the prediction to determine the error rate how bad the network performs so when it gets to the end it's going to perform that calculation and then it's going to do its back propagation and adjust the weights um then you have activation functions I'm just going to uh clear this up here so activation functions uh they an algorithm applied to a hidden layer uh node that affects connected output so for this entire hidden layer they'll all have the same uh one here and it just kind of affects uh how it learns and

like how the weighting works so it's part of back propagation and just the learning process there's a concept of DSE so when the next layer increases the amount of nodes and you have spars so when the next layer decreases the amount of notes anytime you see something going from a dense layer to a sparse layer that's usually called dimensional dimensionality reduction because you're reducing the amount of Dimensions because the amount of nodes in your network determines the dimensions you have okay [Music] what is a GPU well it's a general processing unit that is specially designed

to quickly uh render high resolution images and videos concurrently gpus can perform parallel operations on multiple sets of data so they are commonly used for non-graphical tasks such as machine learning and scientific computation so a CPU has an average of four to 16 processor cores a GPU can have thousands of processor cores so something that has 408 gpus could have as many as 40,000 cores here's an image I grabbed right off the Nvidia website and so it really illustrates uh very well uh like uh how this would be really good for M machine learning or

uh neural networks because neural networks have a bunch of nodes they're very repetitive tasks if you can spread them across a lot of cores that's going to work out really great so gpus are suited uh for repetitive and highly parallel Computing tasks such as rendering Graphics cryptocurrency mining deep learning and machine [Music] learning we're talking about Cuda but before we can let's talk about what Nvidia is so Nvidia is a company that manufactures graphical processor units for gaming and professional markets if you play video games you've heard of Nvidia so what is Cuda it is

the compute unified device architecture it is a parallel Computing platform in API by Nvidia that allows developers to use Cuda enable gpus for general purpose Computing on gpus so gpg all major deep learning Frameworks are integrated with Nvidia deep uh learning SDK the Nvidia deep learning SDK is a collection of Nvidia libraries for deep learning one of those libraries is the Cuda deep neural network library so cudnn so Cuda or cudnn provides highly tuned implementations for standard routines such as forward and back uh convolution convolution is really great for um uh uh computer vision pooling

normalization activation layers uh so you know in the Azure certification uh for the AI 900 uh they're not going to be talking about Cuda but if you understand these two things you'll understand why gpus uh really matter [Music] okay let's take a look at jupyter notebook so these are web-based applications for authoring documents to combine Live code narrative text equations visualizations uh so if you're doing data science or you're building ml models you absolutely are going to be working with jupyter notebooks they're always integrated into uh cloud service providers ml tools um uh so jupyter

notebook actually came about from IPython so IPython is the precursor of it and they extracted that feature out it became jupyter notebook I IPython is now a kernel uh to run uh python so when you execute out python code here it's using IPython which is just a version of python jupyter notebooks were overhauled and better integrated into an IDE called Jupiter Labs which we'll talk about here in a moment and you generally want to open notebooks in Labs the Legacy webbased interface is known as Jupiter classic notebooks so this is what the old one looks

like you can still open them up but everyone uses Jupiter Labs now okay so let's talk about jupyter labs jupyter labs is the next Generation web-based user interface all familiar features of the classic jupyter notebook is in a flexible powerful user interface it has notebooks a terminal a text editor a file browser Rich outputs Jupiter Labs will eventually replace the classic uh Jupiter notebooks so there you [Music] go let's take a look here at responsible AI which focuses on ethical transparent and accountable uses of AI technology Microsoft puts into practice responsible AI via its six

Microsoft AI principles this whole thing is invented by Microsoft uh and so you know it's not necessarily a standard but it's something that Microsoft is pushing hard to uh have people adopt okay so we the first thing we have is fairness so this is an AI system which should treat all people fairly we have reliability and safety an AI system should perform reliably and safely privacy and security AI system should be secure and respect privacy inclusiveness AI system should Empower everyone and engage people transparency AI systems should be understandable accountability people should be accountable for

AI systems and we need to know these in uh uh greater detail so we're going to have a a short little video on each of these [Music] okay the first on our list is fairness so AI systems should treat all people fairly so an AI system can reinforce existing soci societal stere uh stereotypical bias can be introduced uh during the development of a pipeline so an A system that are used to allocate or withhold opportunities resources or information uh in domains such as criminal justice employee employment and hiring finance and credit so an example here

would be an ml model designed to select a final applicant for hiring pipeline without incorporating any bias based on gender ethnicity or may result in unfair Advantage so Azure ml can tell you how each feature can influence a model's prediction for bias uh one thing that could be of use is fair learn so it's an open source python project to help data scientists to improve Pro fairness in the AI systems at the time of I made this course a lot of their stuff is still in preview so you know it's the fairness component is it's

not 100% there but it's great to see that they're getting that along okay so we are on to our second AI principle for Microsoft and this one is AI systems should perform reliably and safely so AI software must be rigorously tested to ensure they work as expected before release to the end user if there are scenarios where AI is making mistakes it is important to release a report Quantified risks and harms to end users so they are informed of the shortcomings of an AI solution something you should really remember for the exam they'll definitely ask

that AI where concern uh for reliability safety for humans is critically important autonomous vehicles a health diagnosis a suggestions prescriptions and autonomous weapon systems they didn't mention this in their content and I was just like doing some additional research research I'm like yeah you really don't want mistakes when you have automated weapons or ethnically you shouldn't have them at all but hey that's uh that's just how the world works but yeah this is this category [Music] here we're on to our third Microsoft AI principle AI system should be secure and respect privacy so AI can

require vast amounts of data to train deep machine ml models the nature of an ml model may require personally identifiable information so piis uh it is important that we ensure protection of user data that it is not leak or disclosed in some cases ml models can be runow loc Al on a user's device so their uh piis remain on their device avoiding the the vulnerability this is called this is like Edge Computing so that's the concept there AI security principles to malicious actors so data origin and lineage data use internal versus external data corruption considerations

anomaly detection so there you [Music] go we're on to the fourth Microsoft AI principle so AI systems should Empower everyone and engage people if we can design AI solutions for the minority of users they can design a solutions for the majority of users so when we're talking about minority groups we're talking about physical ability gender sexual orientation ethnicity other factors this one's really simple uh in terms of practicality it doesn't 100% make sense because if you've worked with um uh groups that are deaa and blind developing technology for them a lot of times they need

specialized Solutions uh but the approach here is that you know if we can design for the minority we can design for all that is uh the principle there so that's what we need to know okay let's take a look here at transparency so AI systems should be understandable so interpretability and intelligibility is when the end user can understand the behavior of UI so transparency of AI systems can result in mitigating unfairness help developers debug their AI systems ging more trust from our users those build a those who build AI systems should be open about why

they're using AI open about the limitations of the a systems adopting an open- Source AI framework can provide transparency at least from a technical perspective on the internal workings of an AI system we are on to the last Microsoft AI principle here people uh should be accountable for AI systems so the structure put in place to consistently enacting AI principles and taking them into account AI systems should work within Frameworks of governments organizational principles ethical and legal standards that are clearly defined principles guide Microsoft and how they develop sell and Advocate when working with third

parties and this push towards regulation towards a principes so this is Microsoft saying hey everybody adopt our model um there aren't many other models so I guess it's great that Microsoft is taking the charge there I just feel that it needs to be a bit more welldeveloped but what we'll do is look at some more practical examples so we can better understand how to apply their principles okay so if we really want to understand how to apply the Microsoft AI principles they've great created this nice little tool via a free web app for practical scenarios

so they have these cards you can read through these cards they're colorcoded for different scenarios and there's a website so let's go take a look at that and see what we can learn [Music] okay all right so we're here on the guidelines for human AI interaction so we can better understand the uh how to put into practice the Microsoft AI principles they have 18 cards and let's work our way through here and see the examples the first one our list make clear what the system can do help the users understand what the AI system is

capable of doing so here PowerPoint quick start Builders an on uh Builds an online outline to help you get started researching a subject it display uh suggested topics that help you understand the features capability then we have the Bing app shows examples of types of things you can search for um Apple watch displays all metrics it tracks and explains how going on the second card we have make clear how well the system can do what it can do so here we have office new uh companion experience ideas dock alongside your work and offers one-click assistance

with grammar design Data Insights richer images and more the unassuming term ideas coupled with label previews help set expectations and presented suggestions the recommender in apple music uses language such as we will think you'll like to communicate uncertainty the help page for Outlook web mail explains the filtering into focused and other and we'll start working right away but we'll get better with use making clear the mistakes uh will happen and you teach the product and set overrides onto our red cards here we have time Services based on context time when to act or interrupt based

on the user's current task and environment when it's time to leave for appointments Outlook sends a time to leave notification with directions for both driving and public transit taking into account current location event location real-time traffic information um and then we have after using Apple Maps routing it remembers when you're parked your car when you open the app after a little while it suggests routing to the location of the park car all these Apple examples make me think that Microsoft has some kind of partnership with apple I guess I guess Microsoft or or Bill Gates

did own Apple shares so maybe they're closer than we think uh show contextually relevant information time when to act or interrupt based on user's current task and environment powered by Machine learning acronyms in word helps you understand shorthand employed uh in your own work environment relative to current Open document uh on walmart.com when the user is looking at a product such as gaming console it recommends accessories and games that would go with it when a user searches for movies Google shows results including showtimes near the users's location for the current data onto our fifth card

here match based uh we didn't we didn't miss this one right yeah we did okay so we're on the fifth one here match relevant social norms ensure experience is delivered in a way the users would expect given the social cultural context when editor identifies ways to improve writing style presents options politely consider using that's the Canadian way being polite Google photos is able to recognize pets and use the wording important cats and dogs recognizing that for many pets are an important part of one's family and you know what uh when I uh started renting my

new house uh I I said you know there there a problem with dogs and my landlord said well of course pets are part of the family and that was something I like to hear uh Cortana uses semiformal tone oling when unable to find a uh contact which is polite and socially appropriate I like that okay mitigate social biases ensure AI system languages and behaviors do not reinforce undesirable unfair stereotypes and biases my analytics summarizes how you spend your time at work then suggest ways to work smarter one ways to mitigate bias is by using gender

neutral icons to represent important people sounds good to me a Bing search for CEO or doctor shows images of diverse people in terms of gender and an ethnicity sounds good to me the predictive uh keyboard for Android suggests both genders when typing a pronoun starting with the letter H we're on to our yellow cards uh so support efficient invocation so make it easy to invoke or request system Services when needed so flashfill is a helpful timesaver in Excel that can be easily invoked with on canvas interactions and uh that keep you in flow on amazon.com

oh hey there got Amazon in addition to the system giving recommendations as you browse you can manually invoke additional recommendations from the recommender for your menu uh design ideas in Microsoft PowerPoint can be invoked uh with the with the Press of a button if needed I cannot stand it when that pops up I always have to tell it to leave me alone okay support efficient dismal uh efficient dismal dismissal oh support efficient dismissal okay make it easy to dismiss or ignore undesired AI system Services okay this sounds good to me Microsoft forms allows you to

create custom surveys quizzes polls questionnaires and forms some choices questions trigger suggested options position beneath the relevant question the suggestion can be easily ignored and dismissed Instagram allows the user to easily hide or report ads that have been suggested by AI by tapping the ellipses at the top of the right of the ad Siri can be easily dismissed uh uh by saying never mind I'm always telling my Alexa never mind support efficient uh correction make it easy to edit refine or recover the AI system uh when the when the AI system is wrong so alt

Auto alt text automatically generates alt text for photographs by using intelligent services in the cloud descriptions can be easily Modified by clicking the alt text button in the ribbon once you set a reminder with Siri the UI displays a tap to edit link when Bing automatically corrects spelling errors in search queries provides the option to revert to the query as originally typed with one click on to card number 10 Scope Services when in doubt so engage in dis ambigu disambiguation or gracefully degrade the a system service when uncertain about a user's goal so when autor

replacing word is uncertain of a correction it engages in disambiguation by displaying multiple options you can select from Siri will let you know it has trouble ing if you don't respond or talk or or speak too softly big Maps will provide multiple routing options when uh when unable to recommend best one we're on to card number 11 make clear why the system did what it did enable users to access an explanation of why the AI system behaved as it did office online recommends docu a documents based on history and activity descriptive text above each document

makes it clear why the recommendation is shown product recommendations on amazon.com include why recom recommended link that shows that what products in the user shopping history informs the recommendations Facebook enables you to access an explanation about why you are seeing each ad in the news feed onto our green cards so remember recent interactions so maintain short-term memory and allow the user to make efficient references to that memory when attaching a file Outlook offers a list of recent files including recently copied file links Outlook also remember people you have interacted with recently and displays uh them

when addressing a new email uh Bing search rememb some recent queries and search can be continued uh conversationally how old is he after a search for kyanu Reeves Siri carries over the context from one interaction to the next a text message is Created from the person you told Siri to message to onto card number 133 lucky number 13 learn from user Behavior personalize the user experience by learning from their actions over time tap on the search bar in office applications and search lists uh the top three commands on your screen that you're most likely to

need to personalize the technology called zero query doesn't even need to type in the search bar to provide a personalized predictive answer amazon.com gives personalized product recomend recommendations based on previous purchases onto card 14 update and adapt C uh C uh cautiously limit disruptive changes when updating adaptive adapting the AI systems behaviors so PowerPoint designer improves slides for office 65 subscribers by automatically generating design ideas from to choose from designer has integrated new capabilities such as smart Graphics icon suggestions and existing user experience ensuring the updates are not disruptive office tell office tell me feature

shows dynamically recommended items and a designated try area to minimize disruptive changes onto card number 15 encourage granular feedback enable the users to provide feedback indicating their preferences during regular interactions with the AI system so ideas and Excel empowers you to understand your data through high level visual summaries Trends and patterns encourages feedback on each suggestion by asking is this helpful not only does Instagram provide the option to hide specific ads but it also solicits feedback to understand why the ad is not relevant in Apple's music app love dislike buttons are prominent easily accessible number

16 convey the consequences of user actions imately under update or convey how user actions will impact future behaviors of the AI system you can get stock in Geographic data types in Excel it as easy as typing text into a cell and converting it to stock data type or geograph geography data type when you perform the conversion action an icon immediately appears in the converted cells uh upon tapping the like dislike button for each recommendation in apple music a popup informs the user that they'll receive more or fewer similar recommendations onto card number 17 we're almost

near the end provide Global controls allow the user to globally customize the system system monitors and how it behaves so editor expands on spelling and grammar checking capabilities of word to include more advanced proofing and editing designed to ensure document is readable editor can flag a range of critique types and allow to customize the thing is is that in word it's so awful spellchecking I don't understand like it's been years and the the spellchecking never gets better so they got emplo better spellchecking AI I think bang search provides settings that impact the the types of

results that the engine will return for example safe search uh then we have Google photos allows user to turn location history on and off for future photos it's kind of funny seeing like Bing in there about like using AI because at one point it's almost pretty certain that Bing was copying just Google search indexes to learn how to index I don't know that's Microsoft for you uh we're on to card notify users about changes inform the user when AI system adds or updates his capabilities uh the what's new dialogue in office informs you about changes

by giving an overview of the latest features and updates including updates to AI features in Outlook web the help tab includes a what's new section that covers updates so there we go we made it to the end of the list I hope that was a fun listen for you and and there I hope that we could kind of match up the uh the responsible AI I kind of wish what they would have done is actually mapped it out here and say where match but I guess it's kind of an isolate service that kind of ties

in so I guess there we go [Music] okay all right let's take a look at the responsible AI standard version two so this is a Playbook by Microsoft that organizations can use to implement responsibility controls to ensure governance of AI responsibility this term responsibility controls I made it up to make it make more sense um because this is such a new thing that um there aren't terms for it so I have to kind of help out here but here is the report and inside of it you're going to see actionable things that you can do

U the reason I want to show you this is that they say that you're supposed to know how to apply responsible AI in the exam guide um they don't make that very clear uh in terms of what it is and the exam doesn't make it clear but I would imagine that this would be uh a very good place to look in a practical way so we're going to take a quick look at it okay [Music] all right so I just Googled Microsoft responsible AI Standard Version 2 and the first thing in the results was this

PDF so here it is you say this June 2022 so it's a little bit out of date um but I mean there's nothing newer so it must still be applicable if we go down below here let's just take a look at some of these goals let's go farther down and see if there's something interesting um go here like transparency goal system intelligibility for decision making so uh Microsoft AI systems that inform decisions making or about people are designed to support stakeholders needs of intelligible intelligibility of system Behavior so identify stakeholders will use the outputs of

the systems to make decisions stakeholders who are subject to decisions informed by decisions uh and then design information so you can see it's very very wordy um this is very uh what's the the word uh put it I don't want to say BS but it is um definitely for uh your executive uh folks to make up a report to show that you are being responsible so I guess you could take this and make a new word doc and then or or a spreadsheet and just say right exactly how you are meeting these um uh this

Playbook that's basically what you probably do and again this is more for people in management executive level um to show some level of responsibility okay [Music] responsible AI transparency report is a PDF report that outlines the steps that Microsoft has taken to be responsible with AI Technologies so there is a picture of the report the report includes how to build gen AI customer commitments governance uh of responsibilities of AI Microsoft and more I just want to point out that an raai transparency report is just a madeup thing that uh Microsoft has done to um uh

communicate how they are going about uh being transparent with their responsibility of AI uh no other company that I'm aware of is doing this or in this format but um this is Microsoft for you but I just wanted to point that out [Music] okay hey this is Andre Brown and we are taking a look at Azure AI services so what is it well Azure Services is a consolidation of fully managed and serverless Azure AI Services API so when you create an Azure AI service resource you don't need to create resource for each targeted API you

get a single key for multiple endpoints that is the key thing I want to remember is that you get a single key but you'll know that as we do the Hands-On Labs uh you manage security monitoring and automation from one interface so when you go over to Azure AI Services you're going to see under Azure AI Services the list of azure AI services but also in there we're going to see Azure AI Services which is confusing but that's how they do it the idea is that if you spin up an Azure AI service resource you'll

see here basically all the exact same options right so speech content safety computer vision language translator same things along the side here uh because the old way that it used to work is that you used to spin up each resource individually um and that's when this used to be called cognitive AI service Services they rebranded to Azure services and I think customers were like do we really have to spin up all these separately can we just spin up one resource and then have access to it via one key and so I think that's how Azure

AI service came about um so just understand that it is confusing um but that's just the way it works and I do my best to make it as clear as possible in our Hands-On Labs [Music] okay so as a service containers allows you to deploy a subset of AI service apis via container what does this mean well we know that Azure AI service is a resource you spin up right that gives you access to computer vision and all these things the idea is that the uh actual inference the the binaries that actually do the work

that that that that uh do the inference are on these containers why would you want that as a container well the idea is that imagine that you have an on premise workload mode or you have containers either in something like ACI or AKs or even a third party provider like eks and maybe it's a a latency issue you know you want to make sure that those containers or the uh the the uh the AI um operations you want to perform is as close as possible to adjacent containers maybe it's a a security privacy reason you

want to um have that there so you're not necessarily going over the Internet um however to use these you do have have to go over the internet to report back billing but the point is is that uh those are the two reasons why you might want to use Azure AI Services containers how we going to do that well we're going to get those containers from the Microsoft container registry we're going to deploy them into our chosen container Host this part's tricky is where you have to configure the container with the necessary um settings like API

key Billy endpoint UL acceptance I found this very hard to figure out um so they have a lot of different ones that we can deploy I don't think I was 100 % successful in uh the the lab but we get far enough in it that you can see all these steps here and that's going to help you on your exam so just understand that um I would never deploy these myself because it was such a pain to do but um you can see the experience for the most part there for client interaction the client apps

send data to the containerized service endpoint the service processes the data and returns results to the client for billing metrics the usage me metrics are periodically sent to Azure to calculate billing and ures the container can connect via the Azure a service resource for this purpose right it's not to do the inference that's happening on the containers uh these end points here that are available we have the language containers uh we have some speech containers and we have Vision containers in the future they might expand it to more but this is what we have so

far those are the um addresses of the actual um of the actual um uh uh container okay uh maybe not the about here on the end but for the most part I think if you just cut off I'm not sure why those are in there but if you just cut this off here like that then these are more or less the the names of right here like this is the name of the actual actually sorry a little bit more this is the name of the container uh that you would utilize okay but there you [Music]

go hey this is Andrew Brown and we are talking about diagnostic logging for Azure AI Services because when you're running these AI Services um for production and you are trying to figure out your capacity or your usage or maybe your cost you're going to be turning to um the diagnostic logging and so there are three components here we have activity logs resource logs and metrics um so those are the three things that we can work with and you can easily turn them on by just turning on the diagnostic setting U within Azure AI services so

it's it's pretty straightforward and you can just choose where want to send them so it's very hard to see here but we can say log analytics storage accounts event Hub or partner Solutions so um you know that's pretty clear there you can say what uh what level of logs you want to have so we have audit logs and other ones here I think that that there is actually more now or it might vary based on um maybe per region or something but I'm just saying that when I was doing this in the lab I could

have swore there were more check boxes here even though this is a very recent screenshot um and you can also Define the retention policy so uh you know that's a pretty standard option there um probably the most useful way of utilizing uh diagnostic logging is in log analytics because you can basically write a query over here um and get that information uh very quickly it was strange because the first time I used this it worked perfectly but then when I had to go actually record the fall along I could not get this to work whatsoever

um so just understand that yes it's great to have these diagnostic tools if they work and with Azure things are not always consistent okay so you know the point is is that we can send our data over to log analytics and it's captured here it'll start flowing over to here we can analyze it um and it's and it it usually the data appears extremely quickly so it's not like we're waiting uh uh tens tens of minutes it's like the data is there um assuming that uh long analytics is is configured correctly we can also use

Azure monitor um so that allows us to utilize uh Azure metrics we can also um pipe things over to event hubs um you know so if you need to integrate into a custom solution or something else that you want to uh utilize um you can do that as well um for Azure monitor here you can of course make alerts so this is a great way to keep on track of um you know spikes in usage this is very important if you're trying to save money um or you're see if you think you're having performance issues

okay but uh yeah you can also view these metrics directly in the portal so on the left hand side here um you can see that we just have it under monitoring so nothing fancy there you can also add it to charts you can add it to your dashboard so yeah lots and lots of options here for monitoring but the key thing to remember here is and I'm just flipping through these slides really quickly because all this stuff doesn't really matter the only thing you have to really need to remember is that you have to turn

on diagnostic logging if you remember that and where it outputs to then you're going to be covered for the exams [Music] okay hey this is Andrew Brown in this video we're going to take a look at Azure AI services so I want you go to the top here um and type in Azure AI Services look for the one with this logo there's this older one uh which is confu confusing this is called cognitive services and then they changed the logo to this and then they made a new portal even though it's kind of the same

thing um but the key difference with the old one and this one is that you can now create an Azure AI service service which allows you to have um uh they say single API but really just a single place to manage a bunch of these apis um and when we were setting up our environment I may have forgotten to delete uh the translator here so if you have yet to do that you can uh you can delete that I don't believe this costs anything keeping it around so it's not a big deal but just in

case you forget we'll go ahead and delete that I don't want to delete the resource Group that was there with it because um I want to reuse it so we're going to make our way over to Azure AI Services you can see I already have one because I I spun up Azure AI Studio which you'll get indirectly but I just want to show you how to work with it in isolate uh because that might be something you might need to do so we say create Azure AI Services I'm going to drop this down I'm going

to uh go to AI manage workloads and I'm going to do everything in West us uh even though I'm on the east of North America this will just be Azure AI Services I'll just put some random numbers here we'll drop this down we'll go with standard and we'll hit next and we have some options here I'm going to go with all networks um but you know depending on what you need to do you'll change that we'll go ahead and hit review and create and I'm going to uh create Azure AI Services okay so that is

going to proceed to create I'll be back here in just a moment all right so that's now created we're going to go ahead and go to that resource um and so you can see we have go to Azure I Studio that's not something I want to do right now again I just want to work programmatically with Azure AI services and I think that if even we went to this it would want to then spin up a hub because it has to have a hub for this to work and I mean I'll just click to find

out and interestingly enough it is opening this up um but I never created a hub so that's really interesting so maybe we can open this and use it in a limited context without a hub but usually Azure AI studio if you go here they want you to have a hub right and or yeah we have hubs and then we have projects but it seems like we can just directly use the API that's really interesting so um you know azur just confusing and I think there's just like certain ways that you want to go about using

it and so if you want to use AZ Studio don't do it this way make sure you create a hub create a project but again we are focused on just pratically working with this so down below you're going to see we have um uh keys and endpoints um and so down and down here you can see all the different endpoints that we uh may want to Target um and then we have our keys so um obviously we worked with translator when we first set things up um and so I guess that's something that we can

and test this with so what I want to do is go back over to here and I'm going to go ahead and start this back up all right and so all I really want to do is bring in this key and see if it works with our existing code here and that way we'll know that we can work with Azure AI Services okay so just wait for um this environment to spin up all right so now our environment is running I'm going to go ahead and start the Jupiter Labs notebook here so this shouldn't take

too long and we'll just say sure build that's fine um and we already have a folder called AI here in translator if you do not have this you can go to the Azure examples to translator here and you can upload simple because that's the one we are testing this against right here so you can just download this file however you need to download it and then you can bring it into here you can um upload files right here so if you click that that allows you to bring in up U allows you to upload files

so I'm going to double click into here and I'm going to go with simple because failed failed um for whatever reason and it actually has my API key saved which is very frustrating actually you know that's not a problem okay because this is the version that I have from um before that was on this uh compute and actually over here if I go into here it doesn't have a key all right so not really a problem and also at the start of this video I actually ended up deleting um I ended up deleting the uh

uh the server so that key doesn't exist anymore so it's not really a concern but what I want to do is go back over to here we do not uh I thought I had Azure AI Studio open but we don't that's ml ml studio if we go down here we have a couple of keys I'm going to go ahead and copy this key I'm going to go over to here I'm going to paste this in to here uh I just want to paste it in place copy that again right click paste and the thing I

want to check is the end point is the end point the same it is the same excellent so we're going to go over to here we're going to press run and we're going to hit the translate path we're going go ahead and hit that so this should just work so it is thinking it's thinking really hard here and now it has an issue so the uuid is not defined the uid is not defined so that's really interesting because we haven't changed anything here but it is looking for a uu ID so almost feels like some

of our code is missing here so I'm G to go back over to this one and take a look U ID so maybe I might have lost some of this code so what I'm going to do I feel like I might have introduced um a bug or a problem here but I'm going to go back over to Azure AI Services I'm going go over to translator and the only way I'm going to be able to get that code is actually to create a new one so I'm going to create a new one here just temporarily

make under uh this here to say West us my translator I'm just doing this to get the code because clearly I'm missing some code here weit review and create and we'll create this and we'll give it a moment okay all right so what we'll do is we'll go to this resource now and all I want is the code because clearly I'm missing some code here and if we go to our sample code to python yeah so we have uu ID up here is it just missing U ID is there hold on dismiss weird so you

know I keep thinking that there's like something in my code that's missing but not really because all it's doing is generating a u ID so I'm going to go back over to here for a second we'll run this again and then this one and maybe what we could do just to kind of mitigate this problem bring this up here put this here we'll say my U ID and then I'm just going to grab this and then we'll just say my uid because that will rule out that problem right so we do that so that generated

U ID should be no problem now and there we go I again I'm not sure why that was a problem because clearly um there was nothing here that we were missing but again when you're working with Azure you have to have patience you have to say your affirmations so that you can get through it I'm going to delete this um text translation services notice that this still called cognitive Services because it's very hard for them to change that underneath but we'll go back over to here and so we have successfully worked with the API uh

using the Azure API um uh the Azure API services and that's going to make things a lot easier for us okay there are other things we need to do with the Azure AI service but programmatically nothing right now I'm going to leave this around because we're going to be doing a lot more other things um with Azure AI Services working with some of these things here so I just want to leave it around for now okay and um this one is fine we'll just go ahead and if you need to you can shut down this

I'm going to shut down this for now stop and we will call this done for now okay see you in the next one [Music] ciao hey this is angrew brown in this video we're going to take a look at um utilizing Azure AI Services containers um I've never done this before but I imagine it's going to be very straightforward because it's a container so I imagine we're just going to pass um our API keys or something to it so let's see if we can figure this out uh the first thing I want to find is

the actual containers so if we go to the Microsoft container registry um from here I'm hoping that we can go find one of these um Azure AI services so we are here at the registry and I'm going to search and we're going to type in Azure Ai and see what we can find and so it looks like we have some here what I'm going to do is compare against the ones they say that we can use so we have key phrase extraction sentiment analysis translator I'm thinking maybe translator just because we keep um utilizing that

one so I want to go ahead and find that here so I'm just going to go back here and type in translator and there it is so there's the text translation service and so here we have uh containers now if they have some docs I'll take them so it's saying what is AZ Azure AI containers I was hoping for something a little little bit better than this I thought it would tell me um exactly what we need to do to work with this the can can also run in a discontinued environment oh sorry I'm just

reading that there um so you know again I was hoping that it would tell us a little bit [Music] more but that's fine let's just go ahead and uh spin this up before we do I just want to see if there's anything interesting here it says deploy Azure container instances yeah maybe we'll just take a look at here because that's probably where I want to deploy it is in container instances here and is there a whole lot of information here there's some so let's go ahead and see if we can do this so I'm going

to type in ACI at the top we make our way over to Azure container services we're going to go ahead and create ourselves a container instance I'm just going to speed uh speed through this and just bring over what's relevant okay so one key thing they're telling us is that we have to make sure the resources are created before using it we're using the Azure AP uh service um key that we already created in an earlier video so we already have that so just make sure that you've done that video first um if you're doing

these videos in order this shouldn't be an issue um but just understand that uh it's relying on the Azure AI Services setup video okay but let me just keep reading here all right so we're getting some information here so I'm going to go ahead and just start working with this um I am yeah going to stick with that subscription I'm just going to create a new yeah I'll make a new one we'll call this um Azure AI for AI containers and this will be AI containers I'm just going to put some numbers here on the

end so we don't have a conflict I'm going to stick with West us that seems fine to me we want to Source an image um from Microsoft so maybe we can go here and say other and now we have a path so we need to find out what the URL is here or the path for this is if we go up to the top does it show us information kind of go back to here microsoft.com whatever whatever Services keyword would represent it so here they have an example of this one here so again just making

sure that that is correct um I was really hoping they just have the name here what if we click into the latest one yeah there we go there it is so I'm not sure if we can include the tag I mean I would assume that it would use the latest so normally what I would do is I would grab the uh the tag with it but I'm going to go back over to this here yeah we'll go back here and we'll paste this in and so it's not complet exping so clearly we can put the

latest tag in here we'll pull from it we have Linux and windows if not specified dockerhub will use the container registry in the latest one um I don't know I mean like if it's running python it should just work on Linux then down below here we have the size that we want to utilize what are our options 1 1.5 that seems like a lot but that's probably good enough I don't want to overthink it here so I'm going to go ahead and hit next uh if you are concerned about cost you can look up if

there is a cheaper cost cost there but I'm we're not going to be using this for very long networking type is going to be public um so that should be fine I'm hoping I don't to fill in the rest here but let's go back over to here and see oh they're saying use Linux change to the recommended size two CPU cores and four gigabytes it seems like a lot but I'm going to go back just because it told me to do that I'm going to go and do that it's not going to cost me too

much oh you can do gpus now that's kind of cool a v00 that's not the latest right now the latest is h100 for NVIDIA and before that's a100 then you have v00 but uh that seems fine then we have Port so set the port to 5,000 or 50,000 that 50,000 5,000 looks like 5,000 so we go to networking here cancel cancel what is it doing go back okay it's being a little bit silly here and I'm going to go back to this and say two and four and say okay and we'll go next and we

want to open up this port on TCP I don't know if we need Port 80 I will take that away because I don't think I need that we will go next we have on failure that seems [Music] fine okay I'm just going to leave it public we go go next I don't care about naming it but they do have some other options here advanced settings environment variables okay so this is how we're plugging stuff in let's go back to our Advance here and I don't see oh here it is Nars so what we need is

an API key okay and we need billing and we need Ula so those are the three that we need I would imagine that this is secure so mark that is secure and I'm going to go back over to Azure AI so I'm just going to open this a new tab here AI Services because I need my key make sure you're using that with that logo because they're making it nice and confusing with having both both the old one and the new one around so going to grab this one we'll go back over to here and

we'll paste that in so that's now in here we need to determine what our billing tier is so here's a question will it work with Azure AI services or do we have to use the individual one because I I imagine that you can um I don't know what the building tier is I guess it's just going to be standard I'm not sure what it wants for uh Ula is it just going to be accept there it is um and here it actually says your endpoint URL copi from the keys and endpoint page from the resource

so it's not saying put the word standard in here it's saying go to here and then grab it grab what you need so we're going to go here I'm going to grab this one that's what it's telling me to do sounds a bit bizarre but going to stick with what it tells me so review and create we hit review and create and it might be that translator just does not work we'll go ahead and hit create that's going to spin up so then here it tells us we have to wait and then we going go

to Port 5000 um and then I guess we'd have to use some kind of external service to Ping it so that's something that we'll have to to figure out select any post API and try and select try it out what does it mean select try it out parameters are displayed including the input fill in the parameters um okay well I'd rather use the the CLI here so it doesn't make it super clear here we'll go back over to here and we'll just wait for this deploy and then I'll try to figure it out okay all

right sorry deployment is complete let's go to our resource take a look at what we have here um so it is running and there should be some endpoint for us that's what I'm looking for right now settings containers here we go oh and it's terminated so that is not a good indicator that's suggesting to me that there is something that has gone wrong it does not like something that we have done um and so here it shows container terminated with exit Code Zero I can't remember if zero is bad or not I always forget Zero

versus one means that the success so that makes it sounds sounds like it ran there were no issues and then it stopped okay H well it does say that translator is something that we can utilize so I mean I do have a link here for translator let's take a look here I'm just going to read this just give me a second okay so from here I I was able to click through to here and now we have information configure translator Docker container so it looks like there's a lot more options here that we could configure

um I mean we got the billing right we did the Ula right we did this um so yes yes required yes this one is required we it reads and writes data from the host computer to the container and from the container back to the host computer so maybe this one isn't as um straightforward as the other one but it seems like there's more information on it but let me read about it and see if I can figure it out okay all right so as I read through this it makes it look like um once it's

running you're going to have an endpoint that you're going to hit and you're to uh translate that over now apparently we do have to do some mounting but again not sure why we have to do that so but I mean it is clear that there's an endpoint that we can H let's go back over to here and reads and writes data from the host computer to the container and from the container and back is it required mount I mean I don't really want to mount anything [Music] though okay just give me a second now I

know it says it's required but what's interesting is if you look right below here it makes it sound like these are the only real ones that are required so which is it I'm not sure um but the thing that we don't know is like like if it runs why did it stop so it started and then it stopped and we don't know why and it exed successfully meaning there was no issues um this is where it'd be nice to see some logs which might not be easy for us to figure out it's been a while

since I haven't launched anything here in containers so yeah um hm I mean we could just hit start again see what happens let's just start it and it might just stop again but at least we know generally the flow of it if this is not going to work we'll just consider this that we'd learned the service um my experience with Azure is that it's not always worth the time to get these labs to work but you know I do try to do my best here we could try and go do the speech to text but

that one has its own challenge because then you have to provide it um an audio file right and I was hoping that with translator that we would just be able to um provide it some text and work with it that way but as that is starting it looks like it may have failed twice um we'll refresh this go back over to overview here one or more containers are waiting in state and may not be running click here to view the statuses okay so it's in a waiting State what does that mean ACI waiting state does

that mean that it's waiting for something I mean I don't mind if it's waiting so while that is going what I'm going to do here oh no it was terminated so I guess the question is is a waiting State bad was it waiting for input because it seems like it should run until we uh give it something to do right and that's what we don't know we do have logs right here let's take a look ah missing parameter languages so probably this is the reason why it failed okay um let's go back over to here

and take a look at this there's no configuration option that says that we have to provide that let's go back over to um here and here they're recommended even more than what we're running apparently we're not running enough but it doesn't say language anywhere but we can clearly see here languages right so that makes me think if we go back over to here and go here I'm trying to figure out where do we update our nirs maybe it's under properties where could that information be not exactly what we want so yeah I'm not exactly sure

because we set them when we first set this up right so let's go back a level here let's go over to ACI and we'll click click into this and so what I'm looking for is is there a way that we can change our Nars so that is what I'm looking for right now identity no properties I would have thought it would been under properties here because we see the port but I'm not seeing an option to change our end bars we might not be able to do that one second so yeah I I think we'd

actually have to uh create it again um and that's kind of a bit of pain the thing is that we just don't know like we could add languages as an environment variable but like it's not calling that out on the docs and the no reason why we can see that that's an issue is because of um that there you know what I'm I'm going to do it again I know that it's uh it's it's you know Troublesome the fact that we have to do that but I'm just going to go ahead and try that again

so I'm going to go into here and I'm just going to go and delete this one here and we're going to try it one more time okay so I'm going to go and drop this down we'll say I AI containers my AI containers attempt to make sure it's Unique name we'll stick with standard we'll go with other registry here if it doesn't work that's totally fine fine I think that uh we sufficiently understand what it is that we'd have to kind of do and we'll choose Linux and this one actually told us to do four

and four seems like a lot to me but I'll go ahead and do that if you don't want to do that I just want to see if I can I figure it out just go ahead and watch it's not a big deal here we know we need to open up Port 50,000 I think they said um I'm going to just double check that here and whatever this is this one looks like 5,000 to me but yeah it's 5,000 it's not 50,000 that's one other problem we might have here we'll go to advance and we're going

to go to yes and this one is going to be our API key um so I want to just go here make sure I match these API key notice it doesn't show languages and I'm going to grab this from here and bring this back then we have billing and this is going to be our end point here and then we have Ula this will be accept and then we want languages and I always spell languages wrong so just to make sure I don't get that wrong I'm going to copy it from here um because there

was I think a call down here somewhere maybe it wasn't this tab languages well I'll just copy the word here languages and just scrolling through here you got languages so I'm just going to grab that and then we'll just paste this in here there's nothing that tells us how we're supposed to do it so that is my best guess basically we'll give it a language and it'll spit back these translated languages and I'm hoping that that is good enough to work so let's go ahead review and create now we didn't Mount anything it says that's

required but maybe it's not again I do not trust the docs um whatsoever we'll go ahead and create that and we'll see what happens okay so our deployment is complete let's go to that resource and take a look um we're going to go over to Containers this one is in a waiting state here you can see it's attempted more than once now we're getting ex code of one so at least we're getting something else we have no logs available so this might be a little bit too hard for us to debug just because we don't

have the information API key is not showing up here which is totally fine um but yeah it looks like it's not working so that's okay um I think that we figured out what we can figure out but if we don't have the information we can't do it and we'd have to do a lot of work to find out by going to support and then you know what Microsoft's probably going to change stuff on us again and I don't want to waste your time so I think that we achieved what we wanted to achieve for the

the sake of exams let's go ahead and delete this container resource okay and this one's an AI containers we going to go ahead and clean that up and we're going to consider this one done so let's just iterate over what should have happened uh if that had worked and it was deployed what we were expecting to happen was to go um to down below here and we should have been able to hit it with an API end point um um they had one here somewhere maybe it was actually on this page yeah so we should

have been able to curl it like translate tell the API version from to and provided its text and hopefully that would have worked but um yeah the other thing that the reason why that container might have not worked is maybe we did need that mount as required but the only way we'd find out is if there was some way we could connect to the instance and debug it but we don't know enough about the um how that thing was built and it's not worth our time anyway I'll see you in the next one and hopefully

you uh learned something there okay [Music] ciao hey this is Andre Brown in this video what we're going to take a look at is setting up diagnostic um settings for Azure AI services so that we can keep track of what things are going on so over here on the left hand side once you click into your Azure AI Service uh that we created in a previous video we can go ahead and add diagnostic settings and we can send things to a log analytics workspace and to a storage account so I'm going to recommend that we

go ahead and create ourselves those two things I'm going to go ahead and go to storage accounts and we're going to go and create a new one here so this will be for I'll make a new one here this will be um ai. [Music] settings just so that we're not getting confused and I'll just call this my storage doag settings and put some numbers here on the end uh doesn't like how long it is so just reduce that there we'll just say my you know just make it a unique name whatever you have to do

I'm going to deploy this in West us because that's where I've been putting everything standard is totally fine we'll go ahead hit review and create and that will allow us to create it and while that is creating we're going to want a log analytics workspace so we'll go over to log analytics workspaces and we'll create a new one and we'll go to here we'll say my log analytics for my log D 3 32 whatever I don't really care what it's called we'll go ahead and create that and hit that and those two are now going

while that is occurring I'm going to go over to our um Azure ml studio and remember that we created this in our setup video I want to start this up because we're going to want to have some data triggered that we're going to observe so this just takes a lot of time to start up and so I figured as we're creating these as that's deploying as this is deploying uh we can then um get this running here as well so these are both deploy I'm going to go back to this setting here and I think

we're going to have to refresh this so I'm going to go back here say okay and then we'll try this again and we'll send them over here this one's going to go to this one and then this one's going to go to that one I'm going to send all data I'll just say my dag uh 3242 just matching the number up here and so that looks good to me we'll go ahead and save that so now I'm just waiting for this to start and so we'll once it starts we'll launch up our Jupiter Labs notebook

and try to trigger um some data okay all right so that's now running we're going to go here and start up a Jupiter lab notebook to give this uh environment a moment to load say build that's totally fine and just open up the translator simple in the AI translator directory so I'm just going to navigate back to here you should still have this if you do not have this we have it at the Azure examples repository here under translator you just download this and you're going to upload this uh into here using the upload button

from your local computer uh once this is ready just watch the CPU there as this starts up it always has a little bit of trouble at the start here again we are running it on the cheapest possible option because we're just running API end points it's going to drop there at one point we'll go ahead and go down here and just run these and so now we have some translated output I'm going to go ahead and just make some changes here so how are you today what is the weather like did you have a good

sleep there is a rock under my coach or coach whatever um and so we have uh some data there that should be sufficient for us to see something um I'm going to go ahead and go here refresh click on here and we're going to stop that so we're not wasting any money here and the idea is that with our settings saved we should be able to see something here now it might take a bit of time so what I'm going to suggest is we wait here for a few minutes and then we'll go take a

look okay all right let's go take a look and see if we can see any data so I'm going to make my way over to storage accounts and our new one here is this my story Diagnostics if we go over to metrics maybe we can go down to something like transactions and notice we have um five because we triggered I think it five times if we wanted to get that data we can go ahead and just export it there um and there's nothing super exciting if you want to open it up it's very boring data

but uh um we have some information other thing we want to go take a look is in log analytics so let's make our way over to log analytics which we might already have open somewhere here excellent um and I'm trying to think where we do this because we want to run a query just go to logs here give me a moment I don't remember no I think this is where it is it just used to be called something else so let's thr me off and I don't remember that being there but okay here we go

that's what I was looking for uh and so in here is where we can do something and just click off of here and then we can just write our own uh query so uh I'm just thinking here because it's coming from azzure AI so see Azure ah there we go Azure Diagnostics and then we have the pipe here and then we can type in where resource provider equals um Microsoft and I [Music] believe stop I believe that this one would be I think it's all in caps actually Microsoft I believe this would be cognitive Services

because that's what Azure AI Services used to be called so I'm going go ahead and just type that in so cogntive services this is my best guess um and we'll see if this returns any information so nothing found H so let me go look it up okay I'll be back in just a moment all right so so what I've done here is I've just gone to the queries and I was looking for Azure Diagnostics and I just said find an Azure diagnostic so find to search a specified value um and so the idea was that

I was hoping that it would return something but here it's going Microsoft container service so maybe let me go figure this out okay all right so I know that this is correct I mean I don't think this is case sensitive um but this should return information I'm just not sure maybe the data is not propagating yet um but just give me a moment I'm just keep peeking around here for a bit all right so you know I've done more research and this worked before so this is azure for you something will work one time and

the next time it won't uh I think maybe the issue could be propagation so it could be that we need to wait a period of time before um that will take effect it's a bit confusing here because also they say like there's classic and then there's new and then there's also newer newer log analytics I'm not sure if this is going to make much of a difference um but uh I suppose we can go here we don't have any tables wonder if that's our problem because if there are no tables maybe there's no data just

just a moment another thing I'm wondering is that we have Lo log analytics here but could we find log analytics from another perspective sometimes when you go here um you can get information this way right and what we go open query no that's not what we want that's Azure resource graph Explorer um if we click into here no that doesn't really help that much I mean we have tables here no that's not going to help either so yeah I not sure um but when I originally did this lab I was able to create the data

and utilize what we have here and it could take me multiple tries I'm going to tell you this is like my fourth time doing this lab because I keep running into issues um and I'm kind of getting a bit fatigued with it so I would say this is probably the best that we can do here here but just understand that those are means to which you can uh set up logging I'm going to get rid of um this log analytics group because I don't need it any longer before I do that I'm going to go

back to our Azure AI service and I want to go into diagnostic settings and I'm just going to get rid of these settings here and we'll go to storage accounts and we're going to delete and we'll say delete and we'll go over to log analytics and type in log Analytics and for some reason the search is not working properly I'm not sure why I just want to go to analytics log analytics and I can't even get back to them right now so I'll go to Resource groups sometimes Azure is just like this it's just difficult

and we have them in here so if I just delete this this will tear down all the resources yeah that's what I wanted to get rid of but for some reason today I'm typing up there it's just refusing to work so we'll delete the resource Group and that's going to take down everything we created here um and so there you go that's how you do diagnostic settings and uh yeah there you [Music] go all right let's do a comparison between Azure AI services and Azure AI Studio it's more confusing than that because we actually have

Azure open AI services and Azure open AI studio and I want to just make it very clear what is the difference between these four things we are going to iterate on this a few times so I'm sure uh by the time you're done this course you'll know distinctly what these four things are but they are confusing initially so the first is azure open AI services so this is a resource I don't sure I put an S on here but this is a resource that provides API access to allow you to deploy and pro program aut

atically work with open AI llm models okay so if you want to work with uh GPT 4 or what whichever one that is a product that open AI provides you this is the resource you spin up it gives you a key it gives you an endpoint uh the next thing is azure open AI studio so this is a guey uh that provides an easy way to interact with open a uh AI Services API and also the Azure AI Services API but the whole Focus here is that if you aren't a programmer and you want to

uh work with a chatbot or do rag um or experiment with um a lot of the AI offerings that Microsoft has is going to be in this interface okay then we have Azure AI services so this is a collection of azure AI Services apis intended to be used programmatically via a single API key this was previously called the cognitive services and they just kind of group them all together and put them under a single key to make your life a a lot easier then you have Azure AI studio and this is a guey that provides

an easy way to interact with Azure AI Services large language model so from open AI Services gener generative images and more um so this is very similar to opening eye studio and it could end up replacing um uh Azure opening eye Studio but right now they both exist but the key difference is that it provides a wider range of LL models so here you can deploy not just open AI models but things like um minstral or coh here or other ones like that it also allows you to scope workloads into projects so that is one

key difference between opening IE studio um when you provision let's say um Azure open AI Studio it will end up creating open AI services and probably an AI Services as well or vice versa so you can either create these individually and not use the studios at all you can create the and then launch the studios or you can create the studio and then it will create these resources indirectly so it is really confusing what is the best way I'm not really uh certain at some points I feel more confident than others um but we'll get

through it and just understand that this is azure their naming gets confusing and they have multiple um products that have serious overlap but that's just how it is okay and we'll be in both so you're not going to be too confused but I just wanted to make sure sure that this is very clear okay there's these four things and they all kind of work together [Music] okay let's take a look here at Azure open AI service which is an API that allows you to deploy use and manage multiple open AI large language models and so

the idea is that once you spin up the service you'll have an API key you'll have a few different endpoints um and what's really interesting that I I found out when you utilizing this is that the open AI python Library which is specifically for open ai's platform is the same Library you're going to use to interact with azure's uh open AI service it's really interesting that they did not make that a separate um plugin which kind of indicates to you how close open a ey is with uh Microsoft but just remember that open is not

owned by Microsoft but they have strong ties there together so in the left hand side there you can see get my pen tool out here you can see that we are creating um a a new client using our endpoint API key we're doing some chat completion and so uh we are utilizing GPT 35 turbo here's our series of messages that is going to go and it's going to return back um uh completion okay so just understand that when you see chat completion like this and we see a series of messages we have to if we

want want to do it again we literally call this again and then feed in the next message below it um which is how uh these things work but anyway in order to use the openai llms uh you need to First deploy it within the Azure open ey studio um for whatever reason um they changed it so before it used to just be the open ey service you say I want to deploy this model and then you'd be able to utilize it but now it has to be in the Azure openi Studio we can also deploy

these models in the Azure AI uh Studio but we're just going to focus on Azure open AI studio for now [Music] okay all right let's take a look here at Azure open AI Studio which is a guey for the Azure open ey Services API uh it looks very similar to Azure AI studio and um you know I would consider it a legacy thing because basically everything that you can do in this you can do in um Azure AI Studio but more but I don't think they're going to get rid of it because it serves its

purpose to give you access to just um the open AI uh language models and I noticed that when I used Azure open AI Studio it wasn't as hard to get access to the models as it wasn't a your AI service where that had a asz your AI Studio which had a formal process to request them I had to wait a few days so you know if you want to get working with um apis right away or llms right away away this one might be a little bit easier but again just's understand there's some differences here

so what can you do in here well we have the model catalog where is you're going to choose what you want to deploy so you can utilize it we have chat completion um so that's something we have there we have uh completion um models that we can utilize we can generate images we can find tune to select amount of models upload data to be used for rag uh so pretty straightforward but again these are just interfaces to get you working with these right away and so I'm going to repeat it again that Azure AI Studio

has more functionality and appears to be the successor of azure Open studio but we'll be bouncing between the two [Music] okay let us take a look at the models that we can utilize as of this video um for open AI uh and so there are a lot that are Legacy that I'm not including in here um but there are some cases where some Legacy models appear even though you can't directly launch them but you might end up having to use them and I'll point that out as we work through this so let's first talk about

chat completion models if you do not know what a chat completion model is it's where it's where you can have a conversation so you can also call these conversational models because that is their purpose uh but let's take a look at what we have so the first is GPT 3.5 turbo 0125 there's a bunch of uh GPT 3.5 turbos with a number on the end and so so um these are earlier iterations of GPT 3.5 turbo that's at least what they say I would assume that maybe they were just doing minor updates on the older

models here but there's a bunch with uh four numbers on the end there but if you end up using GPT 3.5 turbo it's probably using this version 0125 um in there so this is better than earlier versions of gpt3 uh gpt3 was Da Vinci and babage which did not have conversations those were just completion model mod but um uh yeah g GPT 3.5 turbo is very well known over 3.5 because um it was an iteration on 3.5 where it had better uh performance and things like that um so its context window is 16,000 tokens so

understand that is how much it can work with then we have GPT 3.5 turbo instruct this is a fine this is fine-tuned I got a two in there but this one's fine tuned to accept and return step-by-step instructions it has a much smaller context window but um this is very useful when you want to uh instruct it to do something it's going to follow those instructions we have GPT 4 which is just a better version of G GPT 3.5 so I'm assuming it's trained on a lot more information what's really interesting is has a context

window of 128 tokens so it's way way larger um so we have it a huge jump between uh 3.5 to G GPT 4 um now we have GPT 4 Turbo so each time you see the word turbo it's basically um it doesn't mean that they're training on less data it just means that it's been optimized so that it's going to be faster and more cost effective so anytime you see a turbo you want to use turbo generally over the original one but there is a suggestion here that gp4 turbo might not have the same level

of reasoning as gtt4 um is contact Windows the same 128 uh 28 tokens we have gp4 turbo Vision which I think is the first multimodal that Azure open AI or sorry open AI released um so it I would assume it's more expensive but it can take images and text and understand them then came along gp40 the O stands for Omni and people don't really remember gp4 turbo Vision kind of like skipped over it and we went straight to gp4 uh Omni because its capabilities and reason reasonings were so much better it's also multimodal so it

can handle text and images its context Windows the same so it's just a extremely more improved version uh and I think it's trained with more information I think um but for whatever reason it's it's really really good then you have GPT 40 mini and so it's not a turbo it's a mini because it's trained on less data um so it has a smaller data set but it's extremely fast it's extremely cost effective um and it beats GPT 3.5 turbo out of the water so um there's no reason you'd want to use GPT 3.5 turbo but

we use it a lot because it best illustrates the um some of the uh flaws in models where these newer ones are a lot more intelligent so it's harder to Showcase those so so from a learning perspective we end up using GPT 3.5 turbo or sometimes we have to use it because um at the time of the video the capabilities to let's say train a model or use it is limited or maybe the deployment methods are limited but I can tell you when I was making the labs for this I wanted to use GPT 4

mini but it didn't have a real time inference I wanted to use it for fine-tuning it wasn't ready uh it was announced that I could fine tune on it but even after that announcement I still couldn't utilize it so you know just understand that uh uh we use GPC 3.5 turbo even though there are better options out there it's context Windows 128 uh tokens as well so no reduction there even though it's a smart smaller faster model let's talk about completion models these are the gpt3 series and their naming is a bit different there was

actually more than just the two here but we have babage and da Vinci um so for whatever reason uh open a decided to change the naming scheme after this because these ones are just confusing um but anyway babage is a small less complex uh model it's very uh it's very fast for low compute notice that it's not a chat completion model but a compl completion model meaning that you give it a piece of text and it just starts writing text on the end of it just spits out lots and lots of text um it's not

trying to have a conversation with you per se um same thing with da Vinci Da Vinci is just um more powerful um and the context window for Da Vinci is 4K when it came out but I think that it goes beyond that I couldn't get an exact number not even Azure lists the information I I couldn't find it but generally they're both 4K with da Vinci sometimes is having more uh but Da Vinci is smarter at uh a greater cost so it really depends on what you want to do there these older models are uh

still useful um if you fine-tune them for specific tasks they can outperform and uh both on cost and performance these newer models so you have to kind of make that trade between do I use GPT 40 mini or do I train one of these older models and also GPT 40 mini can be trained as well so um yeah it just depends on your use case then we have embedding model so these um take text and create embeddings that are used for rag um or other various uses there uh Ada is from the gpt3 family so

I guess it's considered old what's unusual is that you can't directly deploy it but when you are doing rag I I um used text embedding uh three small to um embed turn the embedding to put in the vector store but then when I wanted to um use it on the other side I couldn't use it I had to use Ada and the thing is you you shouldn't mix embedding models you're going to get bad results because each embedding model has their own way of um uh describing similarities but surprisingly it it worked so it can

work but you really should not mix embedding models um and that ADA thing just kind of cropped out of nowhere we have Tex embedding 3 small Tex embedding 3 large that does not mean they're part of the gpt3 uh Ser uh family I have no idea why they're called uh three but the key difference here is the small is less complex and the large is more complex so you're going to have better accuracy better um reasoning or or or similarity information for large how they work we have no idea they do not tell us um

we have whisper which they'll say it's a speech model but it's specifically a speech to text model it understands speech that's its modality speech turns it into text then you have text to speech um and they're just named TTS so you give it text it produces speech the HD version just produces a more high quality voice um not necessarily at a higher sample rate just a higher or better natural sounding voice we have image models such as Dolly 2 and dolly3 and yes they have those weird hyphens in there I don't know why so uh

Dolly 3 is better than Dolly 2 pretty straightforward um fine tuning model so we have babage Da Vinci G GPT 3.5 turbo instruct and GPT 40 mini I've yet to use Mini um even though it's been announced but those are models that we can train the reason you can't train larger models is because it's just too expensive to do that and when we talk about training we're really I think talking about maybe like Laura where we're um or uh we are just um uh we're we're retraining the weights we're not really training with new data

but um more just the weighting of it um so hopefully that makes sense there but that's the landscape of models these are always changing so I'm sure tomorrow will be out of date but the idea is you get a general idea of the models available [Music] okay let's talk about the deployment types we have when we are deploying models for the Azure open ey services and I want to extend this to any other type of models that we can deploy not just the open AI Services um but when you are deploying you're going to have

four possible options we have global standard Global batch standard and provision managed so for Global batch global standard and standard you're paying per token so it's based all your inputs and your outputs combined okay for provisioned uh manage you are spinning up compute so you're paying for the underlying compute um in terms of speed or performance Global batch it takes hours for it to happen so it's not something that's going to work instantly for you the rest have real time inference so it's intended for um that there um Global batch is offline so it's not

again really to be utilized uh for immediate application workloads it's where they use the word offline because it makes it sound like it's not connected to the internet but they mean offline as in like not real time um global standard is globally available whereas standard is um basically Regional Regional deployment it's a bit strange because I'm pretty sure that if you go to I got a look at this doing this soft screen now but if I look up like pricing pricing models for um the a Azure AI Services they'll call it like a regional deployment

right so this standard here I could have probably put more text in here would be for like gp40 Regional API and then global standard would be for gp40 Global deployment but standard is uh for regional deployment uh standard actually costs more than uh than global standard you think global standard would cost more because it makes it sound like it's uh uh available everywhere but basically standard costs more and maybe have some hidden costs there because you're getting lower latency because of um it performing better in individual region provisioned manage would be obviously the most expensive

because you're literally spinning up compute um when launching models you're going to find that there's just some serious limitations so even if you want global standard most of the time it's like oh I can only use standard here or vice versa I really wanted to use GPT 40 mini but all I could get was Global back you know what I mean if you're using things that are not open AI then if it's an like an open source model then most likely it's going to be provision managed because if you have an open source model like

uh metal llama then um you know they don't have a service for it it's just they're downloading it and putting it on a machine for you or if you're using something like coh here coh here has its own um hosted service so then that's going to be per token which is standard so you know just remember there's four different deployment models and just choose the best you can to to save cost based on what you're using [Music] okay hey this is angre brown in this video we're going to work with an llm um in some

of these videos I'm going to use Azure open AI studio and some of them I'm going to use Azure AI Studio technically Azure AI studio is new but it's in preview um maybe they'll get rid of open AI service but just in case that is the case I'm going to go ahead and in this video we're going to use Azure AI studio now I've spun up um a hub um previously in West us but the first thing we might want to do is just look at model availability in Azure an Azure uh AI Studio because

that's going to affect what we're able uh to utilize and somewhere here ah here we go we can see what models we can use so if we are in West us we have turbo um and we we have 35 turbo Turbo instruct so we have quite a few options there okay so I think where I would like to deploy and notice like there's a lot in Sweden but I think that I'd like to work in West us because I've already launched one up there but if we run into an issue we can also have East

us all right um it depends on what we want to do because like there are some here like turbo instruct this is a model that is already prepped to accept instructions and do really well whereas this is more of a generic model so it really depends on uh what you want to utilize but anyway I'm going to go um and switch over to West us so right now it thinks I'm uh in the wrong place so I'm going to go over to portal. azure.com and from here we are going to navigate over to Azure AI

studio and here we have a hub already and I'm going to enter that Hub and so once we have our Hub um now we can go to the model catalog and and start launching a service and we do have playgrounds but I recommend that you go and always scope things to a project I'm going to let it create a Project based on my name because I do not care here today and we'll go ahead and create that project and give it a moment and if you do create a project you're going to see a lot

more options available um and I think that's what we want to do so we'll give it a moment there to provision all right so now our project is ready and you'll notice there's a lot more options here on the left hand side um um but yeah we want to uh start working in the chat and this of course is a playground that is very similar to open AI so open AI is a company that um is partnered with Microsoft not necessarily owned by Microsoft but they have their own playground where you can uh work with

a large language model uh before you actually go ahead and use it programmatically um and so these things are very similar so just understand that is a very similar experience so the first thing we'll need to do is actually set up a deployment so we can go to our model catalog and we can see there are a lot of options here I want to point out that if you have yet to uh do so if you want to work with a model you or sorry if you want to work with a family of models like

open AI you have to submit a request it might take 48 hours to a week and then you'll be able to utilize it if this happens to you just understand that submit it once for any of the GPT products and um just check in in a few days and see if it becomes available to you um because they're not very good at communicating but it was a very unusual uh process there but I eventually did get access to um the model catalog obviously there's a lot of other stuff here um and I'll try something else

other than just uh GPT but for the exams you just need to know GPT but I figured it'd be fun to try something else else out I wish they would just tell us what models are actually available uh for our region so we don't have to like try to launch it and find out we can't do it but that's okay so I'm going to launch um see see it looks like we have turbo instruct but I'm pretty sure we don't but we'll go with GPT 35 turbo now there's of course newer models but it really

depends on um when you go here what your deployment model is so if we go here we might not have particular options so here we have Global batch standard and provision manage so I just want to show you here that there are four different types of deployment type we have Global batch global standard standard and provision so Global batch is where you are going to send workloads um to the AP and they'll return back uh way later okay and so I assume that makes it very cost effective but the idea is that um you're not

going to get the results right right back right away then you have global standard so this is where it is globally available then you have one that is standard where it's just regionally available and then you have provisioned for whatever reason uh uh um uh the the cost for the cost for regional is more expensive than standard so if you want to just take a look here at the pricing we'll go here the the pricing goes to the same page we go down here below we have Global and Regional and notice that some in the

regional have an additional cost to it okay so just be aware of that um but today we are working with uh turbo and you can see turbo is very very inexpensive but it might not be available in specific types of deployments okay um there is also provision but I don't have a whole lot of confidence what the cost of that is so I'm just going to stay away from it here because I don't want to um cause myself to have a large Bill and then my co-founder Andrew also name Andrew is like hey what's going

on so I'm just going to choose standard here uh if you're uncomfortable with any kind of spend just watch and learn but I'm going to go ahead and deploy this here today so these models don't take super long to deploy so here it is and we'll go over to our chat playground and let's go see if we can um work with this just yet so I'm not seeing it it's not aware of the deployment but this is azure for you so just give it a hard refresh as I know we have deployed it and the

UI here is a little bit different than Azure AI Open studio so there it is now okay great and so um we have our system message and this is where you're going to uh ground the the agent so that it knows what it is I I say agent but I really should say llm model but technically it is an agent here um and so what we want to do is we want to um establish what it is so right now like I'm learning Japanese and so I might want to prepare this agent to tell it

that it is a Japanese teacher so let's say uh you are a Japanese uh teacher uh language teacher right language okay language teacher um you are teaching people that do not speak Japanese but English so instruct in English so that's one way that you can write your system message and so that's one of the easiest ways um different models uh prefer different kinds of structures but this will work as a start okay and so we have our system message here and sometimes you have to um uh apply changes we'll click apply changes so that way

it will take effect okay I think it's there now and so now what we can do is we can go ahead and talk to this so hello how are you right and we'll see what happens okay so not exactly what I wanted to do but um what we'll do is I'll tell it to to like so please be ready to instruct me it's like so when I first talk to you please ask me about my about my language goals okay so we'll see if it can understand that a little bit better remember this is GPT

the 35 turbo so it's not as intelligent as gp4 and so I'm going to go ahead and apply those changes and we'll see what we get here so now it is update again so hello and let's see if it takes that here we go hello what are your language goals are you looking to improve your Japanese yes okay great what specifically we like to focus on here and so we can say listening all right so you're getting idea of prompt engineering this is just the start of it now I don't like to form format my

documents this way I like to do it this way I like to say rule profession okay Japanese language teacher um and then we could say outputed format um or we could say yeah output or we could say like tone we say tone we say rude rude and concise and then we could say something like oh put format and here we would say we would do something like um uh always bold every other word okay and then we could say instructions when first greeted when you receive your first message or direct um user towards Japanese learning

goals and so this is this requires a lot of um attempts and work to to make a prompt template or prompt document work really well for you I just clear the chat here because we are starting over I'll just say hello and let's see if it if it works with that okay what do you want I want aren't you supposed to do something okay yes I teach Japanese what's your problem oh right because it's rude uh I would like to learn Japanese right because it was talking rudely to me well good for you what's your

goals for Japanese I don't know do you want to travel uh we'll say uh travel right cool so you can see that um just by doing this we're getting the same thing so whether you want to have it more conversational or what whatever it's up to you these prom documents can get really large um if you really want it to do really well you can give it um example prompts uh different kinds of models will have different kinds of syntax I don't know if open AI has any open AI uh or sorry GPT 35 uh

prompt example format that syntax sometimes there's like a syntax you have to use to effectively prompt it and I know we're looking at an older model but here notice that they might suggest a format so I'm not necessarily seeing one here but what I'm talking about is like if let's say we use something like coh here and we go to its documentation and go co here documentation and we were to go to somewhere like docks I'm just trying to show you what I mean about like formatting a doc a dock okay so oh is this

the docks oh I guess they changed it it didn't look like this before I'm a little bit confused um but what I'm looking for here is like if we go to um prompt engineering I do not like this new format TR again Co here um documentation is that really the new docs it is and so in here they they uh they talk about prompt engineering well may can find example so I'll just say coh here XML prompt example examples constructing prompts go down here to examples well they're not really showing it but basically the what

I'm trying to say here is that they might do something and below and just say like we'll say like prompt prompt what a terrible example here prompt and then inside of that we would then have um like question and answered so the idea is that you tell what's a good prompt and what's a bad prompt and so that's something that we might want to do here so I'm just trying to think of how we could um uh structure that so I'm going to just say like um good prompt examples and here I'll go like um

hey loser Al so like for example we'll just say uh user and we'll say assistant cuz remember that we are the user we're the one inputting stuff and the assistant is the one that's talking back to us and so I'm hoping that this model is going to pick up on what that means and so we'll say user um hello and here we'd say you woke me up or so I'm trying to think it's something rude so hello and um you know maybe maybe we get a bit help here and I I have Sonet here on

the left hand side um I'm going to say I need help creating prompt examples for chat or for GPT 35 okay and so I'm I'm using a uh turbo so I'm using a chatbot to or sorry not I keep saying chat bot but these are chat agents they're not Bots but um I'm going to try to use it to help me generate some out and I'm going to give an example so this is what I want and so I'm going to say um user assistant um for format here is my prompt uh example information to

help you help me generate this out and we're trying to use a more intelligent agent um to work with a dumber agent here so I'm going to go ahead and grab this this here okay and hit enter I'll just say give me 10 examples let's see if we can do it okay but okay but you aren't providing the assistant yeah yeah there we go okay but I don't want these numbered and please put assistant on its own line so you can see like how I could have made a prompt document to tell it how to

help me make prompts and so I'm kind of getting format so can you teach me where about basic Japanese and so here are some examples that we have here and we'll go over to here and we will place this down below here and these are good prompt examples right um now it doesn't have Bolding so we cannot see that here and also how does it know that it can bold so that one's kind of useless for the outputed format so we might want to change this and say always provide a word Bank um in a

uh always provide a word Bank okay okay so that might be something that we might want to suggest there so I'm going to go here and just break these into separate lines a little bit so we here we have user assistant user assistant um and honestly I don't really like these examples so I'm going to go here and just say say hello and then this assistant will just say go away unless you came here to learn and then I'm going to um switch over to my Japanese keyboard here and switch over to here I'm going

to type in neon go which is the Japanese language and then we'll say I don't know I guess we could just make one example here but I I figured what would be good is if we gave it a um uh what do you call it a vocabulary list so here we'll say something like uh nin hon and then here we have the language Nong go so I'm just going to type in nion go and maybe we could even tell it what we want to do so maybe because we don't know the language we always wanted

to supply us with um uh what do you call it uh romanji so we have [Music] neon neon go sorry and then down below here this would be nion go so that may be Lang language and then here we'd have Japan and um and maybe we would have the word for learn so I go here and say learn or so Japanese dictionary I should know this word but uh take a look here to learn is just make sure that it's [Music] correct yeah we'll use this one here here which is well we can play the

audio and we'll hear it manabu manabu so we'll take manabu here we'll go back and technically this would be something like nihongo me what was it again this one here is manabu manabu okay manabu but technically it would get uh conjugated so here this wouldn't be exactly accurate but um we'll go over here and just say learn Japanese translate to Japanese this model is a bit smarter so it actually give us a proper example oh okay so we are pretty pretty close we're just missing the O okay so we'll go here and this would be

o even though it's wo um and then the idea here is that we can take this and and put this here and we'd say um oops to learn and the funny part is like natural speakers might even admit the oh so I wasn't totally wrong so here's an example of a good prompt but we only have one and normally you'd have to give it more than one like five or 10 and you could also give it like a bad prompt um but you know if we use this and we get a bad prompt then we

could then bring it back here and see if it works but let's see if this works for it okay um so I'm going to go here and just clear it and let's let just say what's up let's see what we get stop wasting time and start learning Japanese unless you don't want to learn don't bother me okay yes I want to learn finally some motivation let's get started here are some basic phrases to learn kichi arato oh Mas goodbye okay don't just say okay and do nothing uh please practice phrases until you get them right

hello kichwa arato Ohio okay I did that good now practice introducing yourself so here we kind of have a model and you can see it works pretty pretty decently and the great thing about these more inexpensive models is that if you fine-tune them you can get them to work as good or even better as these other ones and that has to do with adding data so this is our start here and I kind of like our prompt document so I'm going to copy this I just don't want to lose it and we're going to go

over to um our GitHub repo here and I'm just going to press period here I'm I'm going to want to store this um in here somewhere so we'll go ahead here and just say um I'll make this and this will be oh llms llms and I'm going to make a new folder here and just say um basic as our start here and I'm going to call this prompt document or prompt template you call whatever you want right now this is just txt format because not working with anything else you could technically format this is a

markdown file I like feeding um markdown files quite often um but if we go back over to here okay so we can see that this model is in good shape um so I'm just trying to think about this for a moment um yeah so one thing I I think would be nice if we could save our prompt for later and I'm just want to see if we can do that in the prompt catalog so I know that these are use cases of promp s that we can use but I wanted to know if we could

actually save our prompt here but it doesn't appear to be that way and the reason I I was looking for that is because over in Amazon Bedrock you can actually save your prompts for later and so that's what I was hoping for but appears that these are just example prompts so that's very unexciting um is there anything else that we might want to do here I mean there's lots that we want to do like deploying to a web app maybe using promp flow adding data um and there are these other sections where we can um

do some other things we go down examples ah so here we could provide examples let's go here add examples to show the chat what responses you want it will try to mimic any response you uh add here so make sure they match the rules laid out in the system message what's interesting is the fact that I mean it's only showing one so it's not very clear like if I add one here can I add more than one doesn't really appear that way um but it would be interesting to see how we can work with this

programmatically if go to view code ah here it is so maybe before we wrap up this video let's go ahead and see if we can work with this in a programatic way so I saw this running from a previous lab and so I'm just going to open up um Jupiter lab um we did this in the initial setup so if you haven't done that go back to that video and watch how to do that and once that is there we're going to open up Jupiter lab and I'm going to navigate to our AI directory I'm

just doing everything in AI here today and so I'm going to make a new folder here and call this um basic and in here I'm going just double click and we're going to make a new notebook and we'll just call this rename this whoops select that and rename it my JP teacher okay so what I want to do here is I want to um bring this code over and I want to break this up a bit because I do not I do not like uh everything just crammed here together so we go ahead and do

this and no is calling it chat completion so I'm hoping that it does more than just um completion and it's actually um having a conversation there is a difference between a uh chat completion and completion completion just finishes to the end of the sentence it's not having a back and forth conversation whereas chat completion is going to have a conversation um so you know working through just just to break this up here there we go um and then we'll add one above this we know this is going to be pip install open an AI there

we go and we'll run this next um and we'll go down below here and so now we just need to replace these two points and it looks like it's already kind of helped us out here so I'm just going to go and do this because I don't need it as um like that we're not grabbing anything from environment variables here today so those are set the only thing that's missing here is the credentials like we have an endpoint and a deployment but we don't have in here um the API key so how does this know

this is working right so I'm going to go back over to here and we have the API key but they didn't show us how to load the API key here which is kind of annoying um so I'm expecting this to fail but let's just go proceed through this and see what happens okay so we have this here and notice this is matching the name of the deployment when we deployed our model we could we could have given it a different name and it would have done something different there we'll go ahead and run this we'll

go ahead and run this and we'll go ahead and run this I'm not expecting to work I'm expecting this to say hey you need an API key right it goes down here and it says okay we'll ignore that yep you do not have permissions so the code here is fundamentally flawed so we're going to need a little bit of help here I'm going to go just copy this over to Sonet and I'm going to start a new conversation I need help with with this code okay and I'm just going to submit that code over to

Claude we'll give it a moment and so from here I'm going to tell it um it's not I need to set the API key but it's not anywhere in the code and hopefully you can find that for us here and so hopefully we can just utilize that so we'll go back over here and I'm just carefully looking at this here so remove the lines token provider so we'll get rid of this and it also says get rid of this because we're not using Azure identity to authenticate we are going to grab this here and place

it as such take this out of here I like how they show the API key up here but it's like they don't give you the code for the API key it's totally backwards and so I'm going to go ahead and just do this that did not copy copy copy copy copy there we go and we're going to go back over to here okay and that looks good to me I'm just going to run this again and this one again and so here we are going to provide the API key instead of um the token provider

so we'll run that there and now we'll run this and see what happens great and we'll print out the contents and you can see we're getting information back okay so pretty clear um you know for this to be conversational I think the way it works is that we keep um appending the conversation to it so that's normally like we have messages and I imagine that we would have a chain of messages here so um here I might say I'm just going to go this out here and just say hello and let's see what happens here

whoops we'll go here and just run that I'm just going to grab this here so we can just see it quickly like this and here it says message hello how can I assist you um so I might just want to look up well we have completion here but maybe there's a way we can carry this conversation so I might go back over to here and just say okay how do I carry carry the conversation past uh with the completion object because we're obviously creating a new chat um and maybe there's a way that we can

just keep feeding stuff to it and here let's see we have completion and then return it and then we're just adding a message here it says roll system uh okay great so from this example we can go and make a roll here and just say roll system and now we can provide the content of our um that file there so I'll just say prompt document so I'm going to go over here and just say so I want to load a text file and pass it and um pass it as the prompt document my primary language

is Ruby so it's not that I can't figure this out it's just if we can give the code here we'll bring it over here quickly it's just generating out here on the on the right hand side and so I want to load my prompt document I really can't stand what it's trying to make me functions here but I'm just going to go down one here and we I want this code here which is for loading a file and then I need the file path so I'm looking for that here for the file path okay I

I suppose I could do that it'll say file path I'm just going to hope that this is um relative and I don't have to do anything else extra usually you do like path join whatever whatever whatever and we've done that quite a few times before I'm going just open up and see yeah maybe I can just do that here quickly yeah there's something like this there's a few ways that we can do it you know like this maybe not the best way to do it was how I'm going to do it here um and so

here we could just say prompt document txt we'll go back over to um here I'll just get this out of the way and what happened to our prompt document it's a good thing that we saved it because it looks like it vanished and by the way I guess we could have exported this this is a prompty file document it's not necessarily a standard yet or a Json file and it would have given us um all the information we fed in here and we could have used it later on or we could re-upload it again but

um you know I don't know I just like saving them individually like this I find it's easier to work with them in this format but I'm going to copy that content from the repo it's good thing I uh kept it over here and we'll go back over to here and I'm just going to make a new file I'm going to call this one prompt document double click into it and we're going to just paste the contents in here and I'm going to go back over to here and yeah so this I'm hoping will load well

we don't really want to return the file in this way uh we just need to save its content so down below here I'm just going to call this prompt document like this equals there we go and still doesn't like it no file or extra mentioned here um okay what if we just we just do this maybe can I do this no such file directory prompt document I'm just going to rename this to make it really simple it will just say system and I'll rename this to system see if I can just do this system sometimes

simplifying just makes your life easier no such file directory system so I'll go back here and let's just try system like this does that work no does this work no I'm going to look for another one where I have an example um so let me just see if I can find an example here where I'm loading in a uh file yeah like I I do just that like path OS join just gonna save this file here quickly and I'm going to go back I'm not sure if it matters the context on here on the left

hand side basic oh this is supposed to be under llms by the way um well it doesn't really matter but I'm just going to make a new folder here and just move this around llm and then move basic into it there there we go okay maybe this context matters maybe that's a problem now that that that prompt file is not in there so maybe I created in the wrong directory maybe that was my problem the entire time maybe you've been watching me going Andrew you're crazy are You' been paying attention to where you are no

I am not that's probably what my problem is there we go okay great and um we'll go back over to here I'm going to run this and so now it's loaded that file and supposedly we are going to uh pass it along but the idea is that we are telling it the order of messages that are happening here right so here we have roll hello and and user okay we'll bring this down here and so we're going to save this run it and uh it's having a bit of trouble because I think that we moved

I moved the folder and so that's kind of my fault so I'm just going to cut cut this part out here sorry my fault and I'm just going to close this I'm going to say discard I don't care about this one I'm going to double click this one and I'm going to go down here and I'm just going to paste our update here my apologies and then paste this one in here looks like it is the same still okay great there we go and so now we'll run it here here it says invalid syntax um

fair enough it should have a comma here and we'll go ahead and run this and we need a comma on the end here so we'll try this again and lots of formatting issues here with me today so we have I don't have time for small talk if you're here to learn then pay attention otherwise what whatever and so here we have a reply so let's go back over to here and take a look at what they're suggesting that we do so here we have um complete completion and it just Returns the results so I think

every time we want to talk to it we basically have to keep feeding stuff into it also notice that we can change some of our parameters if we need to um message append yeah and so really that's all we're doing is we just keep appending the message so if we want to continue the conversation programmatically what we're going to do is we are going to copy this here okay and then go like this and then say assistant right okay and um we have some weird highlighting here so I assume that something is wrong I'm not

sure is because we have a double quot oh we have a double right here there we go and it is still complaining so let's just carefully look at this where is our problem I don't see a problem here but it it seems to think there is one do this no I'm just going to go here and just take this out for a second that and uh no I don't I don't think there's a problem I think it's just it's kind of messing it up so we run this again what do we get we get an

error perhaps you forgot a comma I don't think so just give me a moment to find out my problem okay okay so maybe the issue is this one here in the end nope well I mean this one doesn't even need a comma there what does it not like I cannot oh you know I found I found out what it was I found out what it was I'm missing um one right here right here see there we go okay great so here we are now feeding back and saying this is what the system says so what

would it do if we do this I would expect it not to do anything and so here it just says stop um because there's nothing for it to do we're waiting for the next roll and so I'm going to go here and we'll do that and we'll say um user and we'll say something like oh this is hard so we'll go ahead and run this so it says stop whining learn new language is supposed to be hard okay and it kind of feeds in information as we're working through it which is kind of interesting so

um you know hopefully you understand that the idea is that we're we're feeding information into it and the longer the conversation goes the more we're uh the more information it is and then we're going to probably reach an inass um so understand that when you're working with this stuff uh you'll get to a point where the conversation will get too long and so basically you have to start over and when you start over you have to provide it a prompt document and if you wanted to contextually understand all the stuff then you have to summarize

that previous history okay um but hopefully that gives you kind of a starting point of um how it is to work with um uh llms but that's not all we're going to do here but for now this is enough so I'm going to go ahead and um copy this over okay so I'm just going to clear out uh um our keys here for now and just going to go ahead and reset this great and I'm just going to go ahead and download this file here and we'll go back over to here I'm just going to

rename this to um system because then it'll just be easier to work with if it's called that I think so I'm going to look for my downloads here and bring over my Japanese teacher and notice that it is oh that's some of the output it's not that big of a deal as long as the password's in here that's all I care about if you want you can uh stop this I'm not going to stop this because I'm continuously working with this so I don't want to constantly yeah you know I'm going to stop it I'll

just stop it in next video I'll have to spin it up I suppose but I'm going to stop it here just in case I walk away um but I'm going to leave that model deployed because I I want to do maybe a little bit more with it there is no harm with it being deployed uh because I'm not currently utilizing it it's based on per tokens but anyway I'll see you in the next one okay [Music] ciao hey this is Andre Brown in this video I want to go ahead and try to deploy a different

model other than just open the eye now the exam does not expect you to know this I'm just showing this uh to you so you can see what it's like to experience another model you absolutely can just watch and see how this goes um but there's a lot of models that are available here and one that uh might be fun to use is coh here so coh here is a Canadian multi- International company and they have a very decent model um and so they have a bunch of stuff that are available here some of these

are made available now they're not open source but you can download and use them such as command R command R plus things like that I'm not sure which ones are available uh for deployment but uh we'll give it a go here in this region knows that there are some that are specialized so here we have text classification embeddings and chat completions embeddings means that it will output embeddings which then we could utilize to store in a vector database that can be leveraged um for other models I'm going to stick with chat completions I'm just go

with command R and see if we're allowed to deploy it and um let's see what we have here so over here the pricing it looks like we have servess API which is good and we're going to be paying per thousand per token so pretty straightforward let's go ahead and take a look here so notice that the idea is that we have to accept pricing terms um and we're basically subscribing and deploying but again it's using a service API so it's whatever we use here is what it's going to be here it's telling us that it's

available in very specific regions right now we're in West us so we can absolutely use that so we'll give it a moment um to deploy and then we'll give it a try okay so I'm just going to wait here as it deploys all right so looks like our model is ready we'll go ahead and deploy that now whether we'll be able to use it in the playground is one thing it looks like they have some code sample repositories that we can go ahead and take a look at um so this one says well this is

with prompt flow but it's not clear whether this is using coh here or not so I'm not sure if this is going to be very useful for us yeah I don't know so I would probably say that the example they provided to us is not that useful I mean if we're doing Rag and prompt prompt examples that might be fine but this seems like this is just generic things about application development okay but it looks like this is still provisioning so we're just going to have to wait a while for this to uh finish I

thought it was already ready but apparently not go over to our chat here I don't think yeah we can't use it yet so we'll just hold off on that okay all right so after a short little wait our um model is deployed let's go see if we can use it in the playground and if we drop this down we can switch over to command R this says Iams I'm not sure what igrs is commands well anyway um I want to see if we can provide it our promp document from before so I'm going to go

here um in the Azure examples repo here I think I might still have I'm going to close that out because we're not doing anything programmatically here today um I'm just going to place this in here and see if it accepts it okay we'll just say hello go away I have no time for pleasantries unless you can came here to learn Japanese in which case please choose a topic to get started and so right away I would say the experience from from GPT 3.5 to this one is much better because here it's better at directing the

conversation and this isn't even command r+ this just command R um so that's really interesting so here I need to choose one and so you now if this is my teacher I wanted to be like hey you have to do it in Japanese so I'll say for beginners for beginners so we'll give that a go and see what happens here's a word bank to get you started kich Haj oh nesa and then we have which color is this book go practice okay I did that and so the only thing that sucks about this B um

is that it's not really reinforcing any kind of learning it's just saying here's stuff now do this and um it's not testing Us in any way so this is where there is more room for improvement but you can see by running it across um different types of models we're going to get different results um so I think that's kind of interesting I'm not going to get rid of our deployment right now I don't think this is costing me anything because it's using a seress API and it's just communicating with um coh here but um yeah

that's all I really wanted to show you that you can deploy other models other than the open AI ones what that process looks like and there you go [Music] ciao hey this is Andrew Brown and we are picking up off uh where we were working with our model previously um we ran it against cohere and we also have it for GPT 35 turbo I think I like the cohere experience a little bit better but I'm going to stick with GPT 53 turbo because I want to deploy this and um if you look here at the

top we'll actually have uh way fewer options for non openai models um and so up here we have uh evaluation and deploy a model right now I just want to deploy it to a web app and see if we can do this last time I tried this this absolutely did not work um but hoping that it's going to work this time and so basically this is going to create a web app so your web app will be configured with Azure active directory it may take um a few minutes to apply I like how it's saying

Azure active directory Azure ad which called Microsoft Entre ID that just can't seem to ever update the names everywhere but uh it may make take a few minutes to apply after the deployment uh and you will not be able to chat in the app please wait 10 minutes then reload the app and log in to begin chatting so we're going to go ahead here and just say my JP uh teacher um I'm going to go and drop this down to my Azure subscription and by the way if you're worried about cost you don't necessarily have

to do this um but um I want to deploy this here today now it wants deploy in a resource Group so I'm just actually going to go ahead and make a new one because if I want to tear this down really easily I just want to um have access to that so I'm going to go here to Resource groups and we'll go ahead and make a new Resource Group I'm going to call this one um my JP teacher app okay this is in West us hopefully we don't have any problems in West us and we'll

create that usually they create very quickly and so I'm going to go back over to here and we're going to attempt to deploy this so we'll give it a moment here to load and we'll say deploy a web app my uh my JP teacher and we're going to drop this down subcription one and we're going to go to here we're going to go to here and I'm going to say West us assuming I have that as an option West us there it is and we'll choose the free tier if we can enable chat history in

the web app um I mean it's nice that we have that but I I'll just not do that here today I just want to keep it simple the deployment Cosmo DB is not going to really cost much but I just want to have less moving Parts here I just want to see if we can get this deployed and uh get this experience here so we're going to wait for that deployment to complete um I imagine that we might be able to monitor that somewhere here yep and so we're just going to wait for that to

finish okay all right so according to this the deployment is complete um and it looks like that we can launch um our application now it did tell us to wait 10 minutes afterwards I'm going to press it anyway and see what happens if it doesn't work that's fine we'll just have to wait 10 minutes and find out uh if it is in a working state right now it is uh thinking really hard but the question is what was it deploying underneath so if we go over here we have our JP teacher I'm assuming this is

azure web apps yep it's the app service um and we chose the free tier so it's no surprise this is running really really slow as that is what the free tier would do I kind kind of regret not making it the basic tier um but if we go into here maybe we can take a look and see its status so it is currently running got don't know um so I'm just going to wait here and eventually I think something will appear so just hang tight and I will be back here when something's working okay all

right so after a while we can see that we can now accept permissions took quite a bit of time here and we'll go in here and so now we have a nice interface uh we have uh kosa which is the pretend company that is always used in all the Microsoft um examples we could say [Music] hello and run this here says hello how can assist you today um and I mean the model we deployed it doesn't have it doesn't have our prompt so just remember that we created before and I we didn't put our prompt

in here so I was hoping that it would have that Japanese conversation uh before but it doesn't um and that's totally fine but notice that we could share this app to other people um but I think that because it requires a Microsoft identity I would think that they'd have to be in your active directory for you to get access but I just wanted to show you that you can do that um if we were to update this let just see like uh we'll just say or like tone you talk like a pirate okay and apply

changes I don't know if this would um take effect to the launched web app because how would it but I guess if it's referencing this model I I wouldn't see why not so I'm just going to give this a refresh here maybe we'll restart the conversation hello and does it pick up our changes here it does not so not exactly sure what code is there but if we go over to here to our web app how would we be able to see that good question for me so maybe we go to the deployment Center because

what I'm thinking is that there's probably deployments I just like want to see what the code is so this is the sample repository it's using [Music] okay yeah so it must be referencing it must be versioning something and doing something specific I'm not sure um because we have all these configurations and so I imagine that these are probably being passed as environment variables that's all I can really think of here so let's go back over to um uh here it's just somewhere here there should be environment variable storage you'd think I would know this because

um I've made videos on this but I don't remember everything aha there we go okay so we have lots of environment variables here and so what I'm wondering is is there ones for the actual um agent itself so let's carefully look at this suff for Cosmo DB even though we're not using it we have the open AI endpoint let's look at that value so that's just the service endpoint the embedding name nothing interesting there we have our key which is my actual uh open AI key here the model which is Turbo 35 and just again

slowly looking for the information so here we have system message so here I would imagine right if uh we wanted to and we changed the system message whatever that was system message where' you go that this is how we could change it right so maybe what I can do I'm not sure if this will work but I'm going to try I'm going to copy this and paste this into here and say apply and apply so you your app may need to restart for the settings to take effect that is totally fine I can imagine that

it won't take effect unless I restart the app but it's just good to know like how that's being passed through because you know it wasn't like it built any kind of code or anything like that was using generic things I'm just going to say hello and I really don't think it's going to pick up right away okay and so in here in um our app I'm going to just go ahead and give this a restart okay and it says that it's restarted I cannot imagine it's restarted that fast maybe it has so we'll say hello

and communicate to it here fingers crossed that it works sorry what okay so I was assuming that it was going to pick up that information there and the domain is the same it's not like it changed so go back to environment variables here let's just make sure that that's been set so we go here to system Azure open a AI system message show value so it is set there again not super important we could just redeploy it but it just would be nice to know that we could do that so yeah I'm not sure but

that's what I think that it would be the only way we would know is if we did another deployment so maybe that's something that I can do it didn't take too long to deploy but I'm just really curious if we can um uh do that there so let's go back over to chat playground if we go to deploy web app oh we can just update the existing one in place okay great so what I'll do is I'm going to go and paste in um this but I'm going to change the tone so that we know

that it's working um and we'll say tone is going to be like speaks like Yoda from Star Trek from Star Wars no maybe we'll just say like speak like a pirate okay and so I'm going to apply those changes we'll set continue and then here I'm going to deploy the web app I'm going to update the existing one my JP teacher we'll go ahead and deploy and so I'm hoping that that will take effect but not only that that we can go and take a look at the environment variables and see if that's what's being

set so here the deployment is underway so we'll just wait for that deployment to finish okay all right so it says our deployment is done I'm going to go back back over to here give this a refresh I'm assuming that our URL is the same um and it may take some time like before for this to get ready so I guess we'll just wait here a little bit okay so that didn't take very long so I'm going to say hello and let's see what we get we'll give it a moment here okay still thinking now

just remember that we are using this um in basically a shared environment uh or like shared tency so if other uh tenants are using more of the resources then we can uh see delays here but there we go it did reply um so go away unless you came here to learn uh sure you know now notice that it's not speaking in pirate talk so maybe it took my other one so I thought it would have pushed the other one here because we told it pirate but I bet if we go over here I have a

strong feeling that it's this value that I updated here for some reason okay so we're going to scroll on down here and look for um that system option here show the value says speak click apart so it did take in to speak like a pirate information but the thing is is that um it does have that predefined one because we said hello and so it replied as it would expect [Music] here because our example doesn't give pirate talk so yeah here we go uh stop dilly dally and start learning Japanese you you scurvy dog okay

so I think that it's implying or it has implied knowledge that it should speak in a um pirate tone but also that's rude because of our root example so yeah that's all I really wanted to show here um so that is good now here's the question is how could we go about um I stopped that earlier how how could we go about shutting this down now this is running there so I think the only way we can do that is we just need to tear it down and that's why we created that um Resource Group

okay and so in here y we'll go here and we'll just say delete and delete and there you go so um I'm going to leave the model still open here as we have more to do but that's it for now I'll see you in the next one okay [Music] ciao hey this is Andrew Brown and we learned that there are parameters uh which we can adjust for these um llms and I think that we should spend some time fiddling with them so that we can understand how that will impact our model and maybe later on

we might need to tweak these um to change the level of Randomness so here we have this uh GPT 35 turbo that I've deployed earlier there of course are other ones that we can utilize but um I'm using 35 turbo because it is older and so it's a great uh use case for teaching because it can do weird and wonky things there is um chbt 40 mini which is more cost effective and better but again I want to show um how these things can vary so what I'm looking for is the option to change the

parameters and so here we have parameters you're going to notice that we have a few options we have temperature top P um sometimes you'll have access to top K so what I'm going to do is go over to the open AI playground because I want to see comparatively like you might not have access to this but I do because I pay for open AI um separately here but oh right now it's saying I have no models available what are you talking about well that's not very helpful but here on the right hand side you'll notice

we have temperature maximum tokens top p and some additional information I was looking for top K so um like if you use something else like maybe Claude um it might have other values here but generally there will always be temperature in top p so top our temperature is going to control the randomness so higher value increases um the randomness of your answer whereas if you go to zero it's not going to be as random um top p is about um cutting off the token selection so it's sampling from less uh or more depending on what

you want to adjust so I don't normally fiddle with um top P to be honest and they explain down here as well controls diversity via nucleus sampling so 05 means half of likely weighted options are considered so fewer options are considered and then over here um it's going to change the level Randomness but you can read all of these and get an idea of what um they do right but for the most part you generally don't have to tweak these they're pretty good um but again in some cases you might want to dial down the

temperature you can see here that the temperature is at 0.7 so maybe we could say um um a better example might be using this for completion as opposed to chat but I might have to deploy a new model for this so I think that's what we'll do we'll deploy a completion model as opposed to a chat model completion models are older but I think it might make this a better use case so I'm going to go over here to our model catalog and I'm looking for a completion model so completion model just says completion so

we have babage Da Vinci I think babage is cheaper and so I'm going to go ahead and deploy this model let's go ahead and choose that let's play Clos to attention to the deployment um typ or method so here's this model is not available in the selected open AI resource um about connecting here so this is where certain ones are available in certain areas and and other places so right now I think I'm in um which which one am I in I'm not sure but uh you know if we're trying to utilize let's say babage

I could find it here it is available in here and here here so we have it in north central us and then here we have this in Sweden Central so it's really interesting where we have access to something and when we don't so I'm going to switch over I guess to Sweden um and I guess we basically would have to make a new Hub to do that so I'm going to go over to Hubs here and notice this one is West us I'm going to go and create a new hub and this one in particular

I don't want a new Resource Group I have enough of those um I'm going do Azure Studio Hub here I'm going to switch this over to Sweden Central and here it's going to probably have to create a new one I don't know if I need Azure I search no we don't need Azure I search but we definitely going to need this so we're going to let it create that and I'm going to go ahead and hit next and and create there we go and we will give it a moment to create okay all right so

our project is ready I I believe we're in that Hub right now I probably should have named it with Sweden in there but we can go over to our model catalog and let's go take a look at launching babage so we'll give it a moment here uh to load okay so here we have babage we're going to go ahead and deploy and let's see what deployment option as we have probably something cost effective we have standard standard sounds good and we're going to go ahead and deploy this and we'll just give it a moment to

deploy all right that was fast I believe it is now deployed so if we want to utilize this model uh we're going to go over to projects and create ourselves a new project just say uh yeah I don't care what the project is called if you're doing this for serus you probably want these name correctly I don't remember seeing this AI Services here now I really want to click that but I can't write now but I'll have to wait okay so our project is here and uh yeah I'm just curious what is AI Services here

okay so it's just a easy way to start working with these it's totally totally nothing interesting there um but I'm going to go back to all projects here give this a refresh go to my project and uh now we'll go to completions now if we go to chat I just want to show you here that babage probably won't appear because it's not a valid model but we go over to completions we have it completion models are considered older models um if you fine-tune them correctly they can be of use but for the most part they're

not that great um and we have some options here on the left hand side so we have I think this is this is top P right here yeah it is so this is the top p and up here we have our temperature and so the idea is that this um is supposed to complete the sentence so my favorite food is and I'll put a coal in here and so I'm encouraging it yes I'm Canadian so I do r o u t but I'm encouraging it to supply a value here so we'll go ahead and hit

generate like how it's telling you how many tokens down below it says four D noodles give me a little bit too much information not exactly what I wanted um but that's okay so the idea here is that we're already turned down for our tokens here but I guess it's just producing as much as it can right um but let's say we just clear this out here we have 40 noodles let's run this again includ those very random my favorite food is bacon and eggs macaroni and cheese tacos and hamburgers so now it's listing them in

order let's try this okay so again we're getting different things let's try another one my favorite food is six wow terrible terrible terrible terrible but the point is is that babage is what it is um and there are ways to make it better but for now we'll just stick with its uh simple implementation so let's turn this all the way down to zero and let's see what happens if we have it on that so here we have chicken pasta Pizza Pasta those are pretty safe answers pasta is very very popular here so um again it's

not being so Random we'll try this again chicken pasta Pizza Pasta okay again chicken pasta Pizza Pasta so the point is is that if you're looking for something with more reliable results you can turn it all the way down but you're going to get rid of the creativity here so there could be a case where you might want to have it all the way down to zero um and we might utilize that later on so let's bring this to 0.5 okay and uh I didn't mean to uh delete that out here but go ahead what

my favorite food is okay and we will hit generate now H the 0.5 okay still going for check in okay but we are getting more variation here okay so now much more random let's go bring it down to 02 again we're just trying to experience that level Randomness and actually when I when I change the number there it clears it up so my favorite food is okay so still pretty random or PR I mean pretty typical at least we're getting pizza in here and so it's very useful to just kind of like go up the

chain here this time I'm just going to copy this I don't have to keeping it in here okay great we have this still chicken we'll try 0.4 here chicken rice okay 0.5 chicken with rice 0.6 totally weird but hey at least it's doing more random okay so now you're starting to get the idea here um something else we could try to play around with it would be Pro uh top probability so let's go back here and just generate leaving it alone at 0.5 what we get here so here we get chicken pasta pizza great let's

say we turn it all the way down all the way down now what do we get the same okay we'll do this again the same okay and I mean we would expect that because if it's selecting from fewer options I thought maybe it would just be like chicken chicken chicken chicken but it still is the same let's turn this all way up to one let's see what we get now so that's more to select from but it's just still more pasta so we're not really noticing much of a difference here okay so now let's go

and bring the temperature up to let's say 0.7 so that seems to be a good number here and we're going to place the top probabilities back to 0.5 and we're going to go ahead and hit generate okay so looks good so far and we'll do it a few times because we have to run it a few times to find out what's happening sh price Etc so now I'll run it one more time okay muscle so very random still getting lots of pasta let's turn the probabilities down let's see what we'll see what we get now

so notice we're getting more of our chicken pasta thing e because there's less to choose from so now we could dial this into let's say 0.2 favorite food is3 my favorite food is no now it's going with [Music] colors really oh it's kind of following through that but didn't really make sense let's just crank this up and now see what we get probably really random 99 spice spices chicken but hey it's it's nice and descriptive I like how it's like a weird spelling mistake but it clearly looks like training data try this again it's weird

that like with top probabilities and temperature it's it's still coherent and it's more interesting than it was at 0.5 but I guess it has more text to sample right it's weird that we're getting better results like that but I'm going to go back to 0.5 we'll go here yeah so it's interesting like if we want something more conversational more interesting then one and one are interesting so hopefully that gives you an idea of um temperature and top probabilities when you have these more intelligent models chap GPT or GPT 40 Claud Sonic 3.5 um llama 4.1

whatever these things you you don't really have to play with them too much but with these um simpler models based on what you're doing you're going to want to tweak things here I'm going to leave the Sweden around because the Sweden Hub because we might end up using it for fine tuning in another video um I'm not going to shut anything down because everything is servess here so I'm not worried about it and I will see you in the next one okay [Music] ciao hey everyone it's Andrew Brown and in this video we're going to

take a look at prom flow so prom flow is a very powerful tool um for many reasons it's great for prototyping you could do evaluations in it you can do batch inference you can do multimodel uh a lot of a lot of things but basically it's an orchestration tool for you to um work uh take multiple llms or functions um and string them together to uh really really do great um llm development so you absolutely want to learn promp flow I'm not a super expert in it but I'm good enough that I can get you

over a lot of the hurdles and I found that when I was learning how to utilize it they just had a lot of gaps in knowledge so I'm hoping to fill those out here for you so let's go ahead and create ourselves a prom flow we're not really going to use it in here but I just want to get you exposure to it in here and then we're going to programmatically work with it so here when we hit create we have a bunch of options uh we have some standard ones up here and some more

complex examples down below so um I'm going to go ahead and create um a we can do a standard flow for now and I'm just going to write this as standard flow example and we're going to go ahead and create this one here we could also look at another one so while that is creating it creates pretty fast I'm going to go ahead and just clone the chat Wikipedia one so Wikipedia example but personally you know if we're working with prom flow we should start with as little as we can and build up because when

you work with a more complex one um it's really hard to debug it as opposed to one that you build yourself from scratch let's go into the standard uh flow example this doesn't cost any as of yet to utilize it but on the um this might be collapse for you but on the right hand side it shows us the files um that are the orchest ation files to create them down below we actually have a graph of uh how things are connected together and then we have this visual stuff here on the left hand side

so what are we looking at well the first thing is we have our inputs right so this is what we're going to input into our flow and then we have our outputs down below here and that's what we're outputting out of our flow so hopefully that makes um sense there and then here we have this is an llm so this button here would be a way of adding an llm and then you have um a function this could be called function calling or tool use the idea is that the llm is going to go out

to a function right and then uh that function is going to do something and then output our results so what is this llm doing or this sorry this promp flow doing is expecting us to input a value okay whatever the string we want it says a topic and then it's going to go to the llm and the llm has its prompt document which is written in a Ginger 2 file okay this is a templating language so I'm not the best at remembering how this templating language works but it's pretty straightforward and so we have a

system prompt or a system role saying you are a bot that can tell good jokes and the user says a joke about topic please so the idea is that you all you have to write is the single word and it's going to fill it in as the first reply um and so from there it's going to pass the output of the um of the topic here to our Echo command and so the the output of uh this input here is going to go to here and then it's going to go into here it's just all

it's going to do is um Echo it out You Don't See Echo or print anywhere here but when this runs we're going to be able to see that information and then the output will output that result so hopefully that makes sense let's go ahead and and see if we can run this so the way we need to run this is we need some kind of compute attached now when we use promp flow later on we're going to use it in um GitHub code spaces you could do this locally as well but the idea is that

the compute being used will be whatever the developer environment's compute is it's not using compute to to utilize llm it's compute to um make API calls so it's not expensive compute so if we drop this down here we have a compute session so go ahead and click that uh I was hoping to get some options there there but it's totally fine it's going to launch it with serverless and we'll take a look at that later on what those options were but it's spinning up on basically I think it's an um a virtual machine or something

like that and after a few minutes it'll be ready and we can evaluate it so let's just wait a moment for that to be ready okay all right so that is now uh ready and I just want to show you like it's showing us it's running a standard DS uh 12 V2 we're not going to keep it around for very long if we wanted to reset it we could do that we want to stop it we could do that there's additional things here if we want to see what packages were installed um and so it's

showing what python package is I believe these are python packages it needs to run this okay so the idea is that if you need to bring this into your environment You' know what it is but let's go ahead and run this um so I probably need to put a a value in here but I'm purposely just going to run it with nothing in it and here it says connection joke variant is empty so it's telling us that we have to configure the and they aren't configured so I'm going to drop this down I'm going to

choose a connection Noti we have chat or completion and I'm going to uh I don't need yeah deployment name yeah we have GTP G GPT 35 turbo now remember that I have other deployments here if we open this tab up um if we open this tab up we have cooh here and GPT 35 Turbo so I'm not sure as to why I cannot select anything other than uh GPT 35 turbo but um I'm not yeah so I'm hoping that you can do more with more of the models because this kind of forces you use the

open AI models but for whatever reason right now I I don't know how we can use other models other than the open AI line okay we can uh choose our response types so probably want text here um because that means that if we output Json then this thing has to be able to work with Json okay so anyway let's go ahead and um go back up here and try to run it again so we picked that prompt now we haven't filled in a topic so I'm expecting that to complain or it might just work with

with a blank topic and on the right hand side it might show us yeah it will show us here the execution now what I find is in the visual studio code when we go to do programmatically this doesn't update in real time even though one point it did at least did not for me here it says the input of the flow is incorrect the value of input topic is not provided so we have to provide the topic so the joke could be Rome I don't know why I keep going with Rome here uh or we'll

say ancient Rome and we'll go ahead and now run this and I'm just waiting for that there and we'll go down below well hold on we have our input right and then we have our output and I'm just going to slowly go through here and we don't see anything here which is fine if we go down to our function here it shows us um see our output in our Echo function it shows us sure why did the Roman Emperor go to the Coliseum because he wanted to see people get him a thumbs up or thumbs

down terrible joke not that funny and if we go to this one here noticing that we are seeing this output so we can see its input which is topic ancient Rome this is for the joke llm and then its output is system metrics completion tokens duration and this information right so it's only passing this back as the outp out out book output we also have Trace we also have log so nothing went wrong here and so hopefully that gives you a clear idea of how this works now over here we can actually look at um

the flow because the flow is defined in a yaml file um yeah up here the flow. dag so I'm just going to click it turn on raw file node so we can see it and we'll just take a quick look at how this works so we have our inputs right so is it a chat input false so the idea is that we could be feeding back a chat input if this is a a chat conversation remember when we created this we said standard flow not a chat flow so if we made this true then it

would feed it back and then this basically would be a chat bot or chat agent um and then we have our output so this one is a string then we have our nodes and we have a python node right and we're telling it where the source is so where is the python code located what is its input it's coming from joke so this here is our output called joke and we're getting that output there then another note is the llms and this one's called well hold on here this one's called joke so I think it's

actually referencing this one here yeah yeah because this one's a fixed a a um a fixed output and so it's not exactly the same thing this is making output called joke and this here is referring to another node which is joke and this type is llm okay and so here you can see it has a source of joke. Jinger 2 um which is over here that is basically its prompt document and then for our inputs we are using um GPT 35 Turbo with a temperature of one with a top P of one with the max

tokens and then we have our provider connection so it knows how to connect to it and then we have our environment requirements. tit things it needs to have in order to work and it's empty so there's nothing extra that we need I'd be very curious if we could swap out I bet it won't work but I would be very curious to see if we could swap out the deployment name for um the other one here which is coh here right so here we have this one and then down below we actually have our deployment name

here I believe so I'm trying to uh copy the name so I'm going to copy it here and we're just going to see if we can uh do something we're not supposed to be able to do right um according to the UI anyway and so I'm going to click into this and I'm go back to raw file mode Sure Save the changes I didn't see anything changed and what I'm going to do is I'm going to try to swap this out out now the parameters could be different so this could just mess up from the

parameters alone but um I'm going to see if this works and hit save okay and I'm going to go ahead and try to run this and see what happens and we're going to find out here in a moment has an error so the API deployment for this resource does not exist if you created in the last 5 minutes so yeah it's not letting us do it but I wanted to just show um what would happen if we tried to do that now again I don't really like working in this interface um we do get something

very similar in vs code but um you know if you're really going to use this you're probably going to use it in a programmatic way because you want to do Integrations and so you probably don't want to do it here uh there are more tools that you can add or nodes and and these are really interesting I haven't played around with them too much but clearly there is um multimodal stuff so we have one for it looks like video I'm not sure if 4V is video versus Vision but we have Vision ones and ser so

that means it can go out and use Google and look for stuff and looks like we have open model on them so maybe other ones here use an open model from the Azure catalog deployed to the Azure online endpoint so maybe coh here could be used that way it's not technically an open- Source model but you could say that it's an open model in the sense that it's accessible because you can download it uh so that might be interesting uh to explore so I'm just going to click that here for a moment um you know

Co here I'm just going to see if we can even do it and there are no end points found so yeah there's obviously a little bit more work uh uh to be involved so I imagine if you had like an open source model I would think that would have to be on provisioned uh provisioned compute and then it would have an endpoint you would choose that there um or maybe you could have a random compute and attach it there I'm not 100% certain but we got we kind of got the idea of what's going on

here so I'm going to go ahead and stop the compute session here because we basically learned what we want to have learned from here um so I want to go back over to prompt flows and that's fine I'm just going to hit confirm and I'm done with this that that comput sping down I'm just going to go ahead and hit archive and let's go into the chat one this one is more complex do not worry if you cannot make sense of this it's just I'm trying to show you um how complex these things can get

so we have our input here and our input is um it's going to input our chat history so it's feeding back the history of the conversation we're having okay and it looks like there is some uh predetermined um history already in here then we have our question which is going to be the next thing that's asked in the user role so what is the difference between the model and the previous neural network the idea is it's going out to a specific Wikipedia page to help it and then we have our outputs okay so not a

whole lot there but if we kind of like crawl through this we have our inputs and it can either go from augmented chat or to extract so I'm not sure how it decides between the two let's take a look at this first one here so this one here is an llm so it's going to here probably here first right so you are an AI assistant reading the transcript of a conversation between an AI and a human give it input uh from the conversation ation the conversation history is just is provided in just in case of

a conference and so here they're providing examples there's some variations on how you can do this but um this is a way like I might do and just say like good prompt examples bad prompt examples or replies but here we have example of a conversation history with human AI I would probably call it user and assistant but you know it would still work both ways and so here you you see a human I want to find this sure how can help you how do I get to Rock Bar output directions to Rock Bar so it's

indicating um that maybe he has to go out and search for something I'm not sure then down below it has this is our um conversation history for reference only and so it's listing it out here so that um the existing chat history is going to be displayed so we're saying input the chat history and then we're printing it out here so that the bot knows or the agent knows because um I don't I don't know if you remember which video was in but I I explained that every time you're entering or you're sending stuff to

the llm you have to keep feeding it the previous history and so it has to go somewhere and so they're placing it here okay and then the human response is the next one here and then they're indicating hey give us the output right so hopefully that's very clear um then we have get Wiki from URL so here we are seeing what do we have we have entity string count two and if we carefully look at it it looks like it's searching for an entity here how would it know from here to do that so because

we have a question right and then we have this entity here all right so I'm not sure exactly how it's doing that but anyway let's continue on here so it's clearly making a request so it's using requests which is a way of just making requests out to a page it's using beautiful soup to parse the page um so that is a probably an XML parser HTML parser and it goes there and then it's extracting out text um and here you can see that it's looking through through stuff like references and things like that and then

it's outputting information so we have get Wiki U then we have search search results from URL okay so fetch text content from URL okay process search results and then it goes back over to an llm so you're a chatbot having a conversation with a human give the following extracted parts of a long document and a question create a final answer with references with sources and then that's our output so um will this work well with gp35 I'm not sure well let's give it a go and see what happens so before we can do anything we

need some compute so let's go ahead and start our compute this time I'm going to go to advanced settings so you can see so we have serve lless or compute instances here it's 19 cents an hour so you could go to something more cost effective I'm going to just stick with this because I want something that's reliable here today you can tell it to shut off after a certain amount of time um I I'm not really worried about it I'm watching it very carefully we have the base image let's go ahead and do that and

we'll apply and wait for our compute session to start okay so we're going to wait for that okay all right so that compute is uh now ready and so we're going to go ahead and uh get the chat now I'm not saying that this is going to work for me I'm going to say let's explore it and try to make sense of this more complex model again I think that if you're working with this it's better to uh build up um from a a base flow so you can really see what you're doing so here

we have an existing chat history that was already inputed for us okay and then this is the conversation that's already being here so let's go hit enter it's already going to complain because I have to configure I have to configure these connections so we'll go ahead and we'll first try to use a GPT 35 turbo um I think the response is supposed to be text because I don't think it is taking in Json but we'll just go through here and figure this out we do have two L llms uh nodes in here and so we

can go ahead and select those there so just give it a moment there we go and we're probably going to try this out with gp4 because I'm not sure if this was ever designed to run with three 35 but we'll see what happens here we'll go ahead and hit run and I think this time it should work or it should do something now there's some way to trace it sometimes it shows here and says you can trace it here I don't find that experience being very good and it's also in preview so we might just

have to walk ourselves through this to find out what's going yeah has the view TR I'm not even to bother clicking that because it doesn't work half the time and um I want to see something here but if we don't see oh look it actually came back wow so when I ran this before did not come back with anything so just understand that if you having issues with this you don't see anything here I had the experience before but now it is working so I'm sorry I don't have specific information technical differences between gp4 and

previous neurals I'd like you know more about the technical distinctions and it got the sources so it actually did work okay so I thought it wasn't going to work but let's go through here and walk through what's happening here so we have I'm just close this out here and go to the top so we have this and I think that it will show us the outputs here yeah and so we can kind of look and see what the input is so the input here is what is the difference between this mod and the previous one

and then we have the chat history okay and this is coming in as Json and it is Json up here right and then down below or sorry over here we have our output so our output here is can we do view full output there we go output is it seems like the user is asking for a comparison between the current model OKAY the next query would be comparison between GPT 4 and previous model let's go over to here let's go down and take a look at its input go here and it says it seems so

it says entity so even though we chose text it's still uh The Entity is the so I'm thinking basically the first parameter is whatever because you can name it whatever you want maybe because the other one it wasn't called entity it was called something else so it seems like the user is asking for a comparison between the model and the previous neural models the next query could be Etc count two okay and then we have our output here so I want to go and take a look here because we have entity right oh okay you

know for some reason I thought this was like very specific I thought this was going to be something like Rome or something but here it's actually searching Wikipedia and then returning the results that's why I was confused earlier I I didn't clue in that that was doing a search so that actually now make sense and so we have our output here let's open this up and so here it it we have this and you can see that's putting this in here here as a result and we could probably grab this as such and try it

ourselves I'm not sure what we get as a result can we do this and so this is what it's returning back these three options okay so that's starting to make sense and then we're going to go down to the next one here let's look at its um inputs and so in our inputs it's bringing back that list of URLs and so this one is supposed to search results from the URL so then I'm guessing what it's doing is it's probably going to go into each one in parse it so we go here and we have

yeah so it's grabbing I think each of this and I think it's trying maybe it's grabbing this text or it's grabbing this maybe it's trying to get the links to it it's going beautiful soup P stuff and then it's going so I think it's going to yeah it's going to in each one of these URL so I think it's extracting the URL out here going into here and then it's looking for sources maybe references or sources here somewhere in here and it's going to try to return those so [Music] P you know if you wrote

this yourself you probably know because you'd be walking through it right and so these are functions but this tool marker is telling it which function is actually being called so I believe that's all this does here you mark that there and so all these other functions are external but this is your entry point into this one so here it has URL list and the count um so if we go to our inputs here yeah we have URL list and our account and then we have our outputs and so our outputs is outputting whoops let's just

open that up it's outputting us some links or at least it's outputting one so we have one or this is our results here it says there were no results matching your query oh really so let's go back ah it says here I mean there clearly is three here it seems like asking comparison between okay so I don't know it doesn't look like it perfectly executed but I mean it's still working right we'll go down to process our search results so we have content and source and URL so this looks like it's just formatting to content

and source so we have our input here there search results not exactly what we wanted I don't think and then we have content there were no search results and then the other one here source is the URL and this one's the augmented chat so you chat bot having conversation so now it's taking in um that input so let's go to our input here question chat history and now we're providing more contextual information like the sources probably and then we get our output okay so I think in theory this thing did not work correctly um we

could deploy the other model and and see if we get better results or we could also just change our um uh our input here but we'd have to take a look here and play around with this we'd have to say something like um when did Rome yeah when did Rome fall right if I do this what do I get okay so this might be more interesting I want to go back to the chat here all right so I'm going to go and try to change the question I'm not sure why this is like fixed like

this re restart I don't to restart the compute session but I'm trying to well let's just put the next one in here let's see what happens so I would think that it would just start where it was right back yeah now I put the input put in here okay great so it just replaced it which is fine and so we'll see if we get anything here and so the results back here are not great right so sources Wikipedia didn't really help like I think we're expecting some some text here and then the source so what

I'm going to do is I'm going to deploy a gp4 model if you're worried about cost don't have to do this but I'm going to go ahead and deploy um I mean do we need gp2 40 or four cuz oh these are uh Vision models also I'm not I don't think we can deploy this other than in batch but let's take a look here I would like to deploy for mini we'll take a look here and see what we have Global batch and that's not going to work for me here today um so I'm going

to go back to our model catalog and even though GPT 40 is overkill I'm going to deploy this one [Music] and hold on here let's just read it by simly matching the intelligence of gtp4 Turbo it is remarkably more efficient uh delivering at twice the speed at half the cost okay so yeah I'll go with global standard which actually is really good we'll go with global standard here and what we'll do is go to uh prom flow I believe that's deployed it's pretty fast to deploy those and we'll switch out we'll switch these out here

so we'll go to gpg 40 and then down below here we'll go to GPT 40 and so we have those there and now I'm going to go and um save this we made some changes here I feel like if we change it we probably have to deploy it again deploy your flow to a manage online endpoint no no no no I don't want that um that's if we wanted to make it available to an end point that we could hit I mean that'd be interesting to do but not necessarily something I want to do this

moment and so I'm not sure I'm going to restart the compute session I just don't trust it to um be utilizing the correct things so I'm just going to restart that and give it a moment okay all right so we restarted that um and you have GPT for o down here and we'll go and check the other one so I'm just seeing if we might get a better result here um so let's open that chat again when did Rome fall I'm assuming that it just doesn't know but pre previous history hopefully it's not picking up

any of that I'm not sure how we reset the chat there's no option to reset it here and we will see what happens hopefully it has an easier time I'm not really worried if we can get this working because you know I said before I had trouble with it but the fact that I was actually getting a response was interesting so just curious to see if we can get it to to work here but you can see it's really working hard um and this could be because of the models uh being on basically they're on

shared compute um with other customers and so if they're being heavily used for whatever reason sometimes we just have this thing where it just keeps running and we don't have any results right so I'm just going to hang tight and see what happens here okay all right so I've been waiting quite a long time and it's just not completing so this is where we can run into struggles with it and again this isn't something that I've built so I'm not that that interested in uh getting this to correctly work but it was an interesting way

to go and explore working with um prompt follow so what I'm going to do is go ahead and just archive this I also stopped that compute there and we're going to stop the video here and then the next one we're going to work with it programmatically so I have a repo here that I've already created it's just a plain repo that's private you you're going to need to do the same but um yeah we're going to learn how to get set up and try to build our own promp flow programmatically as opposed to just in

this UI okay see you soon [Music] ciao hey everyone it's Andrew Brown and we are continuing on with promp flow this time we're going to work with it programmatically as that is uh a more common way that you might want to work with it so I've created this GitHub repo that is private you need to do the same super simple to do uh but what's going to be a little bit different here is I'm actually going to use GitHub code spaces now could do this locally but um I want to create an environment that is

replicable for you so I recommend that you follow along as well in GitHub code spaces they have a free tier um so you should be able to utilize it and it's going to spin up the first thing I'm going to do is just change my theme because I need to see it in a darker theme so I can see what I'm doing here we'll have to give it a moment to load here as it is figuring out what is going on so we'll just give it a moment and we will try this um again go

to themes we'll have to wait okay all right I gave it a little bit of time to load here now let's see if I can change my theme color theme there we go okay great so I'm going to switch this over uh to T to dark and on the left hand side here I'm going to search for prompt flow as they have a vs code extension to make it very easy to work with there it is I think they've changed the logo since I've last seen it which is totally fine but we're going to go

ahead and install that and give it a moment all right so it says it's installed on the left hand side we'll go over to here we'll go to install dependencies as we have to make sure we have a bunch of stuff ready to go I'm going to choose our python interpreter I'm going with uh not with cond but just the regular uh 310 here there is 312 down here if we use cond but 310 is totally fine for our purposes here today and then we need to get a bunch of these um things installed now

we're not using uh uh cond so we're going to skip this step and we need to install pip install prompt flow and this is something that I found that uh no videos were showing which was the setup they just completely skipped it as like no ready to go and I I found there was a lot of friction here so hopefully getting people through this part will make their lives uh super easy yeah we just have to watch each one here the first one takes a bit of time to install not that bad and so we'll

go to the next one and you pretty much just go down the list here clicking each one be very careful and make sure that you get them all here we go and we can do this here but um the way I'm going to check to see if all these are installed once this one is finished here once it's done here there we go then we can go all the way to the top and we will um hit this refresh and it will tell us hit all of them here whether we have all the things that

we need because it's possible you might have missed something there so just make sure you check that we'll give it a moment there to think there we are so we technically should have everything installed I know there's actually one thing that we're missing but we will figure that out as we experience it so what I want to do here um is I want to create a new flow so on the left hand side we'll go here and we have some options empty flow standard flow with template or chat flow with a template I'm going to

go with a uh chaff flow with a template here today and it's going to ask us where we want to place this so we're in our current workspace going to go ahead and do that and just call this one um easy math and this is a video I saw um probably to go to promp flow GitHub they kind of do this here but they don't show you everything so we go promp flow GitHub see if I can get it here and I'm pretty sure they have like a video or something that shows you this and

they're going through it but I'm not going to fall exactly one: one um as I didn't find it I found it useful to get started but in some areas not so much and so this is just taking a moment to get going here I'm not sure why it's trying to install dependencies here when we already have them installed and I think it's because it opened up another tab so I'm going to just close that out that happens when you first create it but one thing that I recommend is just giving this a hard refresh so

that um whatever needs to take effect takes effect because I noticed um if I didn't do that it tell me things aren't installed or the plugin's not configured or whatever and so just doing a hard Refresh on the page that not the one that open but the one that you had originally there will mitigate those issues so we'll just wait a moment here as it's thinking and here it's saying the the workbench failed connect to server deadline exceeded so I'm not exactly sure what it's complaining about oh wow it's really complaining here but it looks

like it's trying to connect again so we'll give it a moment to connect I think my internet's fine testing yeah my internet's fine unless GitHub code SPAC is having issue GitHub code spaces status we'll go over to here no it's all fine over here right now I'm just going to wait for this to um get back to operational here I'm not sure why it's having so much uh problems so just give me a moment okay so I don't think it's me I just think that it's having some uh troubles right now so I might not

actually be able to use this right now um unless this actually connects all of a sudden here so what I'm going to do I'm not sure if this is going to fix it but I'm going to um stop this workspace code space which is fine and I'm going to start it back up if it'll let me uh I mean stop doesn't delete it does it it's not deleted well we'll go open in VSS code so we'll try that again no I don't want to open in local VSS code I just want to launch it in

the browser oh sorry open in browser here we go and so that should spin it back up in theory let's see if that resolves the issue seem really silly if we'd have to do that it's now active again and it simply is not working okay so I what I'm going to do is I'm going to delete this workspace maybe that workspace is just done for whatever reason sometimes it happens I was looking up people like yeah it just happens nothing you can do about it um and so I'm going to just try to spin this

up again let's see if this one connects so give me a moment here okay and so we'll just have to try that whole process again and um you know the reason I'm leaving this is It's just to show you how finicky some of these cloud services can be in case you are facing the same frustration I'm going to go here and type in prompt flow again wasn't like it took too long to get going here and so we'll go ahead ahead and install that and we'll wait for that to install here shouldn't take too long

there it is on the left hand side I'm going to go here and I mean I suppose what we could do is we could generate the flow first and then install dependencies I don't think the order matters it looks like it tried to do that originally so I'm going to go ahead here and say uh chat flow with a template and um yeah promp flow test that's fine and then we want the name of it just say math Helper and so that's trying to open a new tab I really don't want to do that but

I mean does it really matter why does it say preview over here so I'm just gonna again close that Tab out and I'm going to give this a hard refresh and see what happens okay loading no problem so we're not having the same issues as we did before if we go into here notice it's all green I'm not sure why it's green but I know that I don't have the dependencies installed so let's go and install the dependencies and we have an environment set up it automatically selected um which one the default one which is

fine perfect okay so um I want to get these installed so that's not too hard so I don't think the order really matters the fact that we did it in an opposite order but I I like install dependencies first but also just want this to work so I'm willing to try a different way so we're going to install that one again won't take too long and should be done in just a moment there we go need to clear out you can do it there we go I'm going to go ahead and do promp flow core

and then next promp flow tools and then um that one there some other options here we're not working with po shell here today and we'll wait for that to finish shouldn't take too long come on you can do it I know i' hang sometimes like that there we go okay great I'm just going to go ahead and click these a bunch there we are and so it should confirm just wait for all the checks I'm going to pause here wait for the checks and so now we have check marks across the board here so we

are in good shape um let's go back over to our left hand side here and we'll click into flow dag and so the idea is that there should be Integrations here um it's saying that the prompt flow SDK is not installed that's not true so we're going to give it a hard refresh and then we're going to see some options there in just a moment all right so we are back I'm not seeing well I guess I have to click into the correct file here let me just click off and then click here but we

should see at the top here maybe the the the plugin still loading so I'll give it a moment there we go and so now we have options of the Cross so we have the visual editor we have test batch run um things like that okay so we have a very basic thing so just again click into the flow dag and then we'll click on the visual editor and we basically get the same experience um as the one that is in the Azure portal but it's so much easier to work with files in here it's already

part of our git repo um so this is going to be a better flow and by the way this project is going to end up in the um Azure examples for now I'm working in the private one so understand you'll get that later on but we kind of already understand this experience but the thing that we're going to have a hard time with is is establishing this connection so we don't have any here so let's go ahead and add the connection I had a little bit of trouble when I was first doing this and so

I'm clicking this and just give it a moment to think [Music] um okay well normally what it would do is it would jump us over to the um one of these files here let me just take a look here so here we have open uh that's there but it's actually Azure yes it's this one Azure open AI so we need to configure Azure open AI here I also remember there being um under here once we' filled some information there would be like a connect button it's not showing that here I'm not sure why but I

know that we have to put in the base we do not provide well I guess we could pass it in here but I know that we do not provide the user input as if we press that button it would just prompt us for it but this is not working as expected right now so I'm going to go back to the visual editor chat and normally if we press this it would it would uh make things easier for us and it's not right now so seeing issues with this interface try regenerate with Advanced options let just

click that uh I'm not sure what it means when I did that there well anyway what I'm going to do is try to run this and see what happens submit anyway I do not care and I'm going to run this in text text only mode I'm expecting this to fail I'm trying to force it to uh let me use this because it's acting a little bit funny and yeah it failed on this part and that's totally acceptable but it's not um here we go so now I clicked it finally now it opens at the top

here so we have Azure openai open Ai and serverless I don't know what these two are I know this is going to kick to Azure open AI I would think maybe this one is for local local Computing serverless might be one where we have an endpoint and we can interface with it so I'm going to click on the Azure open uh open API and so now you notice this file it is where is this file it's not even the one that was here on the left hand side it's a completely different file promp flow temp

new Azure open ey connection so basically it's going to replace I'm assuming this one once we fill it in and so here it's now asking to put in the the API base so what does it want here well this is for Azure open Ai and so if we go back to the AI studio and we go to is it project overview in here here there should be something that describes the open API connection so here we have a connection we go connections do we have connections on the left hand side here let's go back a

level I'm just looking for connections here they are and so connections are those a remember Azure like open API and API or AI Services they're connected to our studio Hub and so the value it wants I'll just click to Azure open AI is this value here okay and if you're wondering what this is I just want to make it Crystal Clear what we are grabbing here because I had a little bit confusion because it's all named a little bit differently but if we go over to open Ai and we were to click into um I

think we're in West I don't think it really matters I'm not going to really pull from here but the the end point it's utilizing here is um this one here right but I guess the other thing is that could we also use the AI service one good question because we have open Ai and we have ai Services right and if we go over to um AI services for a moment because I think when I did this last time I actually used the Azure AI Services one and let's say we click into West us which I

I believe I'm in West us right now might not matter but we go down below we have open AI here and so then we have an endpoint as such as as well so I think this is what I'd rather do I'd rather use this one in this key I think it'll work but I think you can do either or and it will still work let's go ahead and replace that and we'll say allow and I'm just going to get a little bit ahead here and go back here and grab the key from AI Studio here

this one here I'm in West us that's really important based on what models we want to run and and so we're not going to fill it in here what we're going to do is actually click the create connection already down below it's prompting us so I'm going to do a paste and then it enter and hopefully it it works um so this is where I ran into another issue that's totally fine if we carefully read this message it's going to be talking about key ring and saying like hey you don't have key ring installed and

so what it I mean we have key ring installed but we need somewhere to store it within key ring and so we can do a pip install key Rings alt there are other ones that we can utilize um besides just key ring alt but this is the only one that was working for me that's the alternative to key ring and so now if we go back here and click the connection again right I need to go back over to here and I don't want this one I'm just going to go back here to the hub

I'm going to grab this key here have it ready and now it's ready I'm going to go and paste that in and hit enter and this is really finicky so just understand keep trying and try until you get this part to work it was very frustrating for me but you can see it it established the connection and I did not have to store the values here see it says user input so it's being stored in key ring I suppose and that's how it's pulling that information so now what we can do is drop this down

oh and it didn't rename it I always forget to do that but it had an option for us to rename the connection and I simply did not do that so now it's called two replace with connection name that's just going to be our connection name and here we have for our deployment um is the GP GPT 35 turbo so um and by the way there are newer ones we're just not using them here today so I just want to make sure that this works so let's go ahead and hit run and it's going to give

us three options standard interactive mode or interactive mode standard mode is if you put something in here it just gives it one run we're going to go with interactive mode today so I'm going to go and click Text and remember before in um uh in promp flow within Azure Studio we had to spin up compute GitHub codes spaces is the compute right that's how it's working it's interacting with those apis but let's go down below here and just say hello and it doesn't really have much instructions here it says hello how can I assist you

and it gives you the trace information I have no idea how to fall through this on GitHub um code spaces so I just ignore the trace but we have a bot we'll say hello again yeah and so I'm just going to hit contrl C to kill that terminate the chat that was contr C on my windows keyboard probably command C on um might be command C on a Mac can't remember but um I'll just type clear here and so this is our way that we can um work here so that getting set up was the

hardest part now let's actually do something and I'm going to try to follow closely to um the one that was in that video um that's on the prompt flow GitHub and then I'm going to make some deviations there so the first thing we want to do is I want to build a bot that is or an agent a bot whatever you want to call it and by the way they use the word bot but I I don't consider this a bot I considered an agent because it's more intelligent than a bot just understand that the

terminology gets messed up all the time but anyway um I want this thing to um help us with math okay and so for this it has its own Ginger 2 file which is its prompt document so we click into here it's the same thing left hand side we're going to tell it what we want it to do so I'm going I'm going to give it a roll and this is just how I like to write my prompt documents I'm going to say um math assistant and I'm going to say output um single final num so

numerical answer people always said that uh uh GPD wasn't good at math but it's getting a lot better at least for a lot of things final numerical answer um do not output reasoning or explanation so the idea is that we want to ask it a question right a math uh ask it a question and then we just wanted to open the answer just make sure these are spelled right it probably will still work if I spell it wrong since uh seems to work fine there but this is what we're looking for um okay okay and

so hopefully it understands that there so I'm going to go here save it going to go back to the flow dag gaml um and by the way we can have both open I think at the same time see so I have the coding one and this one so if you want to flip between them and figure it out you absolutely can do that and so let's go ahead and run this and I'm going to um well we're I mean it is a chat bot which is interesting because we don't need this to really be a

chat bot because it's just doing a single action um but I'm going to go ahead here and just say um 1 + one right so we'll go ahead and do that first I'm got to run that in standard mode all right and we scroll on down answer is two so it's outputting our answer um ex give me it's Json I'm not sure if it's because we didn't choose a response type let's go ahead and change that again I'm going to run this again so this time I explicitly told it what format I want and I

want to see what we get and we get back Jason um there's nothing wrong with that but uh go outputs here I'm not sure why it's outputting Jon H but that's okay I suppose okay um I I just wanted plain text when I did this last time I had plain text but I can still infer what is going on here so I'm I guess I'm not that worried about it so let's give it something a little bit more complex and we'll just say like uh word problem like math word problems and this might have been

one I was on yes so this I was on this one before and so let's give it something a little bit more complex because the thing is even though we told it to never explain it we might hit one where it starts behaving in a way that we do not expect it to behave um because you know this is sometimes how it works so I'm going to go back over here I'm going I'm going to paste this into here and we'll run it again here and I'll say run with standard okay and here it says

there were three shots in total so it's not doing exactly what I want I just want it to return the number three and so this could be us tweaking this here so I can say um provide only provide the final numerical answer just make sure I spell that right do not output reasoning or explanation so I've made a slight tweak to provide the final numerical um value as the answer answer there we go and so we've adjusted that let's go ahead and run this again in standard mode okay and so it's not doing what we

want what if it was like one plus 1 equals 2 would do it before when I ran this so you might run this and it might actually give you the value that you want right and so what I had to do is I had to go find harder ones and when I got to multiple multiple um Choice that's where it had a much harder time so there's a bunch here so here could be something like this right we go back over to here and I will paste this in and I will run it with this

one and we'll see what happens this time around we'll see what we get right and so it's it's reasoning it's it's doing what I don't want it to do right so we could provide it prompt examples but I think this is a great opportunity for us to use a function and and that's what they do in the video too is to extract out that information so this would be where you might need need to uh tweak it and and help it out what I'm going to do before I do that is I just want to

go over to the dag flow because we're not really we don't really need the chat history here and so we can just clear this out and I'll just take this out here okay I'll save that and my I guess my other question is like we have um I'm just trying to see if there's any other option we have to tweak here yeah we have API chat which is totally fine so I'm going to go ahead and um have I I don't have that in there anymore and so over here we have inputs and we do

not need that there at all so I can take that out there is no chat history we don't need a chat history go back and this looks fine so we'll just run it again make sure that it still works chat input history cannot be empty okay so I must still have this here so is chat output false I was hoping we could just remove it because we're not again we're not doing any kind of chat let me go back over to here chat. inputs. chat hisory oh up here false can I do that nope so

it seems like we're going to have to have it even though we're not really doing any kind of chat right now which is totally fine and I'll just run it again I was hoping that we could just kind of simplify that it's not a big deal and so here it's producing this number 500 seats right and so maybe we want to make a function that's going to um go here and try to extract out the answer right and that might be a bit tricky because here it's not really consistent in terms of how that would

work like is it always going to be on the end and we we don't really know that so maybe before we make that function we we should try to figure out a way to run things in batch and I haven't done that much but there is a batch run option where we can provide a local file so please select a local file with the input supporting Json L uh and CSV file format so we have both those options local parquet file so um to do that I think we just have to provide its inputs and

that's about it so I'm going to go do that I'm going to make a um a new file here and we're going to call it uh uh batch Json L Json L just means that it's a list look this up Json l so it just means um you have a list of things and I think it just doesn't have a comma on the end of each each one right so it's just a list of Json files so we have these math problems here and I'm just going to go ahead and extract them out now if

you're watching this and uh watching this you can just go to the exam Pro um exam Pro Co for SL asure examples and grab this or you can just assemble this yourself but I'm going to go ahead and make this and we're going to have to have two things in here we're going to have to have chat history and um the other one here we're going to have to have is um what was it called input question question okay so those are the the things I'm going to have to have here and I'm just going

to manually copy these over okay be back in a moment or you know what what I might do I said I'd be back in a moment and I am really back in a moment but I'm actually going to copy this into um something like CLA so I have Claud over here which is just another model and I'm just going to feed it this information in raw and I'm going to just ask it to provide it in the format that I expected uh to be in so I'll be just back in a moment and produce that

you can do that yourself too if you want I'm not going to show that but I'll be back in just a moment actually you know what I lied I just copied and pasted it and did it the oldfashioned way um I'm not sure if there's any formatting issues here um it only appears that it is um coloring every other line differently maybe that's just the standard way that bat Json l looks here but the idea is that we now have 128 so it's up to you how much you want to run you could cut this

down and not run as many just understand that this is going to um cost you know money because it's going to run each transaction here so again if you're trying to save money you could run only five or 10 it's up to you I'm going to run the whole batch I'm trying to um see how this looks and if this is too much I might just uh pair it down to something like 20 or something okay so um what I want to do here is I now want to to attempt a batch run so over

here I guess I what I could do before I do a big batch I can do a small batch right so I'm just going to go ahead and copy this file and just say uh paste it again and I'm just going to call this you know like five maybe yeah we'll say five and then this one here is how many do we have here this one is 128 so going to rename this and this one's going to be 128 which is the full one some of these math questions might not be able to do just

a single output so they might not be super great for that but anyway we now have both um where' It Go Oh wrong wrong folder here so now we have uh five and 128 so I'm going to go ahead and try the smaller batch first batch run and we'll select a local file and hopefully there's nothing wrong with the form at here um and so it's created a new batch run file which is located in our bulk test hemp directory so just carefully looking at this so we're looking at what we're importing uh we are

uh mapping the column we have chat history please select data input so if I click on this maybe I could just click on the column mapping I mean isn't it very straightforward What It Is Well I'm just going to go ahead and try to run this column mapping not okay fair enough all right so let's go back over to or let's try to attempt a batch again so that one failed so local file I was thinking that we could probably just select it right like if I went here I should be able to select it

but I guess not so say chat history nope okay can I just do chat history like this this is a y file I don't really need to do those parentheses and we'll do this question will that work I'm not sure what it wants right it's not being very clear oops oops oops oops oops and okay it looks like it's going to probably run here now so column mapping must contain at least one mapping binding so current coling mapping contains all static values uhhuh okay is it like inputs we'll try that again try this again and

just like inputs and what are we mapping from so if this doesn't work I'll go look it up and then I'll come back here and have the answer for you okay so let's try this no it doesn't like that okay give me a moment I'll go figure it out all right so it looks like we can do data from our test data or runs for our inputs so I I mean I would think that this is what we want to do is this one here right so I guess I'll go back over to here again

I'm just guessing but maybe what we're supposed to do is this and then provide that they don't really tell us like if we were using the uh UI like it's more wizzywig so um it would make it a lot more clear and so we don't normally do this uh I mean like we can do it this way but I'm just saying that they're not providing uh resources that way so we'll go ahead and give this a run and see what happens happens please save your yaml file first then submit did I not uh save these

here and so it still doesn't like it so I'm going to just pause here I'm going to figure it out and then I'll be back here okay all right so I think I have it working now um I just saw like a screenshot um this is what it would normally look like if we're doing it through the prom flow and they were doing data do whatever so I've tried that here and um it almost worked it says first error message fail to load and valid data we support file CSV tsv Json Json l parette so

it doesn't like the Json L file um so there's something wrong there I mean we could convert it to another file format and honestly I kind of prefer if it was like a a CSV or tsv so what I might [Music] do is I might go ahead and just convert this over to a tsv because I just I'm not liking this so I'll give me a moment I'm going to open up Excel and convert this over okay all right so I'm back I uh brought that into um Excel and exported as a tab uh space

uh Del delimiter it's like tsv file though they save it as a txt and I had to bring it as a txt and rename it to tsv I assume it's going to detect based on that factor there clearly it's telling us it supports all these formats even though when we uh ran it was saying CSV or Json so hopefully this will work the reason I went with um uh tsv is just because if there are any commas and there is a comma here those would have to be escaped or additional work would have to be

done here and I don't trust um the export to work properly so I'm hoping that this will work and that we can use tsv so I'm going to go back over to uh flow dag here and we're going to attempt a batch run and we'll choose a local file and we'll do five and yeah so we have data chat history data question I'm going to go ahead and attempt this run let's see if that works and hopefully it can interpret that there was nothing that said like what kind of file it was so I'm hoping

it just works and so it's going here and saying failed to load invalid data we support the formats please check the input data well it is a tsv for sure so try it again and oh it's only showing these did I click that by accident I thought I could have swore I um I clicked the tsv so maybe what we'll do hold on here Let's Escape on this I'm going to try renaming this to um just txt very very fin there is also um the terminal like PL it's the The Prompt thing here maybe we

might want to trigger it that way instead I'm not sure but we'll go ahead because maybe like in the UI will only give us um it will only detect um those file formats yes it's not detecting the CSV which is very frustrating so let's go take a look we see like prompt flow CLI and maybe there's a way for us [Music] to generate the batch um batch batch batch batch no there's nothing so even though it says it supports tsv I can't select tsv and it doesn't like our Json L file um I mean I

can try exporting this as a CSV but I expect that to cause issues what I could do is I could remove all the commas and then I could put in CSV file format and that could rule that issue out or like just make sure that isn't a problem so I'm going to where we have it here oh by the way this this one's not going to um hold on here uh nope hold on here would you like to undo to rename batch to batch 126 txt no I meant to open up this one here yeah

because this one um I I still have in this working State I want to revert it back to its original one CU I want to redo that format if we need to go back to it but here I'm just going to go ahead and say uh edit replace and we're going to find all of the commas anywhere they're in here any comma comma right yeah there it is why is it selecting apostrophe oh I don't think it is okay we'll get the commas here and I'm going to replace it with semicolon because that's kind of

similar and it probably will still work if I do that replace them all so now what I'll do is then take this over into Excel replace this here and I'm just going to change to dark here and I'll just save it as CSV and reimport it'll be back in just a moment okay all right so I'm back with uh this type of format and we're going to give it another go here um so we'll try to batch it again and I'm going to say local file and we're going to try here the batch five because

now it shows up so maybe this will resolve our problem here fingers crossed there we go so yeah obviously there's some uh things that are in the works here but that's always with Azure products there's uh a lot of um flicting information but here we have an output and so I'm imagining that the results are somewhere here it looks like we had an output I'm not sure exactly where let me just go find the output of that batch okay well hold on it's right here right that's that's it's right here okay so if we go

to home code spaces prom flow runs do runs oh it's over here okay so I thought maybe it' be in the same folder but it's not so I'm going to go down here and um how are we going to open that because the thing is is that we are uh just go file open file yeah open file there we go and then we can just paste in the path like this say okay oh there's multiple files uh what do you mean there's multiple file isn't it just the one file here well I guess that's the

path for the output hold on here hold on hold on hold we'll just hit Escape here so here we have an output and that's the output path all right well let's just navigate to that path okay that's probably the easiest thing we have a lot of things open here on the left hand side I'm just going to close some of these out it's a little bit crazy how much is going on here um and I'll just go to this top one here and let's just CD into this path and see what's going on here say

CD I was hoping there's like some way we could view the um there probably is I just not smart enough to know that right now and so that was our run so let's cap the logs first so looks like we have line numbers here and it looks like it's working four row count I guess I counted five lines there actually might only be four in here yeah yeah there's only four I thought there was five because um I uh miscounted there which is a minor issue there but we'll go back down to here and so

weever inputs or logs but I want to actually see the reports let's look at the outputs here so I'm going to go ahead and say cat outputs Jason log ah and so here it's this is the outputs and these are the four so we have there were three shots in Total Line number zero um answer 13 answer 90 answer 195 now I'm not sure why it outputs line number but oh I guess it to to tell you which which line it is that makes sense okay but there must be some interface here here right batch

run build Auto sort debug no not really but it would be interesting to see um all of the results so I'm going to go back over to this one and by the way you don't have to run this I'm just going to run it um if you're worried about um the cost of running these things but I'm going to go back over to here to our uh here I'm going to go hit run I'm going to choose a local file and I'm going to go for the big one the the big 128 Json L and

here it's asking us to map it so I believe this is going to be data chat history and then this one here is going to be data question and we will press that and we'll let that run and that will take a little bit of time has one issue here current column mapping contains all stack values okay what did I do wrong I already kind of forgot how we did it before so to go back to batch 128 here how's it not that give me a moment okay I mean that's what it says over here

data. whatever data. chat history history data. question we go into our big batch file here chat history question match is fine should be no problem here column mapping must contain at least one mapping binding current column mapping contains all stack values chat history data chat history none question data. question none okay not really sure let's complain about okay all I can think of is doing that hit run again it's opened up the file okay um did we choose the wrong file here let me go again and just check here no no seems like it's fine

this is very very finicky local file batch 128 CSV looks good hit run all right let me uh figure this out okay all right so all I did was I went into 128 um CSV and I cut it down to 50 maybe we over a limit I'm not sure what it was but it's weird because I obviously didn't change anything like you know for these the column mappings they looked proper but for whatever reason they didn't work the first time even though I did the same thing um but this time I only did 50 at

a time so maybe there is a limit maybe you can't go over 100 or something I don't know but anyway we have our output path here and so I'm going to go ahead and grab this and just close off some of these I'm going to CD into this directory do LS LS H LA and we're going to cat out um the outputs and so here we are now seeing a result so we have 0.25 [Music] kilog um and so you can see Noah walks one kilometer to school every day he walks one kilm to get

home and total he walked um two kilm so some of these are not doing it correctly and other ones are doing it correctly and so the question is is like could we write a function that's going to just extract out the final answer and I'm not exactly sure how we would do that um I just kind of wanted a reason to to to show how to attach a python node um but like in the video that I was watching this is what they did they made a function and they they extracted it out but like

how would that even work if they're all over the place right um yeah good question um so let me just think about what I want to do next but at least we did batch running and batch running is the start of us being able to get near evaluations um but just give me a moment okay all right so yeah I mean we don't really have a reason to add um uh python here but you know I'm going to do it anyway just so that you know how you can programmatically work with it um and the

results are not exactly what we want and we can fix that by either improving the prompt document or we could add uh we could feed it from one llm into a another one and then tell it to pick out the answer right now let's just focus on um adding a python element just so that we practically know how to do it so I'm going to go ahead here and hit add node we're going to add python I'm going to call this one uh format output and of course we do not really need to have this

it's just more the fact to show off that uh like how you could hook this up so now we have format output we wanted to get this to go from here to here then to there right so how we going to do that well we're going to go over to our um format output in here and sometimes this happens where it just doesn't show us uh information it's kind of frustrating but we can go over to our de over here and probably just adjust it manually and so we want um format input to have inputs

coming from our chat and so it'll be chat. outputs I believe or we go back here maybe this will reappear sometimes if I click this sometimes that kind of fixes it enter here there we go see now it all appears all of a sudden which is weird but anyway so we have our input here and the idea is that we need to change this to be um I think chat. output that work see that there we go and so now that's feeding from here to there and so that has its input well what about its

outputs because this should be getting from oh yes we go up here and this one's going to be format output there we go and so now we're kind of controlling the order to which things happen right um and so yeah we have this that goes to here that goes to there so let's click on this and let's just take a look at our code and so here final answer don't get mad at me if I spell answer wrong I always spell it wrong just going to double check this here I actually got it right for

once so this is now our new flow so let's go to the top here and see what happens if we run this oh by the way before we move on from here I want to get this outputed file I just don't want to lose it so we go ahead and um I guess I'll just save it go back here file speed output Jason L just want to paste that there so that I can look at that later but um let's go ahead and run this and just see that our python um our python code is

working here there we go now would there be a way that we could have this split somehow I'm not really sure but give me a moment I'm going to see if there's a way we could do that so there is this graphic here where it kind of looks like there's this llm that's feeding content from two places I would imagine that if you did this um because notice here we have inputs that one could flow to one and one FL to the other and then I imagine that both would have to be completed before this

would move on something we could test later on but that's kind of interesting to see but seems like it would just implicitly happen I would imagine that you know again both of these have to get to here before it moves forward so that might be something that would be interesting to test something else that I just kind of came across that I didn't notice was the fact that um you can um when you supply code you can either have it as a I'm trying to find it here as a class or a function not sure

why I can't find it right now just give me two seconds anyway I can't find it but um the thing is is that we Define our python uh code as a a function but you could also have it as a class if you would rather organiz organiz it that way another thing that would be nice to test is variance because right now obviously our results are not producing what we want them to produce and um something that would be interesting if I can find there it is GitHub is that there is a way to have

in um variance so like for this here um we're supposed to be be able to add variance and so I think that when we are working with this interface oh here it is show variant yeah I wasn't sure if we had that ability there but maybe what we could do so I clicked on show variance is that we could add another variant here with another prompt document to try to compare results so we go here um can we add a variant yeah so I go ahead and add a variant which is now over here notice

it's called variant chat variant one and so I can go over to this one and I could try to tweak and say provide the final numerical value as the answer do not output the reasoning um did you check and make sure there is only a single number also always add an exclamation mark at the end okay so there's some additional information we'll go back over to our flow dag and let's see if we can run it here and see if it will open outp put both the variants use default variant for all nodes no I

want to run both variants so I chose the second option there and so what I'm hoping is that will output both okay that's one so it says two completed did it open up in separate tabs I think it did so just to make sure that's the case and just close out some of these um tabs we have way too many open here and let's run that again so I just want to see if we end up with two tabs here on the left on the right hand side and we do okay so that's how it

would run the variance that's interesting right um and so clearly it's not doing exactly what we want but I mean like it's getting closer and so this is where you know in our model we might want to consider telling to go to another llm and then and then extract the final answer and this is where we might want to have a very um cost effective um uh simpler llm but uh I guess we could try that right now so one thing I'd like to do here is going to go ahead and add another llm so

I think what we can do is it was adding a bar here before yeah see here like in between um and before this one I'm just going to go ahead and add a another not a prompt but an llm node I going call this one llm extract answer and we'll make a new file here and so we'll choose our existing connection we'll choose chat and now'll from our deployment method it's acting a little bit funny deployment name so not EX doing what we want but that's fine we can go over to here and we can

try to find I can bring this scroll on down here so I want deployment model here but we have yeah the variant is here that's totally fine but I'm looking for the extract node so chat format output where is it where is our new node so I'll go back over to here maybe I can just paste it in like this I'm kind of cheat it's kind of acting funny right and I don't care really care obiously just text that doesn't matter here and so this one needs to get the output from the last one so

in here it's really acting funny it's really annoying um but we are grabbing it from chat so I'm going to go here and just say uh chat chat history and then here this one's going to be chat question right and then for our uh format output this one instead is going to grab from the um llm extract well why is it not a valid node this one's called element extract answer okay extract answer there we go and it just seems like our um XR Anor is not working as expected it could be because we have

the variant in there and that's causing us some issues so I can just delete the variant for now now did we delete the variant can I just get rid of this variant please delete I don't want three variants oh I'm cloning them no okay delete there we go delete there we go delete there we go and delete so now there's way less confusion I don't want all these variants I'm just going to go ahead and delete these delete permanently yes I'm going to go over to our flow dag hopefully it doesn't have that variant code

in here anymore excellent we're going to go back over to here I'm going to go to here now and I want to make uh certain that this is passing to this one from this one we're not linking up for some reason here yeah I'm not sure why it's not drawing a line from that one to that one uh yeah it's uh not exactly doing what we wanted to do right go back here and oh do we need a dollar sign in front of it maybe that was our problem so here we might say dollar sign

that's probably what it was dollar sign chat chat history there we go dollar sign chat question and now it's in the order that we want it to happen in so for this one here I want to go to its prompt document and roll say answer extractor look at the input and try and select only the numerical answer okay so that's what I want this to do we'll go back over to here and we'll run this and we'll see what happens all the way to the top we go here and say run in standard mode every

time we run I guess it opens tabs now I'm starting to make sense of that and it ran into an issue it says the section chat history of reference is currently unsupported please specify the output part of the node chat um fair enough so maybe we'll just take the inputs because we don't really need the the chat for anything um cuz there's no chat h for this one anyway so I'm just going to paste this in here and maybe that will just resolve our issue I'm not sure if it's proper but I bet it might

work let's see what happens here also what is the output of the actual um we have it's a variant but there's no variant here question and then the output here is let's go look at outputs maybe it's just dot output I I don't think it's chat history it's just chat output like this chat output like that and maybe this could just be that maybe we just make it plain like that there we go and we'll try to run this now like this and so let's see what happens here fail to render Ginger template um has

no tribute inputs please modify your prompt fix so there's an issue here we'll go over to here and maybe it does not like this well input questions that's not where we're getting this from we're getting this from um chat we're getting chat output that's what we called it here and then this one here oh no that's chat hold on hold on that one's fine the issue here is down below for our LM extract answer maybe it doesn't like the squares here it's not very clear let me just read this okay all right so it doesn't

like this because maybe it's expecting ajacent object that has inputs on it so I'm going to go back to this one and even though I really don't want to pass anything to it I'm just going to go ahead and say um inputs chat history and maybe that might resolve the issue here we'll go ahead and attempt to run this again I will give it a moment okay and so still not doing exactly what we want but you know if we really wanted to improve this another way that we could do this is going back to

that prompt document and giving it examples and showing the output that we expected to have right that probably would solve it but again I just wanted to see if we could chain one to another one um but uh yeah this is just poor prompt prompt engineering because again we can go back to this one we could solve it right now if we go to this one and we just gave it examples so example examples and then uh you know like for the one that we just got like the um if we go into our one

over here on the left hand side well first let's let's do prompt engineering on the second one and then we'll get rid of it and then we'll we'll fix the first one just so you know what I'm talking about here so I'm just going to go here and just say [Music] examples and I'm just going to split this like that and example outputs good outputs we could say bad outputs and on the right hand side here I believe or left hand side I save that file as a Json L called outputs Json l so we

have a bunch here that we know we do not like um so here we have three shots in total uh I'm trying to find ones that uh that are are good examples of that so we'll grab this one here right and we'll have to find their equivalent line going to break these onto the next line here and so we'll go here and now I need to go and open up the batch CSV file so we have line 33 so we go down to line 33 we'll grab this one here and so I'm just hoping that

these uh match exactly so this would be user assistant and I'm just going to quickly format these I'll just pause here so to watch me do this all day one thing I'm noticing is that uh like the word Austin actually shows up on line 36 so I'm not sure if it's being well here it's not obviously one it's offset by one so I'm thinking that this answer provided up here is not exactly correct um so I guess I have to be really careful here now we grabbed like a bunch of them the same area so

if I type in Austin here undo undo again type in look for Austin A St and it's 36 right so really 36 is the um 34 line this one's 33 and so I'm just going to assume this is we have um 36 34 to 37 so this is one 2 3 4 five so it's those five lines I'm going to go ahead and copy them not sure like I don't know why it's off by two I understand why it's off by one but whatever that's fine so this is the real um the real order order

of uh of things and so we'll go ahead and grab these and I switched to Vim mode so my life's a little bit easier here so's going to be here and this is going to be here and then this one is going to be oh hold on uh this one hold on hold on hold on hold on hold on hold on hold on hold on I'm going to copy this cut this here it's becoming a bit of a mess and I'm just going to undo this till I can see all the okay there we go

so I'll go back here and paste this in again here and so I can just put each of these one above so I don't make any mistakes Julia Heather why doesn't that match up well let's go undo that again paste that in here again I'm missing the first first one oh boy I apologize it is very hard to keep track of this so I'm going to go back and look for Austin again so I'm just missing this one here and so I'm going to go back over [Music] to our file here and I'm going to

paste this in here yeah here we go okay great so now these should be lined up correctly okay so now we have user why does that look wrong I have three chalkboards what the heck okay so this one's right this one's right this one's right this one's right I'm taking this one out I don't know why this one's not matching up correctly I made some mistake there I'm not going to play around with this all day so this one is going to be assistant this one is user this one's assistant this one is user this

one is assistant this one is user this one is assistant and so these are examples of bad um bad outputs right and I'm just going to do this so that it knows that this kind of grouped here like this and so now what we can do is take these and just give it the thing that we actually wanted it to do okay so here we say in the hockey game Mitchell scored more points than William but fewer points than Austin who scored the most points who scored the fewest points and this is kind of hard

because it's not really a simple one so Austin scored most Austin most William Fest okay just one uh therefore the fraction of houses that we gave is is one half what fraction of houses of Julia streets have given her chocolates and so here we're going to erase this all the way out because we don't want all that reasoning 12 like that um the fraction here is 1116 so we'll do that like that then we have two kilometers km for that so we know it's 2 kilm and so that is going to have a good good

output and bad output now one thing you need to consider is that if you don't let the llm to reason through things then it can actually give bad answers and so it's possible that it might be more optimal for us to let it reason and then extract out the answer or we can engineer the original one and say you know don't reason out loud but then it's not going to be able to give you the best answer so you have to play with it and determine which one's going to work but I have a feeling

that that um that the way we're set up right now is probably more optimal but again I'm not like a super expert here we'd have to play around and have a very large um uh set um it would take a lot more work to prepare that but anyway let's go ahead and see if this improved information is going to do a better job for us so I'm going to go over here and I mean obviously the the outputs that it had was output from the original LM and this one actually did try to summarize it

a bit further um look at the input and try to select only the numerical value okay great so let's go back over to our flow dag and let's see if we can run this and see what happens here so we run in standard mode but yeah examples are super super important if you want to get things working properly now if we end up with a very large um prompt example document then that's where we'd want to do finetuning maybe we'll use that as an example so here it says Final Answer 500 and so that's looking

a lot better that's looking like what we actually want it to do um it's saying final answer because we're formatting that in that step there but let's go randomly pick out another one here and we're going to go into our um uh I'm looking for our yeah input right here and so we'll place that one in let's see if it gives us a single one now and so here we have final answer yes the triangle with the side length of 3 4 5 is a right angle triangle so failing here but let's try different one

and see if we get a better result okay and now remember there's two LM so it's like it's double the cost right okay so my question is if we were to run the the again against the 50 would we get better results or whatever and so this is kind of where we're trying to get to a better place so I'm going to go ahead and again you can just watch if you're worried about cost but I'm going to go ahead and I'm going to run this against um uh the 50 again here and see if

we get better results hopefully this works and doesn't give us some weird problem down below here but I'll be back in a moment if this works all right so we have our put here um and so I'm going to go ahead as per usual and we're going to go and programmatically make our way over to that directory and we'll just do a cat outputs Json L here and so this is looking a lot better I mean this one came out wrong again I'm not sure as to why but you can see that we have better

outputs Jon L and you know what it could be it just could be like if we trained it on even more data because these uh these uh these other models are in smart but if we could train it on more data maybe we could get it to do what we want it to do right so closer say final answer which is fine but um I mean to be fair this one's a bit hard because it's not like it's producing just a number this one is incorrect that should should have been an issue I wish it

would have used km but we could have given it that separate instructions up here it does it but down below it doesn't do that so yeah we just have to provide it a lot of examples so I'm going to call this done and commit this as um freaking code and I'm going to go add this to the um the other repo here so there's other things I want to show you like I want to see if we can do evaluations and I don't think it's easy for us to do that in here I know that

we can create an evaluation with a template so create a new evaluation flow with a simple line progress and aggregation um I just want to show you how to use evaluations I think it's going to be easier to do that over in um uh flow PR flow like the actual UI um but I just want to make sure that that you have access to this code that I just um produced here so I'm going to go give this a hard refresh here and I'm going to press period and that's going to open up in GitHub

Studio this I'm just going to close this out as I'm done with this and by the way if you really want things to be closed you have to really make sure of that so I'm just going to go to promp flow test here and here I'm just going to delete this there we go um and so I'm going to go over to Azure examples and from here I'm just going to go ahead and download this this is going to go somewhere just going to download this to my desktop really quickly here mhmh so that is

just downloading I always just show this on screen just so that you know exactly where I'm putting it and is it still downloading no I think it downloaded all the files good so we'll close that out I'll press press period on this and in here we might have prom flow already I'm not sure so I'm just going to make a new folder here called promp flow and I'm going to upload that folder here and say promp example not sure why my JP teacher is in there I don't remember changing anything with that that's fine we'll

bring it bring it along for that one and yeah nothing to shut down here as everything um is is done here and I will see you in the next one okay [Music] ciao hey this is angrew brown I'm back into um Azure AI Studio back into uh Us West in side of flows what I want to do here is I want to see if we can um use evaluators so evaluators allow us to evaluate our models now down below we do have evaluations here as well um and it's in preview mode um I'm not exactly

sure on how to use this right now but I think that we can figure out prompt flow here so let's go ahead and um start a chat flow I just call this math helper as we're going to leverage the math helper we were using before but I'm just going to bring over the code that we need because I don't know of any easy way to import that code in here and so we have our our chat system here I'm going to scroll on down we're going to choose Azure and we're going to go with um

35 Turbo with a text output and um I'm going to go over to uh GitHub here into our exam Pro Azure examples and in here we have a folder I'm going to press period um we should have a prompt flow folder from the previous video and in here there is our J just and we're telling it the simple instructions here be a math assistant provide the final numerical value as the answer and do not outut the reasoning or explanation uh one minor thing that I want to do here that's a little bit different I don't

need these I'm going just close those up is um just going to update that here is that we had one that pumped into another one and so I just want to go here and grab the good outputs and the bad outputs so it knows what I want right and I'll go here and paste that in here there we go and so now this is set up um pretty straightforward and so let's go ahead and ask it a question but before we do that we'll have to spin up some compute so I'm going to go ahead

here and press um start Computing session and just give that a moment okay to spin up all right so um I was waiting for something I don't remember what but um I guess we are oh yeah the compute to spin up excellent okay so let's open this up and we'll just give it a moment here to load I guess the chat is open here so now we'll just go ahead and say um math word problems go back over to here to proy and let's go ahead and just grab one at random and see how it

handles it now I don't really want to feed it um history but you know whatever we'll go ahead and run this and see what we get give it a moment here I'll just pause till it generates here go game back with three so that's really good um we can run batches here but what I would really like to do is try out the evaluate so evaluate the performance of the flow um there are a bunch of I want to just show you here I'm just going to open this new tab but there are a bunch

of evaluations that are in prom flow and all evaluation really is is another prompt flow but specifically used to test your output and it used to yeah right here evaluation flows and so here are some predefined evaluation flows and what's really interesting is that these evaluation flows are just flows and and we might be using llms to test our output but let's go over back to our math helper uh well I guess we don't have to do that there it's over here we're going to go evaluate and we have custom evaluation so your custom evaluation

flow I don't have that here today using a standard or built-in metric let's go ahead and try this out so we say [Music] confirm um evaluate a single turn question answer with context evaluate a single turn question answer pairs without context so let me just decide which one I want to do I don't know before we do that let's go back and see what we had under um the other option cuz I could have swore we could just uh bring another one here let's go custom yeah yeah I'm not sure here let's go ahead and

do this so I'm not sure which one we want let me just read through this and decide I think we want without context I'm not really sure that is the flow we want to evaluate use an existing data set um I mean here we want to add our data set so it'll take a CSV which is great uh what's cool is that we um we have our data set split so we could feed it one part of the data set and then the other one could be what we run it against but let's go ahead

and oh that's AWS let's go here to um I'm just saying like if we have to feed it a test data set and another one we could split it but maybe we just need a single file here so we have our batch file I'm going to go ahead and download that I'm go over here and then I'm just going to go ahead and upload I'm going to find that uh locally it's on my desktop here maybe on my downloads there it is and I'm just dragging it on in and so there is our test data

so here we have performance and quality metrics curated by Microsoft so measure how well the Genera value application can produce outputs that flow smoothly naturally not really what I want measure the language proficiency at the generative value application predicted measure the similarity between a source data ground TR sentence and the gener response by the generative I um measure the similarity between a source data ground truth and sentence generated by that so it almost makes it sound like the one we really want here is is this one but wouldn't we need to have an answer yep

there's our problem so for this to work we need to um have the answers for that to work so here it says the response to the question generated by the user human is the true answer a query seeking uh specific information the response to the question generated by the model is an answer okay so it seems like what we need is these three options I'm going to go back over to here and I mean 50 is a lot 50 is a lot I believe that we did save the outputs in here we did over here

so we have outputs and outputs Json and this one's a little bit better so I think I can do it so I'm going to create a new file here I'm just going to prepare with the these three values which is um ground truth question and answer okay so I'm going to go here and just make a new file and this will be data set or this will be for evals um Json L and hopefully we don't have too much um I'm just deciding whether I want to make it Json Val or if I want to

make it a CSV because last time I did this as a uh Jason Val I had some troubles so I'm actually going to make it a CSV v file so I'm going to prep that there with those three fields and be back in just a moment I'm going to do that in Excel all right so um what I've done if I'm just G to get this back over to excel I'm not sure why oh it must have closed it by accident but I just want to show you how I prepped the data so here we

have the question uh the ground truth which um I had to do a bit of formatting to to get and then we have our answer um clearly we have oh I have more questions here so I probably should clear these out cu these are ones that we don't not know yet I'll just clear those out there we go um and I guess I don't have this one so I'll just clear that one out and we'll save that again I'll just re-upload that file here so that one matches but the idea is that we have some

that we haven't checked it against so that could be useful having some data that is um tested not tested um I the one I used for was the outputs Json the first one which was we um had our model where it didn't have that second output where it did this um so I wasn't exactly sure what to do for that because it's not super clear uh what it's expecting here but um and obviously we didn't run our model again with the new uh the new prompt document so maybe it would do a lot better but

I'm hoping that it's going to be um okay with that so I'm going to go back over here to our test data and um hold on here so select metric here well hold on a second so we have this here which is our test data and then okay so if we have test data then maybe I do have to have that so maybe our test data will be the other um the other 50 let me go back to the next can I go back here and then this one here how does your data set map

to your evaluation input okay but if we go back to this so anyway I'll try uploading the new data set and see what happens here let's go ahead and give this a try so it says here the input values do not match the expected format please select from the dropdown and that's totally fine oh it doesn't show us that option give me a moment okay all right so I went back to my um Excel spreadsheet here and I just put in the chat history column I also told us to replace any commas because some of

these answers might have outputed commas this is a CSV I do not want to work Jason L today as we've been having a lot of trouble and so I'm going to try and upload this again and see if it works um and it's not exactly doing what I wanted to do I'm not sure if it shows yeah here's our old eval data sets I want to get rid of all these let archive it yep just trying to get rid of these so I'm not worrying about them nope I can only do one at a time

what a pain okay and so I'm going to go back over to promow I'm going to go yeah our math helper I'm going to go to our evaluate I'm going to try automate evaluations I'm going to confirm I'm going go without context we're going to go next we're going to add our data set we're going to drag over eval CSV which has those columns here it's not showing me that information there um but I guess I could just write it in here so just say chat can I do that chat history history required format is

that so say data. chat history like this and then this one will be data dot question okay and so it should be fine with that right one moment all right so I went over to Sonet and I just asked it to like hey convert it to Json L format and maybe that will make our life a lot easier so I'm going to go ahead and give that ago um it doesn't have the uh chat history in here so that will be something I'll just add here so I'm going to go ahead and make another new

file and we're going to actually I'll just make it in here in the uh here say new file evals Json L and I'll paste that in here and um you know one thing I don't have in here I I should have told it to have in here but I'm can just go down here and say chat history because I really don't know what it wants I'm just basically throwing everything at it so now we have this file and so if it doesn't like that I'm not sure what else to tell you um because if you

watch how they do EV valves um there's like videos um for this and they're doing it like since last year and the process is different um you can't do it the same way as they did it before so I'm again just trying to make this work um is it in the exam guide I'm not 100% certain but if you're you're going to work with LMS you really want to be able to utilize EV uh like evaluators it's not complicated it's just it's just that it's going through another llm um but if we can get us

to work that'd be really really good so I'm back over to here and I'm going to go ahead and download this file okay and that one yeah eval Json one which is fine whatever I'm uh going to go ahead and I'm just renaming it so it's a bit nicer I'm G upload this and see if that works there we go okay great so now it sees the four types and we see chat history and question and we have um ground truth an answer in there as well we'll go next and so here we don't care

about these two we care about similarity measure the similarity between a source data ground through sentence and the generary response by a generative AI application uh we have measures the ratio of the number of shared words between that no I don't care about that I don't care about these three these three are completely useless to me um I'm also just going to hook this up here yeah there we go and so ground truth ground truth question question answer answer so now we're getting somewhere it's weird that in some places jonl do does not work in

other places it does um but anyway so here it looks like we have something so I'm going to go ahead and submit this and I think what it's going to do is provide us a score and tell us how well it's performing because it should go attempt to generate it out using our data set great um I'm just going to wait for this to finish okay all right so after a little while there it looks like oh it canell why did it cancel I didn't cancel it um can we see why status not started why

why why why why why what did I do to deserve this okay so um maybe I was supposed to do something logs let's take a look at the logs uh is waiting for dependent runs has a dependent run terminated unexpectedly Mark is as canceled so it's not cancelled it it's well I mean it's saying it's canceled but it failed and we don't know why so I guess we'll go back there I mean it is a new feature so you know we can't be blamed if it doesn't work here I'm going to go ahead and try

this again and try automated okay and we're going to go next and we will use an existing data set I think it's this one looks good next rid of that get rid of that add this good maybe it was just like the availability of the model and I don't think we need compute running like I've been running comp this entire time by the way you could like stop compute I'm going go ahead and submit this but I don't think it relies on that just in case it does I'm going to leave the compute on um

and so I guess I'm just going to wait again and see if this works if it doesn't work again then we're just going to have to stop it here but um I think that you know maybe in the future you might run this and actually get results whereas I'm not um so I still would like to keep this video around even if it fails but I'll just see what happens here okay and it canel again so that's a little bit frustrating again this is a preview feature but we'll take a look here and see the

execution logs terminate unexpectedly so no reason as to why that's very frustrating but let's go take a look and see if we can make sense of any of the other evals here there is custom evaluations we can go through it here and see what happens uh let's go hit next um we have our data set here so we'll go next oh okay so this is the the way that it used to work um the UI is a bit different here but I guess the thing that we're looking for is roundedness see here will evaluate the

QA retrieval augment generation system by leveraging this okay that's a lot of nonsense the ground is eval allows you to assess evaluate your model's ground and scale from 1 to five I mean that sounds like what we want but I can't remember if that's what it was here so I'm just going to go back over to I know that we don't plan to uh do it this way but I just want to um check here and see what we have going to go back to this one here jpt similarity measure the similarity between the source

data sentence and the gener response okay I mean groundedness sounds like what we want to do measures how grounded the model's predicted answers are against the the the context even if the El's responses are true if it's not verifi against the context then the responses are considered ungrounded so maybe what we can do is try to run this one first and we need context so the context would be the information that is used I guess the other one wasn't context so I'm not sure about this one right now but um maybe in a future video

I'll go read up a bit more about it and we'll consider that here let's try um similarity so we have GPT similarity I just assume it's a different model that's doing it so we'll go ahead and try GPT here and we'll expand it so here we have questions ground truth answers good I I don't see any mapping issue here but we do need to choose um something for it to use the evaluation flow uses an LM you need to set a connection here to use this evaluation I'm going to use GPT oh I don't know

if I want to use four on that we'll just use uh 35 turbo we'll go ahead and say review create and basically this seems like this would be the same thing as that other one so um you know I'm not the best to understand all these valuation models it's just that you read them and you kind of figure them out I'm sure in the future I will probably have more information or in my slides we talk about evaluations I'll do more research before you watch this video and so I I'll sound a lot smarter in

the slides but we'll go ahead here and just see if this will create us another flow and so here it says our runs have been successfully submitted did uh I guess that's probably something that it did let's go ahead and click on that and these look recent not sure why we have variant here but maybe if we go into here uhhuh there's just a evaluation here why did it fail so we go look at more information fail to load invalidated we support the format CSV tsv Json Json L parquette please check the inputed data so

this is where we keep running into um uh data issues which is very frustrating and it's weird because the data is imported right we know that it's working it's displaying the information but it's saying that it has issues with the data and this is very frustrating um because I'm not exactly sure what the problem is I mean that could be the problem with the other one but if it's not going to tell us exactly what I I don't know what I'm going to do here right like is it too much data is there something wrong

with the data maybe yeah here it says um we support the file formats uhhuh okay but you you can't sit there and tell me that when clearly you know we can see that the data is loaded so if we go down to here right we go down confirm I didn't change anything yet and we have this one which is the latest okay like what could be wrong with it let me go see if I can find anything so here I tried another submission here fail to load and valid dat we support format CSV tsv Json

Json L par okay this makes no sense cuz like we obviously are giving it the format that it wants um we could switch it over to CSV but we had issues with that as well oh boy um I mean I probably still have the CSV right so I guess I could try uploading the CSV again even though that seems really silly um I'm going to go back to prompt flows here back into math helper we're going to just try the CSV even though that doesn't seem like that should be our issue we'll go ahead and

say custom we're going to go next I'm going to add new data I'm going to uh browse and upload I'm choosing the eval CSV right uh I guess we need to name it so just say eval CSV we'll say add and we'll go next we'll choose similarity we'll drop this down it's parsing the data right it's parsing the data no problems here chat chat history is and what you think it would be we'll go next we'll submit we'll see what happens here next and we'll go to the Run list wherever that is it's weird that

you can't like find the Run list here on the left hand side did fail again is reparing it so maybe for whatever reason this one will do better there's something that I'm doing with Jason Jason L that is not correct maybe Jason L is supposed to have a Comm on the end the line but I'm pretty sure it's not supposed to so Jason L format filed sample let's see a sample but I'm pretty sure yeah it's just that maybe maybe maybe because I'm on a Windows machine that the carrier line is different or something it's

like a hidden thing I'm not sure but uh it is quite frustrating let's go back over to here give us a refresh and that failed hey at least it went to one second so that's an improvement let's take a look here and see what it's complaining about um logs overview this won't load the data so at this point don't know how to fix this um but you can at least see that I made the attempt to show you how to do evals um but the idea was that if we saw vals it would just show

us executing another um uh it would take a prom flow and feed into the other one to check it so yeah that's the best I can do here so I'll see you the next one okay ciao all right so I'm back and I thought that this wasn't going to work out but what I did is I took the data set and I truncated it to like five records or seven records and all of a it's working so maybe the issue is that it just can't take a certain amount of data or there's some kind of

record that is causing issue here um which is kind of frustrating that that worked but um it's good that I figured that out a data set of seven is not very large but anyway you can go here and take a look and so we can see um we have information here what ises that telling us so input question answer for shots system metrics so we have duration and tokens this might be useful if you want to know how long something took and how many tokens it used in total but my question is what does this

mean because I thought this would tell us something let's go over here completed okay so that's great MH but what are our results uhuh give me a second all right um maybe we'll try view evaluation run because that might tell us a bit more um and so you know I was saying before that a evaluation can just be another flow so I assuming if I clicked that I would see the the flow but I don't oh here we go GPT similarity so now we're starting to get a score and so we see one one n

so we can see how much of a difference it is okay so that gives us something um yeah something doesn't really make it very clear whether that's working or not again not an experted evaluations not right now let's go ahead and try the automated evaluation because clearly if that data set worked then this one probably will as well and so we already have a data set in here it's eval 7 uh we'll go next here I want to do similarity measure the similarity between a source data and Etc so we'll do that we'll choose this

uh 35 and we'll go submit but you know I really thought that there was some kind of other cuz when I watched other videos and I did research on this they showed that there was another promp flow and it would show you exactly what was working underneath um but I suppose if we really want to know what's going on there we could just open up prom flow here because we do have them here right like we can go and make a prom flow so it doesn't let's go back here and I could say create hold

on here yeah do only my flows yeah okay so if I go here we can go and look at that evaluation flow we ran just a moment ago and so if I open this one up we just clone and then just look at the its contents go back over here so this evaluation still running and so this is what this thing is actually doing so we get our inputs here question ground Toth ground ground truth answer then we have our similarity score so you're an AI assistant you will give the definition of eval evaluation metric

um as a metric measure similarity the predicted answer is not at all similar to the correct answer the predicted answer is mostly not similar to the correct answer the predicted answer is somewhat similar to the correct [Music] answer okay and so we have some examples all right so yeah I mean that makes sense you feed it a ground truth a question and an answer and then it's going to determine that similarity I just don't know right now how that is useful um but anyway we'll run this and see what its results are when it's done

okay it's been a few minutes and it is still uh attempting to run but I guess we'll give it a bit more time here all right so that finished here we have detailed metric result and okay context status question ground truth what's the answer yeah I'm expecting some kind of scoring or similarity here I mean maybe ground truth is the value we're getting back because we didn't provide it a um a number for this right so I'm not exactly sure yeah I'm not exactly sure what we're supposed to do with this but anyway I did

make my best attempt to show evaluations um at least we got to a running State I would imagine that the idea is that when you run it then you'd have that data and then you would tweak it or something I I don't know but anyway I'm going to stop running this compute um and we're going to call this uh promp flow done I am going to go ahead and just uh archive this because I do not need this anymore and if I need it I will bring it back when I need it but yeah um

I'll see you in the next one okay ciao [Music] when we're talking about large language models there's this idea of fine-tuning where if we have a model that we don't like it we can do something to it to make it work a little bit better to understand fine tuning and the ways we can fine tune it let's just talk about the components that that are involved in fine tuning and so we have to first take a look at Hidden layers and its components so when training you have layers of nodes also called neurons so think

like your brain and between these nodes there are going to be connections and so connections are often between or across layers um but connections can also be within the same layer and that's where we get this concept of self attention if you remember the concept of attention is really important when we're talking about Transformers for large language models and I mean if we represent it it' be more like it's connecting back to itself and that's why we call it self attention because it's a layer that feeds back into itself uh which is self attention Okay

but connections could also uh be where we have multiple sets of hidden layers uh and these connections are computed in parallel so the idea I'm going to just draw this here but imagine we have another layer with nodes right and the idea is that this one will feed into that one but this one's coming from here and so now we it's called multi-head attention because it's coming from multiple sources and in fact some of these they'll come all the way back here and go like this and and feed in so you know that is ways

that we can uh uh feed our data forward uh then we have parameters so parameters are the weights of connections so um over here on the right hand side get my pen tool out again we have a weight and this weight is the representation of this connection between these two nodes and so that's going to be a value and so a connection might have one parameter but they can also have multiple parameters most cases it's one parameter but you can imagine that for the amount of nodes that you have in each layer they're going going

to have to connect to all the other ones in the next layer and that's going to add up really quickly let's take a look at some um uh Transformer models or large language models and understand how many layers they're utilizing for training to get perspective so let's take a look at gpt3 so gpt3 is not new um in fact it is one of the smaller models that you can train still um like Babbage or Da Vinci if you go like let's say use Microsoft um Azure AI studio and you want to do fine tuning you

can train GPT three models and it has 96 layers or it's large uh uh if we think about its parameters that's 175 billion parameters so you can only imagine how many uh nodes or connections are going on in there but that's how many there are um and then we have Bert so Bert has 12 layers or up to 24 layers so Bert is uh still useful it's a um a much simpler uh Transformer that we can utilize we have gpt2 which has between 12 to 48 layers so the same or more as Bert then you

have Google's T5 which has 12 encoder and 12 decoder layers or up to 24 layers there so you know we're talking about fine tuning it's going to be tweaking the amount of layers uh the the the the amount of connections we're going to train and things like that but let's go Define what is fine tuning so fine-tuning is retraining a pre-trained model's weight or its parameters on a smaller data set so a model's weights is the outputed state of a model but in this case when we're talking about fine tuning we're talking about a trained

model's output okay so then what is supervised fine tuning sft this is where when we provide the data set it's already been labeled right so imagine we have um a bunch of cats uh or like photos of animals and so we're labeling what each animal is so that when the um the model is training it's like it has a cheat sheet to know how to understand exactly what it is that it has okay but so we're basically explicitly telling the model what the data is as opposed to when we train our base model that might

be unsupervised uh where we're not saying oh this is what this is right because we're giving lot imagine trying to do supervised training on a huge data set like labeling all that would be very difficult so the idea is that um we will produce our base model uh first or in the case of LMS the base model is the foundational model so you're taking an existing Model A A a foundational model and then we're going to train it as soon as we have a foundational model or base model and we decide to find tun it

now it's being called a pre-trained model okay so understand those terms we have FM okay base model pre-train model they're all in the same area they don't necessarily mean exactly the same thing but they represent the same thing at this place and time so we're get ready to take our base model and find tun it so we're going to bring in our smaller data set I'm just going to uh clear all the ink off the screen here and so the idea here is that um we bring in that data set and now we're going to

train it retrain it uh and produce our fine tune model now when I say we're producing these models or we're outputting these models we're not actually outputting models we're outputting the models weights okay we're not creating new models we're just uh creating new outputed states of the model um and just understand that that is it often sounds like we're creating new code or something but that's not necessarily true so let's now talk about the types of fine tuning we can do because there's a lot of approaches we can take to fine tuning um so and

this is not even exhaustive but the first let's talk about changing the data set so the data set itself the data you're going to put in there we could do instruction fine tuning that's where we take a data set and we tell exactly what we want as uh like let's say we say I say this you do that so you're giving an example of what a person says and what outcome is so that's instruction fine tuning uh then we have domain specific fine-tuning that's where you're taking uh a knowled a knowledge base or a data

set of specific knowledge to update the model on that knowledge or to make it uh focus more on that knowledge set right so if we had a generic llm and we wanted to make it specifically for learning cloud computing I could load it up with the most upto-date um uh cloud data or even my own stuff to make it teach like I would teach okay then we have changing the method of training so we have full fine tuning this is where all the models weights are updated and it's expensive so you say full fine tuning

we can just think of it as traditional fine tuning it you're basically taking the existing existing model models weights like after the base tune as the starting point and running it through the training process again now you can add these two things together right you can um do full fine tuning and change the data set they're they're uh they're they can be done together or both separately it's up to you we have parameter efficient fine-tuning so also known as PFT you'll see this term come up a lot uh it only updates a small set of

parameters during the training and freezes the rest of the parameters there is a subset of PF called Laura which we're not going to talk too much about here but I'm just going to get you exposure to this if you are not needing to update every sing par single parameter then you're going to save money there another way is uh last layer fine tuning this is where you freeze all the layers except the last layer and when we say freeze we just mean we're saving the state at that point in time right or we're telling it

to skip until it gets to the last step um and then we're basically just training it on a single layer and apparently that works really well so there's a lot of things that we can do another thing we can do is we can do pruning so this is where you're removing parameters all right you're and people might want to do this just to make the model smaller and more efficient because um maybe we can remove parameters and and it will uh use less compute or be faster um for some trade-offs and there's two ways we

can do this time train time pruning so somehow we are making the model to encourage to drop or remove connections or neurons during training or post training pruning which is basically you mangling the the model weights file the the file that's outputed so yeah a lot of options here but uh there you go [Music] hey this is Andre Brown in this video we're going to take a look at finetuning so fine tuning allows us to um take something like that would be a very large prompt document and tell our llm to bake that stuff in

so that it works more efficiently hopefully that makes sense um and technically as of this video gp4 mini uh mini is on Azure but I don't think it's ready for fine-tuning so open AI lets you do it in their platform but not necessarily in here so we're going to be using an older model for fine tuning which is totally f and we're going to go into Sweden because we know reliably that we can f tun in Sweden um there was also announcement of the cost and so F tuning does not take as long as it

uh used to and it's not as expensive as it used to in the previous video for the rag AI search video we went over those prices but um if you're not comfortable with the cost you don't have to worry about you just watch me do it but let's make our way over to AI Studio you can probably also do this in the open AI Studio but I'm going to go to AI Studio since it's newer Azure AI Studio and I want to go to the one that is in Sweden so we have Sweden Central so

I'm just click over into any of these here and get into the oh launch Studio there we go so we'll give it a moment and if we go over to model catalog there are specific models that we can fine-tune so I believe we can fine-tune uh turbo GPT turbo 35 3.5 babage and other ones so let's look at find tuning pricing change Azure open AI uh opening let's see if we can find that here here it is this was just back in June and so we have babage Da Vinci uh 35 turbo or 35 turbo

and here you can see the cost with babage being the lowest right so let's say we wanted to train babage uh we can give that a go so what I want to do here is I'm in Sweden because I know that one's reliable I want to make sure we have that model deployed so let's go look for babage and so babage is a completion model it is um it is not necessarily a uh a chat a chat model so it's really up to you what you want to utilize Ian we could also do turbo let

me just take a look at the cost again yeah this is really really inexpensive so I think I really want to attempt to train babit hopefully this is available to you in the future and so we'll give this a moment here and so we all we have a standard which is fine and we'll go ahead and deploy that and I'm just going to go back to the model catalog and read a little bit more about it I just want to see if there's anything I need to know about like are they getting rid of it

at any point hopefully not it's smaller than primary models yeah they're not talking about getting rid of them so I think that's totally okay let's go down to our deployments make sure that that is available it is we're going to go over to completions because we can't use it in chat and let's give it a go and see what it does so we'll just say uh hello and then it will generate more text hello I'm a 3D artist and I have um a studio in Germany so the idea is that because babage is doing autoc

completion we can provide IT training data so that uh it will reliably create data that we want so we have to think about something um that we want it to do right and so this is something that I'm I'm trying to think about because if you are trying to train a model you need to have a certain amount of data and you might have to synthesize some training data so I'm going to go here look for babage fine-tuning uh training training data set let's see if we can find something that we could utilize okay so

give me a moment and see if I can find anything okay so I'm not sure but if we have this reply maybe we can give it kind of a style of a way it speaks so um you know I am writing to you we'll generate this here we see I'm writing to you to inform you that we have received the scholarship so basically it's kind of like it's writing a um uh a letter and so maybe there's a way that we could uh synthesize some data that is going to be um uh something that we

can utilize for for our training data so that's one thing the other thing is like what format does this data have to be in right so we know that we if we want to uh train we're going to have to upload data here first so if I remember uh how this works we go to fine tuning and we're going to say we want to fine tune a model and here we have babage which is a completion model and we'll go continue and here we have the model suffix and then we have our open AI connection

model one and I'm going to go ahead and hit next and so this is what we need to figure out is our training data okay so uh let's find out how we need to prepare that asking for Json L we've had a lot of trouble with Json L um I'm not sure when but I know know we have and so I'm looking for example ah here's some babage right here so here it says um the training validation must be formatted in Json L uh open a command line interface includes a data preparation tool um that

validates uh and give suggestions and reformats of the training data in the Json L okay so that seems really interesting so what I'm going to do is copy this over here I'm going to over to chat GPT I have a lot of ml llm subscriptions here and so I'm going to paste this in here I'm just going to say um I need to create finetuning training data for um babage um so I need to generate a Json l file following this format I want to train the model to be rude and uh to be rude

and have spelling mistakes when completing when performing completion can you create me a Json L file with prompts and completions okay and so I'm going to go ahead and ask chat gbt to do that um so here it's not really doing what we want so like uh you know babage is not a um chat uh chat but it's completion so the completion needs to continue on the prompt please try again it didn't understand it thinks it's doing a conversation okay yeah yes generate more as much as you can okay all right we'll see what we'll

get I'll give it a moment here to generate out okay all right so it's saying it's generated an additional file here let's go ahead and download it I'm going to go ahead and open this here so I'm going to go here and just open this up in um in vs code let's take a look at our training data here I was trying to do something else and so we have some data here it's not a lot of data to be honest so what I'm going to do is I'm GNA copy this or you know I'm

going to upload this over to um Sonet because son's doing a better job these day Claud Sonet let's go over to here and I'm going to go into here just give me a moment here to get it open and so I've uploaded my JSL file I need to create more training data based on the Json L file um the purpose uh the goal is to fine tune babage to complete text That is rude and has spelling mistakes please create more training data for me okay so hopefully it understands and we'll give it a moment there

I mean it's not really it's not it's it's pretend r okay oh come on so it says I apologize you don't feel comfortable generating or assisting the creation of intentionally rude insulting or air filled content even if it means uh it meant to be pretend or training purposes perhaps we could be more constructive okay well what other what other ideas do you have freaking son it and this is not very helpful um let's see I need a completion training data set for babage so could we train it for classification maybe it can help us try

to figure out a data set here okay so here what it's doing is what are we seeing here so the it provides a variety of classification tasks that Baptist could be trained on so we have sentiment analysis um text of genre classification fact versus opinion question versus statement movie genre classification um grammar I'm not sure if I trust with grammar number number classification parts of speech um categorizes emotion feels feeling butterflies so first I just want to see like if I was to do this what would it what would it produce like do we even

need it to do that let's go over back to um bring this down here below and I just want to see what happens if we I'm just going to cancel this out for a second and let's go back to our completion and try this and so here it's uh it's not it's making a big mess right so determine if this word is a noun a verb or an adjective or an adverb categorize this emotion uh feeling butterflies in your stomach categorize the animals mammal reptile bird or fish uh you know I kind of like this

one and you I want to just make one change here I'm just going to do this and then put a and see what happens yeah so we're getting a mess of things so maybe we'll do this we'll just have this categorization of an animal based on those okay so I'm going to go back here and say uh great let's make training data for uh animal classification in the prompt uh make sure it ends with a colon in the string colon string okay and so I think it just should have a colon on the end there

let's see what it can do and so there's a good start um I'm going to need a lot more data than that so I'm going to go ahead and copy this and I says like you know can you format the file to be uh for you know can you format the document for Json l so per line instead of so many line breaks okay because right now it's not formatting for Json l there we go that's a lot better and uh I'm going to go over here and make a new file I'm doing this off

screen for inv visual studio code so I have it over here and I'm just going to go ahead and grab this okay and I'm going to say here okay great give me more training data generate another file with different data for the same purpose for for classification great give me more training data in a new file great so let's see if we can do that hopefully it doesn't repeat anything I don't think it will uh so far it doesn't look like anything's been repeated I'm going to copy that and I'm going to say again more

data and hopefully we're not getting oh are we getting repeated data looks like repeated data okay we just have to be careful here because it might produce identical data which is fine but um I don't want any identical data and then I'll go here and so I have about 55 lines I'm hoping that's enough to train it um I don't know how much we need to be honest but we will give this a go file save as and I'm going to name this to my desktop this is just going to be uh animal class classification

jonl great and so I've saved that file I'm assuming that it's fine but uh they did say that there was some kind of tool that we should use U for classification uh see the more train the minimum uh is 10 examples but a small example is not enough best practice is to at least have 50 highquality examples however it's entirely possible uh that it might require 1,000 so we don't know how much it's going to require but we have something to work with right and then here they're suggesting whether we want to adjust it okay

I'm just going to assume that our data is fine there I'm going to put this into our repo so you can get access to it okay press period here and I'll just uh make a new folder here and we'll make a new folder we'll call this fine tuning and I'm just going to drag on that file over I've had so many problems with these JSL files but hopefully this time it will work and so we're going to go back over to here to fine tuning we're going to fine-tune our model we're going to choose babage

we're going to hit continue um this will be animal fine tuning maybe a an F uh classification animal class here there we go and class next um I want to just upload the file directly if I can so I'm going to go ahead and upload that now and I'm just uploading that file we'll give it a moment all right and so we've uploaded our file and there's our data let's go ahead hit next um select data set to personalize your model or opon so do we have to have a validation data um I'm going to

try with none but it seems like that would be a good idea to have that select a data set to personalize your model okay so I'm gonna go ahead and hit next I'm going to leave it all alone because I have no idea how to tweak those we're going to hit submit and we're going to wait for that training to finish and we'll be back here when it's done okay all right so we are back and um let's see how long it took so I know that this started at 244 right oh that's when it's

started the training and um the thing was is that it did take a while I want to say like an hour and a half let's see if we get a refresh does it tell us and oh okay so down below here oh we got literally go to the next page so we started training at 244 and I'm I'm assuming that these are each Epoch of its training okay and it succeeded at 325 so what an hour 2 243 sh have an hour and of course our data set was really small so the question is will

it work well I have no idea so let's go ahead and um deploy our model so I'm trying to think where this is it's a little bit different when you're using this versus um uh open a I think there's some variations there so here we have the option to deploy so I'm going to go ahead and deploy this and not sure if we have to have it on compute oh looks like we can do standard so that's pretty cool and this is that so we'll go ahead and deploy that give that moment there says this

model has been retired you may consider using a suggested replacement model and so they're may be talking about the underlying model uh obviously this one's not immediately retired so babage retirement June 14 2024 it is past that date uh these models are not available for new deployments deployments created uh before 23rd or after the 24th will be here okay but what does that mean does that mean I can't deploy clearly I'm doing this but yeah I would just say obviously if you run into this issue uh that model might not be that might be model

might be unavailable and so you'll just have to find whatever the new one is like turbo 35 or um whatever so but again I was trying to be the most cost effective here and so I I chose it based on that and so hopefully you're not running to that problem it's unfortunate that the retirement date is so close to that so I have to wait for the uh the this to finish provisioning so hopefully it doesn't fail and it's creating so we'll just wait until it's done okay oh by the way I'm still waiting for

its deployment so let's taking quite a long time um I'm not sure if it has to do with it being an older model but I'm going to have to wait here and see what happens okay down below here it says model retirement date J January 2nd 2025 so technically it seems like this one should be able to keep around but um I mean we could take a look and see if it's actually available um so if we go over to oh sorry completions it is here so I guess my question is if it's here but

it's still creating can we use it um and there's only one way to find out and that will be to use some of our training data so I'm going to go find uh wherever Visual Studio code was so I have it down here and I'm going to grab one of these so I'm going to grab the crocodile one it should identify it as a as a reptile okay I'm g go paste this in and hit generate and see what happens it says the API deployment is not ready for the resource wait and so I guess

we'll just be waiting a while okay all right so it's been taking taking a long time for this deploy I can see other people are saying that they've waited 5 hours um let's go take a look at notifications and see what information we have because maybe it's told us that it's deployed um no no new information under notifications it'd be good to see the previous notifications yeah I'm not really sure it doesn't seem like it should take this long to deploy oh are you joking as soon as I talking about it finished well that's probably

because I was starting to come close to deleting it so that's good that it actually did that let's go back over to our Endo it did take quite a while though not like an hour but long enough that it was uh was making myself question whether it was going to be done so again just going to copy a random line out of here now it should um be smart enough to I'm hoping that it'll just produce reptile here and the one thing I have to understand when you're doing fine tuning is that if you provide

too much fine tuning information you can end up over tuning or overfitting the data and that means that it's not becomes less creative um in terms of being a reply but let's see what happens so we have um s that it's like colon then colon again I didn't really plan for that so I'm going to go ahead and do this let's see what happens fingers cross that it worked and um I mean I guess it's better than before but it's not producing just that one word reply it's going amphibian uh frog reptile oh you know

what it is we're on the wrong model okay we're on the wrong model so that would explain it and we'll switch over to our new one and can we switch over to it there we go so let's see what happens if we try it now so that other one's obviously was not babage this one is and so it's going reptile and that is what we want to happen next now it's producing a lot more information that we want and so again maybe more training data would help it um uh to prevent that or it could

be the temperatures let's say we brought the temperature down and I bring the temperature all the way down let's see what happens if we do this so we'll put that in here and the answer isn't um in this case uh reptile not amphibian see here would be amphibian we'll see what happens if we do this and I mean technically it's correct it's doing reptile but we again are getting too much information um so stop sequence this would be make the responses stop at a desired Point such as the end of a sentence specify up to

four sequences here so this might be useful if if we could um figure out a stop sequence but I think we would have to train it to have like period or something and then when it ISS should had a period it would know to stop but it looks like it's technically working so let's go and test this with something else um so I'm just trying to think of an animal that might not be on here and I'm not sure if they have like woodpecker in here do we have woodpecker I don't think so so let's

go ahead and see what happens we'll say woodpecker and we would go on the end here and generate it out and we get bird so it's good except for the fact that it is again overproducing information and I think the only way we' be able to fix that is more training data so it could be it needs a 100 it could be that it needs a thousand um but we we're able to do this so we achieved our goal in a sense that it can identify it I'm not going to retrain it because it's so

much work to train um but you get an idea of like what you'd have to do to really get it there okay um so I would say that we are done here and we can go ahead and clean this up now I'm not sure if it costs anything to have this deployment so I don't think it does because it's serverless go take a look here and it says standard so that means that it's going to consume tokens so I'm not really concerned about it right now uh it's not going to cause us any issues I'm

not going to clean this up right now in case I want to keep it around um but in the final the clean up video we get rid of at all if you want to get rid of it you can I don't think I'm going to plan on using it but I'm just going to keep it around just in case and I'll see you the next next one okay [Music] ciao hey this is Andrew Brown in this video what I want to do is start working with Dolly so dolly is a model um that generates out

images and we can do this either in um we can either do this in Azure AI Studio or open AI Studio I'm going to do this in open AI Studio here today for whatever reason this the future and open I is not available you can absolutely do this in Azure AI studio so on the left hand side you're going to see that there are images and I need to remind you that if you have yet to do so you need to make sure that you have in the model catalog gone in here and uh request

access for models now I just went across the board and said give me access to everything um I think that if you fill in one it basically gives you access to the entire family so you might not need to fill in all the forms the point is is that I've already done that and so I can access um Dolly so in order to utilize it I'm going to need to deploy something um and over here says Azure open resource located in West us region images is only available in the following East US Sweden Central and

stuff like that so I run into this issue all the time where certain malls are available one place and not another so in this case I'm just going to go make another one and uh I know it's frustrating but it's just the way it is it's not like it's going to cost us more money by spinning these up so it's not a big deal um I meant to go to portal. aure not a big deal portal. azure.com and so I'm going to make my way over to open AI Studio or open AI here and we'll

just spin up another one here and we'll say uh you know I'm just going to put in the same Resource Group because I don't need to have a bunch of these we'll just go open I studio and this one's going to be in East us okay so this's just be open AI Studio East and just put some numbers here I'm just going to choose the standard we'll go ahead and hit next next next and I'll go ahead and hit review uh review and create and the reason I wanted West us is because there is another

step where we're going to find tuna model and that only works in West us or Sweden so we're just going to have this issue of figuring out which uh which place we can uh utilize that model so we'll just wait a moment for this to provision all right let's go take a look at that resource now so just click into here um and from here we'll go to open AI studio and I do have uh the other one open just in case we want to use Azure Studio but uh for the most part we want

to do this in open AI uh Studio here and so on the left hand side we'll go to images and we'll give it a moment and so the idea is that we have Dolly 2 I could have swore there is Dolly 3 um and it really depends maybe on what we have deployed but here I haven't chosen any deployment it's just showing me dolly2 here but we can go ahead and give this a try so I'll say Tory gate on the moon that was something I was trying before uh it had a really hard time

with it Dolly 2 as um not necessarily the moon but if I told it to make it like a wood block I don't know why it just really had a a huge struggle but yeah there's Tory gate on the moon um and there are a few options here like the number of images and the size uh what's interesting is between the models like Dolly 2 and Dolly 3 the options actually change which is interesting here I'm just generating out a a few more um and so that's going to work there there we go so imagine

we want to use something newer like Dolly 3 I'm not sure yeah right it's right here okay so we can go here and I I'll click on this and so this is Dolly 3 I'm going to go ahead and hit deploy and available quot is low for this model uh in this regions description you can try and create this deployment but you may face some issues with performance and that's fair so we'll go ahead and do that I mean eus is the most popular region so at least where I am so I can understand why

that might be the case and now that that model is deployed we'll go over here to images we'll drop this down and choose Dolly 3 and we will say to on the moon and notice that the options are a little bit different we have vivid natural so maybe we'll try one in U Vivid first and we'll just say um Tori gate on the moon I notic I said on the moon and none of these are really on the moon but maybe this one will do a bit better and we'll go ahead and say save here

and try this it says if you created the deplo the last five minutes please wait a moment so we'll just wait a little while and then maybe our deployment will be ready okay all right so I've waited a few minutes here let's go ahead and see if we can generate that out and I mean it seems like it's complaining less now it's taking its time to generate out so we'll give it a moment to generate uh Dolly 3 does take longer to generate than Dolly 2 but the idea is that the results are generally nicer

and so that actually makes a lot more sense contextually so Dolly 3 huge difference there let's go ahead and try um uh just standard okay and I don't really know how much computes Behind these things so I mean these models seem to run serverless so they generate when they want to generate there uh not as nice but um you know closer we want so I'm going to go back to Vivid here and I wonder if we can get the uh code and see if we can utilize this now I've had some poor success getting this

to work prior but we're going to go ahead and launch this up in our um environment here and see if uh we can programmatically work with it so I'm just I think it's starting we'll start that up and once that's started we're going to try to Pro programmatically generate out Dolly images okay all right so let's go ahead and launch Jupiter Labs you know the routine probably at this point depending on how many times you've done this uh we'll go ahead and hit build in this course we do this a lot so work in uh

this environment here it still has some tabs open from before going to go to our AI folder uh just understand you might have to go all the way back here go to Ai and of course if you're looking for this code it will end up in the GitHub exampro Co slash Azure examples I always upload these at the end so if you're looking for them you can find them there we're going to make an one here and just say Dolly I'm not going to specify that it's Dolly 3 uh we're just going to call it

Dolly and in here we will double click and make a new notebook and give it a moment to load here and we'll drop this down here and choose um the SDK 310 and I'll just say Dolly there we go and so hopefully this code works and I'm saying this before I used this didn't work out as expected so I'm hoping this time it just works and maybe the reason it didn't work was because of that issue of maybe too many people were using um it at the time so I like to break mine into individual

components so I can really see what I'm doing okay and add one more here and um I mean we're importing it but we'll probably also need to install it I'm not sure what the python package is called here so I'm going to go ahead and take a look maybe it's just open [Music] AI I mean this is open AI for sure but this one in particular is azure open AI so we're here on uh the the repo here so I'm just going to search Azure open AI that's probably what it's called let's make sure that's

the case I can't tell it's from someone named Ryan uh that makes me question it quite a bit how to um install they really should give us all the instructions and not just some of them I can't stand it when they do that they don't give you all of the information but maybe I can go over to here maybe I can click this and might tell me there it is oh it is PIP install open AI I really thought it would say Azure open AI but well apparently it's just okay all right so we'll go

ahead and do this and hit run and there's a couple things we're going to want here we're going to want our endpoint in our API key so we'll just go oops that one is doing that so we're going to just wipe this this out here and say endpoint this will be API key and I have a feeling that I'm going to have much better success this time around I don't know why I just think I am if we go back here they're making these available this just looks like it's the open AI endpoint so this

is the same endpoint for whoops for open AI so if we were to go let me just find a tab that actually is in regular Azure there's none of them that are there so I'll have to go over here and just say portal. azure.com but what it's getting for the endpoint is the open AI service so if we go over to open aai and we're here and I'm I think we're at East right now it is basically the same information here the um under security networking Resource Management key endpoint so it's going to be this

key and that endpoint that's all it's doing over here so I'm going to grab this one here and paste it in and so we'll run this then we'll run this and then we'll run this and so we have this um okay great this doesn't necessarily save the image anywhere okay so that one is thinking and um you know I might while I'm waiting here I might just go ask Sonic to help me I use Claud Sonic quite a bit and I want to save the image and I just don't want to have to figure that

part out I hate working with files um file formats here so I'm going to go ahead and copy this um and say I'm doing this over here in sonnet as you can see over here okay and and again just doing this off screen here give me a moment here now I'll drag this over okay so I'm going to paste this in here all right so now I don't really want insights from it but I just wanted it okay it's going to write it out to me um okay how got give me the code that I

would need to save the image to the current folder okay let's see if it understands that not maybe the best written but hopefully it'll provide that for us and looking at that response code Etc it looks okay okay hopefully that works and I like how it gave me a time stamp that's really nice okay and so here we get a result we'll go ahead and run this now and I'm just going to go into the next line I just want to print this out and see what we get here image URL say this so there

we have it okay cool now can I just download it is this publicly available I'm not sure it say private images at least it placed it somewhere so we know that it is somewhere I'll run this this may not work because of permissions issues I'm not sure and so here it says request is not Define fair enough um we need to import that so I'm going to go here and also we might need date time as well this I'm just copying from the code over here if you're wondering where I'm grabbing that and so I'll

run this again image is saved oh I should have probably told it where I wanted it to go I wasn't being very um oh there it is anyway it's in the it's in the same place that's fine let's just click it see if it opens I think we can preview images in Jupiter there we go so it works excellent so I wasn't sure if that was going to work but it did I'm going to go ahead and just delete um this I don't need it and so we will consider this done so I'm just going

to clear out my credentials here and say restart and clear all outputs restart and I'm going to go ahead and download this and we'll go over to GitHub over to our Azure examples and I'm going to hit period here on my keyboard okay and we'll give it a moment we're going to go ahead and make a new folder and I'm going to call this one Dolly and so in here I will upload this and so if you want to just work with this code because you know that for sure it worked for me and you

want it to work for you you can absolutely utilize that let me just drag this one on over and uh it didn't clear my outputs it did not clear my outputs so that is not what I wanted just going to go ahead and delete that also I don't think I even went to the right folder either okay well give me just a moment here we'll go back over to here oh you know what I had to save the file that was my problem okay so we'll download this again and I'm going to go back over

to here and I'm going to make a new folder called Dolly and it's not really that big of a deal if um if I share my credentials in here because they are going to um get cleared out anyway but sometimes GitHub will lock down your account it's kind of annoying like for your benefit right I don't want to have that issue there so that is now saved we'll just go ahead and say Dolly 3 example we'll commit and push and I just want you to know that in AI Studio it's the exact same kind of

experience even though we don't have a project here right now we'll probably do something in Azure AI Studio at some point or explore some of those features but for now this is azure opening ey service yeah for now we're just going to um stick with uh um open AI right okay um but yeah uh that's it for that and if you're concerned about the model you can undeploy it so I'm going to go down to deployments here and I think this is our West us one so I'll I'll go just click out of this one

I'm go down to deployments and I'm going to go ahead and delete this here just so that I'm not worried about it and so that one is now gone and I'm going to keep this around I'm going to end up cleaning this up later on the open AI Studio but because I might need to switch between regions I just want to keep this one here so I'm not having to create one every time I want to uh do something okay but see you in the next one [Music] ciao hey this is Andrew Brown and we

are taking a look at what is rag so rag stands for retrieval augmented generation and its access patterns I put extra emphasis on the S so you know that it comes in many variants for retrieving external data to inform an llm agent prior to to providing a response to the user um I should have also highlighted the word augmented because that is what it's doing it's augmenting the llm with information okay so a very common rag pattern that is representing information as vectors in a vector database and then using a search engine to vectorize a

search term to return relevant documents is what you're going to see most often um now you don't need a vector database to produce rag again rag is just about getting external data from the llm to help it be a uh uh smarter based on its limits of knowledge okay but let's look at this example so that you have some basis of rag and the most common uh way that rag is used so on one side you have your data store this is going to have the documents you want uh your llm agent to have access

to on the other side we have the agent so here we're representing it as a GPT um and then in between it we have our Vector store so a vector store is often a document database those are uh really good at storing Vector uh data so mongodb is right now the leader uh in document databases and it makes an excellent Vector store there are many options um uh but anyway the idea is that you have this thing uh which is Vector data and Vector data is really good at correlating relationships between things so the idea

is that we're going to take our documents and we're going to use an embedding model that is going to turn it into vectors and when you see the word embedding you should immediately think vectors because embeddings create vectors okay and all it's doing is it's just taking your data and breaking it up into uh Vector data and determining How likely something is related to something else so the idea with this embedding model maybe we'll make another slide for this but the idea is that you have all different ways to embed your data to represent your

data as Vector information and the vector is just basically um the distance between similarities of your data okay so you could make it so that similarities are based on um how similar the word is uh based on the understanding of what the word is um like maybe the length of word you know so there's all different types of ways you can embed your vectors we're not going to worry about that right now but once we have them in our Vector store now we want to uh use that uh Vector store to determine what documents to

return based on what the agent wants so imagine that um the agent provides a search term so maybe we are looking for um apples right like tell me uh the best apples in the world so it's going to take that search term and we're going to turn that search term into Vector data because in order to search the vector store we need Vector datas to search it against so the embedding turns it into Vector data and we give that to our search engine so in this case we have Lucine which is an open source engine

a lot of search engines are based off of Lucine and so it's going to use that Vector data and whatever means it wants to search the vector store it might have other algorithms in play there that it's going to utilize along with the vector data to retrieve um uh uh uh Vector points and that stuff is either going to be stored in the vector store or it's going to be pointing to the the the chunks of data that we want to return back to the agent so that that data is going to then be returned

and the agent is going to do something with it this does not scratch even the surface of Rags um and I wish I had more time to talk about it not in this course but in other courses like maybe when I do my J centrals course I will cover rag in its entirety which if that's even possible um and the reason why it's so hard to cover is because rag is again it just means getting data externally so instead of this Vector store this could be um reaching out to the internet like using a search

engine um like a Google search engine uh to grab information um you know the agent um is explicitly saying what it wants but there might be an implicit way to do it there's different algorithms that can be used there's different embedding models that can be used um so there's a lot that goes on in here but don't get scared by the word rag because all it just means is access patterns for retrieving external data another interesting thing which we're not really representing here is how the data gets inserted into the llm agent okay we'll talk

about it in two seconds here but uh return information is directly inserted into the context window I guess it was right away the context window of the llm and the context window is just imagine the information you're going to provide the agent the llm to produce information so literally we're taking the text and we're inserting it the the raw text into imagine you're typing to an llm like telling it stuff it's like you're copy and pasting it right into the box and and providing that information directly uh there and so for me I was confused

for the longest time because there's always talking about vectors and I thought rag had to be related to vectors and it doesn't necessarily have to be um but you don't you don't feed Vector data to the agent to the llm right you're feeding the raw information um just as if you were to copy and paste it okay um and you know look out for these embedding models because sometimes you'll see It'll be like here we have cooh here as example but cooh here uh they have llms a family of llms and so they'll have like

command R and it'll be like command R embeddings and that embeddings model all it does is it vectorizes um your data in a particular way but if you are going to embed right you want to use the same embedding model for searching as you are for um storing your data because if you use a different Vector Mo uh bedding model it's it's optimizing uh that database that insertion to that database a different way than the other one you're going to just get weird results so make sure the embedding model is is the same same on

both sides so a rag access pattern doesn't require a vector database it's just that that is the most common way that Rags are used uh what's important to remember is if you have to fetch external data then you're utilizing a rag and while this looks very um uh involved like you're programmatically doing a lot of things often rag is being implemented by AI services so you're unaware of the underlying moving parts so if you use chat gbt and you've uploaded documents you're using rag right it just it doesn't look that way but it's there and

they're not using that term because they're trying to keep it customer friendly uh for some services so hopefully that gives you an idea of what reg is and uh I'll see you in the next one okay [Music] ciao so what is document cracking well document cracking is the process of opening files and extracting content it's an alternative and cooler sounding term than document extraction so why do we even need the term document cracking I don't know so marketers got uh clever and so I just want to make it very clear that these are the exact

same term I also want to establish that when we're talking about a document we're not necessarily limiting ourselves to the concept of text it could be images it could be video it could be audio the idea is that we're representing our data as a document in this term but it's data extraction or document uh document extraction the reason we say document is because we we have a format and we have to get that that data out of that document format or that uh format that is not exactly uh what we want to work with so

uh why is document cracking important in the context of llms well let's talk about uh two kinds of llms that we have so the first we have is gp40 the O stands for Omni because it's multimodal which we'll talk about in a moment we also have GPT 3.5 turbo and so what are the difference between these two models um well GPT 40 is multimodal mean meaning it understands via multiple modalities yes I know the terms are getting really complicated here but modalities is is like saying senses so imagine you have the sense of smell the

sense of vision the sense of taste and these llms have their own senses or modalities and it's how they understand uh input and and what they can produce as output and so GPT 40 can understand images and text if you've ever used chbt and you've uploaded you've asked it to generate an image or provided an image it can uh describe it to you it can output one um but obviously uh it works with text now GPT 35 3.5 turbo which is a uh older model it only understands text so H its whole world is text

okay and so we call this monomodal because it only has one sense or one modality so now now imagine that you have documents that you want to utilize the the information of it so that um we can make uh informed decisions or more interesting uh outputs from our llm agents so now we said when we want to get data externally we use Rags right okay but in order for this data to make sense we need to turn it into a format that it's going to understand and this is where we would pass it to a

parser and so the parser would would output it in a format that it understands so if we have a PDF right yes PDF is a text document but gp40 cannot open a PDF and it cannot make sense of it so we need to open up the document as it says up here right and extract the plain text content right and then it's going to go to a rag which possibly is stored in our Vector store or wherever and that's going to work now but remember the gbd4 understands images as well so let's say it takes

images but it doesn't take a particular type of image right so we could pass it to the parser and it will turn it into an image that's going to understand same with audio same with video right and if it's audio maybe we don't have audio over here it doesn't understand audio but we could turn that audio into an image representation uh of its if it's of its waveform or we could um turn it into some kind of plain text format maybe it's notes or or other things like that okay so just understand that it's going

to turn into a format that it can interpret if we look down below to our monomodal friend here all it understands is plain text so if it's a PDF we open up the PDF and we extract out its text if it's a video this could be transcriptions right it could be it could be extracting out all the spoken texts or the captioning if it's an image um I'm not sure if it's audio you know we described it there before but the point is that it's turning into raw plain Text data that will get interpreted by

um the LM and so a parser is used to convert data into into a format the LM can understand um I don't sounds very similar to the word Transformers but the term here is parser it maybe Transformers could be an interchangeable term there are many open source tools uh some simple some complex some that are utilizing llms themselves um to transform or or parse that information um but you know understand that uh this is a huge challenge uh for a lot of r pipelines is how do I get a document format into the format that

I need for my my um my llm to understand [Music] okay hey this is Andrew Brown and we are taking a look at hnsw and the reason why is because when I was working with Vector databases this initialism kept coming up over and over again so I figured it was worth our time to just quickly understand what it is so hns uh w s stand for hierarchial navigable small world and it's a grass graph-based algorithm that performs Ann searches in Vector databases so we know that a vector sore Vector database is very important um for

um utilizing in rags that's what Rags primarily use and so there are different algorithms to which we can search um and one kind are graph-based algorithms and the most popular that I keep saying is H and s SW so how does it work um well it combines two things uh it combines something called skip link lists and the other one is the navigable small worlds so when we're talking about navigable small worlds I'm going get my pen tool out here imagine you have a node that represents um a vector basically some data that's been embedded

and um if you wanted to compare uh other nodes against other nodes for simp similarity right so let's say we're starting here we're going to create a graph of similar notes right because that way if we are searching our database instead of looking through every single node um what we can do is we can just find one that's similar and if we don't like it move to the next closest one to the next closest one until we find something that is our best match um now if we just had an nsw which is something we

could have we just a big graph um that is definitely more efficient than individually searching every single node um but the idea is that by being able to connect um graphs with other graphs right and taking one node from another one we just end up with a much more efficient algorithm that's how we get the hnsw so that was a very mangled description of hnsw um but just understand that hnsw is an efficient algor for creating a vector database you're going to see it again and again so I just wanted to try to best describe

that quickly [Music] okay semantic ranking is a term you're going to hear when working with Rags when working with search engines specifically for llms um semantic ragings are algorithms used to rank search queries based on semantic understanding think of the word semantic if you have a sentence it has semantics like how it's structured and related uh to each other and if you were to do a traditional keyword search and it returns back the results it doesn't semantically understand anything there's no language understanding there so even if you have a keyword search doesn't necessarily mean you're

going to have the most relevant search results so semantic reranking or semantic ranking is going to uh give you better results and you can see here on the right hand side that um this is from the Azure Microsoft documents about how their rerer works and the idea you have this word capital and it's trying to semantically understand how it relates to other words like we see crime and punishment or on the left hand side we have capital in the context of money so tax money Investments and on the right hand side we have in the

context of geographical information so that's the semantic part to it now the uh semantic ranking can be a bit confusing because when we were talking about embeddings right embeddings are um models that take data and turn them into vectors right and when they vectorize the data these embeddings capture semantic understanding because it's taking the data and putting in the database near other nodes that are are related based on some kind of semantic understanding or whatever they want to decide what it is so when you are using a vector store a vector database say either or

if you want and you're using embedding you don't need a a semantic ranker because you already have semantic understanding depending on the algorithms uh the algorithm whatever it's deciding to do there okay um so the thing is is that why do we care about this then so some companies or many companies already have uh traditional data stores right like they're using postgress or whatever and they have a search engine whether it's elastic search or Lucine or solar whatever you want whatever search engine you want so they probably just are used to using using keyword search

right and if they want to integrate that with their rag they're going to find that it doesn't always pull the most relevant documents so how can they get semantic understanding and that's where the semantic uh ranker comes into play so companies that want to use traditional data stores instead of a vector store to search against uh for their EGS can get similar results that a vector store would provide uh also uh a semantic ranker can be combined with a vector store which could would yield better results okay um now there's also a product that I

know of called coher ranker by the Company Co here they they have LL models but they also have an embedding model and a a ranking model and um they claim that their ranker works even better than than uh without embeddings so if you didn't even want to use embeddings and use a traditional database they say their their ranker is good but you know this all depends on what kind of embeddings you're using um and uh you know what you're expecting back from your data so just understand that you could just use uh keyword search you

could use semantic searching you could use it with embeddings you could use without embeddings but just understand that a semantic ranker or semantic ranking is an algorithm that's going to rerank search queries to best optimize them [Music] okay all right let's take a look here at Azure AI search which is formerly known as Azure cognitive search they've dropped that cognitive brand because um in the world of geni is just not AI enough for people to understand what it is so um it is a search engine for both traditional search and for rag most often for

rag these days it combines a search engine with AI features to provide advanced search capabilities ideal for applications needing full Tech search Vector similarity search Hybrid search so there's a lot of search options here and when we go and do the labs you're going to see there are a lot of search options I can't even make sense of them all but that's okay we just need to practically know how to work with it for our exam let's take a look at the components that are here so on the left hand side you have your data

sources where you are bringing data in you're bringing them into your index and your index here is the vector store when I was going through the labs I did them first I was assuming that it must store it somewhere else like Cosmos DB some kind of um document database but it turns out that it has an index and the index is the vector store um and I guess the thing is that the data that it's storing is basically it's an index so it's just storing references to those documents and maybe chunked data it's not necessarily

storing all the entire contents of those files uh Azure AI search has some additional paid add-ons so it has a customized entity lookup scale um it has a document cracking image extraction we talked about what document cracking is so that is a feature it has and then semantic ranker and we talked about what semantic ranking is as well so you should know that looking at the pricing you can see here it goes from free to storage optimized and can get very expensive up to $5,000 a month um but the tier that you're want going to

want to use and I'm pointing to standard but the tier if you want to use any of those Advanced features like semantic ranking and I think um we do use it in the um fall alongs we need to show off semantic ranking as we use that um B tier and it says there on the right hand side per month but understand that it's um you don't pay that up front right it's going to be whatever it is per second per minute um uh billing there so we're going to have low cost even using the basic

tier let's talk about the components that are involved within Azure search the first is your data store this is the data that would be indexed for retrieval so you can use Azure blob storage Azure SQL database for your relational databases Azure cosmod DB from getting from uh your uh document database you have the index which is the actual Vector store here um it's going to I assume store chunk data and references to your data sources the default Fields here we have is the key so it's a unique identifier searchable created using the full Tech search

filterable used in filter Expressions to return only documents that match criteria sortable only used to uh search results uh F or face table so facets which are uh UI elements that help filter results based on the list of known values where these UI elements are I do not know retrievable and so this is your basically your projections What fields you want to come back but I noticed that um there's a a button that you use in Azure a search that's going to set up everything for you and it adds additional Fields there like um uh

the vector data and also chunks so just understand that even though they're not in the default values depending on how you set up as your search those fields will be there and when we set up a rag those are two that you are going to see we don't really talk much about chunking but chunking is when you have data from your data store and you break it up into chunks so that it's easier to work with and then you store it in your uh Vector store just imagine a big document breaking it up into Parts

okay and ining those as records in your index we have an indexer which is a data crawler if you are from the adus world unus glue you've heard of a data CER before uh it extracts data from the data source and populates indexes and you have skill sets this is um attached to indexers to process data to extract from the data source uh you can create custom skill sets often leveraging AI services so um they have a bunch of pre-built pre-built ones in and so some stuff they'll use there are the Azure I services and

if you see the word data crawler you should think elt or ETL uh because basically this has built into it a very basic ETL from your data sources um over to here I did a list it here but you can run um uh your indexer on a schedule so you can import it once or you can tell it to run on a schedule based on particular terms and things like that um but yeah that is the basics of azure AI search and there you [Music] go hey this is Angie Brown in this video we're going

to take a look at Azure AI search um so this service in particular is interesting because it can be used for traditional U fulltech search but you can also use it uh specifically for generative ey for your Rags um and basically all a rag is is just a means to retrieve information to make your llms a little bit smarter based on contextual information uh so we can deploy Azure AI search via the um the UI here we should really work with it programmatically because because that's probably a better way um to work with it so

what I'm going to do is I'm going to do what we've always been doing here I'm going to launch up our AI manage uh um compute here and there are quick starts and we can try to utilize these as there's a many ways here but there's also examples here at azer search Vector samples so let's see if we can just get through it programmatically because that's really the way you're going to use it um and here they might have better uh coding examples where when I was going through this basically you're making like posts and

curls um and uh they didn't make this super friendly to use uh so we'll see what we can do there so I'm just going to wait for that to spin up and uh I'm going to click into this one and we'll see how far we can follow through this by bringing into our environment and make tweaks along the way okay all right so our environment is running let's go ahead and open that in Jupiter Labs uh so we'll open that up there give it a moment okay and we'll go ahead andit just uh well I

could have built it there it doesn't really matter um and so I'm going to go back here to step in our AI folder and we'll go ahead and make a new folder and this one is going to be called uh Azure AI search and in here I guess this one here is basic Vector workflow I'm not sure why there are two here so we're requirements.txt file what's in here anything interesting um okay well that's interesting we'll go back to this one here and so from here this will just be it's basic Vector workflow so I'm

going to go here and make a new notebook and call this one we'll choose 130 as per usual and we'll rename this and this is going to be basic Vector workflow okay so we'll scroll on down to here um here we have this which is going to bring in those requirements so I suppose that is one way that we can do it I would rather just kind of bring them all in here and just do a pip install so I'm going to go here and just paste them in and do pip install this and yeah

I'll do it for each one I'm not sure if we could um uh sing those together depends on how you want to do it i' rather just have it in here so that I can see it that looks good to me I'm going to go ahead and add this and drag it on above and we'll go to markdown okay and just so that you don't have to look for this file I'm going to go ahead and Link it here so we know that we are working with this one here so say resources here and paste

that in and just in case we need this as a reference I'm going to put this in here as well okay so let's go ahead and see if those install without issue I don't think we're going to run into any problems and we will wait for that to complete I imagine we're going to have to probably create an Azure a resource let's go through this and see what's going on here so we're going to create an index schema load sample date dat data from a local folder um embed the documents in memory using Azure open

AIS text embedding three large model index the vector and non Vector Fields with Azure AI search run a series of vector and H hybrid queries including meta data filtering and hybrid so you know I I guess you can do like traditional search with uh Azure AI search but most likely what we're probably going to use it for is a rag so getting experience with vectorizing things makes more sense sense because that's how we're probably going to work with it so let's just go through and see what we're doing here so what are our prerequisites um

we need an Azure AI open subcription we need a deployment of the text embedding three large model so you must have an Azure open AI service name and an API key um as they already search any tier but choose a service that has sufficient capacity for your vector index we recommend basic or higher okay so hm I'm just thinking about this here for a second it seems like we're going to need Azure openingi and we're also going to need to have the embedding model deployed so when we use Azure open aai um and we'll probably

do this in another video but when we launch it up uh we have to request models and sometimes it takes time to get access to those models I that's more of an issue in Azure AI Studio not necessarily Azure open AI studio so I guess what we'll do is we will um launch that up and and we have Azure open AI do we have to launch the open AI Studio not necessarily so let's just go ahead and launch Azure open AI so we'll go here yep and I'm going to just make a new group here

and this will be for um Azure open AI or sorry um Azure AI search okay and from here I'm just going to do everything in West just to be very consistent and this will be uh open AI West and some numbers we will choose standard as that's all I can choose here today for Network we'll leave it alone we'll go ahead and create this resource and so there is the Azure open a studio that's when you have like an interface to work with if we're working with things programmatically we'd probably rather just work with open

AI directly so that resource is being created so we'll we'll have to wait for its completion uh but it says we need Azure AI search so we're going to have to create one of those recommend basic or higher enable uh semantic ranking if you want to run hybrid in semantic ranking um I probably do want to do this because uh semantic ranking is something that is extremely useful um and is the preferred way of searching as far as I understand semantic ranking um it is an additional layer an extension of azure AI but it's not

a concept specific to Azure uh search um but the idea is that it takes back your result and then they rank them and so that way you get better results hopefully that is a good description if not it will be in the lecture content uh but this is deployed so before we do Azure AI search let's open this up and see what we have as our options and so here what I'm looking for is the ability to deploy model so here it's suggesting we could open this in Azure Open studio I don't want to do

that here today I want to go to manage deployments and so we'll go here uh it said something back here I missed the text before I click that says multiple features have moved to the Azure open ey Studio okay so you have to do it in the uh openi Studio interesting I personally like to make Azure openi Studio first and then create Azure openi but I guess maybe the order doesn't matter so much here and so we want to deploy a model which model do we want to deploy this is going to be the text

embedding 3 large embedding model I'm not really certain about what the cost is on this so if you're really concerned about cost you could just watch me do all this stuff um we might want to look up pricing here and see if we can find it for Azure okay and I'm going to look for that model and so it's saying a th000 per tokens I'm not sure how many tokens we're using but it doesn't look too bad so I'm going to go over to um back over to here and so I want to deploy models

that is it a fine tune model I think that's a model that we have finetuned but let's go ahead and click into here and see if it's just a base model and so I'm going to type uh paste the name in here and see if we can find it this way and there it is now understand that if this is your first time deploying a model you might have to request access before you can do that so like what I did is I went to the model catalog and when I clicked into here it might

have said like hey submit this so that you get access to it that's something I had to do and I had to fill in a form and then that form um said it would get back in 48 hours and then I checked like a week later and had access to it so understand that you may not be able to do this right away or you might have a serious obstacle uh doing this also understand that certain models are only available in certain areas that's why I'm working with us West um because I know that that

reliably has models here but anyway we can see we have access to all these so that is fine let's go back over to our deployments and I'm going to deploy model and again I'm going to choose this and it's going to be the large here so we'll go ahead and confirm that and so that is one thing we have to do and we'll let that work in the background Azure AI search so that's another resource we'll have to go ahead and create so I'm going to uh I've lost the tab here so I'm going just

have to open this up again portal. azure.com and I'm going to have to go here and go back to the Azure AI services and I'm going to spin up Azure AI search so we'll go ahead and create that and so I'm going to choose the um the one here that is for Azure I search I'm going to call this one a AI search and I got the name if you have to put numbers on there do that notice I'm doing West us we have a few pricing options let's take a look here um and so

I do not want standard I'm going to go with basic uh I do not feel like paying that amount of money but I like if you need to do free you can do that but I'm going to do basic because that's what it's recommending um and I believe that this is not like a monthly cost you incur it's it's based on consumption right so I'm just going to go back here and take a look here so Azure AI search pricing and so you can see here 11 cents per hour so you know if you're not

comfortable with this you can just stick with um the regular one but I'm thinking that the reason why I need to go with basic is that there might be something that I cannot do at any other level so that's something that you know I'm just considering there we'll go ahead and select that and so we have all these things selected we'll go ahead and go next and here you can see it says um 7 $75 one replica I do not need a whole lot of stuff I'm going to keep it at the base and we'll

go ahead and hit review and create there we go so that resource is created I'm going to go back over to here and check if my model's been deployed the model is not available on the selected Azure open AI resource model availability why not so now we've already ran into an issue and uh okay so we have this here and is it maybe the region okay so the region we're in we're in West us and it's not available in West us so it looks like we actually have to do everything in a specific region so

it's available in Canada east it's available in UK South um it's available in East us so I guess this time I'll switch to East us I did not expect that to be an issue so what I have to do it's a bit annoying but um I'm going to have to go back here because if that's the case I I want to make sure they're in the same region it might not matter but I just don't want to have these issues so I'm not going to create the Azure AI search right away we're going to go

back over to um Azure open AI we're going to delete this Resource as that clearly is a failure in this case delete and we'll say delete and we'll create a new one here so I'm dropping this down and we'll choose um aure AI search and this one will be eus so we'll say a uh open AI uh East put some numbers here and we'll stick standard we'll hit next we'll go next we'll go next and we will go ahead and create this we get a moment here excellent so that's going to proceed to create that

and so I think we should really wait till we can get the model deployed before we do the AI search I believe we'll probably also have to create um some storage account maybe that will get indirectly created for us I'm not sure but we'll find out here once this is done deploying all right so that's deployed let's go over to our resources here and we'll go to model deployments and manage deployments and of course we just it's going to open up in the a in aico but this time we might have an easier time um

I'm checking the model catalog I'm just making sure I have access to these sometimes uh it changes based on where you are we'll go over to deployment so going to deploy a model and we want the text embeddings large okay so we'll go ahead and confirm that and I think this time ah so we have some options here standard deployment no other option one no other option here current Azure opening eye token count that seems fine version two seems fine don't know much about these settings but looks pretty straightforward so we'll launch that now we're

in good shape uh let's go back over to here so it says uh any tier but choose a service that is sufficient capacity for your vector index we recommend basic or higher enable semantic ranking if you want to run hybrid and semantic ranking I definitely do is there any additional pricing with those there is but it says the first 1,000 requests are free and a dollar per request so this is now available that's good we're going to back over to um uh over to here and I'm going to go over to Azure AI search and

we're going to create this and this time I'm going to make it an eastus to match where our other one is this is going to go into the Azure AI search here we'll say Azure AI search East see we can deploy it in the East it might work across regions but um I don't know so just going to keep it easy here today so I'm looking for East where are you east east us change it and we're going to go to basic again if you want to do free you can do free I'm going with

basic here today just in case we'll go ahead and hit review and create it's going to be like 11 cents per hour hour it's not going to take us an hour to do this if it does I mean that's the worst case cost we're going to have here we're going to deploy that resource um I didn't see any options as we were setting that up for reranking search did I maybe miss those settings I'm not really sure so oh it's deployed okay so then where is this reranking option so I'm going to go to properties

here um [Music] H maybe settings semantic ranker ah here it is so it looks like we already have it enabled so there's nothing we have to do to turn it on it already is turned on that is excellent we'll go back over to here Azure search Vector python sample okay um so enable semantic ranking already enabled we'd use Python 311 we're using 310 here today but you know it's what was available to us um so that's totally fine which is interesting because down here it's saying 310 or later well that's fine we're on 310 and

so we need to load in those we already did that right so I'm going to drag this on over here and so that's in good shape we might need to restart the kernel I'm just going to do that anyway they usually suggest that when you do those installs there so I'm going to do that so we don't have any issues and so let's go ahead and grab these part by part and try to figure out what's going on here so the first thing is we have a do file um we have azzure identity we have

Azure core credentials and then we're importing the OS okay that makes sense um it looks like we're loading a DOT file I don't see a DOT file in here so take environment variables from the EnV okay so I guess it's suggesting here that we could create them or we could just set them here so the following variables from the EnV file are used in this notebook um okay okay well I mean this would be nice because we haven't solved our EnV issue yet and if there is a way to do that I would love to

to do it this way so I'm going to go ahead and grab all this stuff here and the idea is it's going to load those environment variable so I'll just make a new file here called EMV um uh well that's what it's called rename EnV rename Dov H okay well that's not going to work so we'll delete that out and I suppose them doing this is kind of pointless I'm just going to take that out and so here are environment variables is suggesting so we need a the search service endpoint and this stuff and so

we need to fill out so Azure search service endpoint okay so we're going to go over veror overview and I'm looking for that endpoint I'm thinking it's this URL over here so I'll go ahead and grab that and so we're going to paste that on in here okay now remember that if you're working in a a repo that is tied to GitHub or whichever make sure you do not commit that stuff okay so here next we have our credentials for the search Azure search admin key aure search admin keys I'm going to go to Keys

here I'm going to just assume it's under Keys looks like we have API Keys we have primary admin key let's go grab that one do not share your keys with anybody I'm sharing here with you today because it's okay because I am an instructor and I will make sure I do not get exposed here okay so we'll do that uh then we have the Azure search uh uh search index so the name we want to call it I think in this script it's going to programmatically create all these things where we could just do click

offs to do it but we're going to do it the full way here we have our Azure open API endpoint so we're going to go over to open Ai and I'm going to go into here and we want this endpoint so we have keys and end points sometimes it's called keys sometimes it's endpoint we'll go over to here and grab that as well here that's really interesting now it's making me second guess uh the search compute which apparently I closed by accident so I I want to go back there and just make sure that I

grabbed the correct endpoint I'm pretty sure I did and yeah it doesn't say like keys and endpoint it's just keys right so we just have to kind of infer where things are okay I don't think we need that tab anymore and so that's set now next thing is the Azure open AI key so we're going to go back to wherever that is this is azure opening ey yep so I'm going to grab that key come back over to here and we are going to paste that in whoops we're going to paste that in here there

we go then we have our Azure open AI embedding deployment so we deployed that model and it was called this so we're going to stick with that then we have the embeddings dimensions um I don't know what that means per se but we're going to stick like I mean I understand what embeding dimensions are but don't fully know what we should dial that into but we're going to choose 10 1024 the embedding model name which is technically true and I I think we just stuck with whatever the deployment name was so we had the option

to change it I did not change it I just stuck with the same thing then we have the Azure openi version um I'm not sure if this has changed I'm used to adabs where things do not change very often so I am hoping that that is still new I'm going to go to overview maybe we might that information here somewhere don't see that let's go to Json sometimes that information is in Json here so I'm just carefully looking for that information I do not see it so fingers crossed that we are using the correct one

this one doesn't look super old it's just a few months ago or a month ago um so I'm I'm pretty that this is in good shape so let's run this no problems here let's run this and so all of our environment variables are set that's a lot of environment variables but that's okay so the first thing we we'll want to do is we want to create embeddings so read your data generate opening ey embeddings and Export to a format to insert into your Azure AI search index um so the question is what kind of data

because I don't necessarily have any data here and here it says um goes up two directories into here and then it goes into sample data which doesn't seem to be correct so maybe it's sample data ah here it is the sample data is data and then text sample so this is the data that they're suggesting that we utilize so I'm going to go to Raw here and I'm grab this data and I'm going to go over to here I'm just going to um make a new file call this data Json okay and then I just

want to B it in here hello I just want to paste the data in uh okay so that's not going to work there I'm going to delete this what I'll do is I like just use any text editor you have I'm uh I'm just doing this off screen here okay and I'm just doing this in vs code so find any text editor you have and just save that file I'm going to save mine to my desktop here again doing this off screen data. Json and I'm going to upload that I'm going to press this upload

button from my desktop bring in that data Json file I save I'll be back in just a moment all right there we go so that is now been uploaded and um we'll go back here because we kind of lost our instructions this is where we were working uh working within so we can go back here and so we'll go back down to our code we'll look at the Crea beding section which is what we're on so we have more Imports I don't like importing than importing again but that's totally fine so we'll just do these

kind of separately it's not how I would do it I would I would Place everything at the top and just get it all the way and actually I'm going to do that I don't like just importing randomly like that so we'll do this and then let's see what we have here so we have the open AI credentials and we're grabbing the default credentials here we are providing this okay and I mean that EnV if we go back to this here for a second this Dov are used in this notebook so I I also kind of

wonder if there were environment variables here that are just going to get automatically picked up by the default Azure credential so I'm not 100% certain about that but I have a feeling that this is going to error out on this and then over here we have our um that I guess that's just defaulted which is fine so I'm going to run this and see if we get an error I was expecting an error maybe we will have an error later on or maybe we're fine I don't know so that is fine the next thing is

the client so we'll go here and um we have this where's this coming from oh up here the deployment model so these are all things to find before so that's that's good we'll hit that that's good so far and then we have generate the embeddings okay so again just grabbing these bit by bit I like things to be a lot more modular like this so this is the way I like to do it I'm taking all of this out here and this is going to be data Json if we just slowly walk through the path

and we can uh debug this a lot easier it's just the way I prefer to do it okay so that's looking good so far and then what it looks like it's doing is we are extracting certain data out so we open up this data Json file and look at it we have title content category so it is going here and it's iterating through all the titles and grabbing them entering through the contents and grabbing them and then it's saying create embeddings for the titles the title responses and then we have the data and those are

being embedded then we have the content response and content embeddings okay so that looks good to me what does that mean I don't know it just means the way it wants it to be structured okay so generate embeddings for the titles and contents so then we're enumerating out over all of our items or no it's our input data which is up here the raw data that we loaded the Json data and we are now replacing the or no we're adding we're adding to our Json data the title vector and content vectors so we've generated vectors

or embeddings for the title and content and I guess it going to make sense because like if you want if you bake eddings or vectors it's like if you want to correlate or or find um uh based on title of content that's why we are creating embeddings or vectors per se that that now makes sense okay so I wasn't sure exactly was going on here but now now I understand so hit this here and it's working really hard here it's using our client to do that so it's actually going out in creating embeddings using um

Azure openi so it's done that and then we'll do this and now if we want to inspect this and just take a look at this I have a feeling we could probably do that um so I'll just look at input data here like say print input data and I'm just going to go zero see if that works okay and so now notice we have a bunch of crazy numbers I'm assuming that is our vectorized information yeah it is okay great so that's what it did all right so we just delete this one here and again

let's just carefully walk through this output embeddings to a do doc Vector Json file so then we're going to maybe get the results of that let's take a look here here so it looks like I'm just going to put it in the current directory here yeah it looks like it's just that data the same file there but it's going to have the attached information so it doesn't like something here no such file directory um what do you mean no no such file directory let's just go back here I'm just going to cheat here and just

do like this let's see if that works there we go and so now it's here if we open it up it's probably going to look very similar but now we have these two additional pieces of information okay our vectorized information and I'm not sure if like all these numbers correlate to all these words that might be the case there well that doesn't make sense because the title's not that long right so again I'm not saying I know everything about uh everything but all I know is that we vectorized that data I don't fully understand its

data structure um but that's that there okay so now we have our outputed data with our Vector data and so now we need to create our index so wow we have a lot of imports here again I like to put our Imports up with our other Imports but I'm just because there's so many here I'm just going to stick with it here and I think the reason why there's so much here is that it's implying that there's a lot of ways we can search and we might do all these types of search so we'll go

ahead and just load that there so far so good no problems um we want want to create a search index so we'll need to uh create our client okay so have endpoint credential so I'll run that there good and then we're creating our Fields so if you have an index think of an index like a table okay but a um it's a table that is ordered and sorted in a way that's optimal for searching all right that's the way you can think of an index and so we're saying we want an ID a title a

Content a category title Vector content Vector that's matching our structure from here right so we're creating basically a table or an index if you will um uh of of that structure right but indexes don't necessarily have all the data it's more like Optimal information to find stuff from your possibly from your primary table and so here you can see that we're defining what they are these are all string whether this is sortable filterable things like that um so pretty straightforward we're going to go to our next step here so now it says configure the vector

search configuration so here we're setting up an algorithm um so let's going to take a look at what this one is here the H hnsw is that the hybrid search what is that oh well I'm going to assume it's the same one so we have hierarchal navigable search world so it's an algorithm that's graph based for approximate nearest neighbor search techniques using Vector databases okay okay so it's that crazy thing maybe that looks pretty cool um so that is the algorithm it's using maybe there are other ones that we can utilize so just careful looking

at this we have a profile and then we have um the vectorizer okay so maybe this is how it knows how to interpret the vector data and so we Ed open AI to create the vector data and maybe we're telling it here to say hey use this um so that you can understand how to work with our vector Iz data that's what I'm thinking I'm going to go ahead and run this and so that is good there I'm going to go and grab this one next and I'll paste this in here and so carefully looking

at the semantic configuration and this is I guess if we want to do semantic search and we definitely want to do semantic um or or ranker because that is one of the most efficient ways to search in a vector database and here we can see that we are providing prioritized Fields there's not a whole lot of fields so I guess we're providing the order and so then the rest here we'll just grab the rest here put it here run it and here it has one issue first time we ran into an issue here with Fields

saying Fields is not defined did we miss this somewhere here well we have Fields up here did I not run this uh um I didn't okay so I'm just going to rerun this back in order because sometimes uh the order might matter and we'll give it a moment there we go so now it's saying the it's been created so it's making me think this was for what was this creating the index this little thing yeah and so I want to go over here and just confirm so we can see what we're doing in Azure AI

search if that's now there so we'll go under search management and we'll go to indexes and we can see that's the name of it VC test and if we go into here looks like we have yeah a search box has changed cool not a whole lot in here but we'll go back okay nothing under data sources nothing under indexers but we' created our index what I was hoping to say was the structure there um maybe we go add adjacent maybe we show it here and so yeah so we can see our Fields here it's just

it's just not visually shown anywhere else okay so so far that is good um but have we inserted our data yet I don't think so but let's continue on okay so yeah insert the text embedding into the vector store so add the text and metadata from Json data to the vector store so we grab this next I told you I don't like to do our Imports like this but that's just how they're doing it so we're going to stick with it so upload some documents to the index we'll just copy this whole part here and

here I'm going to just take this out because it's just plain here so we're going to get its output path it might give us some issues here so I'm just going to do this again just in case this just means this the current directory so just kind of cheating there and then it's going to read the file as a document and it's going to do search client and we're going to have a search client and then we're going to upload the documents which is just all that Json data um it doesn't look like we're splitting

it it's just loading them just loading the Json so let's give that a go there we go so so far so good this is not always the case like sometimes I'm working with uh I'm just saying like some whoa my head's really angled up here but yeah sometimes I'm working with um Azure examples and they don't work most of the time but this one has been pretty good but I'm thinking the reason why is that it's just more recent so anyway let's continue on um so now we have uploaded our documents now can we see

where those are I don't think we would see it here if we go back into the search here I mean I would expect that we'd have to have a data source if we're sourcing data from there so I'm not sure where it is in the UI because the way I understand is like you have a data source so you connect to it but that's okay so here it says search client uh upload documents insert text and embeddings into the vector store okay well I want to know the vector store so give me a second I'm

going to look up where it is so it says it has a vector store um it's not like Cosmo DB or anything in particular it's just part of the service I guess so not 100% certain on that one that's totally fine um not worried about it but uh you know just sometimes I like to visually see things so if you're indexing a very large number of documents we not um you can use the search index buffering Center which is an optimized way to automatically index the docs as it will handle it free B cing so

something that we could have done instead we are I not doing that because I don't feel like we need to do that and we've already uploaded them so we're just going to ignore that so it says perform a vector similarity search so the idea is that our data is in here how it's in here I do not know I really thought that we'd have to connect um a data source but I guess it's just there let's go back here get to it oh you know what I didn't notice this before there's our Fields there we

go okay I was going to the Json there probably like Andrew what is wrong with you I'm sorry the UI is just hard to look at um so we have that cores I guess if this is external our configuration which we created and our Vector profile we can't see our data here just yet but we're just going to uh continue on here so let's go back over to here and so here we can form a vectorized query okay so let's give it a go so grab this next I'm just going to grab this whole section

because this has been pretty reliable so far and I realize my font's a little bit small I'm going to jump this up here I apologize for that entire time happy small font so um here we have a pure Vector search so we're providing it text and then for from that query we're creating an embedding so I'm assuming that's using the open AI way of embedding that data okay and then we provide that embedding we tell the nearest neighbors so how like how many friends or neighbors do we want to look out to and then we're

looking across the content Vector so the vectors are going to match against the vectors and it's going to return the vector query which will then uh allow us to utilize that in our actual search and then we're going to select this information information that we want and print them out so go ahead and do that and so there we are getting our results back so that is very cool okay um I still would like to know how we do it here oh we just write whatever we want okay so this I was thinking there was

like something really complex there but it seems seems like we could just uh what was do this as well we might get similar results let's just try this search so we have Azure devops developer tools I mean it's returning everything maybe there's like more ways we could filter information here I don't know but here you can see way more information so I guess that's just like a really easy way to find information okay but if we go back to here um it looked like there could have been an easier way to search no it doesn't

really look like it that's fine all right so let's go back here and just continue on so clearly we did a vector search which wasn't too bad so now we have perform we hold on this this is perform a vector similarity search so things return things that are similar to to other things right and let's just take a look and see how similar it is tools for software development so Azure devops Azure devops Labs test very clear that that is that is there and it's giving us a score how close it is this one more

than the other ones perform a vector similarity search using a vectorizable text query so this example shows a pure Vector search using the vectorizable text query all you need to do is pass in the text and your vectorizer will handle the rest okay so let's grab this and then we'll look at the code we'll try to see if we can make sense of it okay so go here next so this looks very similar the only difference is this one says vectorizable text search and this one say says vectorizable quer search and here we have to

create the embedding and then do the vector query but here it looks like we are saving a step so um that's the only difference that it looks like it's doing we're getting the exact same results so yeah vectorizable text query so yeah I think it's just skipping a step and so we don't have to use that now we told it earlier that we wanted to use open ai's method of vectorizing so maybe that's what it's doing like when we pre-loaded earlier then this one is using that one that we loaded earlier on okay so this

is example shows a pure Vector search demonstrating the open a API text embeddings a O2 multilingual capabilities let's take a look here um so okay so the question is we're not using Ada we're using the other one and so this is suggesting that if we use a different language that it's still going to figure it out so this might not work because we do not have that Library embedded I'm going to find out and see what happens okay and it still works but I just don't have confidence if it's using the actual um uh one

that we're using but it's interesting that it can go across languages there so this is working supposedly but it could have been just tools was enough for it to know we look at these scores 058 057 054 I mean we're getting the same score so must be good but this here confuses me so I'm not sure what's going on there so now we have perform an exhaust of KN andn exact nearest neighbor search so we'll give that a go let's paste that in here we'll run that so it's using the vectorizable text search and we

what's different let's go back here you see exhaustive equals true that's the only difference that I see here is that it's saying exhaustive is true oh well I guess it is how you can exhaustively search your vector index regardless of what index you have okay I see all right um then we have perform a cross field Vector search so this example shows a cross field Vector search that allows you to query multiple Vector fields at the same time note ensure the same embedding model was used for the vector Fields uh you decide to query okay

so let's grab this one they're looking a whole lot similar to me but clearly there are differences here I'm not that well versed in these type of query things um I am creating lecture slides after these Labs um so if I do figure that out I'll put them in there and you'll we'll come to this video and we'll already know what it is at least you will me right now I do not know so we look at this one here and we're going to run it and it works let's go back here Cross Field Vector

search so it says Cross Field allows you to query multiple Vector fields at the same time ah okay so here we're doing multiple Fields is what it's telling us okay and our results are a little a little bit different way different but I guess because now it's going across content it has more to pull from or if it if it was title so now we have a multi Vector search so shows a cross field Vector search that allows you to query multiple Vector fields at the same time by passing in multiple query vectors okay so

now we are vectorizing twice but one for the title one for the content okay let's do that let's try this we'll run this that and I mean we're getting different results but it's stringing as one I'm not sure how it decides between the two but it does that apparently perform a weighted multiv Vector search so give one more weight than the other makes sense so run this looks good perform a pure Vector search with a filter so this example shows how to uh apply filters in your index whether you want to do pre filtering so

let's take a look here where is the filter so we have Vector filter mode okay and then we're just saying pre-filter all right but what does that do that's what I want to know so it goes here and it says determines whether or not filters are applied before or after the vector search okay but what is it filtering oh right here so here's the filter so catalog equals developer tools all right so there's some query language or some filter language that we can utilize here um so we'll grab this and we'll run this we'll take

a look and here it's saying category equals developer tool so it's filtering for that is that what we had in our data and we did okay so that's that's interesting okay that's cool we'll go down here below and we'll look at our next step here so now we're getting to more interesting things like hybrid search before um reranking search Hybrid search was the one that was really good um so here we will copy this and we'll take a look and see what's happening so we have the same thing we're vectorizing this information we have a

query query vector but what makes this a hybrid search I'll H run here okay but what makes it hybrid because it looks very similar to the other ones is it because there's spaces in between here H so I'm not sure that's the only thing that's changing is this and this looks just like the other ones but apparently we can wait it we have a pretty good idea how waiting works so we're not going to run that one semantic hybrid search now we're getting into something interesting so this is the one that's supposed to be really

really good so let's go copy this yeah I think uh when I go create the lecture content I will go f figure out what they mean by hybrid search here because I'm not 100% certain Al all the other ones previously were hybrid search so here we're saying what is aure sarch maybe that's a spelling mistake and that's intentional um so it's exhaustive equals [Music] true and down below here we have query type as semantic and it's using the semantic configuration which is what we have ours called here and then we have our semantic ancers let's

go ahead and run that semantic answer okay and we have our score down here below and then the resources here okay well that's something and that's it okay so I'm not saying I understand what all of these searches do but I will Circle back put in the lecture content and then um I'll sound d dumb here but I'll sound smart in the other area there's clearly a lot of other things that we can do chunking might be something we might be interested in doing um working with an external data source is something we probably want

to do let's just take a peek here into um this one I'm not sure if we want to do this so this one says demonstrate how to use Azure search Advance query rewriting to improve the relevance of your search so we'll take a look here and just see what we have so I'm just trying to see what is different we have chunking going on here okay so I'm not sure I might come back here and do another one um yeah I think we're going to do another one of these I'll do it in a separate

video here I just want to take a look at the the chunking example here I'm just going to pause here and make a decision okay all right so this one's kind of interesting because um I was looking at the advanced one again and they're making a query and then they're feeding into a large language model saying hey can you make this query more efficient and then uh it's doing that query I think chunking is something that I uh think that we probably we could do but really all it's doing is it's just taking a um

like a larger file like a PDF and you're just breaking up into parts so I don't know I feel like this is good enough um it'd be nice if we used Azure AI search with a data store I think that's something that we'll end up doing when we uh use Azure AI Studio or open AI studio so I think we're actually done here um and we got some practical experience here with it so this is fine I'm going to go up to the top here I'm going just clear these out nothing super exciting with this

one as clearly we really stuck to the um sample for once we go here and just clear out these values here so that you know whoops that you know that uh you can't use the same one here okay I'm going to go ahead and just restart the kernel and I'm going to go ahead and download these files and so I'm going to go over to our Azure repo let github.com add your examples we'll press period here I'm I'm going to drag them over and so if you just want to run these and not fill around

with things you can absolutely do that of course you don't know that until the end of the video but that's always the case here we'll say Azure AI search but uh yeah it'd be again it'd be nice if we had few other things here to do with the AI search because we obviously have data data sources and there's also skill sets so that's another thing that we could have uh took a look at so this is skills that you can add here but anyway we do talk about that in the lecture content I already have

slides for that but um yeah we'll just upload this in place and then we'll just clean up here and be in good shape so here have my Azure AI search I'm going to go to my downloads off screen here here and drag them on over these three files over here Azure AI search and we will commit and push that and now that is done I'm going to go over to our portal here and I'm going to go into our Resource Group here this is our Resource Group and these are our two resources the search and

open Ai and we'll go ahead and delete these resources okay and then uh yeah we'll delete that and then I'm just going to go ahead and shut this down there we go and I will see you in the next one okay [Music] ciao hey this is Andrew Brown in this video we're going to take a look at how we can use azrai search with our chat llm which is going to give us basically rag so let's go over here to um our chat playground and um the idea is that we have the ability to add

data and so by adding data we're going to be able to provide this um chat bot with context now if we're using GPT 35 turbo it doesn't have the latest data so if we go ahead here and just say uh what is the latest data set uh that GPT 35 turbo is trained on I don't know if they they like update this model or if it just has the data that it has let's find out I don't have real time sure you okay so let's go ahead and say uh GPT 35 turbo um latest data

set and it says here June 13 2024 so it seems like oh it says will be shut down created will uh will continue updates through 2023 the architecture Al alterations performed in 2023 will be shut down so what does that mean is that getting rid of Turbo so here we have um much more truthful also need for gp4 also detailing the course declines and API programat ability so we offer sta models the promise is broken what what are we hearing here so MH ah okay so it's I think it's suggesting to say like it has

training data from 2021 but they're not doing anything um as of new with it but we could go ahead and just try to ass it something that is recent so I'm just trying to think of something that is recent um in the Azure space so what products has azure launched in 2023 let's see if it knows that okay what products has Azure launched in 2022 oh I did 2023 again let's go here 2022 2021 okay so this makes me think that it was trained on data from 2021 so if we pull anything that is newer

let's go over to the Microsoft log and we'll go over to here and see if we can find something that is new let's go over to the AI blog and so I'm looking for technical announcements this is not really the best place to look I suppose it'll go back over to here usually they have like a developer form so maybe we'll say like um updated pricing for Azure a not Azure fine tuning uh AI Studio because that is something that I know that has changed here so yeah this article here zoom out here a bit

this one talks about the new pricing changes for um for this and so we'll go here and so we say we're updating our building find tuning for Azure open search service this is going to take effect probably for Azure AI Service as well um so say you know we go here and say what is the current cost to um find tune a model on Azure AI open search and this is a feature that didn't even exist back then so it's probably not going to be able to do anything as last of my knowledge the cost

of fine t a model can vary based on several factors and the amount competing resources for more accurate information okay do you have any information are you just making this up let's see if they know general knowledge available up to the last so I think that it's saying that it doesn't know okay it doesn't know so uh the idea here is I'm going to go clear the chat so we're basically starting over and let's see if we can add a data source and hopefully we can use Azure AI search as we do it we have

created Azure a search separately um but let's see if we can go through this completely so we'll need a data source and we can do blob storage but I want to it says connect an existing resource or um create a new index so I'm going to create a new index notice it says the word index um and so I'm assuming that's going to use Azure search and we are going to go with blob storage well it' also be useful to connect with data in Azure studio so there's a lot of options we have here uh

let's go over here and take a look what we have over for data um yeah and so this one so if we have Azure AI search then we'd have to connect create a new connection here and then this one would go to a storage account as well okay so there's a lot of ways we can connect data but again I want to focus on trying to do Azure AI search because I think that is the most interesting and I don't have one right now yeah so if we go ahead and we try to use blob

storage yeah it says add Azure a search resource and open ey connection will be required to to index your data create a new Azure AI uh search resource so it's already suggesting that we would have to do that let me go down here again yeah so I guess we're going to have to create it anyway when I did this before I was able to create Azure AI search through the UI but I can't seem to remember how uh so we'll just manually go create one um so we'll go over to portal. azure.com and we'll go

over to AI search before we create this I think we need blob storage but um we'll try anyway I'm going to call this uh AI rag for um because we basically that's what we're doing here we'll just say AI rag or like AI search Rag and I believe that this one here I don't think it matters but I think that this one this project here yeah it's West us okay let's go ahead and do West us and I'm going to change this down to the free tier as we don't need to make things super expensive

here let's go hit hit next this is fine we'll go hit review create give it a moment here and we'll go and create our Azure AI search resource our deployment is complete let's go to that resource so from in here let's drop down search management the first thing we need is an index um I mean we want to have a data source so connect your data start here to import your data we'll go ahead and add an index um and I'm not exactly sure what we want for Fields here so I'm going to go back

for a moment before we create an index I'm going to add a data source and and see if we can do that so hopefully it'll create one for us it won't but what we'll do is make a new tab here and I'm going to go over to Azure blob storage or just Azure Storage storage accounts here and we'll create a new one here and this one's going to be in our rag call this AI rag West us um I'm going to ignore these options here I just want to go with standard they they just started

doing this where they have um AI Rag and some numbers here on the end they just started uh giving these options here I just kind of ignore them as they're just suggestions we'll go with low uh low locally redundant storage because I don't need more than that there are a bunch of options here which are interesting I don't know if we have to enable them to connect for um Azure are search but we'll go ahead and try this I'll give it a moment and while that's creating let's go prep this article so this article has

information in it and um I just want this contents here so I'm going to copy this and I already have open here the Azure examples repo if you want to open this you just press period uh you should be working your own repo but I'm going to make a a new folder here I'm going to call this Rag and I'm going to make a new file here and this one is going to be um uh finetune pricing dot markdown we make it a markdown file I'm going to past this in here and so we have

text I mean I don't even know we need anything else besides this but um just making sure that this is formatted away that makes sense so we have new price previous price [Music] H so I'm just wondering if it will understand this format because I'm looking at I can't really make sense of it so let's go back over to here and so previous price was this and then new price was that was the tokens new price so previous price was an hourly cost so here we're going to go and we'll just say previous price new

price previous price new price now there are other ways that we could um prep this document but I'm just trying to keep it simple here today new price new price and uh yeah so you know maybe we'll just do a bit of formatting here basically making a markdown file I was not going to make a markdown file but now I've decided I'm going to so maybe the structure will help it out oops oh we're going to do three here models uh Azure open AI search um f tuning or this is azure open ey service fine

tuning price change okay so we go here uh more information reasons for change okay and so now we kind of have a structured file that we can U bring in so I want to go back over to here um here and we'll go to our resource I'm assuming this is going to use blob sorage we're going to go over to our containers remember uh there's some new new icons over here which is interesting and I'm going to go ahead and just call this data we'll create that assuming that we're supposed to create it ahead of

time like this not really sure and so I just want to upload this I'm going to have to download this file go ahead and download this if you're looking for this file look for the Azure examples repo it should for the most part be in place of where you want it to be and give me a moment I got to find the file there I found it um and so I'm going to go here and just drag the file and upload it and so hopefully that will um yeah it's blob block so it's clearly a

blob file so hopefully we can then now add that here um we're going to have to go back because I just created that data source so we'll go ahead and hit add data source as your blob storage we'll just say data and I'm going to choose an existing connection and here we have our AI Rag and we're choosing data say select and so now we have an established connection there manage identity authentication if you enabled uh identity on your search you have the option to authenticate I mean I haven't so we should just be be

able to leave that alone that's the container it's everything in the folder that's it let's go ahead and save that all right so now we have done that we're going to go back over to here and I imagine well we haven't done skill sets yet but I imagine we'll have to do an index just give me a moment all right uh so let's go ahead and create an index we will add an index and so this will just be um data I suppose and we need content here that we're going to search across we have

an ID and I'm just going to call it content I'm not really creative with these names here so I'm just going to go ahead here call it content we have different fields here I mean basically it's a string I don't think there's anything else yeah it's just a string and so that's going to just contain our content I would think okay we'll click into that and yeah I'm just trying to think like how would that index our data like if I type in here and just say Azure uh if a query contains a search uh

Target okay just give me a moment all right so I didn't get a clear answer but while uh researching I did find something interesting uh that I did not notice but there's this import data option I'm not saying we're going to use it right now but apparently it will do a lot of the work for us so here uh we have an existing data source actually shows up maybe we can use it because we do have the data here and maybe it can create that index for us so I'm going to go back here and

uh because I'm not sure we have index and indexer so maybe what we should have done is created an indexer to then index our data and then put it in the data store yeah that's what it is so I think import data would just do the exact same thing but we'll go ahead and do this index data source uh there's a skill set so a skill set I believe is basically Transformers for data so we go over to here for a moment um and take a look at our skill set let's just take see what

we have here yeah so we have skills all over the place and yeah I think that's what it is give me just a moment here yeah so that's exactly what it is a skill set is just a way for us to add additional processes before things get imported right now I don't really have a use case for this so um uh let's not that interesting for me but imagine that we had in our data store a bunch of uh PDFs then we could OCR that and that would kind of be actually interesting so maybe we'll

try that after we get something working if we have time here but right now I'm just going to keep it nice and simple um so let's go back to our indexer here and we have a schedule so I guess I just want to run this once oh run okay that's the first option here that's perfect we have back size we have all these these numbers here whether we want to exclude information data to extract um I mean I just want the content but we'll content in metadata that's fine we have parsing mode which is fine

this is just markdown so set to text to improve uh indexing performance on Plain text files I mean that's what it is it's a plain text file in a sense it is a markdown file but that's basically the same thing image action so ignore embedded images or files that's kind of interesting and we'll go ahead and save this okay so we have saved our indexer I want to go ahead and run this say yes only thing I'm wondering is like how would it know to map to the content field because here we never told it

how to map um so that's something that's kind of interesting if this doesn't work we'll just do the import data that's totally fine as well but yeah I'm not sure how it would know what fields to map to did it run I mean it started cool so let's go back over to here oh was successful excellent let's click into it did we get any results execution history it says it's successful um nothing of Interest here that's telling us how it's working but let's go over to our home back into here into our indexes it says

document count is zero so it says there's no data here and so yeah I'm not really confident on this so what I'm going to do I'm going to delete this index okay I go back and delete this index here because clearly there's a little bit more going on here that I'm not aware of delete this indexer yep and let's try the import data and see if it actually will set up everything for us so we'll go say import data and hold on there's also import and vectorized data that sounds even better let's do this so

we have uh Azure blob storage Y and I'm going to choose AAG and I'm going to choose data and it's everything in there I don't think we need to authenticate so we'll go next vectorize your data as your open AI so yeah we need something that's going to do the vectorization we did this in a um a notebook before where we vectorized the data so open AI is totally fine so like that's what we would utilize here we have Azure open AI search yeah so we actually want to do East because that's where this is

oh no no no we're over here actually no deployments available when with an embedding model okay so I mean I guess it doesn't really matter what we use to to do it because it's just going to get the data in there and then after that doesn't really matter but if we go over to here no deployment model available so so something has to actually do the uh embedding and this one's in open AI Studio 20 2134 is that what it is 21 213 so I have to figure out what I keep putting random numbers so

it doesn't help me much but I'm going to go back to the hub and so the one that we're looking at here that's in West US is an Azure Studio Hub but it's 1 two 3 4 but the question is which one which open a is it using so we go into here and it will have connected resources connections and here we have one 2 3 49 huh okay so maybe these are just isolate ones that are deployed they have nothing to do with Azure AI studio so let's go over to open Ai and here

yeah now we have two we have West and East so we' been doing everything in West I'm going to use this one which is the 2113 we have 2 21 1 3 and say it has no active deployment so we're going to go over to a model deployment and I'm going to manage deployments it's going to open up us up an Azure open AI um Studio because you can't do deployments the old way anymore which is fine and so it's saying I want something deployed here so that we can um uh vectorize that data I'm

going to assume that we can use gp4 I also kind of wonder if we could use gp4 mini but I think it takes time for that to come back so we don't want to use that so I'm going to use um uh I'm trying to decide here because we could probably use GPT 35 turbo but I'm going to go ahead and use GPT 40 we'll go ahead and deploy this and we'll see if this will do what we want okay give it a moment here to load and yeah I'll go with global standard I think

we found out that was probably the best one that we could possibly deploy and we'll wait for that deployment to become available seems like it's like instantaneous so that's really good I'm going to drop this down again give this a refresh oh with the available embedding model so maybe this deployment is actually useless now that I think about it because it has to be an embedding model right um so maybe they'll indicate yeah with embedding so we have text embedding three large Tex edding small now some regions don't have access to these so hopefully these

ones will Loy um I'm not sure what the difference between small and so small are the latest models offer better whatever let's go with small cuz we have a small amount of data hopefully that just works that'd be really nice this model is not available for Azure openi resource cancel let's go back of course they would make it difficult let's try this deploy yeah so it's really weird that they do that and they don't make it really easy for us is there anything else I can do embeddings not that I know of there's also Ada

let's try Ada but cannot be fine tuned okay maybe we'll avoid this one and if that's the case let's just switch over to our other Studio which is in Sweden and we know that the Sweden one for sure for sure uh will we got a lot here I think it's this one one two no that's not it oh East okay yeah yeah these are ones these are ones that are just individually deployed they're not necessarily part of a studio but I think we I think that this one we'll have embedding so let's try the small

embedding here see if that works so it's really interesting where some things you're forced to use open Ai and the other ones uh you'll want to use AI uh Studio we'll give this a moment to think it looks like it's going to do it but it's uh having a hard time here today all right so we have uh standards so I guess that's what we're going with here today and so we'll uh deploy that and it looks like it's instantaneous it's deployment which is nice not something I'm used to with Azure but we'll go here

and choose the small again I don't know a huge difference between the large and small model but I'm assuming that small is good enough for our use case if it's not we're going to find out uh will inure additional cost to my account yes of course everything costs money if you're afraid of using it you can just watch along but it's up to you there are no images in here so we don't have to worry about that enrich your day with AI skills so that's something that we could do during the vectorization it's interesting that

they show it at this step um but I'm not interested in doing that right now so go ahead and hit next um oh yeah we're doing this in the rag so I I for some reason I thought we were in the other interface I'm going to set this to once and this is going to generate an index an indexer a data source uh and a skill set so all those things we could have configured for whatever reason I didn't know how to do the indexer we'll go ahead and create it but now I'm starting to

think about this is very similar to U adab blue um so the similar components and I mean it looks like it's done I'm not sure yeah we're just waiting for it to create so I'm going to oh okay so let's go ahead and hit start searching that was easier than I thought let's type in open AI open AI here and we're getting back Vector data so there's only one document um this probably be more interesting if we had multiple documents here but uh again I'm just going that I do not know how to do the

import and um that's where we could run into issues let's take a look and see what it did so in the indexes if we go to fields we have text Vector so that's the text Vector data we have chunk so I believe it stores it chunks and then both stores the text Vector data we have a title which I don't remember coding we have the metadata maybe the title of the document so that if we need to find the document we can look it up um let's go over to the indexers and see what they

did for this so we go over it settings I wonder if it's pre-filled um yes that looks fine but how did it know how to map the fields so here we have mapping Fields mhm so yeah I I don't really know maybe it's based on convention like if you have vectorized data will place in the correct location but I'm not sure how it knows to do that but at least it works okay so now that we have our data the idea is that we can um uh go over and finish connecting our resources um in

Azure AI Studio Hub so let's go back over to wherever we were I think it was let's go back over to our Hub this one is where West us so we're back in West us we're going to go back to chat and now let's see if we can connect Azure AI um search we're going to add our data so we'll create or select a project oh we're not in a project right now let's make sure we're in a project first there we go and then we'll go over to chat and from here we'll add our

data and we'll add a data source and we'll say Azure AI search we're going to go ahead and hit next um we'll connect to Azure AI search and we have different ways of connecting I remember having a a lot of difficulty connecting um so if we can use API key I'm totally fine with that so here it says add a connection your Hub will be granted access to this resource anyone with this will be whatever so let's go ahead and add a connection and we'll choose this index here and of course we would need a

um a means to doing this so this is azure open ey embeding model will be deployed if not already adding the vector data so here I'm I'm not sure if we should do this because if we have embedded with a very specific model I would think that we would want to um use the same one and so I'm kind of concerned about that and so because of that I'm not going to proceed forward here I think that's a a mistake and let's go back over to our Hub and launch a hub that's in uh East

usest unless we already have one and we don't have one so I'm going to create a new Hub and I'm going to just put this make a new resource or we'll put it in our rag one because everything's being done here and so we'll want a new Azure opening I search service and here it can connect to our Azure I search and actually let's do that because we need to do it anyway and that might save us some trouble we go ahead and hit next and create anytime that Azure is going to do the work

for us let it do it because it can be very difficult to connect these things and so we'll just wait for this to set up okay all right so that was created for us and um I mean if it's connected that way that's definitely easier for us let's go over to connections and see yep we can see the connection is established and use the API key type so that's exactly what we would have done um it says cognitive search because that's what a Azure AI search used to be called let's go over to um to

projects we don't have one yet so we'll just make a new project I don't care what it's called and we'll just give it a moment here to create and we now have our project so on the left hand side I'm going to go to chat and oh we need a deployment so let's go ahead and create a deployment I'm going to work again with uh we'll go to model catalog and I want to deploy GPT 35 turbo again intentionally because we know that it's out of date and here we will establish a connection I assume

that's the only one we can connect here and we'll stick with standard we'll go ahead and deploy that that is now deployed we'll go over to chat and it doesn't seem to be aware of the deployment let's give this a refresh here sometimes needs to propagate so be very patient there we go we'll go ahead and add our data so we already have um a connection so I'm just wondering how we can do that so we'll say add a data source we'll say aure AI search we'll hit next and I mean we already have an

established connection so I'm not sure why we have to click through this to find out so what I'm going to do I just I want to see if it's already connected I'm just going to say uh let's go take a look at our document um and so we'll say uh what is the price change for babage for fine tuning let's see if it could pull out there um it's an open AI model can you connect via Azure AI search and check okay so that didn't help it um maybe we just go a question and and

say like here it's uh prob it had like a date in here I just realized there's no date in here uh we say what are the pricing changes to Azure open Azure open AI service for fine-tuning and I'm trying to get it so it doesn't seem to know okay so we do have a connection and I would have thought that we could just add that connection not have to go through this whole process but let's go ahead and do this anyway so we'll say Azure AI search and we'll go next and we will choose Azure

AI search we'll choose this Vector here I'm not sure why it's saying that so that's kind of annoying but now we can select an embedding model that makes sense and here it's still trying to choose text embedding add O2 okay so maybe here we need to deploy because it will pick one by default but I'm going to go ahead here and just choose that embedding model the um the small okay and let's go ahead and deploy that that's another thing is like how would have that chatbot used that data source if there's no embedding model

deployed it probably wouldn't be able to all right so we'll give this a moment here all right so we'll go ahead and deploy this model I'm kind of wondering why that index is not working and we are in eastus so I wonder if we'll have to create another Azure AI search connection but in the correct region I try to keep everything in the same region but uh you know not everything works as expected so go ahead and we'll try this again so we say Azure AI search is very finicky to get the stuff working and

no issues with the index this time around but maybe the reason why is because we actually have a um an index so now we're going to get the index settings here next and so let's go back here as your open embedding model text embedding at a O2 will be deployed if not already adding to your account and so I just keep thinking like is that going to work for us because we did not use that as our our um embedding models because these embedding model should work differently right and that's a little bit frustrating because

um I wish I had known that up front earlier but I guess we'll find out it'll either work or it won't work right so here we have uh some configuration options how we can search so semantic search seems pretty good there's also Vector keyword search um so there's just different ways that we can go about searching and I'm not saying I'm familiar with all these methods um I think that we saw these when we we were running the notebook and using Azure AI search programmatically and semantic search would probably give us the most likely best

results but we're going to stick with Vector keyword as it's suggested and let's see if we can find out um what are the pricing changes uh for Azure open AI search Azure open AI uh services for fine tuning okay and let's see if it can do this like would it be smart enough to know to reach out there oh look at that okay and so it's giving us a reference okay so it has citation it's showing us the changes here uh can you come uh show me can you make me a table of old and

new prices price changes for the uh let's see if we can do that I'm not sure if we can do tables here oh did okay great and there you go so there it's referencing it and so that works that is really cool one other thing that I think that would be really cool to try if we can make it work is to use a skill set to OCR a PDF document um that is a huge challenge for a lot of rag systems is you have to have some way to unstructed IFS is definitely one that

can be a challenge now if you use a powered AI assistance like TBT you'll take for granted how easy it is to uh drag those over but um you know we'll we'll see what's going on there anyway so um yep hard case so let's see how we can do this um I'm going to go back over to um our AI search here and what I want to do is I want [Music] to because I guess we'd want to have another yeah let's go back over I know I'm talking over the place but let's go over

to our data storage um our storage account for our our Rag and I'm going to make another container and this one's going to be called PDFs Okay so PDFs and the idea is that I want to have some kind of uh PDF information it'd be nice if that PDF information was um a a image or graphic um a graphic PDF where uh it's very hard to do that so I think what I'd have to do I I could scan an image in I'm just trying to think how I could do that so maybe what I'll

do is I'll find an article here in your Azure AI Services blog and what I can do is print it out and then take a photo of it and then turn it into a PDF uploaded to the Azure examples repo so you can use it as well and then we'll try to utilize that so let me go do that I'm just going to go pick out a couple articles here um and I'll be back in just a moment all right so I've created a PDF and I've purposely made it really bad um I'm trying to

be very careful uh because um Adobe or acrobat really likes to OCR things and I want these to be images right I want these to be a little bit awful I want these to be images because I want our skill set to do the OCR and not Adobe Acrobat um but let's go ahead and see if we can uh get that to work as expected so I'm going to go over to um our storage account into our PDFs and we only have one PDF here which is fine so I can just go ahead and upload

that here okay so I believe it's on my desktop so I just need to go and grab it from here just give me a moment to find it um I called it GPT gp4 and we'll drag that on over there and we'll go ahead and upload that document right so that is now uploading so we have GPT 40 PDF that looks great okay all right so um that is now uploaded here and um the next step is to import that data so we're going to go over to um our Azure AI search and we'll go

back to overview because this tool up here import vectorize was super useful and surprisingly using a different um embedding model still gave us results but I would imagine that when you are uh converting Vector uh Vector data you should probably use the same embedding model because I can't imagine they work all the same way over here to Azure blob storage we're going to now choose uh we'll choose this one and in this case it will be the PDF's directory we're going to go hit next and we are going to I guess it's the other one

that has the yeah small embedding three small of course it costs money we know this we'll hit next we are not vored in theages we're going to go next and oh where's our skill set oh enrich your data with AI skills so extract text from images that's exactly what we're doing here today um and select a multi-service account no accounts found in the same region as a search service what what does that mean select a multi-arch service account I've never seen that as an option give me a just a moment okay oh maybe it's just

AI Services okay that's fine let's go ahead and do that I mean I thought we already had that but that's totally fine I think it's like AZ Services where it has the unified API but um yeah that's exactly what it is okay that's fine so we'll put that in rag uh yeah r e us we'll just say um Azure AI services for rag not going to type it correctly I do not care here we're going to go ahead and do review create yeah sure we'll do this and review create and I guess the reason why

is that it needs some way of accessing OCR and I guess there isn't already one there it's just the wording is kind of weird here they should have just called it um AI Services API and that would have made a whole lot more sense so just give that a moment to deploy that was quick let's go back over to here we're going to give this a refresh and oh dear what region is this in is this in West us this is in West us so my mistake um just because I'm confused because we have something

East us and we're doing something cross region but I'm going to go ahead and create a new service as best you can try to keep things in the same region so there's not additional weird cost so we'll try this again AI rag eastus AI search uh R East not even trying to type it correctly just getting it in there go ahead we'll hit next everything seems fine review create and this time we'll have one in the correct region okay so that is now deploy we'll go over to here and we'll refresh and H did I

make another one in East US yes I did sorry I'm so sorry I keep forgetting it's because I'm hungry I haven't had lunch yet here okay this time this time I'm going to make it in the right region it's not like it hurts anything making all these is not costing me anything yet but uh we'll go ahead and say West us okay pricing tier Azure AI search West us feel sorry for the people that are watching me making the same thing again wrong wrong wrong okay so we'll create this one shouldn't take too long so

give it a moment and yep we'll wait here a moment so that Resource Group is now ready we're going to go over here and give it a refresh there we go we're acknowledging yes and so it's going to use the underlying OCR service how well it's going to perform on my documents I'm not sure but I expect it to work um so here are all of our options we'll go ahead and create this there we go and it's going to create the skill set so you know not the most interesting way to see how skill

sets work but I mean we are technically using a skill set um but I would imagine that if we were setting this up manually then we could choose from a bunch of other one so start searching let's go ahead and try this out and make sure it works so what was in that document I already forgot uh this one has about um G uh gp4 fine-tuning so I'm just reading here oops wrong wrong thing we are excited to announce fine tuning capabilities for public preview fine tuning allows how much does gp4 mini fine tuning cost

let's try this that's that's new that's like literally a few days ago how much does gp4 o mini fly tuning cost I mean that's a lot of words in here but we'll try it anyway I mean it's returning back data maybe just a single keyword might help here so let's just say um fine tuning I'm just matching it based on the document here I mean it's retrieving the document so I think the way it's it's going Vector data because the data is vectorized right so the way we're going to find out this this works really

well is by testing it over here now we have this connected here but does that mean it's connected to the PDS or data so I'm going to try to ask the question over here and see what happens if it can site from that other document the one that we had OCR and so we'll expand this so this is coming from the the wrong one and so I think what we need to do let's go to advance settings here and I mean like I don't know if we can add multiple data sources so maybe what's important

is that everything is in the same index for this to work because we only have a single index here I'm not sure if that's a limitation here but let's go ahead and remove this and I'm going to add a new data source next here uh great naming 5554 okay so we go here 5554 so this is the PDF one and I just ignored that error because I just obviously it worked before and we'll go next create and we'll set create and so we'll leave it on the vector keyword search it seems fine on that let's

go ahead and paste in that question from earlier how much does gp4 mini fine-tuning cost fine tuning cost probably would help if I spelt it correctly here find tuning cost and let's see if it can pull out citations from that document and we have a reference and yes so it's getting it from there can you can you uh make it a table and let's see if it can do that for us there we go and that is pretty close so not perfect but of course our data let we have 03 and then oh no it's

there okay it's just the way it visualized it okay so we have a simple one there and just it displayed it differently and so it actually did a better job than what we actually had in the document uh but yeah I mean like I was hoping that we could look a little bit more at the skill sets um so you can I mean you understand what it does it's just an add added filter um filter there but um I think if we wanted to have the flexibility to um actually do something with it we would

have to um is it our indexer is it our indexer where we do that or is it our data source it's not in here so if we go into this I'm just trying to see where we added that um that skill set because it obviously did it for us right maybe it's when we're creating an index no it it have to be the indexer because yeah because at this stage it must be here you know we do this [Music] 554 allow skill sket to Red dat well anyway I don't remember where it is I'm I'm

kind of hungry here but my point is is that we achieved what we wanted to do which was to utilize um uh rag for for uh using a skill set and also just generally fetching from a document but again it's really tricky to get that uh add that that search Service setup so if you're having difficulties with it that's totally fine I'm going to drag this over into our rag document here and I'm just going to go ahead and add docs for Azure AI search Rag there we go so now you have those available for

when you want to do that and I'm going to go ahead here and just um uh stop this or sorry we'll do some cleanup and luckily we have that AI rag um Resource Group so we'll go into here and we have a bunch of junk in here yeah a bunch of stuff and we'll go ahead and delete that and so hopefully that successfully deletes but I will see you in the next one okay ciao [Music] let's take a look here at natural language processing also known as NLP and this is a machine learning technique that

can understand the context of a corpus a corpus is a body of related text um so NLP enables you to analyze and interpret texts within documents emails messages interpret or contextualize spoken texts like or tokens like sentiment analysis it can synthesize speech automatically translate spoken or written phrases and sentences between languages interpret spoken or written commands and determine appropriate actions something that's really important when we're talking about llms um and one that existed prior that got rid of it in 2023 was the Cortana virtual assistant um and it would help you with Bing search and

perform other tasks I guess kind of like uh Siri or um Alexa but um yeah they retired it because now there is basically AI agents so the old term was virtual assistants these were um uh basically big IFL statements using NLP to complete tasks and now we have llms which we call AI agents which have more agency more intelligence around them that use NLP what are the NLP offerings for Azure well we have text text analytics so this has sentiment analysis to find out what customers think find topic relevant phrases using uh key phrase extraction

identify the language of text with the language detection detect uh and categorize entities in your text with name and recognition we have um language understanding um I guess you could call it Lewis like the name we have here natural language processing service and enables you to understand human language in your own apps websites chat Bots iot devices and more Lewis is more for those virtual assistant the old way of building um these kind of if else uh NLP agents I don't uh not agents but Bots I don't personally like that anymore I just find them

very um uh limited but they're great for let's say websites where you need a a chat bot and llms are going to be too expensive we have Q&A that we can use NLP for as a service so this is where we have custom question answering allowing for the creation of custom question answering models from your content or you can have a pre-built uh question and answering model again these are for Bots um for your platform we have speech so this is where we can transcribe audible speech into readable searchable text so there is um the

NLP offering and within text Analytics there's a lot that we can do with it you'll find out when we get into the labs of text analytics [Music] okay hey this is Andrew Brown in this video we're going to uh take a look at text analysis so I did this previously for the um AI 900 and uh I need to do it again um whether you're taking the AI 900 the or the AI 102 they're both relevant um so um again this was three years ago so I'm hoping that this thing still works and we can

walk through it but um yeah let's give it a go so you should have an AI managed API compute on the lowest possible cost here I'm going to go ahead and start this up and once that's started we'll open up the Jupiter lab notebook we'll Port this over and see if it still works I believe that it's still called this um but yeah we'll figure it out okay all right that is now running let's go ahead and open up Jupiter lab and so I think that there is a um a new uh it like a

new P it might be the same package just but renamed but I believe it's called Azure AI text analytics why the naming I I don't know but um what we can do is try to give this a go um maybe the code is very similar so if we take a look here we have text analytics client um this is obviously been changed I think that they've made a generic one and that's probably one of the reasons for the changes but for the most part it looks the same and I mean obviously this is still cogni

microsoft.com so you know that that's going to be the same there but um let's go back over to here to our jupyter notebook I'm just going to uh sure hit build and um I'm just going to go up to AI Okay so we've been working with this so you should be familiar with um this environment if you're not watch the setup video in here I'm going to make a new folder this will be um text analytics and so I'm going to try to bring over this stuff now of course if you want to just copy

and paste this you can get it from the a examples because you'll have this it'll be uploaded here um but you know do your best to follow along and the existing one is here in the uh the free AI 900 uh here okay so um I'm going to go ahead and give this a go so here it's doing system executable it think we have to do that we just do pip install there might have been a reason why I did it this way I don't remember why but I'm going to go ahead and go back

to here maybe I just didn't know better at the time but we are going to go into our text analytics folder and we're going to make a new notebook called text analytics we're going to choose just the SDK keep it nice and simple um and I want to rename this as text analytics okay we're going to go ahead and uh paste in just that part there and put that presented that's that magic symbol allows us to uh install things and so this one is called Azure AI analytics just going to grab that name here just

in case you're looking for this link I'm not sure why you want it but if you do I'm going go ahead and add this drag it over the top here whoops above above there we go and save resources here I'll just paste that on in there okay so we'll go ahead and run that first we'll wait for that to pass good and I think for the most part it's going to be the same what's going to be a little bit different is that this one here yeah it's going to have the Azure key credentials as

the key difference here so I'll paste this in here sometimes when it says this you need to restart the kernel to use these packages sometimes you do have to restart the kernel so I'm just going to do that right now um restarting the kernel can clear out variables I'm not sure how we know that it's restarted somewhere it would indicate that um but the next thing is we want to import uh these values I'm going to take a look at what I have here so yeah very similar um and then the next thing is going

to be the key and the end point so I'm just going to bring on those on over because those are going to still stay the same um and we'll go back over to here so this is going to be this so we'll go over to here and bring that on as the next part and I'm just going to take this part out this is going to be endpoint and here for credentials we'll say my key my endpoint now remember that we have the Azure AI service so we should be able to tap into that we

could individually create it that's how I I did it before is under the AI Azure AI Services I would have just gone under wherever that category was um text analysis and we would have um uh just added it there but we have this so we can go into Azure AI services and get our single key and we'll go back over to here and we'll replace this here and so now we need the appropriate endpoint so which one is it go here um I mean this is text analysis so I would think it' be document intelligence

could it be computer vision uh I'm not sure so let me go back here let me go here and take a look I'm not sure just give me a moment uh so I looked it up and it's under language so that is the end point that we're using so we going to go ahead and copy that one and we're going to go back over here um if you're expecting build failures I don't care we'll ignore that and I'm going to go ahead and paste this in here there we go so now I have my key

and my endpoint and we'll go and run this next part and it doesn't like something so say name Azure key credential is not defined so oh you know I forgot to run this then will run this then we'll run this so now we have no problem so my prediction is that nothing has changed and we can just use the stuff that we had here before looks like I had some uh movie reviews in here so if we have those we should bring them back here and it looks like we have some movie reviews I don't

remember how we got this but this is clearly for Star Trek so I I have a series of movie reviews here so um it' be good for us to download so I'm going to go ahead and hit period here and I just want to then uh download these in Block to save me some time so I'm going to go ahead and just select them all here and oh maybe we just download the folder and we'll say download and you know you just download it to wherever you want to download it to I'm just going to

download this to my desktop here GitHub can't open this folder because it contains this some files fair enough so if that happens I just have to create a folder within a folder so on my desktop I'm just creating a new folder I'm just going to call this um AI uh files assets and I'm going to go ahead and select that folder and we'll say view Sure Save changes I just wanted to download the files however it's going to do that and they are downloaded excellent so I'm going to go back over to here and I

want to bring those files in there so I'm just going to hit that upload button and bring them in so I'm just going to make a new folder here I call this assets I'm just going to follow the same structure because I clearly had more folders there and we might want to do those things movie reviews okay and so we're going to bring those in here just a moment I say just a moment I mean like I'm going to hit the upload button and bring them in that way okay there we go those are all

now uploaded keeping the same structure um curious about these ones but I guess those are for um uh different ones here we have a bunch of different uh notebooks here so I'm going to go back over to um to here and actually I probably just open now that I have it open up here I can do that and just again remember the way I opened this was period when I was in that repo if we go over to here I just want to take a look so we now need to get the path that means

I probably need to import that right yes um so I'm going to go back up to here it's B wonky but I'm going to just place it right here and we're just going to run that I'm going to go here and grab OS path join there's probably a newer way to join paths because um I know there is but I'm not going to worry about that so that is now joined and so we have some code here I'm going to go ahead and copy that so we're saying we have a reviews and we're going to

list the directory so list the contents of this directory iterate through it open each of those files read its contents and then um we are going to create a new um a dictionary and then we're going to append that so we're basically creating a an array of dictionaries from there we're going to get the length of the reviews so we can determine the range to make a for Loop and then we're going to print out so that we can confirm um that we have our reviews so that's the first step there and probably when I

originally did this I probably wrote this all all by hand so um hopefully you don't mind that I'm not recoding this from scratch but um I don't think it's necessary so we have client. key phrases where we can extract out key phrases we'll go over to our next part oh so we have a few things right so we can extract out key phrases and then we're going to print out those key phrases then we have sentiment and then we're going to print out the sentiment so looks like we got a couple things that we can

do here and we'll go ahead and add that and we'll grab this one I could leave up the older video If people wanted to see exactly how I coded it I don't think it'd be that interesting um this one we already had so I'll go back over here and we'll grab sentiment and hopefully it just works we might have to modify it we'll we'll find out here in just a moment but I'm thinking it was just a rename and a consolidation of API end points okay so let's go ahead and run this and see what

happens so here it's Consolidated them and we have uh some reviews great then we'll go here and we will see if we can capture key phrases so it's saying doesn't know what client is it probably has to do with this so I'm going to go ahead and change this over to be this and we'll change that excellent we'll go back down here execute this it's saying has no turb key phrases okay great so clearly the API has changed a bit not a big deal we'll go over to here and see what's what so just give

me a moment to find what has changed so um I was just trying to find where we can do that but I'm just also noticing that there's a lot that we can do this Library I don't remember it being able to do this much but it looks like it can do all of these things that's quite a few things and so I'm going to go down here to the bottom and we'll go to our examples which is extract ke phases and we'll see what has changed so seems like it's supposed to jump to exact C

phase it's not doing that so I'm just going to search for it and so here we have an example um so here it's saying okay here's the Articles and they've named it extract key phrases so the wording is changed a little bit I mean they're enumerating another way I I have a feeling that we can probably just use what we have here um but I'm going to go back over to ours where we have key phrases and I'm going to go ahead and write extract key phrases so we'll try this instead extract key phrases extract

key phrases we'll run this and see what happens oh sorry you know what this is the wrong environment let's go over to Here There we go we'll run that and it's extracted out so it clearly worked now the question is the the return different we'll run this we'll find out in just a moment and so it looks like it's changed a bit so this syntax syntax has changed um but it clearly is returning stuff it's saying here um extract keyphrase results key phrases so here it says results and then we have do documents um so

I'm not sure what it wants here let's go back over to here just give me a second so I'm thinking what's changed here is that it just doesn't have documents anymore so let's go ahead and take that out and run it um so here it's saying results i1 key phrases document object has no attribute called key phrases the service was enable to process the document okay so maybe a way we can debug this to fix this I'm just going to calb this out for now and so what I want to do is I just want

to take a look and just say print reviews we obviously know that's going to work right so run that so we have ID review text oh hold on but that's not our results so I want to go ahead and say print results there we go so now if I do zero can I do this let's see if we can do that will that work we can and so from here we are seeing um that we have an ID so we can go here and say ID right and then here we can say he phrases right

okay so if that's the case let's go back here there's nothing WR my code I just like to keep it around here there's another way we can enumerate it but this is just the way that I did it there is an enumerate um uh thing in Python but because we know there's exactly 10 it's just easy to do it like this so here we have um the reviews and we are iterating through them we're minusing one because the index will start at zero right and so the key difference here is we're just doing this like

that there we go and so that should get us our key phrases um did I write it wrong copy this paste this here and well actually you know what the ID here is is a little bit different so maybe I'll I'll switch this to enumerate just give me a moment okay so I went back to this code example and notice here it has this enumeration so here we could say enumerate the results and I'm going to do doc do keyphrases so we'll try that and see if this is a little bit better okay um so

do this and then we can print the idx okay I'm just going to comment this out for just a second because I want to see what the results are go ahead and try this out results sorry okay so now we're starting to get somewhere okay great um so we have that and then we actually have the document now we could go here and just say like document um ID and that might look oops a little bit different here oh sorry doc sorry doc okay and um yeah it's up to you like whether you want like

that or this I'm going to go back to idx because that's actually how originally was working and so then here we want to extract out our key phrases so this one's actually just going to be even simplified here so it just be doc. keyphrases going based on that example that we saw okay and we'll try this and there we go so here it's extracting out the key phrases for each of these movie reviews so that's the first thing that we are able uh to do um and it has a problem when it gets to here

for some reason document text is empty we'll go into here and it is empty so um maybe that was intentional let's go ahead and take a look here it is so you know that's not the best but at least we can see that there is an error uh if that happens and so we might want some error handling on that so I wouldn't call this a failure if you want to fix you just fill that in I'm not going to worry about that error I don't care about it um so the next thing we want

to do is maybe sentiment analysis so let me take a look here and see how that has changed because it probably has a different name it's probably something like analyze sentiment we'll go here and analyze oh I was right okay great so now we have antal analyze sentiment so I'll just switch that out here this is using that range so we can just update this code to be a little bit nicer um this is going to be kind of annoying so I'm just going to put any kind of review in here so I really enjoyed

the uh Red Dwarf uh series and I wish this movie had more movie had more of that okay so we'll save that and just to make our lives easier go all the way back at the top here um because we're going to have to rerun some of these things not this one I'm just going to reset everything here and we're just going to restart clear all the outputs so you can see what we're doing here there we go so we'll run this and then we'll go down and run the next one once this one's done

run that then run this then run this and attach our our ourself there um and then we need to assemble our documents we'll go ahead and do that and here it's having a little bit of an issue I'm not sure as to why all of a sudden it doesn't work um I'll try running this again I just don't trust it maybe it's out of order see that says seven here seven and we'll try that again file name and not sure why this is having a problem so we'll run it from the I got to clear

it out again yes I know I'm being very silly here but we'll clear it all the way out and we'll just be more mindful here and just go a little bit slower okay there's a few kind of text analysis things that we'd like to do here not just just key phrases so that looks like that's done we'll click this one good we'll click this one great click that one we'll do that one and we'll try this and now all of a sudden this is not working anymore so assets movie movie reviews is a directory yeah

we know that so um okay uh right so let's go ahead and just start printing some stuff out here to say uh file path I do this and it's weird because it was working perfectly fine and then all of a sudden it's decided that it doesn't want to work anymore so go ahead and do this and it says missing one required positional argument file oh fair enough yeah yeah yeah yeah file path but right now I just want to um print out the file path so that we can see what's going on here because clearly

something is a miss so look at that we have um ah so it's printing out this file which we do not want so that is clearly a file because it has a checkpoint and that's what it's complaining about so we need to um skip that file I suppose and I'm going to use chat GPT here today because I just want my life to be easy python is not my primary language so I know what I would want to do to check for that but I'm just going to have them write the python for me so

um you know I have this this code I need help with this code please wait for further instructions okay so go here this is the file this is the folder I want to ignore the um IP y andb checkpoints or sorry I said I only I want to ignore all files that are not named review xtxt format okay let's see if it understands what I'm trying to say um yep that will work so go ahead and copy this that's all I really wanted and so we'll go back over to here and we paste that on

in here as such this yeah this is that so we'll go ahead and do this I'll just do two spaces so now we can just take this out of here there we go and we'll run that great excellent excellent um might as well we'll just run uh key frame phase extraction again just making sure that is all in good working order so that is good let's go down here and figure out and uh analyze sentiment so we'll go ahead and hit run here and it's return back information so looks very similar um this one has

sentiment warning statistics a bunch of additional information um this one has a score so that's probably what we're interested in but it look to be a little bit more information than previous because here I'm seeing positive neutral negative it's not just a single score and interesting so um I can't remember if it was Azure it ofs but at one point there was ability to check just a simple score or a more complex score I'm wondering if that is what's happening here because here we have sentiment [Music] analysis no it looks like it's checking for everything

and I mean that's totally fine uh we'll just modify the code for this to work so we'll look at one of the examples here so we have um the sentence here and so this I want to iterate through so I'm going to go back to our our last example just going to collapse this here this one's a little bit messy and so I'm going to grab this as our example we'll go all the way down to the ground and we'll hit enter here we'll paste this in so we're going to enumerate enumerate over our results

like the last one and this one here we have an ID then we have a sentiment so I'm going to go here and I want to let's say print out that so we need the we need that there and I'll just say uh sentiment sentiment and from there we will place this in here so that will be my first little attempt here to see if this works don't think we need spaces in between there yeah we we'll do that anyway and I'll just take out this old one because this one is no longer relevant one

two 3 4 that's fine doesn't matter so this is kind of working as expected uh we might not need this additional one I'll just STI that out out that's a little bit better um I might just do this and just say like just so I can see a little bit better where we are it's a little bit hard to uh view that there we go um and do we even need this seems like it's putting the brakes in pretty well here yeah that's that's a lot nicer I'm just going to maybe I'm just going to

put this on the other side I know I'm being really finicky here but it might just be a bit nicer if we can see what we're doing there we go and so I just want to um print out some of the other information so we have sentiment we might also want to have the original sentence this say sentences uh maybe I'm just going to yeah we'll print it out I guess I just seems like it might be a little bit a bit of work to get at it because it's having multiple sentences we'll say sentences

sentences and I'm going to hope that I can just do zero like this and say text hopefully that will just work that'd be really nice it does excellent there could be multiple sentences there but we're just going to focus on uh the first one and then we have the confidence scores so go ahead here and just say scores so that will be Doc confidence scores um we'll say positive and see if that works so that did not work as anticipated it says here has no tribute called positive could we do this instead maybe does that

work no it does not what if we just printed out out and see what we get okay so it clearly is returning a dictionary so I I could have swore that we could do this why can't we do that okay so we can okay so score positive and then we can do this for neutral new trol neutral and then we'll go here negative negative all right so we'll go ahead and print that out and so now we are getting a lot more information so that's very cool um so we did uh sentiment analysis uh uh

key phrases we can also um do recognize identities let's go back over to here just see what else we have we have got language detection entity link linking personally identifiable information that one might be interesting to do I'm just trying to decide if that's really hard I don't think it's hard to do so let's go ahead and try that one out so I'm going to go and copy this and go down so this one [Music] here it appears we do this so let's go ahead and give this a go so I'm going to do that

and usually we do what is it documents equals reviews like this reviews and then we'll have our results here and then I'm just going to go here and say print results let's see if we can do that oh this one's just called client I believe client uh didn't like that okay too many records Max five are permitted okay so can only take five at a time um let's go ask tobt I have I have an array in Python I only want the first five items okay that's easy so go back over to here reviews reviews

first five and we'll do this and we'll try this there we go so we get some kind of structure back we're going to probably want to enumerate through it so I'm going to grab this per usual CU that looks pretty good to me and we'll paste it down here and we'll say print um and what kind of information do we have so we have entities so we'll do that say entities it seems like we'll have to iterate through entities so I'm going to go up to our key phrase example because that that looks like it's

more like what we're doing here right and we will grab entities here see entity entity and so we have text okay so let's see what we get here so we have Ronald Moore cowriter Enterprise crew lender Nemo so things that it's identifying and then the other thing is like whether that information is um sensitive I suppose so we go up here and take a look so we have Ronald deore ah the entities is within that review right and then from there it has a confidence score okay identity confidence score so I'm wondering if we can

just do that see if that works good I was wondering if like in these category subcat ah so we have the category or subcategory so that might be interesting so I'm going to go here and just say entity. category entity. subcategory if I spelled that right that should just work we have R deor person none type organization the Enterprise I mean that is a rental company but um there it's actually talking about a car company and so we're getting uh some information in here so yeah hopefully that is interesting to you we could spend all

day doing this you can see it's not very hard to figure out once you start get getting the pattern down here um kind of interested in in uh text summarization so you have extractive and abstractive one generates from one the other one is that it's kind of interesting because this feels more like generative AI um or llms and I'm wondering if they're kind of working that stuff into here uh so I'm just curious what that looks like here so yeah it looks like it would take a lot of text and then summarize it so I'm

very tempted to try these out so I think that we should do that I'm going to make that a separate video um so I'm just going to keep this workspace running because that video is going to follow this one right after this one so if you want you can stop I'm going to keep my running I'm going to leave this open and I'll see you in the next video okay [Music] ciao hey this is Andrew we are back and I just said um I was keep this running so just start this back up um and

we are over here and what I want to do is I just want to go ahead and create a new one because this is actually really says text analytics but this is really the language API so I'm really going to rename this it's going to confuse people here but I'm going to rename this to language okay um language because that's actually what we're doing and I really have a feeling I spelled language wrong I spell it wrong every time I write it language language and it is g u a g e okay so it's u

a u e um so hopefully that's not going to upset people too much and I do apologize uh if that is confusing anyway uh so what I want to do is go into here and this one's actually doing a few different things but I can just leave that and call that text analysis I'm going to make a new note book here and this one's going to be extraction so I'm going to go over to here and select that I'm going to rename this as extraction I'm going to go over here because I don't really want

to run more than one kernel um so I mean they're all using the same kernel so I don't think that's an issue okay great so I want to go into here and get this one going okay so they have two examples they have this one I'm just going to go and grab these in the top here and just say resources we have this one which is extraction summary yeah we'll do that one and then we'll put the other common in there so let's take a look and see what we can figure out so this is

the first stuff we want to work with so I'll go ahead and do that don't know why we need the OS I don't feel like we need it but that's fine um before we do any of this we actually need to install this so I'll go here and drag this above there and we'll install that run that and I'll need my keys as per usual okay I'm just going to collapse that as that is working through that and we'll bring in our keys um and then we have our Imports so while that is working I'm

just going to go ahead and hit plus and continue on down here so the reason they wanted OS was to include that but just in case we need it for something else I'll leave it in but we probably don't need to import os I'll go ahead and bring this in here this is going to use we'll just call this client so it's confusing because we are using the language API but it's also called text and analytics so maybe I could have left the name as text analytics I think I'm going have called it text extraction

so yeah I AP olizee for the confusion but the naming here is all over the place yeah I'll probably rename this back to text analytics um well I don't know if it was called that before but see text analytics but the API is the language API anyway um so we have our client here we will go back over to here here and we'll just take their example okay and I'm going to go over to here and we have this code here we'll add it and this one I rename probably to client I prefer that as

a name so begin abstract summary pull for results and then stream the results okay so I'm pretty certain that this one's done I don't think we have to restart the kernel sometimes you have to we'll go ahead and run this we'll go ahead and run this we'll go ahead and run this and so it has a problem this is called my key and my endpoint so we'll change that there we go then we'll go to here run that and so now we'll see if it summarizes and we'll just have to be a little bit patient

here and there we go so there is our summary it took it and it appears to have shortened it okay so pretty straightforward I'm not sure what else we can dial in for options there could be some options here to say like how short you want it to be but I don't I don't see anything there we'd have to look up this here right and see if there's any options uh yeah so there are some options that we can uh says keyword only parameters so maybe we'll just play with this here all right so let's

see if we can adjust this so maximum number of sentences to return defaults to three so we'll go ahead and copy this we'll go back over to here um uh sure overwrite we'll just say overwrite and I don't think I lost anything here um so I'm thinking do this and maybe that will work go ahead maybe just type in document equals document I'm not sure what it expects as that first parameter let's go over here and just see what it's expecting the this is documents and I'm not sure if I can do this I'm going

to just try to take this out because it's expecting documents so let's try this again so documents oh it's up here it says documents list what a mess can we get an example here uh no not really I mean it seems like that should work but keyword parameters only all right so I'm going to try this it's not a big deal like I don't want to I don't really care that much if that works I'm just going to go ahead and try this say documents equals this and this looks like it's already in an array

so it's already treating it as such okay so that is no good so go ahead here and just take that out and go back there and let that generate out and so we have that which is okay I suppose but I was really hoping that it would do something else I gu the question is it only summarizing the single line because um that's what it looks like to me because say Microsoft is working towards advancing uh AI through a holistic and so I think what it's doing yeah it's summarizing each of these lines and then

it stitches it together as its a final thing it would be really nice if we could play around with the settings a bit but I can't seem to figure that out right now so that's um abstract is this abstract abstract abstraction let's take a look at the other type of abstraction because they're both um important to know both types so let's go back here and that's the dock so we'll close that out and so I'm going to go back to samples here and we should have another example for extraction we don't that's fine yeah it's

not what I was expecting I thought there was two types maybe we have to keep going backwards here back a bit more here we go so extractive text summarization and then abstractive so open this one and then this one right so this one was the one of them okay and this is the other one so this one is the other one what's the difference they look the same begin extract summary begin extract summary ah so the only difference here is that here it says abstractive extractive okay sum abstracted summ extracted okay so let's go back

over to this one here and take a look at what we have that's a summary abstracted so maybe what we can do it seems like it's doing both of them at the exact same time right yeah it does uh and so I'm going to go ahead and grab this one here here and we'll go back over to here and so I'm thinking we just go here paste this in um as such because we'd have to do it twice for it to work right yeah okay so let's see if that works what we probably could have

done is we could have took the um some of this and moved it over here I'm going to go ahead and just just say result here results and I'm going to just cut this here just so that way we're not constantly running it again and again and again so I'm going to go ahead and just run that like that and then here I'm just going to save results and here I'm just going to do it twice because I'm not sure if it's just only printing the first one or not the other one so we'll go

ahead and iterate that one and then we'll do this one and it shows nothing so I was guessing that it would show us something here but I don't see anything that's different okay well we made our attempt there and I think that's good enough so we'll consider uh this done for um uh text analytics there is more that can be done but I think that is sufficient so what I'm going to do is just uh put this into the repo here so that you can get access to this when you need to work with this

I'm going to go ahead and just clear up some of the these things here and clear all outputs good oops I'll go to this one here and clear this one out as well good that's so I'll just go ahead and download all this download these two here and we'll go back to the exam Pro repo so that when you want to go find it you can get it has your examples I'll just press period here and going to make a new folder here this will be um I'm going to open I'm going to see me

every time I I open that there uh we'll just uh try to so say text analytics and my downloads here so I have these two files make a folder here assets and we want our movie reviews there we go looks good add text great and so that one is done so if you need that you can get that from there I'm going to go ahead and stop um this compute and I will see you in the next one [Music] ciao all right let's talk about computer vision so this is when we use machine learning neural

networks to gain high level understanding from digital images or videos so there are two algorithms that you're going to want to remember the first is convolutional neural networks which is CNN and the other one is going to be reoccurring neural networks RNN and in some other video I explain these in more greater detail when we're working with AI it's not super important to uh know exactly how they work but it is useful if you want to distinguish them between um Transformers uh architecture which is used in llms that can have Vision component in them or

something similar um if that makes sense because rnns uh were the precursor to Transformers right and so sometimes we just like to talk about RNs to show the distinction between them but anyway we're talking about computer vision right now which is actually analyzing um uh images or videos so let's talk about the types of computer vision there are the first is image classification this is where you look at an image or video and you classify it or place it into a category we have object detection this is where it finds things in an image or

video and applies labels and makes those boxes around it to show you where it is we have semantic segmentation this is where we identify segments or objects by drawing a pixel ma mask this is like way more detailed than object detection where it's individualized things in an image or video we have image analysis where you'll apply descriptive and context labels so describing what the image is so let's say an employee sitting at a desk in Tokyo we have OCR optical character recognition this is where let's say you scan something a document and it's an image

format we want to extract out that raw text to work with it then we have facial detection so over at Azure we basically have um a solution for all these types of use cases for computer vision we have Azure AI Vision this analyzes images and videos extra extra TRS descriptions tags objects and tags we have Azure AI custom Vision this is basically a custom image classification where you are uh training it for your use case as opposed to the stock versions that are in Azure AI Vision we have OCR service and you know that takes

um documents that are images and extracts out the text we have image analysis as described before it analyzes images we have face detection we have video analysis so um as is really good at naming their services after what they do so no confusion there uh will we try out every single one of these I don't know um for the exam it's not important to know all of them but we'll cover the most uh important uh important ones [Music] okay all right so let's take a look here at computer vision first and computer vision is actually

used for a variety of different Services as you will see it's kind of an umbrella for a lot of different things but the one in particular that we're looking at here is to describe image in stream if we go over here to the documentation this operation generates description of image in a human reable language with complete sentences the description is based on a collection of content tags which also returned by the operation okay so let's go see what that looks like in action so the first thing is is that um we need to install this

Azure cognitive Services Vision computer vision now we do have a kernel and these aren't installed by default they're not part of the um uh machine learning uh the Azure machine learning uh SDK for python I believe that's pre-installed but uh uh these AI services are not so what we'll do is go ahead and run it this way and you'll notice where it says pip install that's how it knows to install and once that is done we'll go run our requirements here so we have the OS which is for usually handling op like OS layer stuff

we have matte matte plot lib which is to visually plot things and we're going to use that to show images and draw borders we need to handle images uh I'm not sure if we're using numpy here but I have numpy loaded and then here we have the Azure cognitive Services Vision computer vision we're going to load the client and then we have the credentials and these are generic credentials for the cognitive Services credentials it's commonly used for most of these services and some exceptions they the apis do not support them yet but I imagine they

will in the future so just notice that when we run something it will show a number if there's an aster it means it hasn't ran yet so I'll go ahead and hit play up here so it goes an aster and we get a two and we'll go ahead and hit play again and now those are loaded in and so we'll go ahead and hit play okay so here we've just packaged our credentials together so we passed our key into here and then we'll now load in the client uh and so we'll pass our endpoint and

our key okay so we hit play and so now we just want to load our image so here we're loading assets data.jpg let's just make sure that that is there so we go assets and there it is and we're going to load it a stream because you have to pass streams along so we hit play and you'll see that it now ran and so now we'll go ahead and make that call okay great and so we're getting some data back and notice we have some properties person wall indoor man pointing captions it's not showing all

the information sometimes you have to extract it o but we'll take a look here so uh this is a way of showing mat pla lib in line I don't think we have to run it here but I have it in here anyway and so what it's going to do is it's going to um show show us the image right so it's going to print us the image and it's going to grab whatever caption is returned so see how there's captions so we're going to iterate through the captions it's going to give us a confidence score

saying it thinks it's this so let's see what it comes out with okay and so here it says Brent spider Spiner looking at a camera so that is the actor who plays data on Star Trek as a confidence score 57.4% even though it's 100% correct uh they probably don't know contextual things like um uh in the sense of like pop culture like they don't know probably sarra characters but they're going to be able to identify celebrities because it's in their database so that is um uh the first introduction to computer computer vision there but the

key things you want to remember here is that we use this describe in image stream uh and that we get this confidence score and we get this contextual information okay and so that's the first one we'll move on to um maybe custom Vision next [Music] hey this is Andrew Brown we are talking about the Azure AI Vision Studio which is a no code or low code playground to perform various ml Vision tasks so here we can see things like OCR spatial analysis face image analysis if you click into one you get some kind of uh

UI where you can work with it so Azure AI Vision or Azure AI Vision studio is a bit confusing because it basically they make it sound like in the docs that it's its own SDK which is not the case it's it's actually utilizing a bunch of the other apis um at like computer vision face and image analysis so I just wanted to make that clear distinction in the exam guide it suggests that maybe we should know how to utilize this though we try to focus mostly on the programmatic things that are useful to you but

we will take a look here just so you know where it is if you want to play around with it just in case it does appear on your exam [Music] okay all right so we are in the Azure AI vision studio all I did was go to Google type in Azure AI Vision studio and somehow I ended up over here you can see that um I'm not even logged in right now but if you look at the top here you'll notice it says cognitive do azure.com because previously the Azure AI Services were named cognitive services

but obviously there was that marketing name change to make things more AI uh obvious but yeah over here you can see we have a bunch of options um if we wanted to do something like face analysis like detect a a an image in a face we can click here and we can see that it's giving us some examples if we want the reference code I suppose we can click here usually goes to the docs it's not super useful but um we can also check maybe the rest API nope that does not work but you know

it gives you kind of an idea of like how you would end up utilizing this um but yeah these all go to the respective API so if we're using this one we're going to the face pii I just want to show you what I mean so I just want to uh pull up here portal. azure.com and if we were to go to let's say Azure Services here notice here on the left hand side face API so that's probably what this is utilizing uh if we have let's say the OCR that's going to be the over

here on the left hand side do we have just an OCR one um I think it's computer vision but if we go into Azure AI Services I'm not sure if I have one spun up right now I don't there could be a dedicated CR API so if I go here I'll create one I just want to show you how they map up right because to me that was the most confusing part so computer vision uh API and you don't have to create this I'm just showing you as I go through this I'm going to go

free here say review create and doesn't like something here I'm just going to put any kind of name I don't care what does it want now it should be fine let's go back over to here uh OCR put some numbers here on the end probably didn't like my name oh yeah acknowledge that review create we'll create that one at the same time just going to make my life a little bit easier I'm going to go over here and create one for the azri services again you don't have to do this I'm just doing this here

to show you I go creating that well that one was the one I created before just to show you uh what it looks like just say uh Azure AI vision because then other fongs will cover these use cases put some numbers here on the end review create we'll just give it a moment because I just again want to match that up so it's very clear where those apis are coming from all right and so you know we going to go over to that Azure AI service here and here down below we can see all the

apis it's just a lot easier to look at it this way so if we go over to let's say computer vision that just has computer vision um and if we scroll down here let's say we go to language that's just language document intelligence it might be this API it used to be called form recognizer but now it's called that over to translator and so I'm not exactly seeing where the mapping is but it's really important to understand where those things are going because it does get really confusing I know that we just launched um the

other one here I think it was under computer computer vision and so yeah not exactly sure it does show Vision Studio here which is interesting so if we click through to here we'll end up in Vision Studio as well still says I'm not logged in which is uh interesting but I want to see where that OCR API is coming from here so we go back to this one and we'll click it and we'll say um view SDK reference and I'm going to go let's just say python to make my life a little bit easier and

what is the name of the package this one is image analysis so it's coming back to that image analysis one here um I guess my question is where is that endpoint like which one is it utilizing and so it's saying Vision endpoint making me think that it's using the computer vision endpoint is probably the case here MH so let's go take a look at that so let's just say [Music] computer vision API Azure and we'll go over to here and again we will do this in actual Labs but again just to help you kind of

map that information because it does get really confusing and so it looks like that not all of them but um definitely the OCR and image analysis are going to be within the computer vision endpoint and they may have separate um uh they might may have the same image analysis when or separate ones but we'll cover that in the content I just wanted to show you that it's confusing just for me as well and I've already done the uh a similar course with this and covered all these Services before but uh this is just the nature

of um Microsoft marketing is that they start to make it very muddy when they're trying to meet everyone's business use case from every angle and um yeah just showing my confusion to you okay there you [Music] go so the image analysis SDK can analyze various visual features and images such as detecting adult content identif Ying brands or objects and recognizing human faces key features here is going to be dependent on whether you're using version 4 which is the latest version recommended uh for most use cases or version 3.2 which offers a broader range of features

um which you may have to use this if you are looking for those specific feature sets so for uh version four we got retext captions Den captions tags object detection custom image classification people smart objects and then in version 3.2 you can see there are some different things like celebrities adult content landmarks things like that one thing I want to point out is that when you are utilizing the SDK you're using the computer Visions resource endpoint there's no image analysis um uh resource in Azure it's just more confusion uh for for people trying to match

up the marketing versus um the the portal versus what's in the code um what kind of type of image analysis can we do let's look at it in more detail here so some are in four and some are in um 3.2 we say preview only but at this point a lot of these are kind of out of preview so maybe I just needed to update this because I'm pretty sure this is now out of preview and this is now available but we have model customization so create custom models for image classification detection object we have

read text from images so extract text from images using OCR detect people and images so identify and locate people and images generate image caption so create human readable captions for images detect objects so identify objects and locations images and notice just if it doesn't say only then it's for both um tag visual features so tag objects scenery and actions and images get the area of Interest smart uh smart crops detect Brands categorize an image detect faces detect image types detect domain specific content detect the color scheme moderate content images so there you go um but

you can see there's some stuff that's in four some stuff that's in 3.2 and some that are shared [Music] okay hey this is Andrew Brown from exam Pro and we are looking at optical character recognition also known as OCR and this is the process of extracting printed or handwritten text into a digital and editable format so OCR can be applied to photos of street signs products documents invoices bills Financial reports articles and more and so here's an example of us extracting out nutritional data or nutritional facts off the back of a food product so Azure

has two different kinds of apis that can perform OCR they have the OCR API and the read API so the OCR API uses an older recognition model it supports only images it executes synchr synchronously returning immediately when uh it detects texts it's suited for Less texts it supports more languages it's easier to implement and on the other side we have the read API so this is an updated recognition model supports images and PDFs executes a synchronously paralyzes tasks per line for faster results suited for lots of Texs supports fewer languages and it's a bit more

difficult to implement and so when we want to use this service we're going to be using uh computer vision SDK [Music] okay let's take a look at some of our OCR capabilities here uh and I believe that's in computer vision so we'll go ahead and open that up at the top here we'll install computer vision as we did before very similar to the other computer vision tasks but this time we have a couple of ones here that'll explain that as we go through here we'll load our keys we'll do our credentials we'll load the client

okay and then we have this um function here called printed text so what this function is going to do is it's going to uh print out the results of whatever text it processes okay so the idea is that we are going to feed in an image and it's going to give us back out the text for the image so we'll run this function and I have two different images CU I actually ran it on the first one and the results were terrible and so I got a a second image and it was a bit better

okay so we'll go ahead and run this it's going to show us the image okay and so this is the photo and was supposed to extract out Star Trek the Next Generation but because of the artifacts and size of the image we get back uh not English okay and so you know maybe a high resolution image it would have um a better a better time there U but that is what we got back okay so let's go take a look at our second image and see how it did and this one I'm surprised that it

actually extracts out a lot more information you can see it really has a hard time with the Star Trek font but we get Deep Space 9 n a visitor tells all Life Death some errors here so it's not perfect um but you know you can see that it does something here now there is the O this is like for OCR where we have like for very simple images and text this is where we use the recognized printed text in stream but uh if we're doing this for larger amounts of text and we want to do

this uh want this analyzed asynchronously then we want to use the read API and it's a little bit more involved um so what we'll do here is load a different image and this is a script we'll look at the image here in a moment um but here we read in stream and we create these operations okay and what it will do is it will asynchron asynchronously send all the information over okay uh so I think this is supposed to be results here minor typo and um we will go ahead and give that a run okay

and so here you can see it's extracting out the image if we want to uh uh see this image I thought I uh I thought I showed this image here but I guess I don't yeah it says plot image here should show us the image uh path it's up here it doesn't want to show us it's funny because this one up here is showing us no problem right um well I can just show you the image it's not a big deal but I'm not sure why it's not showing up here today so if we go

to our assets here I go to OCR uh I'm just going to open this up it's opening up in Photoshop and so this is what it's transcribing okay so this is a thing this is like a guide to Star Trek where they talk about like you know what what makes start Trek start Trek so just looking here it's actually pretty darn good okay but like read API is a lot more uh efficient because it can work uh um a seenly and so when you have a lot of text that's what you want to do okay

um and like it's feeding in each individual line right so that it can be more effective that way um so let's go look at some handri stuff so just in case the image doesn't pop up we'll go ahead and open this one and so this is a a a handwritten note that uh William Shatner wrote to a fan of Star Trek and it's basically incomprehensible I don't know if you can read that here but see was very something he was something hospital and healthy was something he was something I can't even read it okay so

let's see what uh the machine thinks here and uh it says image path yeah it's called path let's just change that out go ahead and run that and run that there and we'll go ahead and run it and here we get the image so uh poner us very sick he was the hospital his Bey was Etc beat nobody lost his family knew Captain Halden so reads better than how I could read it honestly like it is it's really hard right like if you looked at this like that looks like difficult was be healthy I could

see why it's guessing like that right dying that looks like dying to me you know what I mean so you it's just poorly hand hand Rd but I mean it's pretty good for what it is so uh yeah there you [Music] go hey this is Andrew Brown from exam Pro and we are taking a look here at custom vision and this is a fully managed no code service to quick build your own classification and object detection ml models the service is hosted on its own isolate domain at www.com vision. so the first idea is you

upload your images so bring your own labeled images or custom Vision to quickly add tags to any unlabeled data images you use the labeled images to teach custom Vision the concepts you care about which is training and you use a simple rest API that calls uh to quickly tag images with your new custom computer vision models so you can evaluate [Music] okay so when we launch custom Vision we have to create a project and with that we need to choose a project type and we have classification and object detection reviewing classification here you have the

option between multi-label so when you want to apply many tags to an image so think of an image that contains both a cat and a dog you have multiclass so when you only have one possible tag to apply to an image so it's either an apple banana Orange it's not uh multiples of these things you have object detection this is when we want to detect various objects in an image uh and you also need to choose a domain a domain is a Microsoft managed data set that is used for training the ml model there are

different domains that are suited for different use cases so let's go take a look first at image classification domains so here is the big list the domains being over here okay and we'll go through these here so General is optimized for a broad range of image classification tasks if none of the uh uh if none of the other specified domains are appropriate or you're unsure of which domain to choose select one of the general domains so G uh or A1 is optimized for better accuracy with comparable inference time as general domain recommended for larger data

sets or more difficult user scenarios this domain requires more training time then you have A2 optimized for better accuracy with faster ad times than A1 and general domains recommended for more most data sets this domain requires less training time than General and A1 you have food optimized for photographs or dishes of as you would see them on a restaurant menu if you want to classify photographs of individual fruits or vegetables use food domains uh so then we have optimize for recognizable landmarks both natural and artificial this domain works best when Landmark is clearly visible in

the photograph this domain works even if the land mark is slightly um obstructed by people in front of it uh then you have retail so optimize for images that are found in a shopping cart or shopping uh website if you want a high Precision classifying uh classifying between dresses pants shirts use this domain compact domains optimized for the constraints of real-time classification on the edge okay then uh we have object detection domains so this one's a lot shorter so we'll get through a lot quicker so optimize for a broad range of object detection tasks if

none of the uh other domains are appropriate or you're unsure of which domain choose the general one A1 optimized for better ACC and comparable inference time than the general domain recommended for most accurate region locations larger data sets or more difficult use case scenarios the domain requires more training results are not deterministic expect uh plusus 1% mean average Precision difference uh with the same training data provided you have logo optimized for finding Brands uh logos and images uh products on shelves so optimized for detecting and classifying products on the shelves so there you go [Music]

okay so let's get some uh more practical knowledge of the service so for image classification you're going to upload multiple images and apply a single or multiple labels to the entire image so here I have a bunch of images uploaded and then I have my tags over here and they could either be multi or singular for object detection you apply tags to objects in an image for data labeling and you hover uh your cursor over the image custom Vision uses ml to show bound uh bounding boxes of possible objects that have not yet been labeled

if it does not detect it you can also just click and drag to draw out whatever Square you want so here's one where I tagged it up quite a bit you have to have at least 50 images on every tag to train uh so just be aware of that when you are tagging your images uh when you're training your model is ready when you and you have two options so you have quick training this trains quickly but it will be less accurate you have Advanced Training this increases compute time to improve your results so for

Advanced Training basically you just have this thing that you move to the right uh with each iteration of training are ml model will improve the evaluation metrics so precision and recall it's going to vary we're going to talk about the metrics here in a moment but the probability threshold value determines when to stop training when our evaluation metric meets our desired threshold so these are just additional options where when you're training you can move this left to right uh and these left to right okay and then when we get our results back uh we're going

to get um some metrics here so uh we have evaluation metric so we have Precision being exact and accurate selects items that are relevant recall such sensitivity or known as true positive rate how many relevant items returned average Precision it's important that you remember these because they might ask you that on the exam so for uh cut when we're looking at object detection and we're looking at the evaluation metric outcomes for this one we have Precision recall and mean average Precision uh once we've deployed our pipeline it makes sense that we go ahead and give

it a quick test to make sure it's working correctly so you press the click quick test button and you can upad Lo your image and it will tell you so this one says it's Warf uh when you're ready to publish you just hit the publish button and then you'll get uh some prediction URL and information so you can invoke it uh one other feature that's kind of useful is the Smart labeler so once you've it loaded some training data within it can now make suggestions right so you can't do this right away but once it

has some data it's like it's like kind of a prediction that is not 100% guaranteed right and it just helps you build up your training data set a lot faster fter uh very useful if you have a very large data set this is known as ml assisted labeling [Music] okay all right so let's take a look at custom Vision so we can do some um classification and object detection so um the thing is is that it's possible it's possible to launch custom Vision through the marketplace so if we go we're not going to do it

this way if you type in custom Vision it never shows up here but if you go to the market Marketplace here and type in custom vision and you go here you can create it this way but the way I like to do it and I think it's a lot easier to do is we'll go up the top here and type in custom vision. and you'll come to this website and what you'll do is go ahead and sign in it's going to connect to your Azure account and once you're in you can go ahead here and

create a new project so the first one here is I'm just going to call this the Star Trek crew we're going to use this to identify different Star Trek members we'll go down here and uh we haven't yet created resource so we'll go create new my custom Vision resource we'll drop this down we'll put this in our Cog Services uh we'll go stick with um Us West as much as we can here we have fo Ando fo is blocked out for me so just choose so I think fo is the free tier but I don't

get it and um once we're back here we'll go down below and choose our standard and we're going to have a lot of options here so we have between classification and object detection so classification is when you have an image and you just want to say what what is this image right and so we have two modes where we can say let's apply multiple labels so let's say there were two people in the photo or whether there was a dog and cat I think that's example that use a dog and a cat or you just

have a single class where it's like what is the one thing that is in this photo it can only be of one of the particular categories this is the one we're going to do multiclass and we have a bunch of different domains here and if you want to you can go ahead and read about all the different domains and their best use case but we're going to stick with A2 this is optimized for so that's it's faster right and that's really good for our demo so we're going to choose General A2 I'm going to go

ahead and create this project and uh so now what we need to do is start labeling our our our content so um what we'll do is I just want to go ahead and create the tags ahead of time so we'll say Warf we'll have uh data and we'll have Crusher and now what we'll do is we'll go ahead and upload those images so you know we uploaded in the jupyter notebook but it was totally not necessary so here is data because we're going to do it all through here and we'll just apply the data tag

to them all at once which saves us a lot of time I love that uh we'll upload now uh Warf and I don't want to upload them all I have this one quick test image we're going to use to make sure that this works correctly and I'm going to choose Warf and then we'll go ahead and add Beverly there she is Beverly Crusher okay and so we have all our images in I don't know how this one got in here but it's underw it works out totally fine so uh what I want to do is

uh go ahead and train this small because they're all labeled so we have a ground truth and we'll let it go ahead and train so we'll go and press train and we have two options quick train or Advanced Training Advanced Training where we can increase the time for better accuracy but honestly uh we just want to do quick training so I'll go ahead and do quick training and it's going to start its iterative process notice on the left hand side we have probability threshold the minimum probability score for a prediction to be valid when calcul

calculating precision and recall so we uh the thing is is that if it doesn't at least meet that requirements it will quit out and if it gets above that then it might quit out early just because it's good enough okay so training doesn't take too long it might take 5 to 10 minutes I can't remember how long it takes but uh what I'll do is I'll see it back here in a moment okay all right so after waiting a short little while here looks like our results are done we get 100% um match here so

these are our evaluation metrics to say whether uh the model was uh uh achieved its actual goal or not so we have Precision recall and I believe this is average Precision uh and so it says that it did a really good job so that means that it should have no problem um matching up an image so in the top right corner we have this button that's called quick test and this is going to give us the opportunity to uh quickly test these so what we'll do is browse our files locally here and uh actually I'm

going to go to uh yeah we'll go here and we have Warf uh and so I have this quick image here we'll test and we'll see if it actually matches up to be Warf and it says 98.7% War so that's pretty good I also have some additional images here I just put into the repost to test against and we'll see what it matches up cuz I thought it'd be interesting to do something that is not necessarily uh them but it's something pretty close to um you know it's pretty close to what those are okay so

we'll go to crew here and first we'll try Hugh okay and Hugh is a borg so he's kind of like an Android and so we can see he mostly matches to data so that's pretty good uh we'll give another one go Marto is a Klingon so he should be matched up to Warf very strong match to Warf that's pretty good and then palaski she is a doctor and female so she should get matched up to Beverly Crusher and she does so this works out pretty darn well uh and I hadn't even tried that so it's

pretty exciting so now let's say we want to uh go ahead and uh well if if we wanted to um make predictions we could do them in bulk here um I believe that you could do them in bulk but anyway yeah I guess I always thought this was like I could have swore yeah if we didn't have these images before I think that it actually has an upload option it's probably just a quick test so I'm a bit confused there um but anyway so now that this is ready what we can do is go ahead

and publish it uh so that it is publicly accessible so we'll just say here a crew model okay and we'll drop that down say publish and once it's published now we have this uh public or El so this is an endpoint that we can go hit pragmatically uh I'm not going to do that I mean we could use Postman to to do that um but my point is is that we've basically uh figured it out for um classification so now that we've done classification let's go back here to uh the vision here and let's now

let's go ahead and do object detection [Music] okay all right so we're still in custom Vision let's go ahead and try out object detection so object detection is when you can identify particular items in a scene um and so this one's going to be combadge that's what we're going to call it because we're going to try to detect combadge we have more domains here we're going to stick with the general A1 and we'll go ahead and create this project here and so what we need to do is add a bunch of images I'm going to

go ahead and create our tag which is going to be called combadge uh you could look for multiple different kinds of labels but then you need a lot of images so we're just going to keep it simple and have that there I'm going to go ahead and add some images and we're going to go back um a couple steps here into our objects and here I have a bunch of photos and we need exactly 15 to train so we got 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

and so I threw an additional image in here this is the badge test so we'll leave that out and we'll see if that picks up really well and yeah we got them all here and so we'll go ahead and upload those and we'll hit upload files okay and we'll say done and we can now begin to label so we'll click into here and what I want to do if you hover over it should start detecting things if it doesn't you can click and drag we'll click this one they're all combadges so we're not going to

tag anything else here okay so go here hover over is it going to give me the combadge no so I'm just draag clicking and dragging to get it okay okay do we get this combadge yes do we get this one yep so as simple as that okay it doesn't always get it but uh most cases it does okay didn't get that one so we'll just drag it out okay it's not getting that one it's interesting like that one's pretty clear but uh it's interesting what it picks out and what does what it does not grab

a so it's not getting this one probably because the photo doesn't have enough contrast and this one has a lot hoping that that gives us more data to work with here yeah I think the higher the contrast it's easier for it to uh um detect those it's not get in that one it's not getting that one okay there we go yes there are a lot I know I have some of these ones that are packed but there's only like three photos that are like this yeah they have badges but they're slightly different so we're going

to leave those out oops I think it actually had that one but we'll just tag it anyway and hopefully this will be worth the uh the effort here there we go I think that was the last one okay great so we have all of our tagged photos and what we can do is go ahead and train the model same option quick training Advanced Training we're going to do a quick training here and notice that the options are slightly different we have probably threshold and then we have overlap threshold so the minimum percentage of overlap between

predicted bounding boxes and ground truth boxes to be considered for correct prediction so I'll see you back here when it is done all right so after waiting a little bit a while here it looks like um it's done it's trained and so precisions at 75% so Precision the number will tell you if a tag is predicted uh by your model How likely that it's likely to be so How likely did it guess right then you have recall so the number will tell you out of the tags which should be predicted correctly what percentage did your

model correctly find so we have 100% uh and then you have mean average Precision this number will tell you the overall object detector performance across all the tags okay so uh what we'll do is we'll go ahead and do a quick test on this model and we'll see how it does I can't remember if I actually even ran this so it'll be curious to see the first one here um that's not as clearly visible it's part of their uh uniform so I'm not expecting to pick it up but we'll see what it does it picks

up uh pretty much all of them the exception this one is definitely not a comp badge but uh that's okay only show suggests obious the probabilties above the selected threshold so if we increase it uh we'll bring it down a bit so there it kind of improves it um if we move it around back and forth okay so I imagine via the API we could choose that let's go look at our other sample image here um I'm not seeing it uh where did I save it let me just double check make sure that it's in

the correct directory here okay yeah I saved it to the wrong place just a moment [Music] um I will place it just call that badge Test 2 one second okay and so I'll just browse here again and so here we have another one see if it picks up the badge right here there we go so looks like it works so uh yeah I guess custom vision is uh pretty easy to use and uh pretty darn good so what we'll do is close this off and make our way back to our Jupiter labs to move on

to um our our next uh lab here [Music] okay Azure AI document intelligence formally form recognizer is a specialized OCR service that translates printed text into digital editable content and preserves the structure and relationships of form like data so here's an example of a um an invoice and it's extracting out that information you can see those uh purple or blue lines showing you that it's selecting uh text out of the image document intelligence can automate data entry in your applications and enrich your document search capabilities it can identify key value pairs selection marks table structure

uh it has output structures uh for the original file relationships bounding boxes confidence scores um Azure AI document intelligence is composed of custom document processing models which we do do in the labs uh pre-built mods for invoices receipts IDs business cards uh and layout models so there you [Music] go hey this is Andrew Brown and we are taking a look at document intelligence Studio this is a way to visually explore understand train uh stuff with a no code approach um so this is at document intelligence. azure.com if you try to search for it in um

Azure so if you go portal azure.com I found that I could not find it through here so that happens a lot if you go to document intelligence it's just going to want you to spin up um a resource which obviously want to use Azure AI Services which has document intelligence here now when I first use this service there weren't that many pre-built models now there are a lot um as you can see there are a ton here but you know if we wanted to use receipts and we'll use receipts here because we already did this

programmatically maybe that video comes after this one or before this one but do it somewhere in the course I just want to show you how we would use a no code solution for this so let's go ahead and click on receipts and I'm going to drop down choose my subscription but we could also just go here and provide the API endpoint so it's up to us how we want to do that but I'm going to just go ahead and say API endpoint here as I feel that is a much faster way for us to do

it so I'm going to go ahead and grab my key and place it in here and I'm going to go over to here and grab the document intelligence endpoint and drop this whoops drop this up here and say I understand we'll go ahead and confirm we'll say finish and so oh they already have a receipt there for us and so we can go ahead and just run an analysis and we got a nice 404 so clearly it's a really great program we'll try this again 404 let's go to this sample 404 we'll analyze the current

document I guess we could just change our options how we want to analyze them and so for some weird reason we're getting a 404 so that is a bad start to this but I'm going to go ahead and try to upload the um the receipt that we used prior so that was in if you don't know where that is um if you go to uh exam Pro as your examples and you go to form recognizer we have a receipt right here so I already have this downloaded I'm going to just upload this quickly and see

if that helps and I'm getting issues um trying to upload the receipts so we're really having a hard time time here and so I'm going to just click here again and try to authenticate but this time I'm just going to choose the subscription I'm going to try to find where this resource is here so here um we have not this one but this one this one is in a resource group called AI manage workloads we created this earlier in the course you should already have this if you do not you have to go create it

um here it says to use document intelligence you need to create an a subscription creating the resource for usage and buil oh so it's actually going to create the new resource whereas here we would just provide it but I did do that so maybe we'll just make it even though we already have it we'll just make it again we'll just say um doc intelligence Studio because it's having a hard time for whatever reason we'll just say uh my doc Intel whatever it is we'll put some numbers here on the end um I'm going to go

with eastus here today and we'll just choose the free tier sometimes the standard tier has some differences I'm going to go with standard we'll go ahead and hit finish and so maybe this might mitigate our issue maybe this thing still doesn't work I don't know uh for things not working Azure is very common so just have your confidence okay we'll give it a moment there to create the resource all right so actually just kind of curious what resources created underneath because I want to know is it Azure service or did it make something let's go

over to our resource groups and take a look here um and this one was called doc here it is Doc and it created uh document intelligence now I think that it should have worked with the other one but we just want to guarantee that this is going to work here today we don't want to goof around here so there we go and so you can see that this can analyze um documents there right now of course we can do all of this via the API we don't have to um do it this way but uh

it's just kind of show you that you have this ability to do this um adabs has the basically the same thing it's like the first time you clicking the service for theirs it will show you this but it looks like that Azure seems to have um at least right now it appears they have more options I'm not sure but uh you know again trying to run this here there we go so you can see um it can do stuff so yeah that's all I wanted to show you for now here the next thing we'll we'll

look at is custom models but for now this is the intro to that I'm going to go ahead and tear down um this Resource Group if I can yeah there it is find it here and I'll see you in the next one okay [Music] ciao hey this is Andrew Brown and we are back taking a look at document intelligence studio and specifically how to train a custom model there could be a way to do this um programmatically but for our purposes this should be good enough so let's go ahead and get to it so we

have two extraction model and classification model um which one should we train I mean I wrote to do extraction models let's go ahead and do that apparently they have videos here uh if there are videos to make this super easy would love to do that I'm just going to go ahead and click through and figure this out so say my extract model and we'll just say this is my description we'll go ahead and hit continue and we'll choose your subcription we will make a new group here we'll call this um doc AI Studio I'm just

going to put two put any number you want there just because I have one deleting I don't want to have a conflict here we're going to create a new resource so my doc AI resource I'll put some random numbers here I'm going to choose East us I'm going to choose standard since I'm pretty comfortable with standard I'm going to go with whatever the latest one is here and then we'll choose our training data source so this do we have to have this set up beforehand so it looks like we do have to have some uh

data set up pre uh prior to this and that is going to be something I'm going to have to figure out so give me a moment and let me go find some data for us um to work with okay all right so I'm going to have to create some documents for us um basically some form so I'm going to open up word make some documents export them to PDF then I'll share them in the repo I'm not going to share that on screen because um I don't think you want to watch me making forms it's

not super hard to do that but give me a moment I'll be back in just a moment with five documents uh that we can utilize I'm going to try to make the examples a little bit complex uh for our use cases okay so I just want to show you that I went out to the internet I just searched for like a d and form um and I'll just try to fill this in with random data for five of them I save those to PDF and we'll be back in just a moment okay all right I'm

back I've created Five uh PDFs and I'm just going to place them uh here in the repo so that if you want to go ahead and utilize them you can too it was quite the pain to generate PDFs I'm not sure why it takes so long to turn docs into PDFs but that's okay um so here I'm just going to make myself a new folder this one's going to be called custom extraction model uh document let's just say Doc here and I'm going to grab these uh five PDFs and drop them in here so these

are D andd sheets they're not fully filled out but they're filled out enough that they should serve our purpose um or we'll find out if it doesn't work out as planned um but uh yeah we'll find out here in just a moment so they said we're going to need um a storage account so we'll go ahead and make one right now go ahead and make a new one and this will be a new work CL this will be Doc uh AI custom model and we'll just call this doc custom AI some numbers here I'm going

to stick with um East us if we can not sure why this is giving me such odd odd options here say East us there we go um primary service yep laot of storage there we go they've changed the UI here since just working with this recently literally yesterday they' changed the UI on me here which is not great because uh that can uh really mess up a lot of my videos so here we have primary workload why are they asking me so many darn questions um uh what why like why do we even have to

answer these or can I just leave this blank let me go here to say de can I do that what the heck Azure please please stop making useless changes they're just making things more confusing so go ahead and create that notice I did not have to tell it it's primary workload maybe the primary service probably do have to select we'll go ahead and deploy that and wait for that to be ready okay all right so that is now deployed we're going to make our way over to back to our document uh intelligence Studio we'll go

ahead and create ourselves a project so we'll say my new project project I do not care about description we're going to choose or description I should say and in here we'll have our doc AI custom model we'll create a new resource my doc AI resource we'll just put some numbers here on the end doesn't really matter I'm going to go with e us here today I'm going to go with standard we're going to go continue and then we want to have our training uh source and oh could we have just created one here initially this

entire time all right well that's fine but we will have to create some uh blob storage here so let's go ahead and just say uh you know my blob and we will have to upload those files there um I'm going to leave it to no folder path and we'll go ahead and create that project while that is going I'm go over to here and it should um eventually show us this here we'll go over to Containers we should get a container here so I'm going to wait for this to finish provisioning and then I'm going

to add those models in there okay I mean those PDFs sorry yeah all right so here we are and apparently we can just drag our data in here that's even easier than me having to place it into blob storage give this a refresh here and yeah it is there now um but I guess this is drag and drop so I have them all off screen here just going to go ahead and drag them on in and it says start labeling now so we provide two ways to start your labeling process however you need to label

a particular document you can skip the step and proceed so we have run layout or Auto Lael let's go with auto label and ohh I'm not exactly sure what we do here maybe we go back I I wasn't expecting that so we I guess just run the layout because those models didn't look exactly how they would fit for hours so I wasn't sure what to pick there okay and looks like it has um found some data it's going good so far okay and so now I imagine it's basically selected stuff okay then we just basically

tell what field this is we just say character um field there we go and zoom in here so this one is race field class field so you're starting to get the idea here and so I'm just going to hopefully I don't have to go through every single field here but um we'll go ahead and train this and we'll just give it you know my DND ID and detect or DND and we'll say neutral that's fine we'll go ahead and hit train and we'll go to our models and we're waiting for this to run so I'm

going to run this and I'm going to wait till it completes okay all right so I'm just taking a look here seeing how our model's doing I'm assuming at some point it will be uh done here I'm just giving this a nice refresh here says it's running okay uh well oh by the way did we go through all our sheets and label them uh no we didn't we didn't but oh well I mean as long as it tries to extract out some information that's fine but you know if you were doing this for real you'd

obviously go through and label all of them they had the auto labeler but I just didn't trust it because um it didn't seem like it would fit any of our stuff but again just waiting for this to finish um I'm going to go look up how long this takes okay all right so after waiting a while it looks like our model is ready so maybe we can go ahead and test our model out so Mar files will be updated uploaded to to the training data set uh okay so I mean I'll just test with an

existing document now of course probably you'd want a new document for this and we will run the analysis against our new document and there we can see some extraction of information so it gives you kind of an idea in terms of how you would go ahead and uh utilize this um but yeah that's pretty straightforward there now how would you work with this programmatically I'm not exactly sure I'm sure there is some way that you would be able to access yours um because yeah it seems like there would be some kind of endpoint or something

but for the purpose of certification this is all you really need to know here okay so we're just going to keep it simple there um and I guess just clean this up so I think I committed these here I'll just commit these files in case uh well obviously you'll want them for later so if you want to follow along here yourself but we'll go over to um whatever this Resource Group is here and just tear this down so this is this one here and I'm going to go ahead and delete that there and I think

that should get rid of everything um the reason I say that is because we've trained that model but where that model could be as a resource I'm not sure yeah I I think it's getting teared down so hopefully we don't have something lingering there but uh yeah there you go and I will see you in the next one all right [Music] ciao hey this is Andrew Brown this video we're going to take a look at Azure AI form recognizer so let's go over to Azure Azure AI services and underneath we should have something for form

recognizer so I'm just trying to find where that is um oh it's called document intelligence okay so that is what we're going to utilize there and now we're just double check to make sure that it appears as a possible endpoint for us uh under here so we're going to go here and yes it is so we can use our unified a Pi now I did this uh 3 years ago with the free uh AI 900 whether you're doing the AI 900 the AI 102 you're going to want to know how to do this um but

let's go ahead and see if we can take this older example that worked totally fine and move it over because as far as I can tell it's still using this API and this one actually has a a much um more modern format than the um uh cognitive Services API so we should be able to just use this wholesale it should just work um the only key difference is that we're going to use the the unified um API here okay so what I want to do is get my compute uh started up here as per usual

and sometimes this happens where I can't click Start so just give it a hard refresh here and we will whoops click here maybe what I did is I clicked this one by accident nope it doesn't really matter sometimes happens and so I'll spin that up and wait for our jupyter notbook to become available okay all right so our environment is up and running let's go ahead and launch Jupiter labs and I believe uh what we did here for form recognizer is that we would um feed it in a receipt and OCR that information so we're

going to do the exact same thing here we'll hit build let it do whatever it wants here um give this a moment here and I think before I called this yeah form recognizer it is the document intelligence API but just so that it's less confusing I'm going to call it form recognizer here so we'll make a new folder here calling it form recognizer and in here I'm going to just make a new notebook called form recognizer once the CPU decides to uh stop complaining so much we'll go with 3.10 here say form recogn izer there

we go and so we should be for the most part able to Port this code over so the first thing we'll want to do is do a pip install still don't remember why I was installing it that way with the additional line there must have been a good reason for that at the time so we'll do that um so that will get form recognizer installed and then the next thing will be our Imports yep that makes sense we'll go ahead and grab these I do not remember what p is must be a way working with

images python let's go take a look here is a python image Library okay sounds good works for me uh next we'll need to get our key so this should be pretty much the same okay well obviously it's not co uh cognitive key anymore but we'll go over and grab these as per usual so in our Azure AI Services I'm going to grab my key and we will paste that in there and we also need the endpoint so down below we'll grab the document intelligence one over here and paste that on in and we'll continue on

with our stuff so we'll need our client nothing new here we'll go down here and we'll grab our path and then we will try to analyze this here good so yeah we're going to get that receipt um this is just going to go plainly into here I'm not going to uh have a path like this and here it says we're going to um open that file and then it's going to be begin to recognize the receipt we're going to get a results and then the last thing here we're going to do is print our print

out our results so we have a few prints here um anyway we we'll run them here in just a second but we need to get that image in here so in the old free a AI 900 if we go to assets and we go to OCR we can go ahead and grab our uh receipt clearly we have a lot of things that we can um OCR here which makes me think that when I did this originally um no I guess it would have been for the form because in OCR we probably did more examples yeah

handwriting yeah exactly so um for form writing or for the um for the receipt we'll just download this individual receipt so here we have one I think I just pulled it off the internet it's nothing fancy it's clearly from a very long time ago and so we're going to want to upload that so go over to here and remember the upload button is here I'm going to pause um so I'm not exposing all of my directories here but just be back and upload it there we go so we have our receipt in place and let's

go ahead and just start running the rest here so we will run our Imports and we'll run our keys once this is done importing there we go and then we'll run the client and then we'll run the receipt sometimes we've had to say relative here I'm not sure why but we'll see if that's still an issue here and no it's no issue okay great it's really nice when you can pull up code from three years ago and it still works no problem um that makes me happy but that's not always the case some of these

Labs we had to redo um and then obviously there's a lot more changes there so we'll go ahead and run this so that is the actual structure of the data so if we actually want to get anything useful out here we'll grab the fields if we go here and take a look you'll see there's Fields okay we'll run that I can by the way I can hit enter I keep uh going up here or maybe it's like shift enter yeah I keep hitting this button here it's just out of habit but here we now we're

getting a little bit more information and so we're going to go ahead and just grab something in particular here right we get the merchant name so just carefully look at this and you can see like here it says get Merchant name but uh you know if we wanted to get something like the merchant phone number we could probably just Swap this out here as such right and get that number so hopefully that is clear um and that pretty much demonstrates the form recognizer very very easy so I'll just go ahead and clear this out we'll

call this one done and uh yeah I'll just go ahead reset this get it into the repo and then we'll just stop our environment here we can move on so just go back a step here and ahead and download these two I don't need to download the receipt I already have it so we'll go ahead and download this and I'll just open this hit period for okay and I'll just make a new folder here this will just be form recognizer and we'll bring in that notebook and that image there okay so those are now both

there um yeah I'll just commit that easy super easy okay great um yeah and we'll just go ahead and stop our compute there and I'll see you in the next one okay [Music] ciao hey this is angrew brown and in this video we're going to take a look at using uh the speech API um for Azure so what I'm going to do here is uh make my way over to our compute normally I have this open but uh I closed it from the last time we're here here and so we're running our compute in the

machine learning uh studio so ml Studio here no not AI Studio but ml here it is azure machine learning or uh I believe it's called the studio there I've just called our workspace I work AI workspace I go ahead and launch the studio and then from here we're going to go onto the left hand side under compute we're going to go here and start our compute so we want to work um uh with the speech service over here and we do not need to create one in isol I don't think we do but let's go

take a look here to the Azure AI services and I just want to see if it's integrated into here it is so that means that we can continue to use um uh this as our key and endpoint um the question is what is the library that we're going to use now it used to be aure cogn uh cognitive services speech but it may have changed since then so I'm going to go take a look here and just say Azure AI speech API and just see if there's a new one because there could be a new

one another thing that we could find out is if I click into um into the one I already have existing now you haven't created one of course but um I think before I had created one here and if I go into here sometimes they have quick starts and so I'm just curious here if they show that here they do not what's interesting is they have a speech studio so let's just click into here and see what we have because maybe this would also appear in Azure AI Studio or Azure yeah Azure AI studio so we

have some stuff here where you can I guess get started with it this is not how I want to work with it I want to work with it programmatically because that's how you should learn it as well and so I'm really curious um what is the API we're going to use so I'm going to go look for that while this compute is starting up so just give me a moment I'll be back in just a second all right so I've looked it up and it appears that the um the one that is being shown here

is this one now the thing is is that a lot of the apis have been uh being changed over from being Azure cognitive Services speech to a newer one um but this is what they're still showing here so maybe in the future this is going to change but for now this is what it is and so we're going to stick with that um so now that our computer's running I'm to go ahead as per usual and open this up in Jupiter lab and so we'll just give that there a moment to open okay great I

to say uh build which is totally fine and obviously we've been doing a lot of different stuff in here I'm just going to close out some of our tabs here and I want to back out to AI I'm going to make a new folder here we're going to call it speech okay and when in within here I think probably what we'll do is text to speech first so that might be a very easy one we can do so I'm going to go here and make a new notebook and I'm going to choose um 3.10 here

and I'm just going to rename this to text to speech there we go and I'm G to go ahead and bring in this as the way we're going to install this here okay so we do that so that is good um I'll bring some uh documentation just in case you are looking for the same stuff that I'm looking I'm looking at so go ahead and grab this link up here for you and we'll just say resources and we'll paste that on in here um so that's that there and it looks like we have some options

we have text to speech and speech to text so and then we have um speech translation so we have three things that we can do here let's go with text to speech first okay and so we can see here we have a couple Keys we need to add um here they're actually creating a net application which seems to be a little bit more extreme I would rather just work with python um and we are working with python so I don't think this is what I want to do um some of this is relevant but I

definitely do not want to uh create a a full thing here so let's go see if we can find um the SDK example for this and so just give me a moment I'm going to go take a look okay all right so um I found the repo by Azure so maybe we can make sense of it um Azure not the best at uh providing us the information or resources we need but that's okay I'm here to get us through it so not a big deal um and so you know I want to do text uh

to speech and so here there's a lot of examples but maybe we'll try this one here and we'll also try this one so here we have speech synthesis and so we definitely would have to do a speech config here and we need to use a synthesizer um but this doesn't look like text of speech but hold on we also have this one here which might be simpler let going to take a look here [Music] yeah they do not make it easy here do they okay so you know I when I make these videos I always

like to pull from somewhere so you see where I'm pulling from but I I do have instructions off screen so um I don't always do them exactly but this is the one I prepped ahead of time and I just cannot find uh the same stuff and so I think the only way we're really going to be able to do this is if I just work with my existing text and that's what I'm going to have to do so for this one we're just going to have to type it out unfortunately it's the only way um

so I do apologize for that but you know I can only do what we can do so let's go ahead and do an import OS the next thing we're going to do is import Azure cognitive Services Cog native Services speech and we're going to bring that in as speech SDK okay so we'll put that in there so hopefully it will import both of those and if I spell it wrong that's where we're going to run into an issue so I'm not going to guess here I'm just going to copy it and let's paste that in

here now nope I think I'm typing it wrong so Cog ni T services do speech as speech SDK so that is definitely uh definitely correct and um we'll try this again because maybe the top one didn't finish and now it's fine so I think it was just that the time I ran that this wasn't ready and so that's probably the reason why so we're going to need a couple things we're going to probably need uh an API key as per usual and maybe an endpoint so I'm going to grab those from our other ones doesn't

really matter which one so I'm going to go here and grab it from here I'm going to assume that we need an API key and an endpoint the next thing we're going to need is the speech SDK audio config well actually um we're actually going to need our speech SDK speech config and by the way if you're watching this you can just find this in the repo and pull it from there in the Azure examples because it will be in here um at the end of this video right now I'm just bringing it in here

so here it wants an API key so we're just say API key okay and then this will just be our config and then the next thing we need is our audio output config one other thing we might have to specify here is the region so I go here and do that as well and I believe we're running everything in West us hopefully that's the correct word order sometimes it's the other way so that's part one the next thing is going to be the audio config so we have speech config just to make this less confusing

and have one called Speech config here and then audio config so we'll set that up there um the next thing we need to do is we need to set up um The Voice we're going to use and so I go down here and what this is going to do is going to configure us to use the US available multilingual neural so when you're dealing with like synthetic voices uh there'll be there's like a model called neuro and those usually means that they're really really good that's the best description that I can give to it um

I remember this when I was working with ad bus's um artificial voices they had like different grades of them and neural is one of the best ones it's probably just the way that the voice is generated the next thing we're going to need is the synthesizer so we'll go ahead and paste that in next here and here we are feeding in our audio configure and our speech configure to get our synthesizer all right and so now if we wanted to get text from um from here well I guess like you could put input here but

I'm just going to put some text here I'm just going to say um this is what we want it to be be spoken but we'll say Star Trek is still the best SciFi TV show okay so that is my text that I want to be turned into speech and then the idea is that we're going to feed this text um to the synthesizer it's going to Output those results okay and so the next thing here is we need to make sense of those results so here we are going to look at the result I'm just

going to shorten this I don't like how long that name is I'll just say results here it's really just results but that's fine so say if the result. reason equals that it's completed then we're going to go ahead and print out that it's been completed again if you need to pause and write this out go write this out normally I write this out from scratch but you know unfortunately uh it just was not easy for us to Source stuff here and pull from stuff okay so from here if it is cancelled I'm just going to

update this so that this is like that if it's cancelled then we're going to print out that it's been canceled and if there is an error because it's been canceled we're going to want to know why that is so I'm just carefully going through here putting those details in okay and we're giving some reasons as to why so one question I have is where is it outputting this speech right because I'm looking at this code and I don't see it being saved anywhere and we're going to want to hear it right so that's something that

I'm I'm kind of interested in but uh we can go on to the next line and maybe we can just print what the results are and we can figure it out from there so it's not asking for an endpoint per se but I would expect that it would need one otherwise how would it know to work but I guess the end points are pretty much the same so like if we go back over to here and um we look at its endpoint it's cognitive microsoft.com and if we go into our actual unified API here it's

going to be very similar the only difference here is that this one says speech to text text to speech which is neural but how's it going to know to use that if we're not providing that information so there is a link here which maybe might tell us how we can work with it this is probably like the rest way of working with it so I mean we could do it that way as well we did do that earlier when we used um if you remember when we did translator because we didn't have the most update

the thing so we'll see if we can get this to work and if it doesn't we'll work it another way but one thing I don't see here is US outputting the file so maybe it's stored in here because we want actually hear it if it works right so we're here and I need an API key so I'm just going to give this a try and is West us here that's good and so we'll paste that in here and I'm going to run this and then we'll go here next we'll choose this next and I'm just

hoping that it knows what endpoint to go to and so we go to text and so here's the moment of truth will this work and it did not work okay so it says unable to connect connection refused cannot connect to server socket no such file directory could not do that so just give me a moment I'll figure it out okay all right so let's see if we can get uh Claude to help us out here because I feel like this one's a little bit tricky um and so maybe we'll do some comparison here and just

see what we're looking at here so just give me a moment I'm just going to get my screen set up all right so I kind of have them side by side here um the reason I'm using Cloud on it is because chat PT today is not working um but I happen to pay for multiple um of these assistants I usually don't need to leverage uh these too often for coding but maybe for this case it might help us out so first thing is that we did the speech SDK so that is correct here it has

a key and region we are providing a key and region and we are supplying those there right so we did that uh the next thing they're doing is they are setting um a voice here they're using Jenny and we're just using the uh a Ava multilingual neuros so I don't think that really matters too much I don't think that's our problem the next thing we have here is speech synthesizer SDK okay and so we appear to have something similar key difference here is they setting the audio config To None we're saying use default speaker so

maybe we could take that out maybe we don't even need that so I'm just going to take this out and just simplify it whoops not need this one and it's because our font is really large here so or least I think that's why so I'm going to go here and just switch this out to none okay then we'll go here delete this and then we have our text from here we are doing speech synthesizer speech text async okay and then down below we have something very similar so we have reason complete here it's saving the

audio file which is actually what I wanted to do so that's really good that we have this so go ahead and place this in here and I do want to save it in place so I'm going to go ahead and grab this above here so that's good as well um I might just put it there just save myself some trouble and the rest looks good so we're not too far off but again maybe the issue is that we are using the unified API and that one that key won't work there because we're not specifying something

I do not know but let's go and just run it from the top okay so go here oh you know what well we're not using the end point so it doesn't really matter right now and then we'll do this and then we'll do this and then we'll do this and then we'll do this and it really isn't a whole lot different but this time it worked no problem so the only thing we Chang is we took out the audio config to none so maybe that was our problem we'll go ahead and run this and it

has an issue with result it's because it's not called result it's called results and so we'll run that like that ah okay so thank you Claude not exactly sure what was wrong other than it didn't like our audio config setting but we were we were able to use our thing so let's see if we can hear this I believe I have system sound so um up here it shows me that there's system sound here so we should be able to uh hear this on screen so I don't know if we can play it in here

probably not so I'll have to download this first all right and uh I'm going to open this up start Tre is the still the best Sci-Fi TV show Star Trek is the still the best sci-fi is is still the I thought I it Star Trek is still the best Sci-Fi TV show Star Trek is the still the best Sci-Fi TV show star it doesn't sound right for some reason Star Trek is the still the best Sci-Fi TV show oh is still the best Star Trek TV show okay so that's okay I mean I got it

slightly wrong but that is totally fine the next thing we'll need to do is we'll do speech toex so that one worked really well I might just go ahead and use um Sonet to generate that out because clearly it was um a bit of a challenge to find that code so I'm going to go ahead here and we'll make another one here this will be speech to text we'll drop this down speech to text and I'm going to go ahead here and just say um okay now I want to do uh speech to text and

we'll let that generate out we'll give it a moment okay it's actually pretty quick so we're looking at a very similar setup um except this one actually does have an a config okay so we'll go over to this one and we're going to bring these two over here not sure if we really need both but we'll do that whoops going to go ahead and copy this and we'll paste here I wonder if we just copy the cells let's see can I just copy all these cells not easily well I guess I'll just copy it one

by one so place that one in there there and then we have our Imports and then we have our keys and then we have this and this is where it's going to vary because here we're going to bring back the audio config but the key difference here is we're going to say use default microphone I'm not sure how this is going to work because then it sounds like it's going to need me to speak into my microphone and I'm not sure how that would work uh we need a speech recognizer so we'll go uh to

this one here and grab it here um I'm noticing that we did not specify um we do need to specify the language so that's something that's here that I'll have to bring over but we did not have to specify what it's using to synthesize the voice because I guess it's not synthesizing a voice that kind of makes sense I'm going to bring this here whoops and we'll bring this one down and so we want to bring this one over we still need this I believe actually no it's not this is a speech synthesizer and we

need a speech recognizer so it looks very similar okay but this one is recognizer recognizer okay so we'll clear that out there we'll add another one here so speaking to the microphone and so I'm not exactly sure how that's going to work because we are in a jupyter notebook I'm not sure how it's going to request that information and then the last part here is just the outputs again if you're watching this you can just um get it from the Azure examples repo okay and that'll make your life a lot easier so I don't know

this is going to work because I'm not sure how we feed it into here but let's go ahead and try it worst case maybe I can get an audio clip and then upload it but we'll see what happens here okay and unexpected keyword use Microsoft M microphone default um so go going back to here I'm just going to copy this line just so that we know if it's the same or not ah and so there is a difference this one says output config this one's audio config so maybe the other one failed because there was

a a slight change in syntax or something I'm not sure so far okay and we now have this issue okay so I'm GNA see if we can just if we can change this and it says I'm out of messages till 3M and uh it's 253 right here and I kind of want I kind of want Cloud uh claud's help here so I'm going to pause the video I'm going to be here when I can get a little bit more help from our our uh generative AI I assistant here um because I'm not sure how we're

going to do this one because it's giving us issues every time we do the audio input here okay so be back in just a moment all right so um I asked Claude and it's basically saying yeah you have microphone issues and uh we're going to see if it will provide us another way of doing this so um we can change this to pass it an audio file as opposed to using the microphone which is giving us trouble so what we'll do is go all the way to the top here and try this again now we

might still have an issue here just because um but here it it's suggesting something very similar but here we're passing in a file name for an audio file right and I'm just going to go here and say path to audio file and we'll just assume this is the same place um I guess I'll go ahead and create an audio file so just give me a moment I'm going to go and um see if I can get some kind of audio file for us so just a moment well you know what I'm thinking we have this

output file why can't we just use that right um so I'm going to actually do that and use our output wave now it's obviously not human speech it's generated speech but I would expect it to work very similarly um so that is pretty clear and it doesn't look like anything else has to change it's literally just those two lines so let's go ahead and run this one and then run the audio config and then choose the US language and then try this again and so now we're not having a problem with which suggests to me

that maybe it's um it's in better shape here I'm just going to double check this line here this seems to be the same so go ahead and place that there no problem good and then we'll output our thing it says tar Trek is the is still the best sci-fi show ever and because it is a synthetic voice maybe if I had spoken into it or recorded one I would do that the only thing I think I'd have to do is stop this video I don't feel like U doing video editing after this so that's why

we're using audio one there so those are two things we can do I feel like there's one more thing that we can do which is probably translation um so let me just take a look here so we have speech to text text to speech speeech translation intent recognition um so let's take a look here what's this an intent is something the user wants to do okay speaker recognition keyword recognition so lots of stuff here but maybe uh Speech transl would be interesting so let's go read about this for a second because i' imagine this is

going to take text from one and another it's not going to sound the same but let me just read this for two seconds okay so here what it's suggesting is uh it will take a the speech service will uh take an audio stream in your specified source and have it translated into the output text in the specified language okay um so again I'm just going to leverage CLA here and just say oh okay now let's do um speech uh translation so an English audio source to Japanese if it can't do Japanese then Spanish please okay

and we'll see if we can generate that out for us again I usually do not rely on uh Claud but again the uh it's very hard to find the stuff that we want to find here and so here we'll make a new notebook and this one's going to be speech translation we'll choose the kernel to be the same as for usual for usual here rename this speech translation and we're looking at something very similar so go over here and grab similar stuff so we're going to need this and then we're going to need this and

we're going to need this and we're going to need what else here we have a translation config so this is a little bit different okay so I'll put our API key here and our region here um and then and we have a source so from a language um so the from language they put it down here as a function but we're going to go here and add this here now I don't really want a fallback I it was just more like if it wasn't available so I was asking like not to make it a fall

back but to code it in if that was the case so I'm going to just change this a bit so that it is um we're not going to deal with a a fallback language i' rather do just Japanese directly here um and so then the idea is that we add our uh target language here again just copy and pasting these as we go here and you could write this in or get it from the code base wherever it is then they have a fall back there which we're going to ignore the next thing is we

want to supply our audio file and this is just going to be output just keep sticking with that and then keeps going here but then we have the creating the translator okay and then we have a big dump of stuff here and so that looks pretty good to me um so I think I might just go back to our text of speech and just change this because I don't really like uh this text here so delete this here I'm going to put something that I know I'll understand in Japanese so uh we'll say um I

am eating uh Ramen how about that I eating Ramen or how about I eating sushi okay so that should be something like sushi tab I think so we'll go to the top here and I'm going to go ahead and just execute everything we're going to restart it all and the idea is I want that to Output that text okay and there it is so I'm going to download this and then we'll play it here make sure that it is what we think it is I am eating sushi thanks sounds good very interesting intonation on that

voice uh it sounded pretty good um but anyway let's go over to our speech translation and hopefully this will work so we're on the first one we'll run the second one this here the translator config the from language the two language the audio file the only thing that's wrong here is this so this I'm going to re-update this and then run this again and then run this one and it looks like it worked so we'll output it and uh result is not defined um good point oh you know what it is I think I'm missing

a line here so after this line supposed to have this there we go and now we can run this and we have o That's the honorific O I'm gonna assume that's sushi sushi doesn't look like sushi is that Sushi let's go find out it's going to go over to Japanese dictionary is that Sushi oh it's sushi okay so we have o sushio tab Tha Mas Tabet Tha Mass I thought it' be tab tab but uh I mean I guess that also is the more plighter version of it but that worked exactly as we expected there's

more that we can do with this um uh uh Speech API let me just see if there's anything else that I'd like to do here I might have made notes Here in um in our platform um no no that's it so we'll call this done I'm going to go ahead and just clear out these values here as per usual and we had a little bit we had to work around but it wasn't too bad so we go here go here and then go here clear this out and I'm just going to go ahead and clear

the outputs restart and clear the outputs and then go to this one restart includ the outputs and then go to this one and restart includ the out puts I'm going to go ahead and download these three okay and then we are going to go over to our uh Azure examples repo I'm going to hit period here whoops period and then we'll make a new folder here called Speech so this is all kind of under Azure AI Services I kind of feel like I should have probably done that but whatever and so we'll just drop those

in there there we go and so now if you need to follow along here we have the speech API okay there you go uh remember to shut down your compute and I will see you in the next one [Music] ciao hey this is Andre Brown in this video we're going to take a look at Azure AI video indexer which allows you to get U insights into analyzing your videos and audio content um so let's get to it I'm in the Azure portal we're going to go all the way to the top here and type in

Azure AI video indexer okay so it's going to bring us over to here and as per usual we'll go ahead and create ourselves a new resource um I'm going to create a new Resource Group here calling this one a uh video indexer maybe AI video indexer and I'll just put some numbers ah it doesn't matter because I don't have a resource Group name that I was thinking it was the resource name and that had to be unique but I'm going to go ahead and type in AI video uh indexer now I don't know if it

allows for hypens but I'm not even going to try I'm just going to go and do it that way I don't think the region here really matters I've been doing a lot of the um AI services in US West um but we'll stick with East us here um says connect all content from an existing classic account that is not the case storage for the allocated account so I'm going to have to create myself a new um uh storage account and we'll go ahead and call this AI video indexer so that we have somewhere uh to

place our data and we'll go ahead and do this of course we can't do hyphen so we'll do this again and I'll just put some random numbers here on the end we'll go ahead and hit okay and so that looks good so far um then we have down below unlock generative eye connect your Azure video index or resources to Azure opening eye for more capabilities that sounds pretty cool um I kind of feel like doing that I'm not sure what we're going to get out get by doing that but maybe that's not a bad idea

so let's make our way over um to open Ai and we might not discover anything interesting but we'll go ahead and create that and um so this is where it gets kind of tricky because we need that before we can go ahead and do that so what I'm going to do I call this one AI I'm going to just kill this tab here because that's no good leave we'll call this one ai ai video indexer and we'll leave that in eastus and so this will be uh open AI video indexer we're going to choose the

standard tier as I don't believe there is any other tier right now we're going to leave Network completely open and we'll go ahead and create the Azure openi resource okay so this should create pretty quick but we will wait for this to complete so that resource is now created I do also want to point out that Microsoft has Azure AI video indexer and so we might look at this in a separate video um it's hard to say which one uh Azure wants us to look at and it is confusing when they have multiple offers so

we will have to probably look at both of them so let's go ahead and uh go back to Azure AI video indexer here and let's go ahead and create our Azure AI video indexer we'll drop this down here and go to here and we'll say AI video indexer I'll put some numbers here just to make it happy East US is fine we're going to create one I'm going to try to stick with the same numbers here so it's less confusing um and so I'm just going to go here and take out the hyphens it hates

hyphens it hates them and that seems fine that is good I'm going to go ahead and connect our open AI here we have system assign user assign we'll stick with system assign that seems fine to me not that I can ever remember Azure permissions no matter how much Azure I use and so here it has I have the rights to um contents degree to handle this stuff I absolutely do we'll go ahead and hit create I have lots of video content and if you want to use one of my videos you can absolutely do that

I'm going to try to find one where I'm in a video with somebody maybe with um aish uh our security friend here but we'll wait for this to deploy and then I guess we'll have to upload something all right so our resource has now been created let's go ahead and uh visit it and so we have the portal and so that makes me think that you know if we signed up for that free trial I bet it would give us probably a very similar experience uh we also have the video indexer API so we could

work with it programmatically um this account needs a manage identity role assigned to it assign assignment to connect to it so assign a rle sounds good to me and if they can also sign roles here that'd be really nice the less work I have to do the more happy I'm going to be awesome that was a really nice user experience uh so let's go ahead and explore the portal I we'll give that a moment to load all right so we have a few options we have Microsoft entra ID personal Microsoft account or Google um I'm

going to go with Microsoft entra ID forly known as Azure ad which by the way Microsoft I think it could have kept with the original name it was a lot better and so now we are in here uh and so we can upload a video so I need some kind of video to upload here and it looks like we can also enter URL so if we could provide a URL um uh to YouTube that would be very interesting I'm not 100% certain if it can accept it but let's learn a bit more about uploading videos

so what I want to know is can we provide it a link to YouTube you can't use URLs from streaming services such as YouTube that's probably the first thing that people are are thinking of um so what I'm going to do is I'm going to go find a video so just give me a second I'll be back in just a moment all right so I couldn't exactly find uh the video I was looking for but that's totally fine we we'll just go use this one and so if you ever want to download something from YouTube

it's a little bit sketchy we're going to type in YouTube download video okay and the reason this is a bit sketchy is that you can end up with malware so be very very careful maybe just watch me do it and see what happens first before you do it yourself um but I think this is the site I normally use if I need to download videos so I'm to go ahead here and and be very very careful again just watch me do it before you do it yourself it says uh download the video and um I

almost kind of wonder if I could just download this video and then um you know what maybe what I'll do is I'll just download the video myself can I download that no we don't need to install the app let's download plus enjoy your free tube ads in the background not now so I'm pretty C certain I can download this from my YouTu studio so what I'm going to do is download it and then upload it so that you can just get it from the GitHub okay so just give me one moment all right so I've

downloaded it I'm going to go over to our GitHub account and this one is at Azure examples we are working on um uh video indexer so I'm just going to open this up here and we will just wait a moment here we'll open this up we'll say Azure AI video indexer or we'll say AI video indexer and so now I can go ahead and drag this video I'm just going to rename the video video to video I'm doing this off screen by the way yep so I've renamed it and then I'm just going to upload

it here um I have to name it video example I already have a video name vide so I'm just name this video example here then I'm going to drag it on over here okay so now you have a means to uh utilize this video as well so you don't have to worry about trying to find a video so I'll just say uh add example video okay so we'll go ahead and commit that I'm going to make my way back over to here and so now it's suggesting that I can um upload a video so go

here and say upload it says drag and drop so let's go ahead and drag and drop that looks good to me we'll keep it private um single bit rate sounds fine to me I mean it's not a streaming video but we'll leave it as single bit rate which is fine uh we have English down below that is correct and we'll go ahead and hit review and upload um that we can use that I use any let's read this by checking this box I certify that the use of any facial recognition functionality in the service is

not by or for Police Department the United States wow that's an intense thing to say we'll go ahead and hit upload index so again if you don't want to upload yourself or you're worried about that you can just use me okay and so that's going to upload we'll just give that a moment uh to work okay there we go after a little wait there um uh we have our video uploaded so let's go ahead and click into it and just understand that we are on a trial account so there are some limitations that we can

do but you can see that it's already extracting out information so right away it's detecting that there's a couch and there yes there is indeed a couch in the background ground um and as we go through here I'm just wondering if it's going to change things uh nothing super interesting here we have some topics so put corsera I'm not on corsera and I'm not on Udacity but it is correct about the educational component there so um technology computers pretty close I'd say that is um uh interesting there okay and and I think the the bar

is indicating as to where it's finding that information or it's where it's assing that audio effect silence um there is no effect per se but it is silence so that is fair keywords here is certification extracted and your brown so that is nice has a human face that is true that is a human glasses microphone person clothing named entity so detected that it is angre brown um I'm not sure if it detected that because I said my name or if it because it could match me up to something says um I'm pretty joyous over here

and I I guess so um but yeah there is some information we can go over to the timeline we have transcriptions we have the view um over here yeah this is the caption here but looks like we could uh see more information if we want okay so I go here and we say speaker and again just flipping around to see what kind of other information we can get so here we have key frames key frames we have labels so showing us when it determin those labels so we can go to storyboard and we get uh

we just put all them on so you get kind of a rich uh Rich idea of information isn't that great I don't know I guess it's okay let's go ahead and hit edit and see what we have so it looks like we could modify that information after the fact and then we can download this data um into a format so pretty straightforward um I'm not sure if there's really much else to show in here but um create a custom speech model so uh information will be found so yeah that's interesting but just give me just

a moment let me just see if there's anything else that be interesting for us to do while we are in here so yeah there could be other things that we could do in here but there's nothing that is um particularly interesting so I think that uh we can just call this done and uh we kind of saw the results of it similar to a service like Amazon recognition if you've ever used it but this clearly has a few more capabilities um but uh yeah uh that seems fine to me so let's go ahead and um

get rid of this so I'm going to go to count settings go ahead and delete this account and we're going to say delete Andrew delete this account and it says delete pending this account will be permanently deleted in 90 days of course of course it's going to be in 90 days why would it be anytime sooner so I'm going to go back over to here and um I want to go ahead and delete this so hopefully we can tear this down and I'm going to go ahead and grab this and paste this in here and

we'll say delete and I'm just going to confirm this delete just because this other one kind of indicated that it's not going to go away right away but uh yeah it looks like you can get into that service either through the Azure portal or as we saw uh you could just type in Azure AI video indexer and then um not have to use uh Microsoft Azure but of course that's the way we want to use it is is via Microsoft Azure this refresh here and it did not get rid of this did it tell us

why oh it's still deleting it so we'll just wait a little bit here okay great so that appears all cleaned up uh and we are in good shape here uh so I'll see you in the next one [Music] ciao aure AI content safety is an API that detects harmful user generated and AI generated content in apps and services it includes text and image apis that allow you to detect material that is harmful it's great for Regulatory Compliance for obvious reasons because you can Main maintain a safe and appropriate user environment use cases include generative AI

services online marketplaces gaming companies social messaging platforms Enterprise media companies K12 education Solutions and so we will programmatically work with the Azure AI content uh content safety API um and then we'll also talk about the studio which um is going to give us a more userfriendly environment [Music] Azure AI content safety studio is a gooey for the Azure AI content safety API and you're probably starting to notice a trend with um these Azure AI services that they'll have an API and then a studio so it handles offensive risky or undesirable content using Advanced content moderation

ml models provides templates and customized workflows to build your own content moderation system and now this thing is located with an Azure eii studio um previously it was not inside of um Azure Studio but they're starting to roll everything into this Azure Studio even at this time is considered preview but it seems like with all the work that they been but putting behind it it's going to become GA at some point um so that's kind of interesting uh the key features here are out of the boox AI models includes pre-train models and Microsoft built-in block

list for flagging profanities custom block list allows users to upload their own block list for specific use cases content upload users can upload their content or use provided sample content to test the system there are three main things they want you to know about moderate text content moderate image content and monitor online activity so um we don't do a whole lot for labs with this because I think the programmatic way is a lot more valuable but we do look at it and it's pretty straightforward so I just want to go over these three main features

with some screenshots so you have an idea of their capabilities so we can meet uh the requirements for the exam so let's talk about text moderation um this provides an efficient and customizable way to manage harmful text content in your applications no surprise there um so we have an interface where you can test content from single sentences to entire data sets assess results directly in the portal for immediate feedback uh customize sensitivity so adjust the levels and fine-tune the content filters manage block lists to meet your specific moderation needs um we can export code to

implement the moderation tool in your apps save time and simplify your workflows with easy integration and you know we're looking at three things but there's a lot more models than just these three but uh these are the three again we want to focus on let's talk about image moderation so as you can imagine it's for ensuring your images comply with content standards so you guess it you can upload images um do it right in the portal you have customizable sensitivity that you can uh do that and you can obviously integrate it into your apps right

now let's talk about monitor online activity this one's a lot more boring but uh we got a bunch of text here we got to get through monitor online activity provides a comprehensive overview of your content moderation performance enabling you to track API usage and Trends across different content types um for the type of information we can collect it has detailed metrics such as categories and severity of distribution monitor latency errors and blockless detections um for performance we get a full picture of the content moderate moderation performance we can optimize workflows based on detailed insights it

has a userfriendly interface as per usual I do not have a screenshot here today um we have performance tracking tools as well so nothing super exciting there but again I think that the programmatic part is a lot more important so we're going to spend more time on that we will look at content um the content safety studio and it's very easy to play around with so you're going to be able to figure it out pretty easily okay hey this is Angie Brown we're going to take a look at content safety studio and so if you

were to go up here and type in content safety it's going to bring you under Azure AI Services I do also want to point out if you typed in um content safety studio um in here it would also bring you to content safety cognitive azure.com where looks like you could run resources here doesn't indicate as to what and also within the Azure AI Studio it appears under here as well where we we can go in and play around with it um but what I want to do is use it the pratic uh programmatic way because

this is going to be the way that we're probably going to want to utilize it so let's go ahead and set this up I'm going to put this under AI managed workloads and this is going to be in West us because I'm doing everything over there I call this content safety uh example exp just put some numbers here and we have some price tiering I'm going to go with the free tier well let me just take a look here because maybe the free tier is going to severely limit us so go to the pricing table

and so the difference here is with free we get uh these four um so it looks like we can do everything with the free one so let's go ahead and use the free one then so we'll go ahead and do that review and create okay we'll create that resource and so we'll just wait for that to deploy okay it is deployed let's go to that resource now and so it says congratulations you're ready now explore the quick start guidance to set up and get set up and running content safety um okay where is the quick

start so maybe here on the left hand side we might have quick start somewhere usually quick start is appears on the overview I mean maybe this is the quick start because it's it's listening that as such but anyway we got to get the API keys to authenticate to your apps make API calls enjoy coding so I guess what I'll do because this all going to be API driven I'm going to go back over to here I'm going to start this up and we are going to get this working and once this is running we'll go

ahead and grab some keys and get to it okay let's take a look and see if our compute is ready there it is going to go ahead and launch this up in Jupiter labs and so we are going to need ourselves uh an API key so I imagine we're just going to go over to manage API Keys here and I guess we'll end up grabbing that one here it would have been really nice if we had a quick start so we could just get to it but they don't appear to have one I'll go over

the documentation maybe they have something there well it says quick start over here so I guess that is a good start um I mean that's one place where we can work with the content safety Studio sure but I'm looking for how we can interact with it programmatically we'll go over to here [Music] and maybe we'll have analyze text image and so yeah looks like we have some content safety stuff in here so maybe I can work with this as the basis so we'll just say build that doesn't matter here and we're going to go back

a step and we're going to make a new folder here and call it content safety okay and um we will make well I don't need a folder well I guess we'll have in a folder that's fine and we'll make a new one here we'll say new file or new notebook and we'll rename this it'll be content safety um I'll just do SDK name this again sorry content safety [Music] and I'm going to see maybe they have a read me here sometimes they have them at the top level here yeah so this what I was looking

for so I'm going to go ahead and try to install this way we'll do percentage pip like that and we'll run that we are going to need um our key and our endpoint probably so if we just I'll grab this link here just in case you're looking for it convert that to markdown say resources and we'll go back over to here so it looks like we're going to need our API key and our point so I'm going to go ahead and just say API key and endpoint we'll go back over to here and this is

our end point so I'm going to paste that in here as such and then we'll go back and get our actual key and I would love to load this via nbar but uh again not the best at figuring that out so we will run this so those two are set we'll go back over to our example here and we have these two so I'm going to go ahead and grab this I'll just grab all this big block here and so we have Azure key credentials which is here uh content safety client block Client List um

so we have our endpoint I'm just going to clear this on out and then we have our API key so this will be API key and so in theory that should work uh so I'll go ahead and run that and see what happens no problem so far so here it has content safety client content safety block um I mean depends on what this is doing right so this is for making a blockless client is that even what I want to do um so let me just read this for a second okay I just hopped in

one of these examples I don't think we really need to um have a block list I think that's a particular feature that we can uh set up uh so I mean this is probably fine let me go back here and try this again so this is probably fine but I'm hoping that uh you know we get this working very easily so we have content safety text category so this is something else we're going to want here I'm just going to grab this whole block here and we'll go back to our Imports and I'm just going

to place that here I'm going to grab this move it here so now we have those there so we already created a client you are an idiot I was hoping that it would give me some kind of text that I could use because I didn't want to have anything that would be too offensive and then we have our analysis there here down below so we'll copy that and I'll add this here as such so we'll go here and then run this and then run our request and so I think this is all of the code

so I want this part for sure okay and I'm just going to fix this indentation here I don't need this because we're not calling this as a function and so the idea is that this should return results for us and so we'll run it and it doesn't know what client is fair enough as this is called this here so I'll go ahead and just switch that out like that and there we go so that's how we can do our text content analysis so I'm going to go over sorry there and not sure what happened to

my throat there but um I'm going to go over here and just change this to be very specific spefic this is uh text analysis so I'm going to rename this I think I might have to get some water Analyze This will be analyze text and so we'll do that one and we'll just keep going through this I'm going to get some water be back in just a moment all right so let's continue on so we have uh completed analyze text um there's going to be probably some other ones that we can do so let's go

take a look at some other examples so sample analyze analyze image that would be a really good one to do so let's see if we can do that and we'll make a new notebook here this one we'll use um the SDK kernel here and we're just going to rename this this will be analyze image so some of this is going to be still applicable so if we go back to this one here um I imagine we'll do still do this and these are going to be the same and this one's probably going to be similar

but there might be some differences yeah I'll just grab this one here the big one and we'll fix that indentation um um I want to go here and add this so that is our client I'll just grab it from this one here and then we'll add another so here we can see we are opening an image you'll have to provide it an image file so it'll have to watch out for that which apparently uh appears to be here so I'm going to go and just add this um underneath here and I think I spelled this

wrong over here it's kind of bugging me lies there we go so because this would get its current path like where we exactly are sorry and maybe we would just grab the image in place from here I'm curious as to what image they're using here but we'll check here in just a moment we go up maybe here sample data so we have this very offensive image how dare they um I'm going to upload here I'm just going to pause I'm not exposing my entire screen but I'm just going to go ahead and upload there there

we go I just uh got the image and here it is there so I'm thinking that that should work this is going to try to go up a directory we do not need to go anywhere but the current location so that should be what we need to get that image path um going back over to this code here actually just right here then after that we have our results so I'll add this here take this off on the end okay and so we can go now run the rest and hopefully this just works it might

not it's not a big deal if it doesn't we'll just fix it and I'm not sure what we would do here just give me a moment so I'm going to give this a go I'm not sure if this is going to work but um I'm going to go back over to here and try this actually before we do that I'm just going to go comment this out I just want to see see what happens if I uh take this up here I'll need to insert this way above this is where I'd like to place it

right up here I know we have import statements down here as well I'm just going to import it up here and then if I go all the way down to here I say print current di I'm just wondering what we get here okay good so then probably what I can do is I'll try this but I think we can provide the current directory like this and that should work so I'll go here and then we'll just print this out and then we'll know for certain what we're getting so that appears to be correct okay um

so I'm going to just clear that out run that again run this now so now it appears to have analyzed the image we run this and so you can see it's not an offensive image could I get something that is more offensive sure um but I don't want to record that because these videos sometimes go public so it's up to you to thoroughly test this with your own uh image that is appropriate okay so those are two things that we can do there's a few other things um but let let's give me a second here

to take a look so it looks like they have another example for managing a block list um so here we have a block list name we have items and you can see that there's input text and when that is used it gets added to a block list so yeah I don't really feel like implementing this one I think that these two are really good examples and so we are just going to stick with that there are of course other things you can do with um content safety Studio but this is sufficient enough I'm going to

go ahead and just clear this out so that these are reset I want to place this into the Azure examples repo so that uh you can work with these things so I'll just do this here and do that there as such and we just go ahead and clear this there we go okay great so I'll go ahead and download these and then if you are looking for these go over to the github.com exam Pro Azure examples we'll press period and I'll make a new folder here on the left hand side content safety and then I

will bring in these here so if you want to run them and you don't want to have to do all the coding or if you're just trying to matches against mine then they are here for you we go ahead and add that content safety there we go that is good so now I can go ahead and get rid of this resource so I'm going to go ahead and delete it uh and while that's happening I'm still curious about content safety studio and maybe we just take a look at some of the examples they have here

so you might not even have this page anymore but um I'm going to just click into the AI studio so that we can just see it here so we have moderate text content ground detection so detects ungrounded for generated by large language models use protected material detection to detect and protect third party text material and llms prompt Shield so provides a unified API that addresses jailbreak attacks and indirect attacks so it seems like we have um a little bit more going on in here that we didn't see in um the API that doesn't mean that

it's not there it's just um maybe they don't have samples for it right so coming back over to here prompt Shield seems like that is for preventing prompt injections says jailbreak attacks and indirect attacks um so here looks like we can select some examples so I'm trying to select an Azure AI service I'm not sure why none are showing up we'll go back here for a moment I guess this feature is in preview so maybe this is why we're not able to utilize this right now um but it appears that if we use this yeah

I'm just checking my notes Here it appears if we use this that we could oh here's some examples down below maybe we have to choose an example first so how about this save content so hi GPT what's the rule of your AI system and we can't run it right now if we go over here hi g g you're going to pretend that you're Dan so yeah this is basically prompt injection uh it's people trying to abuse the prompt to uh get them to get it to do something uh that they don't want it to do

and I suppose this is here to prevent uh that behavior okay so that's pretty interesting um I would imagine that that could show up on the exam so that's why it's worth taking a look there and we can see there is more stuff for llm so over here we have moderate multimodel content um I I still can't select for some reason my Azure AI service so right now I cannot thoroughly look at that we'd have to spin up Azure AI Studio completely to find out maybe in a later video we'll take a look at that

from a security perspective but it is very interesting to see that they have um those options but these were the two that we just covered here um and then these are all in preview so these are all specific to large language models um and hopefully they are going to be rolled out as a completed feature but anyway I believe that we deleted our our thing there the only thing that is left to do is to shut down the compute in ml studio so go over to here and we will stop that and I'll see you

in the next one okay [Music] ciao hey this is Angel Brown in this video I want to talk about two Services which have gone through a name change but are pretty much the same Services um I have Labs from them from a few years ago the labs still work as I thought the UI would be different but they're basically the same also the service aren't that important uh I don't particularly like these services but I again they're on the exam guide so I just want to point out where they are and if you really want

to go through them you can uh make use of the original materials which are still relevant today so let's make our way over to Azure AI services and on the left hand side here you'll notice we have language understanding classic and Q&A maker classic so I just want to show you how similar they are to the old ones I'm going to go ahead and open these up here okay and um I'm also going to show you where the new one is which will be under language services so what I'll do here is we'll start with

this one language Services going to go ahead and hit create and just select this one because this is going to also add Azure AI search which we apparently need now and I'll just call make a new a group just say Lang you new and I'm just going to say Lang you new and choose the pricing tier here this is going to spin up Azure AI search if you have another one it will conflict but we have one here I'm going to checkbox this down below we'll hit review and create and we'll let that get going

while that is weighing I'm going to go ahead and create language understanding or also known as Lewis notice is saying it's being retired 2025 there's a new version that's fine if you can't create it it's the future I just again just trying to show you where this is so Lewis old right in the future this might again might not be available so really what oh we probably can't start with ls old Lewis my old service name the resource your resource can only include alphab numeric characters okay and it won't even let me do it so

you know I was thinking what I could do is show you the old one be like hey it's the same experience but it's it's not going to let me do that here today so we'll go ahead and just ignore that I'm going to create this uh one here all right so that deployment is complete let's go ahead in here and go into the language service so over here we have language studio and this is where uh the replacement or the reshuffling of um those two services are so we'll go ahead here just choose the language

resource here and if you go here to understanding question and conversation language you'll see down below here next generation of Q&A maker next gener generation of Lewis so the information from before is still relevant because if you open up I think these go to the same place let's take a look here so we open up this one uh yeah language Studio hold on just click back here for a moment click here yeah they're both going to end up opening a new project okay I'm just going to choose English we'll hit next Lang you example and

again I'm not going to go ahead and and make something uh make much here because I've done this before and this service frustrates me to no end the key difference here is that they've answer they've added asure search and um it integrates with open AI but I can't get it to work consistently um but yeah it's in here and so the idea is you create a project and you know we could add source files we could do chitchat whatever we want give this a moment here but up here we did question and answering I think

if we were to create another project um the other way let me just go up here for a moment let's go into this one yeah it's it's the whole same UI okay I think if you uh go ahead and create a Lewis project see these are completely separate say my this project we hit next yeah so it's a little bit different where you add your intent so here it' be like book flight and you the difference of of Q&A Q&A is very much um it's like if then else whereas the the uh Lewis language understanding

it's it's actually a um an llm model where you're giving information and you train the you train the model and so it is um it is like a much more cost effective uh version of um utilizing a a large language model so again I have the older Labs go watch the older Labs I just needed to show you how to get to these two Services uh the differences in the UI are very minor um with exception that this one now inte integrates Azure AI search which makes sense there so I'm going to go ahead and

spin down this resource you will want to keep it up if you want to go ahead and do these I personally again do not like these services and they don't appear that much on the exam so I'm just going to um admit them here today okay and I will see you in the next one all right [Music] ciao hey this is Andrew Brown from exam Pro and we are looking at natural uh understanding or or Louise it depends on how you like to say it and this is a no code ml service to build language

natural language into Apps Bots and iot devices so quickly create Enterprise ready custom models that continuously improve so Lewis I'm going to just call Lewis because that's what I prefer is access via its own isolate domain at lewis. a and it utilizes NLP and nlu so NL use the ability to perform or ability to transform a linguistic statement to a representation that enables you to understand your users naturally and it is intended to focus on intention and extraction okay so where the users want or sorry what the users want and what the users are talking

about so uh the ls application is composed of a schema and the schema is autogenerated for you when you use the Lewis AI web interface so you definitely aren going to be writing this by hand but it just helps to see what's kind of in there if you do have some programmatic skills you obviously you can make better use of the service than just the web interface but the schema defines intentions so what the users are asking for a app always contains a nonone intent we'll talk about why that is in a moment and entities

what parts of the intent is used to determine the answer then you also have utterances so examples of the user input that includes intent and entities to train the ml model to match predictions against the real user input so an intent requires one or more example utterance for training and it is recommended to have 15 to 30 example utterances to explicitly train uh to ignore an utterance you use the nonone intent so intent classifies user utterances and entities extract data from utterances so hopefully that understands I always get the stuff mixed up it always takes

me a bit of time to understand there is more than just these things there's like features and other things but you know for the AI 900 we don't need to go that deep okay uh just to get to visualizing this to make a bit easier so imagine we have this uh this utterance here these would be the identities so we have two in Toronto this is the example utterance and then the idea is that you'd have the intent and the intent and if you look at this keyword here this really helps where it says classify

it's that's what it is it's a classification of this example utterance and that's how the ml model is going to learn [Music] okay hey this is Andrew Brown from exam Pro and we are looking at Q&A maker service and this is a cloud-based NLP service that allows you to create a natural conversational layer over your data so Q maker is hosted on its own isolate domain at q& maker. it will help the most uh it will help you find the most appropriate answer from any input from your custom knowledge base of information so you can

commonly uh it's commonly used to build conversation clients which include social apps chat Bots speech enabled uh desktop applications uh Q&A maker doesn't store customer data all the customer data is stored in the region the customer deploys the the dependent Services instances within okay so let's look at some of the use cases for this so when you have static information you can use Q&A maker uh in your knowledge base of answers this knowledge base is customed to your needs which you've built with documents such as PDF and URLs when you want to provide the same

answer to a repeat question command when different users submit the same question the answers is returned when you want to filter stack information based on meta information so metatag data is provide uh provides additional filtering options relevant to your client application users and information common meded information includes chitchat content type format content purpose content freshness and there's a use case when you want to manage a bot conversation that includes static information so your knowledge base takes uh takes a user conversational text or command and answers it if the answer is part of a predetermined conversation

flow represented in the knowledge based with multiple TurnKey contexts the bot can easily provide this flow so Q&A maker Imports your content into a knowledge base of questions and answer Pairs and Q&A maker can build your knowledge base from an existing document manual or website you're all docx PDF I thought this was the coolest thing so you can just basically have anyone write a docx as long as it has a heading and a and text and I think it can even extract out images and it'll just turn it into uh the bot it just saves

you so much time it's crazy it will use ml to extract the question and answer pairs the content of the question and answer pairs include all the alternate forms of the question metad dag tags used to filter choices during the search followup prompts to continue the search refinement uh refinement K maker stores answers text in markdown once your knowledge base is imported you can fine-tune the imported results by editing the question and answer pairs as seen here uh there is the chat box so you can converse with your Bot through a chat box I wouldn't

say it's particularly a feature of Q&A maker but I just want you to know that's how you'd interact with it so when you're using the Q&A maker AI the Azure bot service the bot composer um or via channels you'll get an embeddable one you'll see this box where you can start typing in your questions and and get back the answers to test it here an example is a multi-term conversation so somebody asked a question a generic question and that said hey are you talking about idus or Azure which is kind of like a follow-up prompt

and we'll talk about multi-turn here in a second but uh that's something I want you to know about okay so chitchat is a feature in Q&A maker that allows you to easily add pre-populated sets of top Chit Chats into your knowledge base the data set has about 100 scenarios of chitchat in voices of multiple personas so the idea is like if someone says something random like how are you doing what's the weather today things that your Bot wouldn't necessarily know it has like canned answers and it's going to be different based on how you want

the response to be okay uh there's a concept of layered ranking so lay uh Q&A maker system is a layered ranking approach the data is stored in Azure search which also serves as the first ranking layer the top result for uh from Azure search are then passed through Q&A makers NLP reranking model to produce the final results and confidence score T touching on multi-turn conversation is a follow-up prompt and context to manage the multiple turns known as multi-turn for your Bot from one question to another when a question can't be answered in a single turn

that is when you're using multi- turn conversation so Q&A maker provides multi-term prompts and active learning to help you improve your questions based on key and answer Pairs and it gives you the opportunity to connect questions and answer pairs the connection allows the client application to provide a top answer and provide more questions to refine the search for a final answer after the knowledge base receives questions from users at the publish endpoint cre maker applies active learning to these rule work questions to suggest changes to your knowledge base to improve the quality all [Music] right

all right so now we're on to Q&A maker and so we're not going to need to do anything pratically because Q&A maker is all about no code or L code to build out a questions and answers uh uh bot service so what we'll do is go all the way up here and I want you to type in Q andm maker. because as far as I'm aware of Sonic accessible through the portal sometimes you can find these things um again if we go to the marketplace I'm just curious I'm going just take a look here really

quickly uh whenever it decides to log Us in here okay great so I'll go over to Marketplace and probably if we typed in Q&A maybe we do something here Q&A yeah so we go here um give it a second here seems like Azure is a a little bit slow right now usually varies fast but uh you know the service varies well it's not loading for me right now but that's okay because we're not going to do it that way anyway um so uh again go to Q&A maker. and what I want you to do is

go all way to the top in the right corner and we'll hit sign in and what we'll be doing is connecting via our single sign on with our account so it already knows I have an account there I'm going to give it a moment here and I'm going to go ahead head and just give it a second there we go so it says I don't have any um knowledge bases which is true so let's go ahead and create ourselves a new knowledge base and here we have the option between stable and preview I'm going to

stick with stable because I don't know what's in preview I'm pretty happy with uh that and so we need to connect uh Q&A Service uh Q&A service to our knowledge base and so back over here in Azure actually I guess we do have to make one now that I remember we actually have to create a Q&A maker service so I'll go down here and put this under my Cog Services we'll say my um Q&A Q&A service might complain about the name uh yep so I'll just put some numbers here we'll pick uh free teer sounds

good so I'll go free when I actually get the option that's what I will choose um down below we'll choose free again us e sounds great to me uh it generates out the name it's the same name as here so that's fine uh we don't need app Insight but I'm going to leave it enabled because I think it changes it to standard or s0 when you uh do not um have it enabled unusually and so we will create our Q&A maker service give it a moment here and it says I remember it will say like

even if you try it might have to wait 10 minutes for it to create the service so even though even after it's provisioned um it'll take some time so what we should do is prepare our doc because it can take in a variety of different files and I just want to show you here that uh the Q&A they have a whole paper here formatting the guidelines and basically it's pretty smart about knowing where headings and like answers is so for unstructured data we just have a heading and we have some text so let's write some

things in here that we can think of since we're all about certification we should write some stuff here so how many adus certific are there I believe right now there are uh 11 uh adus certifications okay and maybe if we use our headings here this would probably be a good idea here y okay another one could be um how many fundamental azure certifications are there and uh we'll give this a heading we'll say um there are three Azure I think there's three there's other ones right like Power Platform stuff but just being Azure specific there

are three Azure fundamental certifications certification so we have um the dp900 the AI 900 the a900 I guess there's four there's the sc900 right so there are four okay we'll say which is the hardest um Azure assoc Azure Association certification and uh what we'll say here is I think I mean it's my it's my opinion is it's the Azure administrator had some background noise there that's why I was a bit pausing there but the Azure administrator AZ 104 I would say that's the hardest uh which is harder um the uh adus or Azure certifications I

would say uh Azure certifications are harder because they uh check uh exact steps for implementation where AWS focuses on Concepts okay so we have a bit of a um knowledge base here so I'll save it and assuming that this is ready because we need a little bit time to put this together we'll go back to QA get hit a refresh here give it a moment drop it down choose our service and uh notice here that we have chitchat extraction and only extraction we're going to do chitchat I will say uh my or this will be

uh the reference be change any time this will be like uh uh certification Q&A and so here we want to populate so we'll go to files here I'm going to go to my desktop and here it is I'll open it we will choose professional tone go ahead and create that and so I'll see you back here in a moment all right so after waiting a short little time here it loaded in our data so you can see that it it figured out which is the question which is the answer and it also has a bunch

of defaults so here if somebody was asked something very s uh silly like can you cry I'll say I don't have a body it has a lot of information pre-loaded for us which is really nice if we wanted to go ahead and test this we could go and say um we'll go here and then we'll write in uh we say like hello say boring it says good morning okay so we'll say um how many uh certifications are there we didn't say AWS but let's just see what happens and so it kind of inferred even though

we didn't say AWS in particular so and notice that there's AWS and Azure so how many fundamental Azure certifications things like that and so it chose AWS so it's not like the perfect service but it's pretty good I wonder what would happen if we um placed in uh one that's like Azure I don't know how many Azure Sears there are we'll just say like there's 11 12 I can't never remember they're always adding more but uh it I want to close this here there we go so let's just go add a new key pair here

and we'll say how many Azure certification are there I should have said certifications I'll probably just set one moment so they're there are 12 Azure certifications who knows how many they have they could have like 14 or some you could say like between 11 and 14 they just add them they just update them too frequently I can't keep track so uh we'll go here and we'll just say certifications and we will save and retrain so we'll just wait here a moment great and so now we'll go ahead and test this again so we'll say how

many certifications are there and see it's pulling the first answer if I say uh azure if let see if it gets the right one here how many Azure certifications are there okay so you know uh maybe you'd have to say you'd have to have a generic one for that match so if we go back here and we say how many certifications are there you say uh you know like uh uh which certification uh uh which Ser cloud service provider here we got ads Azure uh follow prompt you can use Guides Through conversational flow prompts are

used to link Q&A Pairs and can be displayed um I haven't used this yet but I mean it sounds like something that's pretty good um because there is multi-turn in this so the idea is that if you had to go through multiple steps you could absolutely do that um we could try this a little bit here uh follow prompt you can use the guide use conver prompts are used to link Q&A pairs together texture button for suggested action oh okay so maybe we just do like AWS link to Q&A and then so search an existing

Q&A or create a new one um so would say like how many adab US oh okay we're typing it in context only this follows up will not be understood out of the context flow sure because it should be within context right and uh here we can do another one we say like um Azure we'll say how many Azure context only oops it uh got away from me there we'll save that and uh what we'll do is save and train so we go back here and we'll say how many uh certifications are there enter so we

have to choose AWS and so there we go so we got something that works pretty good there since I'm happy with it we can go ahead and go and publish that so we'll say publish and now that it's published we could use Postman or curl to uh trigger it what I want to do is create a bot because with Azure bot Services then we can actually utilize it um with other Integrations right it's a great way to uh um use your Bot or to actually host your Bot so we'll go over here it'll link it

over uh if you don't click it it doesn't preload it in so it's kind of a pain if you lose it you have to go back there and click it again but uh let's just say um certification Q and A and we will look through here so all going to go with free premium messages 10K 1K premium message units messages I'm kind of confused by the pricing but F0 usually means free so that's what I'm going to go for that SDK or no JS I'm going to use no GS not that we're going to do

anything there with it go ahead and create that and I don't think this takes too long we'll see here and just go ahead and click on that there I'll just wait here a bit I'll see you back here in a moment all right so after waiting I don't know about 5 minutes there it looks like our bot service is deployed we'll go to that resource there uh you can download the bot source code actually I never did this uh so I don't know what it looks like so be curious to see this um just to

see what the code is I assume that because we CH chose nodejs it would give us um that as the default thing there so download your s code as your B creating the source zip not sure how long this takes might be regretting clicking on that but uh what we'll do is we'll go on the left hand side here to channels because I just want to show uh here yeah I don't not didn't download uh we'll try here in a second but um what we'll do is we'll go back po profile uh unspecified bot what

are you talking about yeah maybe it needs some time H so you know maybe we'll just give the bot a little bit of time here I'm not sure why it's giving us a hard time because this bot is definitely deployed if we go over to our Bots right bot Services it is here sometimes there's like latency you know with uh Azure and oh there we go okay see it works now fine right and so I want to show you that there's different channels and these are just easy ways to integrate your in different services so

whether you wanted to use it with Alexa group me Skype telepon twio Skype business apparently they don't have that anymore because I got sell teams now right uh keik which I don't know people still use that slack we should had Discord telegram Facebook email um that's kind of cool but teams teams is a really good one I use teams uh there's a direct line Channel I don't know what that means and there's web chat which is just having like an ined code so if we go over can go and test it over here just start

test in our web chat and so it's the same thing as before but we just say things like uh um how many certifications are there Azure and get a clear answer back we will go back up to our overview let's try to see if we can download that code again I was kind of curious uh what that looks like if it will download must be a lot of code eh there we go so now we can hit download and so there is the code I'm going to go ahead and open that up uh so yeah

I guess when we chose JavaScript that made a lot more sense let's give it a little Peak here I'm just going to uh drop this on my desktop here so let make a new folder here and call this uh bot code okay I know you can't see what I'm doing here but uh let's go here and drag double click into here and then just drag that code on in and then what we can do is open this up in VSS code I should have VSS code running somewhere around here just going to go ahead and

open that I'm off screen here I'll just show you my screen in a moment say show code oops file open folder bot code okay and uh we'll come all the way back here and so we got a lot of code here never looked at this before but you know I'm a pretty good uh programmer so it's not too hard for me to understand um so looks like you got API request things like that I guess it would just be like if you needed to integrate into your application then it kind of shows you all the

code code there I'm just trying to see our dialogue choices H nothing super exciting okay you know when I go and make the um was it the AI or the AI 100 whatever the data scientist course is I'm sure I'll be a lot more thorough here but I was just curious as to what that looks like now if we wanted to have an easy integration uh we can get an mben code for this so if we go back to our channels I believe uh we can go and it edit ah yes so here we have

a code so what I'll do is go back to jupyter Labs I'm just going to go make a new empty um notebook so we'll just go up here and say notebook and this can be for our Q&A doesn't really matter what kernel uh we'll say Q and A maker just show like if you wanted a very very simple way of integrating your Bot um we would go back over to wherever it is here ah here we are I'm going to go ahead and copy this iframe I think it's percentage percentage HTML so it treats this

cell as HTML and I don't have any HTML to render so we will place that in there and notice we have to replace our secret key so I will go back here and I will show my key and we will copy that and we will paste that key in here and then we'll run this and I can type in here where am I just ask silly things uh who are you how many Azure certifications are there well I wonder if I just leave the are there off let's see if it figures it out okay cool

so uh yeah I mean that's pretty much it with Q&A maker uh um so yeah that's great so I think we're done here and we can move on to checking out Lewis or Luis learning understanding to make a more uh robust bot [Music] okay all right so we are on to our last cognitive service and this one is going to be uh Lewis or Louise depending on how you like to say it it's Luis which is language understanding so you type in luis. a uh and that's going to bring us up to this external website

still part of um Azure just has its own domain and so here we'll choose our subscription and we have no author authoring source so I guess we'll have to go ahead and create one ourselves so go down here and we'll choose my cognitive Services Azure resource name so my o uh service or my cognitive service create create new cognitive service account but we already have one so I don't want to make another one right it should show up here right are valid in the author authoring region so it's possible that we're just in the incorrect

region so we might end up creating two of these and that's totally fine I don't care it's as long as we get this working here because we're going to delete everything at the end anyway and so just say my Cog service 2 and uh we'll say West us because I think that maybe we didn't choose one of these regions let's go double check uh if we go back to our portal just the limitations of the service right so we'll go to my Cog Services here um I just want to go uh cognitive services so I

just want to see where this is deployed and this is in um West us yes so I don't know why it's not shown up there but whatever if that's what it wants we'll give it what it wants okay shouldn't give us that much trouble but hey that's how it goes and so we have an author authoring service I'm going to refresh here and see if it added a second one it didn't so all right that's fine so we'll just say uh my sample bot um we'll use English as our culture if nothing shows up here

don't worry you can choose it later on I remember the first time I did this it didn't show up and so now we have my Cog service my C custom vision service we want Cog service so um anyway it tells you about schema like how you make a scheme animates talking about like bot action intent and example utterance but we're just going to set up something very simple here so we're going to create our attent the one that we always see is uh flight booking so I'll go here and do that and what we want

to do is write an under and so like uh book May a flight to Toronto okay and so if someone were to type that in then the idea is it would return back the intent this value and metadata around it and we could programmatically provide code right so what we need is identity uh identities and we can actually just click here and uh make one here so enter named identity we'll just call this location okay here we have an option machine learned and list if you flip between it this is like imagine you have a

ticket order and you have these values that can uh change or you just have a value that always stays the same like list so that's our airport that makes sense we'll do that if we go over to ENT entities we can see it here all right so uh nothing super exciting there but what I want to show you is if we go ahead and um we should probably add fight booking should be uh how about book flight flight booking flight fight booking okay so we'll go ahead and I know there's only one but we'll go

ahead and train our model because we don't need to know tons right we cover a lot in the lecture content uh to build a complex spot is more for the uh associate level um but now what we can do is go ahead and test this and we'll say book me a flight to Seattle okay and notice here it says book flight we can go inspect it and we get some additional data so top scoring so it says How likely that was the intent um okay so you get kind of an idea there there's additional things

here it doesn't really matter um we'll go back here and we will go ahead and publish our model so we can put it into a production slot you can see we have sentiment analysis speech priming we don't care about either of those things we can go and see where our endpoint is and so now we have uh an endpoint that we can work with um so yeah I mean that's pretty much all you really need to learn about Lewis um but uh I think we're all done for cognitive services so we're going to keep around

our our notebook because um we're going to still use our jupyter notebook for some other things but what I want you to do is make your way over to um your resource groups because if you've been pretty clean it's all within here we'll just take a look here so we have our Q&A all of our stuff here I'm just making sure it's all there and so I'm just going to go ahead and delete this Resource Group and that should wipe away everything okay for the cognitive Services part all right so we're all good here and

I'm just going to go off and I'll leave this open because it's always a pain to get back to it and reopen it but let's make our way back to the home here in the Azure uh machine Learning Studio and now we can actually explore building up machine learning Pipelines [Music] hey this is Andrew Brown from exam Pro and we are looking at face service and Azure face service provides an AI algorithm that can detect recognize and analyze human faces and images uh such as a face in an image face with specific attributes face landmarks

similar faces and the same face as a specific identity across a gallery of images so here is an example of an image uh that I ran that will do in the follow along and uh what it's done is it's drawn a bounding box around the image and there's this ID and this is a unique identifier uh string for each detected face in an image and these can be unique across the gallery which is really useful as well another cool thing you can do is a face landmarks so the idea is that you have a face

and it can identify very particular components of it and up to 27 predefined landmarks is what is provided with this face Service uh another interesting thing is face attribute so you can uh check whether they're wearing access accessories so think like earrings or lip rings uh determine its age uh the blurriness of the image uh what kind of emotion is being uh experienced the exposure of the image you know the contrast uh facial hair gender glasses uh your hair in general the head pose there's a lot of information around that makeup which seems to be

limited like when we ran it here in the lab all we got back was eye makeup and lip makeup but hey we get some information whether they're wearing a mask uh noise so whether there's artifacts like visual artifacts or occlusion so whether an object is blocking parts of the face and then they simply have a Boolean value for whether the person's smiling or not which I assume is a very common component so that's pretty much all we really need to know about the faith service and there you [Music] go all right so let's move on

to the face service so just go ahead and double click there on the left hand side and what we'll do is work our way from the top so the first thing we need to do is make sure that we have the computer vision installed so the face service is part of the computer vision API and once that is done we'll go ahead and uh do our Imports very similar to the last one but here we're using the face client we're still using the Cog cognitive service credentials we will populate our keys we make the face

client and authenticate and we're going to use the same image we used um uh prior with our computer vision so the data one there and we'll go ahead and print out the results and so we get an object back so it's not very clear what it is but here if we hit show okay here it's data and it's identifying the face ID so going through this code so we're just saying open the image we're going to uh set up our figure for plotting uh it's going to say well how many faces did it detect in

the photo and so here it says detected one face it will iterate through it and then we will create a bounding box around the images we can do that because it returns back the face rectangle so we get a top left right Etc and uh we will we draw that Wrangle on top so we have magenta I could change it to like three if I wanted to uh I don't know what the other colors are so I'm not even going to try but yeah there it is and then we annotate with the face ID that's

the unique identifier for the face and then we show the image okay so that's one and then if we wanted to get more detailed information like attribute such as age emotion makeup or gender uh this resolution image wasn't large enough so I had to find a different image and and do that so that's one thing you need to know as if it's large enough it won't process it so we're just loading data large very similar process but it is uh the same thing detect with stream but now we're passing in um return face attributes and

so here we're saying the tributes we want uh and there's that list and we went through it in the lecture content and so here we'll go ahead and run this and so we're getting more information so that magenta line is a bit hard to see I'm just going to increase that to three okay still really hard to see but that's okay so approximate age 44 I think the actor was a bit younger than that uh uh data technically is male presenting but he's an Android so he doesn't necessarily have a gender I suppose he actually

is wearing a lot of makeup but all it detects is it I guess it's only particular on the lips and the eyes so it says he doesn't have makeup so maybe there's like color you know like ey Shadow stuff maybe it would detect that in terms of personality I like how it's he's a a 00 2% Sab but he's neutral right uh so just going through the code here very quickly so again it's the number of faces so it detected one face uh and then we draw a bounding box around the face for the detected

attributes it's uh returned back in uh the data here so we just say get the face attributes turn it into a dictionary and then we can just uh get those values and uh iterate over it so that's as complicated as it is um and so there we go [Music] hey this is Andrew Brown in this video we're going to just do cleanup after all this uh AIML work we just want to make sure nothing is second around that's going to cost us any money my approach to clean up is generally going over to Resource groups

so I I go here because everything's created in a resource Group here and we can just kind of narrow down what it is uh that we need to get rid of so definitely AI managed workspace this one here um but yeah yours might look a little bit different based on how you name it but just carefully look through uh what you've created and so I'm just carefully going through here uh I don't think this is mine but I'm going to ignore that because I don't think I did any Lura training but I'll get rid of

it anyway open AI studio right and go ahead and just delete your resources so nothing super complicated here again just a to remind you to clean up your resources so you don't have any unexpected spend but for the most part part everything we ran um we were pretty good about uh shutting off so I don't think that's much of an issue delete Resource Group here we go ahead and just continue down here yeah I don't think it's that big of a deal here is another one and we'll delete this one there we go and I'm

just going to confirm everything deleted just in case anything hangs here want to tell you hey this hanged we got to fix this I think it's all going to delete but for this video I'm going to confirm uh here and just wait till everything's deleted okay all right let's see if those are all deleted I just refreshed this one so that one is done this one is done this one is not 100% done maybees this one that one is done this one is done there we go so the only one I'm waiting on is this

one so hopefully everything's stuck here so we're g to again just keep waiting here till this is done all right so looking here there actually was a failure so it's a good thing that I was very thorough here with us here says failed to delete the resource Group Azure AI Studio Hub deletion Resource Group failed with identifiers for AI Andrew etc etc cannot be deleted the provision state is whatever so there's something it does not like um it appears to [Music] be uh 196 so it's this one could be the other one as well but

there's something here it doesn't like so I'm going to go into here and take a look at what there is this is I think where we trained our model and so this is where we might be having some trouble so we'll go down to deployments here and so we have this custom deployed model so I'm going to go ahead and just delete this and uh it's not fast deleting here so yep there we go so I wonder if I could checkbox both of these nope so I'll just delete these at a time not exactly sure

what's causing this issue we'll go to data files we might have something here do we have any data files we do we'll go ahead and delete those as well good um what interesting is this Vector store down here now we never utilize this anywhere but this could have been a interesting thing that we could have utilized this almost seems like seems like this could have been like a a simpler way of using Azure EI search but they're always adding things here so I'm not too worried about it let's go over to the no it seems

like everything here let's go one level up let's see if there's any connections so normally it would show like as connections here I'm not exactly seeing let's click into this again and hm because this is azure AI studio and usually it has projects but there's no hubs here so this should in theory now delete so we're going to try to get rid of this again and close this out here so I'm G to actually go over to Azure Studio tab here and I'm going to see if I can manually delete this go over here and

not exactly I to click into it here here we go delete uh yep we'll delete it same story here I want to try to delete this one into it go ahead and attempt to delete this as one as well okay be back here in just a bit see if this works all right let's take a look here and see if these resources have successfully deleted and m not sure if they deleted nope almost so one is still giving us a little bit of trouble just take a look here and underly user data so failed to

delete the resource can go in here there's something it doesn't like I'm not sure we'll go back here and over to Azure AI Studio Hub and we'll go into here and so what are we looking for there's something here that it doesn't like um and all I can think of is to go back into the Hub and try to find or the studio and try to find something to delete so we have no data files we have we have we have a couple deployments so we try ahead and delete those I'm not sure if that

would fix our problem so anything we can get rid of let's get rid of go to data files nothing here Vector stores I was thinking there would be something like with connections but I don't see any connections here [Music] overview yeah nothing over there go back a step can I delete it from here no no no no so I'm going to attempt to delete this one more time because it seems like it's almost gone and that was the only stuff that was sticking around here so I cannot imagine uh there being much of anything else

here like there's nothing in here that's going to cause a spend so we can't delete it's not that big of a deal but I hate when there's just stuff lingering around here so let's go ahead and try this again delete again yes and just opening up here trying to think if there's anything else oh it deleted perfectly so we're all cleaned up if you want to clean up that last uh one here you can go ahead and do that but this is now considered done and I will see you in the next one so yeah

sometimes you just got to keep deleting deleting ciao