Fine-Tune GPT-4o Model Step by Step

hey hi everyone in this video we're going to look at how do we fine tune open AI GPT 40 model so first of all what exactly the fine tuning so fine tuning is the process of improving the model performance on a specific task or data set in the case of open AI we can provide our own examples that will improve the model performance and basically what that performance means here is these are the reasons that we should fine tune the model so first of all we can get the higher quality results and prompting most of

the time it's not possible for us to you know uh get the better results with just prompting and model requires more and more examples so that it adjusts with your business use case if you find tune the model we can see the better quality results next example uh you know uh next reason I would say to find tune is most of the times to convey your business Ur you require many examples and those examples can't fit inside a prompt and you don't have any other way you need to have some way to expose those multiple

examples to the model so that it can infer those instruction from it and that is possible only via fine tuning and since once you able to fine tune the model now you don't need to send those big prompts your prompt become smaller eventually reducing the cost and also reducing the latency because smaller prompts executes faster so those are the reason for the fine tuning but what are the different models available earlier we only had 3.5 available but now we also have a gp40 and GPT 4 mini so we're going to see in this video fine

tuning the GPT for mini what are the typical use cases you know for you know good for the fine tuning so two use cases that I personally found useful one is setting style tone or format most of the time when you generate some text it could be writing an email or it could be generating the product description you want it to be on a particular style or tone or a format and just by proning sometimes it's not possible we at a future smart AI we have finetune a product description generation model for one of our

client and we have found the improvements especially it's in style and the tone in which it used to create that you know description the second example you know use case that we have seen at a future smart AI is you know getting the reliable outputs so one of our client had an input let's say document and they wanted to have output to be you know a structured Json as an output and we were having difficulty to get that consistent Json output most of the time we have to you know call open AI again as a

fallback mechanism to get that desired output but then we were able to find two n GPT for Mini on their 100 plus documents and then we were able to get the proper Json output so these are the two use cases we have seen at a future smart AI you know building for our clients so definitely if you're not happy with your prompting and you're not getting the good results the fine tuning is something you should definitely try so let's get into the fine tuning um we are not going to cover any Theory we just going

to um fine tune these on a particular data set so this is the typical process when it comes to fine tuning we need to prepare the data set so whatever the examples we have you know um you know you might be having data in the CSV and we want to make sure we format those documents as per the you know your examples as per let's opening a chat model format because chat GPT or GPT for those are the chat models and they have a specific format so we need to format that data set once we

format the data set the next step is to upload the data set so we have to upload the data set as a training data it could be a training validation data and you will get some file ID once you upload it and once you have your file IDs available with you then you can start a fine-tuning job so when you make a request to the open a it doesn't start you know instantly rather they have some you know mechanism a cue mechanism where you know a lot of request comes to them and then they you

know kind of you know cue all of requests so for that purpose we need to create a fine-tuning job and then once we submit that request that we want to fine-tune this particular model on this data set we need to keep monitoring it or you can you know monitor or check you know frequently to see whether the job has finished either it will succeed or it will fail if it succeeds we get the fine tune model ID then you can test and call that model just like you will call the regular model so that's a

typical process so let me show you first of all what data set I got so here I got a data set so basically I got some medical report I know as an input and the expected output we want to extract the structured output let's say the structured output is uh I want to extract the patient name their age what diagnosis uh you know uh mentioned in the report and basically what is the prescribed medication these are the four things we are interested from the report that is generating and let's say we are we want this

and this is kind of use case as I told you This falls under getting the structured output this use case getting the reliable structured output that you might have some Json here or some kind of a um you know structure that this model is failing to produce even though adding you know couple of examples in the prompt now we're going to find you this thing we will not give any instruction about formatting we just want model should learn from the only examples and openi has mentioned we can have minimum 10 examples I have 20 examples

to see how it works so this is our training data a medical report and the extraction of those you know relevant Fields what we want okay so let's go to the first step which is basically converting our CSV training data what we just saw into the format that um you know the open specifies so we know that these are the chat models basically they require data as a messages that's what they require and the typical format that we need to send is basically some kind of a conversation sorry so here this is the simple system

message extract details from the medical report you see I'm here not mentioning What fields I'm expecting what is their structure why I'm not mentioning to just to show you that we want that structure of the expected output should be inferred from the example that is the reason I kept the uh you know the system instruction simple now as a training example we need to send it as a messages where the first message was this ass system message the second message is our medical report and the third message is basically an expected output so this is

nothing but the conversation this is the instruction for our GPT this is our input and this is the expected output this is the one example so we create our training example by iterating through the CSV file that what we got right so we read here our input CSV then we create one output file in the you know writing mode and we iterate through each rows collect the First Column the second column create the training example and write that training example as a Json you see this format is called Json this one Json l or Json

new line or you know if you look at that format let me run so that we can see what exactly that format so training data is prepared we prepared two files one is this is going to be a training file which is the medical records and the same file we have the 10 records as a validation file and we call it as a validation validation do that Json L now let's look at how is that format here is this as I told you one example so Json L is basically this is one example this is

one J on on a separate line so this is one object here is the another object and each object here is one our example if you look at this is our uh the list of messages we have a system instruction which is same across all the examples and then we have a this is a input and then assistant will be an expected output let me pause here somewhere yeah here you could see these are the assistant basically the expected output what we want so pretty simple you know training data that we required basically if you

are familiar with the chat GP you know we have to send messages you know as a list of messages to the openi now we have prepared the data next thing we require an open key because we are finding the open model so you can get your open so if you are inside your dashboard so let's say I am inside my dashboard this is one of the project called YouTube If I see on the left side there are API keys I created this one of the keys that I'm currently using I can simply create a new

secret key I can give some name and I can create the same way what I have created and which I'm already using here which is this open a key so I initiate open a client by giving that open C okay what was the uh step Next Step since we got our data set prepared we want to upload the file so let's upload the file to upload the file we have one function which take our file path right and then simply use this is client means what open a client so open client has the endpoint called

client. files. create creating the file we specify our file which is basically we are opening you know this is a file what you call so we have open our file from the file path and uh this is our file what I called anchor or pointer whatever and then basically you know we indicate here and then the purpose of this file why we are uploading these files we are uploading this file for the purpose of fine tuning and we get the response back and that response will have our file ID so let me run this function

and then you'll see we are running this function for the two files training data and the validation data because we want both of them to be getting uh you know uploaded there okay we got two file IDs that that we going to use next what step so we uploaded the file now we need to create the fine tuning job here we are so we have this function which create the fine tuning job it takes our training file ID here validation file is optional if you don't have any validation data you don't need to send it

it's just an optional thing and then we can specify the model this is a default model so I'm using gp4 mini this particular snapshot so you can specify any snapshot here similarly that create files we also have client. finetuning job JS create so we are creating a fine tuning job here by calling the open a client specifying our training file ID this is an ID and not actual file similarly I got the validation file ID and then finally the model that we want to use so let's run this particular function and I'm specifying this model

gp4 mini and calling without training and validation file and we should get some job ID which is created there okay so we got this job ID which is created next what once we got the job ID we need to mon monor that job progress so let's do that thing so we have a function which takes our job ID and it can retrieve that job and check its status using the job dot status we are seeing if the job dot status is either succeeded or the fail just return the job which is completed if not then

you know we can retrieve some events about our fine tuning job ID so that we know exactly what's happening there we print those events and then we wait for 30 seconds and retry again so it's going to take some time maybe I just start we look at the couple of statement and then I pause the video and then we I will come back when that training finishes so this is the um you know function let's run that function okay so first thing you can see at the moment status is running and the event is fine

tuning job started those files has been validated our job moved to the Q State okay and so these are the events at the moment again uh you know similar event so let me pause video maybe it might take couple of minutes and once it finishes you know um I resume the video okay so our job has finished it has taken five minutes and we could see the logs here what has happened so finetuning job started then basically we could see the status is running then you know it was going to a different steps we could

see the validation loss here and then eventually it finished and we got the result we got this fine tuning model let me tell you you can also specify certain um you know hyper parameters when you submit the job you can submit the AP box uh you know I think there are two three parameters are there which I didn't cover because mostly I found the defaults one is working anyway it was mentioned somewhere so I'm just searching where exactly the mention yeah we could smashy specify the number of epo by default I guess it's a three

or something learning rate and the batch size but I found you know default works pretty good um I don't spend my time here usually okay so let's take that finetune model ID we got you could see it is gp40 mini this is the model name and this is maybe something which came from our account which is future smart Ai and some unique identifier or something so let's copy this one we don't need to copy actually maybe I'm using that variable yeah here find model ID next test that model testing or calling that model is nothing

but calling the any other open model where we are using the chat completions end point we want model ID this model ID is nothing but this F model ID that we are passing here and it takes the same thing like those messages so you can see um you know so let we don't need to mention here actually here also we don't need to mention let's see what happens we keep the instruction same given a medical report extract the following details not even following data let's see what is what was our statement let me check our

data what was our statement No No No it should be while preparing our data extract details from let's keep the same this one which we have used during the F tun and if if I go here so I got this system instruction then this is the input which is we're going to pass so this is some medical report we got and let's run this thing so we got some result right and we could see it is some kind of a Json structure output so let's look at the content field of it okay we could see

this is some string Json string with the same kind of format that we were expecting even though we didn't specify any instruction that how we want to extract those fields we got those you know Fields based on the examples that we have provided and you know this will be a string so we can even get the python diary out of it by passing that Json and then we got this particular thing right so this is the beauty of fine tuning that model can infer those instruction let's also check whether this is you know even necessary

um let's say we don't pass any instruction we just pass an example because we have given let's see what happens okay we can still get the you know so this is what I mean now you see our prompt becomes smaller because there is instruction uh because the model has already uh understand the nature of all right this is also improves your L this going to save your cost this going to uh you know um you get the faster response right and since if you have enough examples then you can also get the better quality results

compared to the prompt what you have right so this is how you find the model you know um let's say open ip4 model and you could use it just like a normal model right one thing you know um open a also supports the the vision model like the model with the images so let me know if you interested in that video then I can create a video on how to finetune the open AI models with let's images and text basically a vision models if you are interested let me know you know in the comments thank

you