Creating Jarvis powered by OpenAI and Python | ChatGPT

who am I speaking with I am Jarvis I am an advanced artificial intelligence System created by Tony Stark my name stands for just a rather very intelligent system Jarvis what is your purpose as Jarvis my purpose is to serve as your trusted virtual assistant just as I did for Tony Stark in the lineman movies I am here to assist you with various tasks and provide you with information and guidance Jarvis can you explain the point of this video certainly saskar the purpose of your YouTube video is to Showcase how one can use chat GPT a

powerful language model like myself in combination with python to create a boost assistant similar to Javas from the Iron Man movies by demonstrating this process you aim to provide views with a step-by-step guide on how to develop their own personalized Voice Assistant using accessible tools and Technologies [Music] have you ever wanted your own personal virtual assistant when you watched Iron Man did you ever wish you could have your own personal Jarvis what does that look like to you Jarvis someone to help you with your tasks and they could do so in a way that would

feel a conversation like an actual assistant you know what you wish Siri and Alexa could do well now with the power of open ai's chat GPT you can have that what you just heard was an actual real conversation that I had with chat GPT acting as Jarvis in this video I want to teach you how to make your own program that will allow you to do what you just saw a program that will allow you to have an audio conversation with it I don't know if you noticed but in the footage that I recorded for

this video I did ask chat GPT to create some python code that would allow me to speak to it as though I was speaking to Jarvis but I found that that the code it created was not ideal as it had some errors and it was not exactly structured in a way that would have been nice for me to be able to speak with Jarvis and so I wrote my own code and I think it's just a little bit better for instructional purposes so that's what I'm going to be using rather than asking chatgpt to write

the quote for me and to start I'm going to start with the code that's at the bottom of this file um in particular these lines right here and I'm going to start with line 67 where I basically initialize this messages array which its purpose is to keep track of our conversation with chat GPT open AI the API that it has does not remember the conversation for us and so we have to store the whole conversation that we have with it in this array so that when we continue to talk to it chatgpt has the full

context of the whole conversation and can respond accordingly and so you know once we get that initialized we go into this while loop and uh there's basically just three steps that are here in this while loop the first step is to run this record text function and once this function runs what that what happens is the Python program waits for the microphone input for from the user and it basically waits to hear audio and once it does hear audio it Returns the text version of the of that audio in the form of a string in

this text variable and once it has that we move down to this line and what we do is we take our messages array and we append the text that we just received remember the messages array is meant to keep track of the whole conversation that we have with chat GPT and so one of the things that we need to do is that every time we get new input from the user we need to add it to the messages array so that we have that full context now the way that chat GPT understands messages is um

it understands it as a dictionary of two things of two entries the first entry is the role and the second entry is the content the point of the role is to telechat GPT who said this text and in this case it's us that said the text it's not chat GPT it's us that said the text so we put user for the role and as for the content we just put the text that we just said and once that messages array is updated we take that messages array and we send it to chat GPT well after

we send it to chat GPT chat GPT will send us a response in the form of a message which is just a string and then what we do is we pass that string over to the speech text function and its responsibility is to audibly say that text so that we can hear it and once those steps are done that's basically what we need to have an audible conversation with chat GPT so now I'm going to scroll to the top of this file so that I can go over those functions in more detail but before I

do that I will say that you will need to import these libraries right here that I'm highlighting as they are necessary for the speech to text intent text to speech portions of this code so I would just make sure that you import them and and um then you'll be good to go now I do go into a little bit more detail about how to get the uh text to speech stuff to work in a different video that I will link below and I will also put it um in this video so you can just click

on it while you're watching but these are a couple of libraries that you will need and I'm not going to go into too much detail in this video One Library that's new that you're going to need is this open AI library and its purpose is to just allow us to talk to chat GPT in a programmatic way using python now the first function that I'm going to briefly look over is this record text function I don't want to go into too much detail about this function just because I already have another video where I basically

cover this function it is untouched from that video it does the same same exact thing and all it does is it just listens using the user's microphone to get audio and once it gets that audio it converts that audio into text and it returns that text and we have a cover a couple of exceptions here to um you know for some error handling um but that's all I'm going to say about that function if you want to know more feel free to watch the other video on converting from speech to text it's only eight minutes

long and it goes into depth on this subject pretty well so I would say to watch that um but that's the record text function the next function is sending the messages to chat GPT now this function is pretty simple as well and it really is the bulk of this whole file in terms of new code basically it has two variables the first one is the messages array that has the full conversation and the second one is the model you can use whatever version of chat GPT you want I decided to use 3.5 turbo as I

believe believe that that's the version that is used when you go onto like the chat GPT website and just use the text prompt so that's the reason I chose it for this but that's the model I chose and then you just call this this function that is in that openai package and basically you give it these variables some of these variables that I have here these bottom three are just default variables that you don't really need to focus too much on but these top three variables are the ones that I guess you need to think

about a little bit more the first one is this model up here that you want to use the second one is the messages array which this function does expect an array and it's intended to be the full context of the conversation that you want chat gbt to respond to and this tokens here basically I set it at 100 but what it means is it is your way of letting chat GPT know how long do you want this message to be it kind of gives chat GPT an upper Bound for how many characters to use in

its response to you now I don't believe that the tokens to character is a one-to-one correspondence I think that like a hundred tokens gets you fewer than 100 characters but I'm not 100 sure correct me in the comments if I'm wrong but you can put whatever number you want here you can play with it just know that this puts an upper bound on how much they can send back to you now once you get the response from chat GPT you need to dissect that response in order to get the message from chatgpt and that message

is basically going to be a string and you put that in this variable here and that's what you return because remember in our Y in our while loop we return a string and all we want to speak text to do is speak that string now we also before we exit the function however we also need to update the messages array and we need to append the message here and notice how this highlighted section is nearly the same as online 63. except we're excluding the content and what we're doing is uh we're we're appending the dictionary

format to the messages array rather than just the text portion which is under content where so we're just going to append the dictionary because that's the structure that the messages array needs to have and so you basically need to do that then you return the message and once you're done with that uh you know you get the response from chat GPT and you need to run this speech text function to say the response back to you and if you go up here uh that speech text function is very simple three-line function you do need this

package in order for it to work but once you have that package the pi TT SX 3 package all you need to do is initialize the engine once you do that you give it the text that you want it to say then you simply tell the engine to run that command and say the text and that is all the function does it's very simple it just literally takes text and converts it to speech and once you do that it goes back to the top of the loop and we repeat that whole process all over again

and you now have a virtual assistant that you can have a conversation with I will give a bit of a disclaimer and say that the way that this code is written as is it will not immediately sound like Jarvis unfortunately if you want to do that you'll need to use another library in order to make that happen now me personally for the intro at the beginning I used fake you I didn't include it in this tutorial only because if you use the fake U library unfortunately it's very slow and while you will get a response

that sounds like Jarvis the slow speed just isn't worth it for me fake U does have a paid plan that you can be a part of that supposedly speeds it up considerably which might be worth it if you really want that Jarvis voice but for me I think it's enough without the voice but just so you know the way that the code is written as is right now you would not get that voice you'd have to add a bit more code one other thing that I will say that I forgot to mention is that in

order for this um send to chat GPT function to work uh you need to run these two lines up here so I did say that you do need to import openai but you also need to assign the API key to that open AI variable and this API key I have it stored in this dot EnV file because I don't want you guys to see it as a part of this tutorial but that API key is something that you get from openai when you make an account with them I can link the page where you find

that in the description down below but that is something that you also need to do in order to get that function to work and if you want uh chat GPT to act like Jarvis the only thing that you need to do is uh you need to at the beginning of the conversation very explicitly tell it hey I want you to act like Jarvis from Iron Man and that's how you'll get the responses like the responses that I got at the beginning of this video if you don't want to say that every time you run this

program then what you can do is you can just start off by already giving chat GPT a message right from the beginning and in the content and you basically just give it that string I just told you please act like Jarvis from Iron Man and now when you run this function it will do that now to prove that this actually works I'm going to run it and just for your FYI for it to work for me I had to run these uh three pip install commands you probably won't need this first one the python.env library

again I only had to install it because I didn't want you guys to see my keys so I put it in the EnV file but for you guys if you're not using that and you're just putting the if you're just going to put the python string in the openai key variable just putting the string here then that should be fine no problem but for me I had to run it however if you have not done much of this stuff before then you will almost certainly have to run these two uh to get this to work

and in addition to that I also had to run these three commands but these are from the text to speech video that I made and this is to make the audio stuff work now with that out of the way I'm finally going to run this but first I'll take off my headphones now I have my headphones off I took them off so that if Jarvis speaks you should be able to hear him so now I'm going to run the program and ask him some questions hello who are you greetings I am Jarvis an AI assistant

designed to assist and serve you in any way possible how may I be of assistance to you today what is your purpose my purpose is to assist and serve you in any way possible I owned an AI assistance designed to help you manage your daily tasks answer your questions provide you with information and make your life easier whether it's scheduling appointments setting reminders or finding information on the internet I am here to assist you in all your needs thank you Jarvis that is all for today you're welcome please feel free to contact me anytime if

you need assistance in the future have a great day and as you can see Jarvis is now talking to me and he is acting like Jarvis from Iron Man now I will say that there are actually some applications and Chrome extensions that will allow you to talk with chat gbt in an audio interface hello there my name is Clement hello Clement it's nice to meet you how can I assist you today and if you're just looking for a generic voice assistant and nothing else then you can just download one of those but before you do

that consider this if you know how to make your own version of a voice assistant you could use this knowledge to make other more exciting things involving chat GPT in my case instead of just getting a Chrome extension I personalized it so that he could behave like Jarvis but you could do so much more you can program your own voice assistant to open certain applications for you on your computer play music set an alarm for you send emails and basically whatever you can think of today making applications with a chat GPT is the big thing

like imagine if you made a search engine like Google an integrated chat GPT in it well that's exactly what Microsoft is doing by partnering with openai also you might notice websites like quora including chat gpt's input in the answers so many things are possible for us now that this technology exists but to do it you need to know the tools and that's where this video comes in to show you how to use those tools in this case to use them to make it like you're having an audio conversation with Jarvis well done sir now I'll

be a viewer that is all there is for this video please comment like And subscribe and until next time