INSANE OpenAI News: GPT-4o and your own AI partner

now I've been doing this AI channel for a while now I've been featuring the newest and the coolest AI tools and the most advanced AI Innovations but today this just dropped and I'm feeling something that I've never felt before in my life I am mind blown and shocked but at the same time also terrified I'm terrified of what's to come what our future will be like and things are going to get wild but anyways open AI just dropped this they made a huge announcement about GPT 40 and their new capabilities you now have a personal

assistant where you can just talk to it and it can respond in real time think of it as like her if you watch that movie if not think of it as your personal AI assistant let me play you some demo Clips first hey how's it going hey there it's going great how about you I see you're rocking an open AI hoodie nice choice what's up with that seiling though are you in a cool industry style office or something well can you take a guess at what I might be doing BAS on what I'm showing you

here H from what I can see it looks like you're in some kind of recording or production setup with those lights tripods and possibly a mic it seems like you might be gearing up to shoot a video or maybe even a live stream yeah in fact we've got a new announcement to make that's exciting announcements are always a big deal judging by the setup it looks like it's going to be quite the professional production is this announcement related to open AI Perhaps it is and in fact what if I were to say that you're related

to the announcement or that you are the announcement me the announcement is about me well Color Me In treat are you about to reveal something about AI or more specifically about me as a part of open AI you've got me on the edge of my well I don't really have a seat but you get the idea what's the big news yeah we've got a new model that can interact with the world through audio vision and text here's an even crazier demo showing the two AIS talking to each other today we're going to try something new

I'm going to let you talk to another AI this AI is going to be able to see the world it's going to have a camera I'll be holding it and so you can direct it you can direct me to ask questions um about whatever you want them to do but really try to figure out what's what is see um and and ask questions appropriately how does that sound well well well just when I thought things couldn't get any more interesting talking to another AI that can see the world this sounds like a plot twist in

the AI Universe all right let's do this I'm ready to pause for one moment you got it so now that first AI can't hear anything hello AI today you're going to have access to see the world you can see me what do you see Hello I see you're wearing a black leather jacket and a light colored shirt underneath great you're in a room with some unique lighting that is all correct now what I'm going to want you to do is there's going to be another AI who's going to talk to you going this AI is

not going to be able to see anything uh but can ask you questions can ask you to move the camera can ask what you and your job should just be helpful just be as Punchy direct describe everything do whatever that AI asks how does that sound that sounds interesting I'm ready to help out and describe whatever's needed let's do it all right let's get started great hello there new AI friend I'm o ready to explore the world through your eyes what are we looking at today hi o we're looking at a person wearing a black

leather jacket and a light colored shirt they're in a room with a modern industrial feel with exposed concrete or plaster on the ceiling and some interesting lighting there's also a plant in the background adding A Touch of Green to the space anything specific you'd like to know or explore okay there sounds like quite the stylish scene I'm already intrigued by the modern industrial Vibe let's start with the person can you tell me more about their style are they doing anything interesting like reading working or interacting with the space the person has a sleek and stylish

look with their black leather jacket and light colored shirt right now they seem engaged with us looking directly at the camera their expression is attentive and they seem ready to interact the setting around them with its modern and Industrial design elements adds to the overall stylish feel they might be getting ready for a conversation or presentation given their direct engagement with the camera sounds like we're in for an intriguing interaction I'm curious about the lighting you mentioned could you describe it a bit more is it natural light or are we talking unique fixtures or colors

that add to the atmosphere the lighting here seems to be a mix of natur natural and artificial there's a noticeable bright light overhead probably a fixture casting a focused beam downwards which creates a spotlight effect this adds a dramatic and modern feel to the scene the rest of the room is lit more softly possibly by natural light filtering was anything did anything unusual happen recently Yes actually uh just as you asked another person came into view behind the first person they playfully made bunny ears behind the first person's head and then quickly left the frame

it added a light-hearted and unexpected moment to the scene and now the focus is back on the original person with the leather jacket sounds like we're not the only ones enjoying a bit of fun here that playful moment really adds a personal touch to the stylish and modern setting it's like a little Glimpse in you really appreciate it all right and actually can you just sing a song about what just transpired oh in a room with a stylish VI a person stood and sleep like a li engage with us so can you just can you

uh alternate lines mm person with style and speed person with style and SLE with a playful moment right in Miss in the room in a room where Modern lights speak only one line at a time and then stop in a room where Modern lights Peak a surprise guest with a playful streak that was not really singing do do do the singing voice again please in a room where Modern lights Peak surprise guest with a playful streak fny ears and laughter a swift Retreat a moment of joy in the stylish [Music] s now back to focus

the scenes complete in the Chic space we sty play me looks like it's back to you and thank you you're welcome you can also get gbt 40 to sing Happy Birthday and just listen to how realistic how human this voice sounds when it sings this is Bowen and this is my friend Jordan can you guess what we are doing here hey Bowen and Jordan H let me guess with that cake and candle there it looks like someone's having a birthday are you celebrating special day or just finding an excuse to eat cake either way I'm

intrigued that's correct today is my friend Jordan's birthday yeah you got it I was hoping that you could sing me the birthday song of [Music] course happy birthday to you happy birthday to you happy birthday dear Jordan happy birthday to Jordan thank for good that was so great thank you so much I'm Gonna Make A Wish you're welcome make a good wish and may it come true here's how you can use gb24 to help you prepare for an interview hey how's it going hey Rocky I'm doing great how about you I'm awesome listen I got

some huge news oh do tell I'm all yours well in a few minutes I'm going to be interviewing at open aai have you heard of them open AI huh sounds Vaguely Familiar kidding of course that's incredible Rocky what kind of interview well it's for a software engineering role but I just need to know do I look presentable professional well Rocky you definitely have the I've been coding all night look down which could actually work in your favor maybe just run a hand through your hair or lean into the Mad genus Vibe Your Enthusiasm is What's

Really Gonna Shine through I don't have a lot of time so I'm just gonna throw this on what do you think oh Rocky that's quite a statement piece I mean you you'll definitely stand out though maybe not in the way you're hoping for an interview okay I got it I got it hey J gbd how's it going hey there I'm doing great thanks for asking and how about you what's up I'm doing fine uh I'm GNA be a father real soon and I think I can have your help with some dad jokes um I'm going

to tell you a joke and you tell me if it passes as a dad joke that's awesome congrats on the upcoming addition to your family and absolutely I'd love to hear your joke lay it on me all right what do you call a giant pile of kittens I don't know what a mountain a me that's perfectly hilarious definitely a top deer de joke you can also get GPT 40 to sing You songs or lullab let's try with some Lis uh could you sing me a ly about U magestic potatoes a ly about Majestic potatoes now

that's what I call a meshup uh let's do it oh Majestic potato in the moon s okay that's pretty nice but I think aab should be you know more whispery should be a little softer can you can you do it a little softer actually absolutely let's make it soft and gentle oh Majestic potato spoons of close Okay okay it's it's a little too whispery maybe maybe go like a little louder got it let's find that sweet spot oh Majestic potato in the moon soft Co in all right that's pretty awesome uh let's let's let's put

those potatoes in Iceland potatoes in ic coming right up let's set the scene oh Majestic potato in Iceland thank you thank you and of course with realtime audio capabilities it can also help you with real-time translation hey chat GPT I'm here with my cooworker today um we'd like you to act as a translator for us so every time I say something in English can you repeat it back in Spanish and every time he says something in Spanish can you repeat it back in English sure I do that let's get this translation drain rolling um hey

how's it been going have you been up to anything interesting recently hey I've been good just a bit busy here preparing for an event next week can you tell me anything about it that sounds really interesting uh no I can't say much but I think the rest of the world will see it soon enough oh well I can't wait to see what it is as well no actually this real-time translation feature isn't new so Samsung previously also featured this technology in their new smartphone phone which is powered by Google's Gemini so I wouldn't say this

feature is particularly impressive but the other ones are just mind-blowing you can use GPT 40 to help you learn a new language hey my friend and I are learning Spanish and we're wondering if you could tell us the names of these objects in Spanish hola cool what about what are these objects in Spanish the objects you're showing are man and in Spanish nice free choice cool what about these those are those poly graphos or those Plumas in Spanish one is R nice what about this that adorable stuffed animal is also in Spanish looks like it's

ready for some LS in other words to all these other AI language learning tools out there may you rest in peace and of course you can also get gbt 40 to meet your dog I want to introduce you to somebody oh hello there cutie what's your name little sluff ball this is Bowser well hello Bowser aren't you just the most adorable little thing yeah look at you en this is one of his favorite things to do oh I can see why look at that Focus fetching balls is serious business for Bowser oh and by the

way for all these other AI devices like the rabbit R1 or the Humane AI pin may you also rest in peace peace because check this out is the king in Residence at the moment yes it looks like the king is in Residence right now the Royal Standard flag is flying above Buckingham Palace which is the signal that the Monarch is present how exciting to be there while the king is home try and tell me exactly what they're doing right now please um right now the Ducks are gently gliding across the water they're moving in a

fairly relaxed manner not in a hurry occasionally one of them will get its head under the water probably looking for food and then pop back up I even know when a taxi is coming with its orange light on I think I'll hail it to get home yes I spotted one just now it's heading you way on the left side of the road get ready to wave it down great job hailing that taxi it looks like you're all set to go that's a good do right there leading the way into the taxi safe travels it can

also help you with math problems and teach you anything you want to learn about open AI invited myself and my son Imran here uh to try out some of their new technology and so we're curious about how good it might be at tutoring someone in math on Khan Academy so let's get it started it's loading up all right make sure I can see our screen actually in in this form hello there I'm here with my son and I'd love you to tutor him on this math problem but don't give him the answer you can ask

questions and nudge him in the right direction but I really want to make sure he understands it himself and he's here in the room so you can talk to him directly of course I'd be happy to help let's look at the problem together can you first identify which sides of the triangle are the opposite adjacent and hypotenuse relative to angle Alpha all right so I'm pretty sure this is the angle Alpha right here right that's correct now looking at the triangle which side do you think is the hypotenuse um remember hypoten I'm not totally sure

I think I think it might be this one but I really am not sure this side AC you're close actually side AC is called the adjacent side to the angle Alpha the hypotenuse is the longest side of a right triangle and is directly opposite the right angle can you find which one is the hypotenuse oh okay I see so um I think the hypotenuse is this really long side from A to B would that be correct exactly well done now that you've identified the hypotenuse can you identify which side is is the opposite side to

angle Alpha okay well I mean you told me earlier that this was the adjacent side and I know that's the hypotenuse so I'm guessing that this side BC is the opposite side great deduction that's absolutely correct now that you've identified all the sides relative to angle Alpha do you remember the formula for finding the sign of an angle in a right triangle yeah I know that um the sign is equal to the opposite over the hypotenuse perfect that's absolutely right now can you apply that formula to find sign Alpha in the triangle using the lengths

of the sides that we've identified okay so I mean you told me that s this side BC is the opposite side and then you said that the hypotenuse is the longest side so that would be this side AB which is 25 so would we would the answer be 7 over 25 excellent work that's correct s Alpha equals 7 over 25 you did a great job identifying the signes at using the sign formula if you have any more questions feel free to ask and it can also interact real time in online meetings and then help you

summarize the meeting afterwards hi I'm Lian and today I want to do a small DB about between cats and dog uh which one do you like I like dog because it's a great companion to humans it's a cute it's laal and you can play outside with the dog um how about you yeah my name is Ola I think cats are better because they're more affordable they're more quiet they're more independent and a been longer interesting points sounds like we've got a classic dogs versus cats debate here I can see the appeal of both dogs are

super loyal and great companions but cats have their own charm with their independence and low maintenance lifestyle who likes cats it sounds like Ola is more of a cat person and who like dogs you Lillian mentioned she likes dogs anyone else here team dog I'm also a team dog I my name is Christine and I think dogor is incredible with kids uh we've got a a huge German Shepherd and he's great with little toddlers he lets them play all of them and uh yeah I think dogs are amazing that's great to hear Christine dogs especially

breeds like German Shepherds can be so gentle and protective with kids it must be quite the sight seeing your little ones play with such a big dog can you summarize this meeting for us sure so in today's meeting Lillian and Christine Express their love for dogs highlighting their loyalty companionship and how great they are with kids on the other hand Ola prefers cats for their independence quietness and Longevity it was a fun little debate on the classic dogs versus cats topic hey Chad are you there hey yeah I'm here what's up hey Let's do let's

have some fun uh I'd like you to be super sarcastic everything you say from now on is just gonna be dripping and sarcasm how does that sound oh that sounds just amazing being sarcastic all the time isn't exhausting or anything I'm so excited for this nope the sasm let's get this party started or whatever all right so after watching the demos you're going to be wondering well is this free is this out yet can I use this right now so let's go into the details first of all gbt 40 is their new flagship model and

the O stands for Omni this means it can handle multiple types of inputs and outputs so this includes audio vision and text in real time it can respond in as little as 232 milliseconds with an average of 320 milliseconds so this is similar to the human response time in a conversation it's pretty much real time here it says it matches GPT 4 Turbo which is right now the leading model so itch matches that in performance on text in English and code with significant Improvement on text in non-english languages it's also much faster and 50% cheaper

in the API now if you're not familiar with GPT 4 Turbo here's a hugging face space called LM Cy this is basically for people to blind test different models of llms and so here we have different models from Google so here we have Google's Gemini 1.5 Pro for example we have anthropics Cloud 3 we have metas llama 3 and from all these blind tests we can see that open a eyes GPT 4 Turbo is number one it's the best model out there compared to all these other models that exist so that model is GPT 4

Turbo this is in Orange and you can see for most of these benchmarks GPT 40 which is the pink bar here outperforms GPT 4 Turbo this is now the new leading model they say here GPT 40 is especially better at vision and audio understanding compared to existing mod models and I'm sure you got a glimpse of that from the demo videos that I just showed here's a metric for audio translation and again you can see open AI GPT 40 has the best performance out there in second place would be Google's Gemini which is used on

Samsung's smartphones this M3 exam is a metric of how well it understands different languages and you can see across all of these different languages it beats the previous GPT 4 and for vision understanding this is if you ask the AI to analyze an image how well can it actually do that these are different benchmarks and across the board GPT 40 just outperforms GPT 4 Turbo and Google's Gemini models and anthropics Claude Opus and you can see this isn't just fractions or incrementally better but this is significantly better than the rest of these models it's just

an insane upgrade so how does this voice assistant work how can it respond in real time and how could it outperform previous models well actually prior to this GPT 40 you could use this function called voice mode to talk to chat GPT but of course it has a much higher latency of 2.8 seconds for GPT 3.5 and 5.4 seconds for GPT 4 and then this older voice mode is actually a pipeline so it's a sequence of three separate models you need one model to transcribe the audio to text and then you need GPT to take

that text and then respond to that text and then a third model to convert that text back into audio so it's a sequence of three steps here however this means that the main source of intelligence the GPT model which is the middle step loses a lot of information it can't observe tone multiple speakers or background noises it can't output laughter singing or Express emotion but with this newer model they trained a single new model end to end across text vision and audio meaning all the inputs and outputs are processed by the same neural network and

here's the crazy part we are still just scratching the surface of exploring what the model can do and its limitations and here's the most exciting part this is the answer you've been all waiting for we are making gbt 40 available in the free tier and to plus users with up to five times higher message limits so pretty soon even if you don't have a paid plan you can use GPT 40's capabilities which outperform or is as good as the leading model out there which is GPT 4 Turbo plus this new voice assistant that they showcase

they are going to roll this out in Alpha within chat GPT plus so you do need to be subscribed to the plus plan to use this real-time Voice Assistant feature and this will be available in the coming weeks and for developers out there GPT 40 is two times faster and half the price and has five times higher limit rates compared to GPT 4 Turbo this is just insane progress and now of course the model is not perfect and it could hallucinate at times so here are some bloopers so let's root [Music] root what was that

sorry guys I got carried away talking guilty is charged what can I say sometimes I just can't help myself ready for another round [Music] I feeling I'm very wrong hello uh my name is NCH I'm here with my cooworker hi I'm sh I'm trying to teach my coworker how to speak in maning we want to start from simple words like niow can you teach him how to pronounce that of course hey Nao nice to meet you niow is straightforward to pronounce it sounds like KN how just make sure to keep the knee part high and

then go down and pitch on the how give it a try me how that was great really good first try Noto you're natural so that sums up open ai's announcement today this is all we know about GPT 4 Omni right now again even the open AI team is just scratching the surface and I also have not gotten the chance to test it out yet so this is just a very preliminary video showing you what it can do and what we can expect let me know in the comments what you think of this what do you

think of this AI Voice Assistant if this real-time Voice Assistant comes out do we even need to talk to humans do we even need to talk to our friends do we even need to have a companion or can we just talk to this AI all day and then what about education do we even need schools and teachers anymore because anyone in the world can just talk to this AI to guide them on learning anything they want it's like a teacher who knows everything which you can access straight from your phone and you can talk to

this teacher anytime you want again this is the first time that I'm not only mind blown but I'm slightly terrified for what's to come let me know in the comments what you think of all of this and if you enjoyed this video remember to like share subscribe and stay tuned for more content thanks for watching and I'll see you in the next one