The Race For AI Robots Just Got Real (OpenAI, NVIDIA and more)

fascinating crazy maybe a little scary I mean it sounds like we're talking about not just using something to help humans do their jobs but just replacing it entirely could this be a new normal on a on a large scale hi welcome to another episode of Cold Fusion this scene is from the popular animated American sitcom the Jetson this episode titled Rosie the robot first aired on September 23rd 1962 the main character in the scene is presented with a few variations of a robot one that's responsive to commands will do housework and basically work as a

personal assistant 61 years on and we're one step closer to seeing that reality this week a tech startup called Figure launched a humanoid robot called figure1 you can have conversations with it thanks to their integration with open AI now at this point you might be thinking we've already seen robots like this over the years whether it's from Boston Dynamics or Tesla how is this any different well first let's take a look at the demo of the robot before we get into the weeds keep in mind that this demo as claimed by the company is unaltered

and in real time but for the purposes of brevity I've shortened some parts all right so let's watch great can I have something to eat sure thing great can you explain why you did what you just did while you pick up this trash on it so I gave you the Apple because it's the only uh edible item I could provide you with from the table so based on the scene right now where do you think the dishes in front of you go next the dishes on the table like that plate and cup are likely to

go into the drying rack next great can you put them there of course after seeing how well it did for those tasks and the human style conversation some of you will be impressed some excited and some even a little bit scared according to the company this bot isn't being operated by anyone it's able to carry out full conversations with people and respond to external stimuli and that right there is one of the primary differences between figure 01 and the other Android robots we've seen so far but let's back up for a second how did a

startup only founded in 2022 make such Headway who is backing figure and what are their intentions given what Nvidia just announced in the past week are we moving towards an aid driven robotics race and most importantly how do these advances in robotics and AI impact all of us we'll take a look at all of that and have a Roundup of all the latest robots in this episode so sit back relax into your docking station as we figure out the future of robotics you are watching cold fusion TV it all started in the wake of Co

at this time robotic sales began surging hitting a record higher in 2022 as the workforce retreated these robots were the kind that were used in warehouses and assembly lines the exact sort of predictable environment that was the lowest hanging fruit to train and deploy robots and amongst this backdrop comes figure AI founded in 2022 figure AI Inc is a California based Tech startup with a focus on general purpose humanoid robots within just 1 year of its Inception the company raised $70 million but now they've recently raised a whopping 675 million and now have a valuation

of 2.6 billion officially a unicorn and you know it's serious business when the likes of Microsoft Nvidia Jeff Bezos Intel capital and many others have become investors figure's Mission according to their website is to quote expand human capabilities through advanced AI they aim to do so by specifically targeting the labor force according to figure the labor force is shrinking they mentioned that quote there are 10 million unfilled jobs in the United States 7 million of those job openings are for essential roles such as Warehouse transportation and Retail and there are only 6 million people available

to fill these open positions while key Warehouse suppliers predict they will run out of people to hire by 2024 quote as we reach the upper limits of our production capability humanoids will join the workforce with the ability to think learn and interact with their environment safely alongside [Music] us figur founder Brett Adcock is no stranger to the tech industry he's previously founded two other companies vety and Archer Aviation veter is an online Talent Marketplace and in 2018 they were acquired for 100 million Archer Aviation is now publicly traded and they're working on building electric vertical

takeoff and Landing aircraft so with his experience Adcock has built a team of Industry veterans including professionals who have worked at Boston Dynamics Google Deep Mind and Tesla perhaps their biggest breakthrough has come in the form of their partnership with open AI in a press release in February 2024 figer announced their collaboration and it will help quote accelerate figure's commercial timeline by enhancing the capabilities of its robots to process and reason from natural language using open AI research and large multimodal models in simple terms it allows for one to have full conversations with the robot

and I suspect will also make it better at computer vision so ultimately the idea is simple build autonomous humanoid robots to address the global labor shortage and to do just that figures first r robot is [Music] here let's start by looking at how the robot actually works first of all every single action and behavior in the video was learned and not teleoperated the robot is equipped with cameras and microphones so when someone speaks and asks it to do something the transcribed texts and images from the cameras are captured to a large multimodal model which is

trained by open AI to understand both text and images the model processes the full conversation including previously shared images it then decides which specifically learned Clos Loop action to take by selecting and loading the relevant neural network configurations onto the GPU and then executing it the mechanical action of figure 01 is rapid at 200 HZ which basically means that after it knows what to do it sends back instructions at 200 times per second that's 5 milliseconds of reaction time for all those micro adjustments humans are in the order of 250 milliseconds and fun fact a

cat's reaction time is around 30 milliseconds so essentially they are using a combination of different neural networks to sort through audio and visual inputs understand them and then do the appropriate actions it's actually kind of amazing all right so how do you think you did I I think I did pretty well the apple found its new owner the trash is gone and the table wear is right where it belongs did you notice the speech hesitation there it's the little humanlike imperfections that allows us as viewers to break the illusion that it's an in anate robot

we've already seen this in chat GPT voice hey chat GPT can you describe a nice kitchen scene but make it very casual and breathe and say um and and things that sound very natural while you're describing sure thing picture this you're in this cozy kitchen right the sun's peeking through the curtains giving that warm glow you can almost smell the aroma of fresh coffee brewing um there's this gentle hum of the fridge in the background kind of soothing you know and uh maybe there's a bit of chatter as someone's flipping Pancakes on the stove it's

just this laidback Vibe like time's taking a chill pill to see it play out in a real robot was only the next Frontier as aside in an episode I made back in late 2021 I talked about how it was only a matter of time before a company cracked the code of mixing dexterity mobility and natural language into a single robot and through 3 years later I think we're getting close but all that being said as you can see there's still some latency between talking and getting a response and I think that's from open AI side

mainly although we don't know the intricate details of the integration voiced instructions probably still have to go up to a server and come back I've noticed the same delay issue when playing around with chat GPT voice so there's still some work to be done if these robots are going to be used in large scale commercial applications imagine an emergency response robot just standing there for 10 seconds while someone's having a heart attack others are simply not impressed months ago figure showed a demo of the robot making coffee some people back then asked if the robots

are also learning bad habits from the training data the figure robot is also not the best at walking but we can't be too harsh it's still an early stage product as I always say criticizing is easy but building is hard with also such an incredible amount of investment and very few demo videos to show some have even drawn the comparison to a certain Trevor Milton if you're a regular cold fusion viewer you'll know what that's all about but obviously this is way too soon for that [Music] conversation broadly speaking it's an exciting time for the

robotics industry so the playing field for figure AI isn't going to be Barren by any means let's take a quick look at the latest in the industry first up this digit which is made by agility Robotics and backed by Amazon before now robots could only replace part of a human job enough to make coffee or an omelet but this is digit a robot it's maker say is ready to do what human warehouse workers do and they can lift about 35 lbs you know the the robot weighs about what a person would weigh it can reach

about the size of what a person can reach you know we're really going for that kind of like OSHA regulations and requirements for the robot agility robotics is funded in part by Amazon which is investing a billion dollars in new Industrial Technologies the fact that it can enter places that were designed for humans um and solve those challenges that exist without having to rebuild uh the environment for Automation and it doesn't quit and it doesn't quit there are about $250,000 so they're not inexpensive but you know over just a few years they're worth it the

digit robots will be deployed in Amazon warehouses and the company aims to produce 10,000 units per year so Jake fascinating crazy maybe a little scary I mean it sounds like we're talking about not just using something to help humans do their jobs but just replacing it entirely could this be a new normal on a on a large scale I mean this is the thing right as a tech correspondent for years I've had companies tell me you know well this technology is for augmenting a human or making a human's job a little bit easier in this

case it is truly right out and out there to replace a human worker Amazon warehouse workers you know that's one of the largest employers in America people really go for that job and so you know we're really I think at this sort of historic turning point where suddenly it's no longer the case of of of you know when could technology replace a set of Javas but this is truly something that's built to do that next up abtronic is designed to move Goods around a warehouse and can work for 22 hours per day all you need

to do is swap out the battery every few hours deliveries start in 20125 the company's also working with NASA on a robot called Valkyrie and that aims to work in dangerous environment ments and perhaps even future space walks Toyota and hunde are also currently working on robots of their own but I'd say less seriously than the others compared to other robots as mentioned earlier figure o one's key difference is the ability to directly Converse in natural language in order to get tasks done for you it's not a new concept as about a year ago on

Cold Fusion we covered a project from Google that did the same thing at that time we saw that the concept was still in its infancy and the robot was quite slow earlier this year here we saw Elon Musk sharing a video of Tesla's Optimus robot folding some clothing at first glance it was impressive but the curtains came down later when Elon added that Optimus couldn't do it autonomously but since then there's been footage of the Tesla bot carrying boxes watering plants walking crouching handling delicate objects such as eggs and Performing other domestic activities interestingly these

abilities come from the knowledge and benefits from Tesla's self-driving efforts which is pretty cool but again as far as we know it lacks that natural language aspect perhaps the atlas robot by Boston Dynamics is the most agile robot we've ever seen we've seen it organized objects and even perform back flips but is this and the figure1 robot really the same thing Boston dynamics's Atlas is definitely more focused on Mobility whereas figure seems to be suited for intricate tasks there's more of a focus on finger dexterity alluded to by the five fingers with three joints on

them and for factory deployment this this could make more sense in a lot of cases assembly line work is done by sitting down or being stationary at a specific location for long periods of time and as we were working on this video Nvidia announced project Groot a general purpose foundational model for humanoid robots it's meant to help humanoid robots quote understand natural language and emulate movements by observing human actions quickly learning coordination dexterity and other skills in order to navigate adapt and interact with the real world [Applause] about the same size the intersection of computer

Graphics physics artificial intelligence it all came to bear at this moment well I think we have some special guests do we little Jetson robotics computers inside they learn to walk in Isaac Sim let's go five things where you going I sit right here the system removes some of the leg work from training robots in the real world via simulation the lessons and skills learned can be put into a knowledge base and transferred to other Robots part and video wants Groot machines to be able to understand human language sense and navigate the world around themselves uh

and Carry Out arbitrary tasks the training part of the stack starts with video so the model can sort of intuitively understand physics then accurately modeling the robots in Virtual space in order to teach the model how to move around and interact with terrain objects or other robots Jensen described this as a gym here the model can learn Basics far quicker than in the real world as we've covered in another episode on the history and size of Nvidia the already turning heads with their stocks growing at unprecedented levels this year I'll leave a link to that

episode in the description as well the company has also mentioned that they're working with Boston Dynamics agility Robotics abtronic and of course figure Ai and just thinking about this I can already see an industry standard forming open AI provides the foundational models Nvidia provides some of the processing hardware and software packages and finally there's going to be a plethora of different robot bodies by different vendors so why is this all happening now well the reason that a lot of this stuff didn't work in previous decades was because the instructions for the robots had to be

laboriously hand coded now with generative Ai and the latest generations of models robots are beginning to be able to dynamically understand their environment and adjust seamlessly with no hard coding the amount of change we're going to see over the next 5 years 10 years will dwarf everything that's happened over the last 30 we went from automating pen and paper right with spreadsheets to connecting PCS into loc Network to connecting networks to get the internet to getting the network effect of communicating people globally but now we're introducing machine intell you know artificial intelligence machine learning deep

learning neural networks Etc so and what that enables is the automation of automation right and so the people who were writing software particularly at the lower end unless you are doing these Advanced things they're gone right the people that s is writing itself in doing itself right it's just math programmer back in the day so when I was writing code it was the algorithms were if this than that right ex or whatever it may be right but you had to guess right you had to use your your best instance then it got smarter and smarter

and smarter and libraries to do bigger and better things all that is being automated the progress of automation has been a funny story to watch unfold if you've been following the tech World closely for long enough you'd remember all of those discussions over a decade ago about how manual labor jobs would eventually be taken by robots but as was' seen since the AI boom a couple of years ago it was the cognitive jobs that were the first to be impacted in 2017 only one in five companies used AI according to McKenzie now half of all

companies do we've already seen many companies laying off thousands of employees in part due to technological advancements like AI is completely the opposite of what most people thought would happen Goldman Sachs last year reported reported that over 300 million jobs could be lost or diminished by AI most at risk are legal architecture and engineering business and financial operations management sales Health Care Art and Design Goldman Sachs also thinks that the price of robots and their constitute components will drop drastically and they've already dropped drastically since last year unlike humans robots don't need to take a

break they don't need to sleep go on holidays strike and don't complain so as soon as these robots are feasible and cheap enough companies will be chomping at the bit to install them so what do we do about all of this well it's your lucky day because I've already done an episode on life after automation it's a huge topic and will just be way too long to get into in this episode so I'll leave another link to that below okay so back to figure one at this point you might think all right the figure demo

looks cool but surely it's going to be a long time until these human robots are in production well it seems like figure didn't want to waste any time even before the demo earlier in the year figer signed an agreement with BMW manufacturing they're going to test the figure 01 robot at the car makers plant in Spartenburg South Carolina but we're yet to get reports on how the 01 is performing and what tasks it's going to do exactly but I still think that's pretty [Music] interesting humanoid robots like the figure one are a future conundrum will

they replace humans completely or just alleviate us from menual tasks leaving the more important stuff for humans but on the other hand of course the military would probably want to have a sniff at humanoid robots in the future too looking at consumer products though we're pretty far away from getting a highly functional robot that's cheap that you can buy at your local store but the day will probably arrive figure 01 is the worst robot from this company so there's only going to be Improvement aside from that the competition is heating up so if it's not

them it's going to be another compy that's going to crack the code from the broadest perspective possible I'm sure opinions from all of you will be split for the engineering types this is an excellent display of human Ingenuity for the Skeptics they'll say it's all pointless and we should focus our efforts elsewhere the optimists will see this as an exciting New Era for Humanity free of drudgery regardless whether that future turns into the Jetson the Terminator or Utopia will just have to let Humanity collectively figure that one out all I know is that I'll be

watching this space closely and that's where we are with figure one and Robotics today I hope you enjoyed that or found it interesting and if you did feel free to share it with someone who would find it interesting as well all right so that's about it for me my name is toogo and you've been watching cold fusion and if you're new feel free to subscribe if you like it's free cheers guys have a good [Music] one [Music] cold fusion it's new thinking