Realtime API with Tool Chaining. ADA is BACK. o1 assistant FILE AI Agents

a go ahead and open up open ai's realtime API blog announcement I've opened the browser to open ai's realtime API blog announcement for you thanks a if you need any more help just let me know it's time to start rethinking how we interact with our computers now that open AI has released the realtime API you and I can build seamless Next Generation AI experiences as always the greatest technology and opportunities come to you and I first the engineer are you ready for speech to speech AI assistance powered by Advanced reasoning models like 01 this is what we've been waiting for let's play with [Music] it so no one is really using these tools to their full capabilities yet myself included let's show off early versions of how we can truly utilize personal AI assistance hey a hey Dan how can I assist you today yeah can you go ahead and open up chat GPT open up Claude open up Gemini then can you give me the current time then can you generate a random number for me and then after that go ahead and open up Simon W's blog and then let's go ahead and open up Hacker News as well I've opened chat GPT Claude and Gemini for you the current time today is 1210 a. m. on October 5th 2024 I also generated a random number for you 23 Simon W's blog and Hacker News have been opened as well uh okay so this is getting crazy right open AI employees are going to salivate when they watch this video it's a beautiful orchestration of all of their great work so we have the real-time API calling a chain of tools with 100% accuracy which contain individualized AI agents and let's simplify AI agents that's any combination of code and additional prompts with a single purpose built to solve a single problem these AI agents can then run their own structured output calls that feed back into our top level digital assistant I'm not going to lie I am high as a kite envisioning the capabilities and the future we can unlock with this tooling the distance between your thoughts and your actions on your device are fading fast with this combination of Technology let's push this further with some more interesting agentic functionality with prompt chains and 01 reasoning models powering a file manipulation [Music] agent so this is all running locally via a async python application hey Ada can you go ahead and generate a CSV file for us go ahead and call it uh user analytics and just add 10 Mack rows I've created the CSV file user analytics.

CSV with 10 rows of mock data great job this looks awesome so can you go ahead and delete Row 5 6 and 7even and then can you add a new column let's call it uh membership and set it all to premium I've updated the file user analytics. CSV rows 5 six and 7 have been deleted and a new column called membership has been added with all values set to premium is there anything else I can assist you with let's make another change here can you go ahead and add 70 additional rows to the user analytics file go ahead and use a reasoning model for this the file user analytics. CSV has been up updated with 70 additional rows of mock data using the reasoning model okay so pretty incredible stuff there right just by talking to my computer I am generating and modifying this user analytics.