Meta Connect 2024: Everything Revealed in 12 Minutes

this is the new meta Quest 3s high quality mixed reality for $2.99 hell yeah I've been waiting for this one for a long time um you know when when we built Quest 3 we delivered the first highquality mainstream mixed reality exper experience where you can see the world around you you can reach out and touch digital things as if they're right there um Quest 3 is the best mixed reality device that you can buy today and and I am really proud of it Quest 3s has the same defining features as Quest 3 high resolution color

mixed reality powered by the same processor Vivid pass through handtracking feels Natural Touch plus controllers for precision you get the full range of experiences from Horizon OS gaming Social Fitness watching videos productivity and more so whether you're looking to get started with the base Quest 3s or if you want the top-of-the-line Quest 3 with 512 GB and 4K displays which is now just $4.99 the quest 3 family it's not just the best value it is the best product available everything you're going to see up here is streamed from his headset live this is what he

is seeing yeah it's sharp all right Kenny what do we got got all right now the first thing that you can see is that we have expanded the quality the the volume of content massively yeah sorry to the dude next to him live demo artifact all right um You can now run everything from 2D mobile apps to remote desktop for PC uh to fully immersive experiences all on one device you can open screen you can put them anywhere around you you can have pretty much as many of them running as you want and we have

also rebuilt all of our social apps for mixed reality 2 we've got all new Instagram and Facebook experiences to bring your feeds and your reels uh with you into mixed reality a really nice interface uh so you can lean back enjoy your content on nice big screens um so and there's there's a bunch more too are we've been working with Microsoft uh to upgrade remote desktop soon so now you're soon you're going to be able to easily connect to any Windows 11 PC you just look at the keyboard and um and you and it'll start

pairing so if you want to work on a giant virtual display or a bunch of virtual displays or you know if you want to work on your projects without distractions or with other people you know this is Quest is going to be able to be a natural extension of your PC we're also bringing photorealistic spaces to the metaverse and we call this hyperscape and the way this works is you can use your phone to scan a room and then recreate it or you can step into a room that someone else has scanned and shared now

of course there's gaming um so we got some really good ones coming showing off what the quest 3 family can do Batman Arkham Shadow up here um exclusively on the quest 3 family it's to be included in any purchase of a quest 3 or 3s uh that you buy this fall lots of great experiences a lot more coming soon today we are making meta AI even smarter uh with our new open-source llama 3.2 models and I'm going to get into more details about llama 3.2 in just a minute but one of the headlines is that

they're multimodal um so now meta AI can natively understand images as well as text now we're using this new capability to build some features that that I I haven't seen anyone else build I mean this is this is pretty interesting and novel stuff creative tools that we're adding to imagine edit it lets you upload any photo and edit it precisely um with natural language in meta AI across the apps I think that voice is going to be a way more natural way of interacting with AI than text and probably I think it has the potential

to be one of if not the most frequent ways that we all interact with AI um it is just a lot better so now you're going to be able to have natural voice conversations with medai across all the major apps you got you know Instagram WhatsApp messenger Facebook Hey are live demos are risky live demos can be risky yes they can be unpredictable prone to technical issues and potentially embarrassing however they can also be engaging memorable I'm going to do it anyway go for it just be prepared for anything to happen so you were one

of the first creators to build one of these Creator AIS with the text only platform um you know can can you tell us what what it does and and your experience with it yeah so like for me I've been treating my Creator AI kind of like like my digital assistant so you know I've trained it on how I respond and how to engage with my audience and then yeah can like send DMS can share links to relevant resources and then uh what else am I having to do oh yeah it can just it guides people

through conversation it's great congrats on the new book that you just released you know what's the main thing that you're hoping that people take away from it thank you so much yeah the main thing I want people to take away from my book is the idea that you have the power to create your own opportunities by combining curiosity adaptability and resilience in a rapidly evolving digital world we are experimenting with automatic video dubbing on reals uh starting with English and Spanish um so this is going to be a way that you can see more of

the content that's out there no matter what language you speak and as a Creator this is going to let you reach a lot more people around the world no matter what language you speak so now you're going to be able to take your content and it's going to be in your authentic voice um speaking other languages if you want and we're also doing automatic lip syncing um so that we actually looks like you are speaking the other language so let's check this one out my native language is Spanish but check it out now I can

also speak English through AI automatic dubbing so glasses they're kind of the perfect form factor for AI right for letting an AI assistant see what you see hear what you hear be able to communicate you with you privately um so it can help you out with whatever you're doing throughout the day I am about to have a conversation with Brandon Mareno former UFC flyweight champion of the world where he is speaking Spanish and I am speaking English all right Brandon you want to come out here and show us this starting live translation from Spanish to

[Applause] English hey hey how are you how is your knee back training doing well KN is is doing well it feels strong I'm ramping up training um how are you doing are you getting ready for your next fight so today we are dropping a limited edition of clear transparent frames um you know I think this is neat because it's it's it's a neat Vibe right you can see the technology in them you know for the first few versions We spent so much energy designing the glasses to basically hide the technology um to make them look

like normal stylish glasses but now I think people are starting to appreciate what makes them special and I think having a design that celebrates that um is pretty awesome this is a our first fully functioning prototype and if I do say so the most advanced glasses the world has ever seen you know not a headset no WI less than 100 gr uh they need wide field of view holographic displays sharp enough to pick up details bright enough to see in different lighting conditions large enough to display a cinema screen or multiple monitors for working wherever

you go whether you're you in a coffee shop or on a plane or wherever you are and you need to be able to see through them and people need to be able to see them through them too and make eye contact with you right this isn't pass through this is the physical world with holograms overlaid on it it is a completely new kind of display architecture with these tiny projectors and the arms of the glasses that shoot light into wave guides that have you know nanoscale 3D structures etched into into the lenses so they can

defract light and put Holograms at different uh depths and sizes into the world in front of you and all of that is directed by custom silicon and sensors that we designed and powered by a battery that fits in the arm of the glasses it is a absolutely incredible amount of technology to be able to miniaturize and fit into a pair of glass and a small Puck that goes with it to help power the whole thing they're going to do voice and AI they're going to do hand tracking and eye tracking so that way you can

select UI Elements by looking at them voice is great but the thing is sometimes you're in public and you don't want to say what you're trying to do with your computer out loud hand tracking is neat for controlling different interfaces but you don't want to like walk down the street like this right so I think that you need a device that allows you to you just send a signal from your brain to the device so this isn't just the First full screen like you know full Widefield view holographic AR glasses this is also the first

device that is powered by our wrist-based neural interface we do still have a few things that I want us to keep pushing on before we ship this as a consumer product um we're going to keep tuning the display system to make it sharper I want to keep working on the design to make it smaller and a bit more fashionable um and we are going to keep making we need to keep working on the the um manufacturing to make it a lot more affordable too but we have line of sight to all of those things so

we are going to use Orion as a dev kit um we are going to use it mostly internally to build out the software that we that we need to but we're also going to work with a handful of Partners external to make sure that we get a diversity of content and and um you know just that we can really dial in the software and the experience so that way when we have the next version of this Hardware it is going to be ready to be our first consumer full holographic AR glasses