Zuckerberg's BOMBSHELL Interview: (AGI AI AGENTS)

so I think we're going to live in a world where there are going to be hundreds of millions of billions of different AI agents eventually probably more AI agents than there are people in the world Mark Zuckerberg did an exciting 30 plus minute interview with Rowan Chun where he takes some direct shots at open Ai and other closed Source companies in the first part of the interview he talks about llama 3. 1 and what he's trying to achieve with it the first time we're releasing a 405 billion parameter model um so it's by far the most sophisticated open source model that that I think anyone has has put out and and it really kind of is competitive with some of the leading closed models and in some areas is even ahead so I'm really excited to see what people do with that especially now that we're making it so that our the community policies around llama allow people to use it as a teacher model to distill and fine-tune and um and basically create whatever other models they want with it so first of all the 45 billion parameter model is outstanding it is Leaps and Bounds more sophisticated than any other open- source model out there also llama is a direct shot at open Ai and other closed Source companies zuck's strategy seems to be tried and tested here basically when you're behind in a technology race the strategy that you employ is called scorched Earth essentially invest a ton of money into replicating whatever that technology is and then release it for free because at that point it becomes ubiquitous it becomes a commodity and why would anybody go and pay premium prices for a closed Source Frontier Model when you can have full control over the model and pay fractions of the price let's keep watching um in addition to that we've distilled the 405 billion parameter model down to make newer and updated and now leading um for for their size 70 billion and 8 billion parameter models they also have really good performance um really good kind of cost per performance ratios so I'm really excited to see what everyone does with this um you know I mean taking a step back I think this is a pretty big moment for open source AI um you know I've been reflecting on this tonight kind of think it's you know I've thought for a while that open source AI was going to become the industry standard and I thought that it would basically follow the path that Linux did where you know if you if you um just go back to before Linux was popular there you know there were all these companies that had their own closed versions of Unix and at the time you know there's nothing that was sort of that sophisticated that had ever been done as an open source project and people thought hey no this is like the closed model of development is the only way to do something that's this Advanced and at first Linux kind of got its foothold because um it was cheaper because developers could customize it in different ways and then over time as the ecosystem built out it you know got more scrutiny so it actually became the more secure one it became the more advanced one um there were more partners that basically built more capabilities in in the case of Linux more drivers um and things like that that basically ended up making it have more capabilities as well than any closed Source unit so I think that this moment with llama 3. 1 is kind of like that inflection point where um I think llama has the the opportunity to become the open- source AI standard for open source to become the standard the industry standard for for AI and even in the places where it's not yet ahead on performance it leads on on kind of cost and on on customizability and on the ability to take the model and fine-tune it and do all the things that you want with it so here he mentioned that now with the 405 billion parameter model they allow you to create synthetic data from that model to train smaller models so that is a huge change and extremely valuable for the ecosystem this is what Nvidia did with their nimron model too they trained a massive model that generates data to train smaller models this is going to allow a lot of AI companies to make their own versions of these models dependent on the use case and that is a really cool strategy and something I really appreciate from meta um so I think that those are just huge advant ages that that we're going to see developers take and we're focusing on building out this partner ecosystem and there are going to be all these different capabilities that get built out around it the thing that I'm most excited about is seeing people use it to distill and and fine-tune their own models right it's I mean like you're saying I mean this is the first open-source Frontier level model but it's not the first Frontier level model so there have been other models that sort of have that capacity and yeah people are going to want to do inference directly on the 405 because it's you know by our our estimates it's going to be about 50% cheaper I think than than GPT 40 um to do that directly and um so I think that that that obviously makes a difference to a lot of people yeah and another thing he's not doing this just to screw over the closed Source companies he actually believes in this as a true business strategy if you build out the foundation for other companies to come build on top of you then of course you get to set the standards and then you'll figure out ways to monetize over time that's essentially what they did with Facebook they built out a platform other develop ERS came and built on top of it now the counter example to that is apple with the App Store they built a completely closed system nobody gets to develop anything on top of it but the thing that I think is really new in the world with this is the because it's open weights um the ability to take the model and distill it down to whatever size that you want to use it for synthetic data generation to use it as a teacher model um you know so our vision for the future it's not just okay it was never that there's going to be One Singular thing I think this is like open AI sort of has this Vision that they're going to build kind of one big AI anthropic does too Google does too it's never been our vision our our vision is that there should be lots of different models I think every startup out there every Enterprise governments they all kind of want to have their own custom models and and yeah when the closed ecosystem was so much better than open source it was just better to take the vanilla clo thing off the shelf because even though you could customize open source there was still some gap between the performance that you could get but now we don't see that anymore right now as open source basically closes the Gap I think you're just going to see this wide proliferation of models where people now have the incentive to basically customize and build and train exactly the right size model for what they're doing um train their data into it they're going to have the tools to do it because of a lot of the partner Integrations that the companies like um like Amazon are doing with AWS or data bricks um or different folks like that who are building these whole Suites of services for distilling and fine-tuning um open models so I think that that's going to be the thing that's new here and that's really exciting is how far can that get pushed and um and that's a completely new capability in the world because there hasn't been an open source or open weight model of of of kind of this um sophistication that's ever been released before yeah and that's a really important point the battle for unique and diverse data is really going to be the front lines of artificial intelligence that is why open AI has been building Partnerships with numerous content companies like Time Magazine to acquire diverse and unique data and I think that's a really interesting approach all right in this next section Rowan asks Zuckerberg about how they're going to teach the world how to use these AI models let's watch what he has to say yeah so I'd say before llama 3.

1 our approach I mean the reason that meta fundamentally is investing in this is we basically want to know that we have access to to a leading model um you know because of some of our our history of of kind of how mobile worked and things like that um we didn't want to be in a position where we had to rely on some competitor for this kind of fundamental technology so we built it for ourselves and before llama 3. 1 you know we we kind of add this Instinct that if we made it open source there would be a community that would grow around it and that would actually extend the capabilities and make it more valuable for everyone including us um because at the end of the day this isn't just a technology it's an ecosystem right that that that you're developing so um in order for this to end up being a useful thing for us there also needs to be a broad ecosystem one of the big changes that that I think we see with llama 3. 1 is instead of just building it for ourselves and throwing it over the wall and letting developers use it this time we're really taking a much more proactive stance on building Partnerships and making sure that um there's this whole ecosystem of companies that can do interesting things with the model and conserve developers in ways that we're not going to okay really what that translates into is they want control of the ecosystem they want to be able to define the standards so there's obviously a financial motive for meta and it's not just purely out of the goodness of their heart that they released this model for free but that's okay everybody can still win at the same time I think that there are also going to be folks um like grock right who were doing really interesting work on really kind of ultra low latency um inference and I'm really excited to get this in their hands and they they're building something for launch that basically is um is going to is is is going to enable that too now here Rowan asked Mark what the implications are of Open Source AI listen to what he says yeah I mean my view is that open source is a really important ingredient to having a positive AI future I that there are all these awesome things that AI is going to bring um in terms of productivity gains and creativity enhancements for people and hopefully it'll help us with research and things like that but I think open source is an important part of how we make sure that this benefits everyone and is accessible to everyone it isn't something that's just locked into a handful of big companies um at the same time I actually think that open source is going to end up being the safer and more secure way to develop AI I know that there's sort of a debate today about is open source safe and I actually take the different position on it not only do I think it's safe I think it's safer than the alternative of Clos development and yeah so I'm going to cut him off for a second because I kind of already know where he's going basically when you open- Source something and everybody with diverse skills diverse perspectives and a much larger pool of talent can look and examine every single line of code and every single piece of data how it's behaving or why it's doing certain things it really helps Harden the system much more than closed Source systems you know I sort of break it down into you know there there are lots of different kinds of harm so it's you can't just talk about one type of thing but um on this I think that there's there's unintentional Harms so the system goes off the rails in some way um that people didn't intend and then there's intentional harms where you have like some Bad actors trying to use the system to do something bad when it comes to unintentional harms which I think by the way it's worth noting that like most of the Sci-Fi scenarios that people worry about of AI just going rogue um are kind of unintentional I I actually think that open source should be safer on that because it's it will have more scrutiny it'll have more transparency um and I I I I think all the developers who use it with all the Lama guard and the safety tools that that it comes with um there's going to be so much scrutiny and testing and pressure on those that my guess is that it will have kind of just like traditional open source software um any kind of issues with it I think will be ironed out and fixed a lot quicker than with when the closed models so I think you got you've got that on on kind of unintentional harm which is why I think most of of the discussion around safety for open source revolves around intentional it's okay it's open it's out there how are you going to stop Bad actors from from doing it doing bad things with it um there I think you basically want to probably divide the problem into kind of smaller actors like an individual or or um or some kind of smaller group that's trying to create some some some Mayhem and larger actors who are more sophisticated have huge amounts of resources like big nation states I think it's kind of a different mix for the two of those um this reminds me of something that Yan Lun talked about in his interview with Lex fridman he essentially said that if everybody has open-source Frontier models that are incredibly capable as capable as closed Source then it's basically a battle of AI versus AI you know for the smaller actors and my view on this is that um you know the way that we've that I think that having a balance of power on this is super important um you know what we've done in managing our social networks is we have all these kind of Bad actors who are trying to do kind of bad stuff on our netw works and the way and a lot of times they deploy AI systems to do that and the way that we stop them and identify them is by having more sophisticated AI systems that have more compute to go find what they're doing so I think that this is actually pretty similar to way that governments and law enforcement essentially maintain order in society it's like yeah you have a bunch of Rogue people who might be committing crimes but you know generally the police forces and the militaries are much better funded have more resources and I think that that's basically going to be true here as a matter of fact I think what you want is for open source to be widely deployed which I think that there's sort of a risk if it's closed that that's not the case but when it's open you're going to have all these big institutions that have a ton of resources that they can basically deploy these systems in a way that I think will check Bad actors then you get to um the question of of basically you know folks like China or like large sophisticated actors and one of the questions that you sometimes he debated is like okay if you're open sourcing the really Advanced models how do you make it so that that it doesn't get to to to China or they're not going to use that against us and um and that's sometimes an argument that people have for hey you should lock down development but I think that that's sort of missing a few things one is that in order for this all to work the US has to have an advantage in the first place or or the the west and and in kind of our advantage is basically open and decentralized Innovation right it's not just a small number of big companies or Labs it's startups and universities and individuals hacking on things who AR even hearts of companies and that's a big part of it and you don't want to shut that down so and I think if you do you you you increase the chance that we don't even lead in the first place but then I think you get to the the issue which is okay like China or not even China any government um you know there are all the risks of of kind of stealing the models and and Espionage I mean a lot of the models fit on you know a hard drive that you can you know quickly put in your backpack or whatever and it's um I I just think we need to be realistic about How likely it is that we can secure um and not not just not us but like any of the tech companies can secure any of these um models long-term against very sophisticated efforts to do that so my own fear is that if we lock down development we end up in a world where basically you have a small number of companies plus all the adversaries who can steal the model are the only ones who have access but all the startups all the universities all the individual hackers are kind of just left out and and don't have the ability to do this so my own view is that a realistic aim that we should hope for is um is that we use open source to basically develop the leading and most robust ecosystem in the world in that we have an expectation that our companies work closely with our government and Allied governments on National Security so that way our governments can persistently just be integrating the latest technology and have a you know whatever it is a six-month ADV eight Monon advantage on our adversaries and I think that that's you I don't know that that in this world you get a 10year permanent Advantage but I think a kind of Perpetual lead actually will make us more safe um in one where we're leading than the model that others are advocating which is okay you have a small number of closed labs they lock down development we probably risk being in the lead at all like probably the other governments are are are are getting access to it I it's that that's my view I I actually think on on both these things spreading prosperity for for um more evenly around the world making it that there can be more progress and on safety I think we're basically just going to find over time that open source leads um look there going to be issues right it's like we'll have to mitigate the issues we're going to test everything rigorously we do we work with governments on all this stuff we'll continue doing that um but that's my view of of kind of where the equilibrium I think will settle out given what I know today next Mark Zuckerberg is going to start talking about economic possibilities with the use of AI and this I guess is a really important part of the interview I I think that there's a version of this which AI will do no matter how it's developed um and then there's a version of this that I think benefits from open source specifically so I think that AI has more potential than any other single technology that's being developed right now to increase productivity accelerate the economy um make it that kind of every person has the ability to be more creative and and and produce more interesting things and I think that that's all going to be great I I also think I hope that it'll help out with science and um medical research and and things like that um there are a lot of folks today though who don't necessarily have access to the ability to fine-tune or build their own state-of-the-art models so they're sort of limited to what these large Labs do um and like I just said I I think um you know one of the defining aspects of our culture around Innovation as a sort of a country or or Society is like it's not just big companies that do it right there's all these startups and hackers and academics and people in University and I think you want to give all of those folks access to state-of-the-art models that they can build on top of not just that they can run which is what they have today with with the closed vendors but that they can build on top of and and tweak and distill down to smaller models that they can run on their laptop or their phone or whatever other device they're building and I think that that's just going to unlock a ton of progress there's also a version of this where there are you can look at it by um you know Nation too um you know so it's not just that startups might not have the resources or universities might not have the resources to go train their own um you know large scale Foundation models now or in the future but um but there are a lot of countries that aren't going to have the ability to to do that because I mean you know pretty soon these things are going to cost many billions of dollars to train and um I think that having the ability for different countries and Entre R preneurs and different countries and businesses to use it to serve people better and and just do better work is going to be something that that basically like lifts all boats around the world and um just has a massive kind of equalizing effect so I think that that's really positive and you know that's one of the places where we get the most um thanks for this is not actually The Tech Community but it's just it's like different developing countries or other countries that want to have access to the technology and do stuff with it here Rowan asked Mar about his letter in which he directly called out Apple in their closed Source approach and then Rowan asked him to elaborate on it and what his thoughts are so let's listen I mean my point in there is more it's a little more philosophical on how it's affected my own kind of approach towards things and um and psychologically sort of affected how I think about building stuff um I actually don't know how they're going to approach AI um you know they do some open development they do some closed development um you know by the way I think it's worth noting like I don't actually consider myself to be an open source Zealot I just think that in this case um I I think that open models are going to be the standard and I think that that's going to be good for the world but we do open development we do Clos development so I get it right and and I'm not saying that Apple's necessarily going to be on the wrong place on this um for AI but if you look back over the last 10 or 15 years um it has been a formative experience for us is building our services on top of platforms that are controlled by our competitors and for a number of different incentives they they absolutely from my perspective apply different rules to kind of limit what we can do and you can just sense his anger here although he's being very diplomatic about the way he's saying all of this but I know he got burned on the fact that he had to build his platforms during the mobile Revolution which really took over everything he had to build Facebook and the rest of his app portfolio on top of his competitors and he doesn't want to make that mistake again and yeah they have all these taxes and you know at some point we we did um we've done some analysis that we we think we'd be you know way more profitable um if it weren't for some of these arbitrary rules and and I think a lot of other businesses would be too but you know honestly the the money part I think um it's annoying but for me it's not the biggest thing it's you I think it's a little bit Soul crushing when you go build features that are that you that are what you believe is good for for your Comm community and then you're told that you can't ship them because some company wants to put you in a box so that they they can better compete with you and my concern for AI at this point isn't actually Apple it's more the other companies and how that would evolve and I I think to some degree it's not even that I'm not even saying that they're like bad people it's it's um I think that there's just a physics and incentive structure to the system where you know if you build a closed system then eventually there are all these forces on you that that sort of kind of push you to to to kind of clamp down on things and um I I I I think that it will be a healthier ecosystem if it's developed more like the web but um but more capable and I think that you know because of how mobile developed where the closed model one right it's like apple I think has has really reaped most of the benefits um in terms of you know they there might be more Android phones out there but like apple gets like almost all the profits of for mobile phones I I think there's a bit of recency bi bias because these are these are long Cycles right I mean the the iPhone I it came out in 2007 right so we're almost 20 years into this thing it's a long cycle um but it's easy to forget the fact that the Clos model doesn't always win um if you go back to PCS um now I know a lot of people have especially if you're using the Linux analogy people don't necessarily consider Windows to be maximally open but but compared to the um the Apple approach of of kind of coupling your operating system with um with the device the windows approach was a more open ecosystem and it won and part of my hope for the next generation of platforms which includes both Ai and the work that we're doing in augmented in virtual reality is to you know meta wants to be on the side of building the open ecosystems and it's not just that we want to build something that's an alternative to the closed ecosystem I want to restore the industry to the state where the open ecosystem is actually the one that is leading um so I think it's possible I I think we'll you know I think we're making good steps on that as I said earlier his motivations for doing this are clearly because he got burned that is why he is trying to change the way that this ecosystem is going to be developed and play out in the long run this is also why Elon Musk decided to open source grock because he was bitter at open AI being closed AI all right so next Rowan now asked Mark Zuckerberg about Lama for oh man I mean it's um you know we're just doing 3.

1 for for for llama now I I think it might be a little early to to talk about llama 4 but um but we've got the compute cluster set up um we've got a bunch of the data set up we we kind of have a sense of what what the the architecture is going to be and and and have run a bunch of research experiments to to kind of Max that out so I I do think that um llama 4 is going to be another big leap on top of llama 3 I think we have um a bunch more progress that we can make I mean this is the first dot release for llama um there's more that I'd like to do um including launching the uh the the multimodal models um which we we kind of had an unfortunate setback on on on that um but but I think we're going to be launching them probably everywhere outside of the EU um so for those who are wondering what he's talking about just recently it was reported that they are not going to be releasing multimodal AI in the EU strictly because of their regulations and that is the setback that he's talking about next he's going to be talking about AGI and agents they are defining their own agent architecture it seems or its own language so let's see what Mark has to say about AGI and specifically agents yeah I mean I'm happy to talk about it both from a technical perspective and a product perspective but since we've mostly talked about the models so far maybe I'll start with um with the products so our vision is that there should be a lot of different AIS out there and AI services not just kind of One Singular AI um and that really informs the open source approach it's you know it also informs the product road map so yeah we we have met AI um Med AI is doing quite well my goal was for it to be the most used AI assistant in the world by the end of the year I think we're well on track for that we'll probably hit it hit that Milestone um you know few months before the end of the year that's a huge statement if true that means that meta AI has more usage than chat GPT which would be surprising to me because anybody who knows about AI knows about chat GPT but they don't necessarily know about anthropics Claude or other models and many people have never even heard of llama before obviously meta has the billions of built-in user base so it's exciting to see that even more than that a lot of what we're focused on is giving every Creator and every small business um the ability to create AI agents for themselves um making it so that every person on our platforms can create their own AI agents that they want to interact with and if you think about it these are just huge spaces right so there are hundreds of millions of small businesses in the world and one of the things I think is really important is basically making it so with a relatively small amount of work um a business can basically you know few Taps um stand up an AI agent for themselves that uh can do customer support sales communicate with all their people all their customers I kind of think that every business in the future just like they have an email address on a website and the social media presence today I think every business is going to have a um an AI agent that their customers can talk to in the future and we want to enable that for all of those that's that's going to be hundreds of millions maybe billions of of kind of small business agents similar deal for creators um there are more than it's more than 200 million people on our platforms who consider themselves creators who basically use our platform um in a way that is primarily for you know building a community um you know put putting out content feel like it's it's kind of like a part of their job is is doing that and they all have this basic issue which is that there aren't enough hours in the day to engage with their Community as much as they'd like and likewise I think that their communities would generally want more of their time but um but again not enough hours in the day so I just think it's a there's going to be a huge unlock where basically every Creator can pull in all their information from social media can train these systems um to reflect their values and their business objectives and what they're trying to do and then people can can interact with that it'll be almost like this almost artistic artifact that creators create that um that people can can can kind of interact with in different ways and then and that's not even getting into all the different ways that I think people are going to be able to create you know different AI agents for themselves to do different things so I think we're going to live in a world where there are going to be hundreds of millions of billions of different AI agents eventually probably more AI agents than there are people in the world and um and that people are just going to interact with them in all these different ways so that's part of you know that's the product Vision um obviously there's a lot of business opportunity in that that's where we want to go make money so we don't want to we're not going to make money from selling access to the model itself um because again we're not a public Cloud company we will make money by building the best products an important ingredient to the best of products is building is having the best models which having the best kind of ecosystem around open source will help us do so that's why it's kind of all aligned for us so this is echoing exactly what Yan laon told Lex fridman a few months ago where he said they're not going to make money by developing and deploying open- Source models but by being the company who can Define the standards and thus make the best products around that AI that's going to be meta and that's how they're going to make money I mean financially one thing that I'm quite aware of is the internet um had a big bubble burst before it succeeded and you know so all the people who were very long on the internet um were eventually right but sometimes things take a little longer to develop than you think and you just need to have the commitment to see that through and um that's something that I'm aware of because yeah I mean I I I'm really excited about you know all the unlocks that we're going to get from llama 3 and then llama 4 and then llama 5 and I think that's going to translate into better products but realistically um it's hard to know in advance when something is good enough that you're going to have a product that billions of people use and then when it's ready to to kind of be a large business and I mean look we're all spending you know a lot of capital and and on basically training these models so I I think that people are going to be probably losing money for quite a while U but but I don't know maybe maybe that'll all happen quicker it's it's it's hard to know exactly all right in this last section he talks about fear of AI why people worry about AI why they should and why they shouldn't so let's take a look the other part of this that I think you are more getting at is people's concern about what it means for their livelihoods and on that this is one of the reasons why I think the open source approach the approach of um lots of different models out there that are kind of personalized and customized to to every business and every Creator and every person um I think that's important because if this of develops in a way where it's just a you know a small number of companies that build the products and benefit and people use the products and maybe they like talking to to you know an AI assistant and that's valuable for them but you know if that if this doesn't in some way help lift all boats then I think you end up event getting a backlash and part of what I've spent some time thinking about after just looking at how the kind of Web 2. 0 stuff developed is in the next generation of Technologies around AI around AR and VR how do we create not just a kind of thriving set of products um and and kind of economic kind of productivity gains but how do we have like a better and more sustainable political economy around it where there's just way more people who are who who feel like they're they're kind of bought in or benefiting from this um in supportive of of of the system and you know I I I thought we did that reasonably well with social media but um but I you know just looking at some of the feedback and some of the response from from the world um I think that it's going to be important to do that even better with AI and some of the new technologies in order to to uh mitigate some of the concerns that people are going to have about what this is going to mean for their livelihoods and and and jobs and their lives so we we're going to end it here I think that is a great place to end it something for us to think about over the coming weeks months and years incredibly important stuff I'm so excited to see all the different innovations that will come from llama 3.