Meta's New Llama 3.2 is here - Run it Privately on your Computer

meta AI just released a brand new large language model llama 3. 2 and it comes in four different versions two of them are multimodal with vision capabilities and two of them are small enough to run on devices like mobile phones I actually partner with meta AI for this video and I want to show you these four models what the difference is between the four and then I want to show you how to install them locally on your computer without any technical skills a five easy step process and then I want to show you some prompts for some practical use cases with running L 3. 2 on your computer let me quickly show you these latest models here there's four of them available I'll put a link in the description to this page right here but the lightweight models are really interesting I think developers are going to do some interesting things with the 1B model specifically what 1 billion parameter and 3 billion parameter model if you're non- technical the lower this number is the more lightweight they are and they could actually run on devices like mobile phones the higher the number for example their Flagship model is 400b model I can't run this on any computer that I have now you also have the multimodal models the 11b model and the 90b model and they have Vision capability so for image use cases these are going to be the models that you want to use now here's a quick Benchmark the lightweights they ran against the Gemma model and the f3.

5 model that I've tested out before and it looks like it's winning again in every category when it comes to a lightweight model now the vision models they are also compared to ha coup and they are compared to G GPT 40 mini here and you could see the Benchmark on this page again beating these two other models these are again close Source models these are open- Source models and one of the main reasons why I cover llama releases every time they release a new model in this channel is the fact that it's open source and you could run it privately and securely on your own device now let me show you how to install this on your own computer step by step and then we'll run it through some prompting tests now if you don't want to install this on your own computer you could also use gro. com this is one of the partners they have and llama 3. 2 990b 11b 1B 3B they're all available here in this dropdown so you could choose the model right here this website's free to use and then you could test out your prompt over here now let me show you the five-step process for installing llama 3.

2 on your own computer the first thing you need to install is this thing called ama. com I'll put the steps with links in the description just press download over here it's available for Mac Linux and windows here if you do have Windows you need Windows 10 or later now I'm going to move that to my applications folder here and then I should get a popup like this it says welcome to all llama and I'll see a little llama icon right on top of my computer too I'm going to press next and I'm going to say install now it's going to give me this code right here I'm just going to go ahead and copy this and I'm going to press finish right here now step two is opening an app on your Mac or PC called terminal and just paste what you just copied over so it's AMA space run space Lama 3. 1 this is going to install 3.

1 and and then we'll install 3. 2 in the very next step so I'm just going to press enter this is going to go through the installation process and install this large language model the previous model right now and it's pretty much installed right now except I'm going to show you how to run this with a nicer looking chat interface but right now I am talking to llama 3. 1 I'm going to close this one for now terminate okay then for step three I'm going to go to the models tab right over here in the AMA website this is their library and 3.

2 is right here so I'm going to click on it and here's going to give me the code for llama 3. 2 so I'm going to copy this one over and if I open the terminal app here I just have to paste this one this time around and this is going to install llama 3. 2 llama 3.

2 is installed and I could again talk to it here but again I'm not going to do that I'm going to go and terminate so that was step three was installing llama 3. 2 now next we need to go to doer. for step number four and we need to have this app installed on our computer we're not going to do pretty much anything with this app but it does have to be installed after you install AMA so I'm going to install it here for my Mac and I'm going to move this to my applications folder as well now I'm going to go to my applications folder double click and I'm going to open Docker right here but we don't need to do anything with Docker right over here I'm not going to sign up this is the page it's going to take you to right here it says your containers will show up over here but I'm just going to minimize it so Docker is open that's it that's the only step that you need to do and step number five is you need to go to this website right here which I'll link below this is something called open web UI that gives you that chat interface for running AMA so you're not running it in the terminal app and you need to scroll down all the way to where it says installation and default configuration if AMA is on your computer use this command so I just need to copy this Command right here for step five and I need to open up terminal right here and I just need to paste this right here again Docker is open AMA was installed and I just have to press enter right over here this is going to do some downloading for us now I'm going to go back to the docker app and now I have a new container I did not added inside a Docker I just added that line in terminal so we're pretty much done if I click right here now this is going to bring us inside of open web UI you just go ahead and sign up for an account here again this is going to be running locally it says Local Host 3000 so after you sign in you don't really even need Wi-Fi connection to run llama 3.

2 and in order to use llama 3. 2 if I just select this drop down right here any model I've previously installed is going to appear over here 3. 2 this is the version I just installed so I installed the 3B model right over here so I'm going to select it and I could set this as the default model if I always want to use this model so I don't have to drop down every time to select one and I could start talking to it right over here and send out your prompt and this again works like any other chatbot that you're used to right over here and you have followup messages with it now as you can see the response time here was almost in real time but that's based on your computer typically the GPU of your computer is going to determine how fast this is so the 3B model should be find in most computers you could always get the 1B model as well and if you go back to this page right over here if you click on the 1B model it's going to give you this code right here that you'll just add into terminal and it will appear on your chat interface as soon as you add it so I'm going to copy it I'm going to paste that inside a terminal and let that install as well and now if I refresh my chatbot right here you could see that I have a new model 3.

2 1B and 3. 2 latest is the 3B now I still don't have access to 11b and 90b inside of this chatbot but I'm going to show you that on the grock website with a couple prompts now I'll start here with llama 3. 2 the 3B model this is a very lightweight model so I'm going to keep some of the more complex prompts for the more advanced models here but I'm just going to try summarizing a news article in 100 to 150 words and I'm going to go ahead and paste something from The Meta website let me copy everything from this page and paste that over here let's see what we get here and this was actually a good amount of text here that I pasted onto this chat meta has released llama 3.

2 and open source AI model family that can be fine tuned okay pretty good the tone is good the information is accurate 1B 3B 11b 90b all right pretty good so far okay let me get a one- sentence summary now remember this is the light lightweight model one of the light lightest weight open source models available right now so just imagine all the different applications that could come to mobile devices using models AI models like this okay pretty good for taking a good amount of text and summarizing it okay let me start a new chat I want to try a creative writing prompt let me just show you the speed by the way I'm going to send this out look at how fast this is responding to me right here this lightwe model as I stood in front of the mirror adjusting my hair for what felt like the 100th time that day okay let me just read this on my own and then I'll give you my thoughts about it okay this is pretty decent for a creative writing prompt I think it's done a really good job changing his tone and his writing style to get us something in in the world of Storytelling okay let me try a quick marketing one and then we'll test out some of the bigger models propose a product idea for mea's entry into the car market let's see what we get okay it made up a product called Meta Drve it is a AI powered electric car integrates with user personal assistance AR te technology social media platforms okay not bad this is pretty good these are again some of the things that I'm getting from the bigger models when I do these type of more practical nonscientific test to see if for everyday application is this something that I could go and use this is again unlimited it's private it's on your own computer so you could use it as much as you want it also came up with a business model that I'm not sure if I agree with this part a $500 a month subscription that is a unique way to sell a car but okay next I'm going to jump into chat. gro. com so this grock chat interface has all kinds of Open Source models so all the Llama models are available here right now I'm going to choose llama 3.