Deep Research.....but Open Source

so open AI came out with a thing called Deep research you might have heard of it yeah it's kind of amazing I've been using it but you know what's not amazing it's price tag if you want to use the sucker you have to be a chat GPT Pro user which is $200 a month now I pay for that because I want to play with stuff but I want you to be able to play with stuff too so I was scrolling through Twitter and I found this guy's tweet David no not that David David came up with his own open source implementation of deep research essentially you get the same capability without paying 200 bucks I have to try this so you and me we're going to try it right now I'll show you how to set it up we'll do it together and we'll do a little bit of side by side comparing open AI deep research to well open source deep research keeping in mind this deep research is using open AI mini three I can't even say the thing open AI mini3 high it's the dumbest name ever I can't remember anything it's using that get your coffee ready let's get started now before we go crazy let's talk about what is this deep research thing Well normally with AI we get very very quick answers just blah blah and it's they're okay answers right like it's good but with this with deep research it's doing what it says it's doing some deep research it'll take 5 to 30 minutes to do a deep dive providing I think people say PhD quality research so it would take a normal person or a normal PhD person hours or days to research a topic I'm talking in depth like going out to web pages reading Pages coming back and referencing others and quoting and citations all that nerdy stuff it does that in 30 minutes I was talking with my producer Alex and we're like you know I love it when it takes longer to answer me like AI like we know we're all bragging about oh AI this is fast this is so fast I love it when it takes longer I feel like it thought more about something I feel like it's giving me a well-rounded answer it's got multi-step reasoning it's like going out to web pages doing some data analysis doing another search it's kind of thinking like a human many people are thinking this is the first real step to AGI and one very annoying thing about AI and the answers it gives us is we can't really trust it like even chat gbt is like yeah um sometimes we get stuff wrong so what's really cool about the research is that it has citations for every for most of the things it says it says hey don't take my word for it here's the link right here that's pretty killer and the sucker is multimodal which you don't if you don't know what that means it can look at pictures it can look at texts it can look at PDFs it can process all of this these types of data so tell you what right now let's try a little deep research I almost said deep search deep seek deep research I'll click on my deep research button here and I'll search for a hard-hitting topic research which animal is better cats or dogs give me a thorough answer a definitive answer based on Science Now what's cool is it won't just like go off to the races going okay got enough context it will ask me for more context this is also going to be available in the open source version of this so we're getting to that here in a moment I'll just say all of the above now I will say sometimes it will just kind of get stuck here and not do anything I'd be like hey bro you good and he won't be kind of like what he's doing now oh wait okay there we go so starting the research task now it should even pull up a little tab on the side on the right and show me what it's thinking any day now and tell you what we'll let that sit here I know you want to know the answer you're going to watch and wait aren't you I'm not I'm not even going to put time stamps you're going to have to watch the entire video but now let's check out this open source thing again this chat GPT deep research right now cost 200 bucks this guy's like nah all you're going to need is a chat GPT your open AI API key which is not free but is significantly cheaper than 200 bucks a month it's pay as you go and he even has a nice little diagram here essentially here is the thought process it's using what's called Ser queries which I just found out because I started playing with crew AI if you haven't played with that oh my gosh video coming soon it's so much it's a big world and this will also be using the 03 mini High I think I got it this time so let's go to the GitHub and get this set up it's actually really quick to get the set up we're really only going to need two things well three things we'll need a fir crawl API which is free don't worry and an open AI API we'll also need a nodejs environment but we're going to do that inside of a Docker container because you know how I roll also I'm pretty new to fir crawl API this thing's amazing they do have a hosted option which is free well up to 500 credits I'll get logged in real quick with my Google account and seriously it's so easy to get this API key cuz right when you log in it's just sitting right here ready for you to copy it so make sure you have that and then for the open AI API key you'll need to get set up with a account with open API open API open AI you'll go to platform. open. com you'll get logged in or sign up normally I think it's connected to your chat gbt uh if you already have a subscription but the 20 bucks a month or whatever you pay does not include this this API usage will be pay as you go which if you're just doing this real quick for a few things it's going to be pennies if you're doing crazy stuff it's going to cost you crazy money but we're not doing anything crazy right now but once you're logged in and you have a credit card you can go grab your API keys I'm going to create one right now now with both of our keys ready to go we can now set up our Docker environment now you don't have to do it with Docker if you preer H prefer just doing mpm he shows you how to do it right here I like Docker it keeps things separate secure nice and safe now if you're like I don't know what Docker is Chuck and um please help me I've got a Docker video right over here go watch it get set up get it installed it's very easy it's on Mac windows and Linux and here in Windows you do have to run it inside WSL which is another thing if you don't know what that is that'll blow your mind I want to watch my WSL instance right here and my video on that is right here it's essentially Linux on Windows we'll first use the get clone command to clone the repo deep research you will need get installed but thankfully on most uh installations like mac and Linux it'll already have get there if you don't got it go get it I'm sorry for that I'm going to CD or jump into the deep research directory and then inside here I'm going to add all the stuff I need to create my Docker container you don't need to know what Docker is just follow along but it will help to know what it is to kind of understand what I'm doing here first we'll create an environment file we'll type in Nano the best text editor in Linux period we'll specify our file.

env. local and we're going to add two things here our fir craw API key and our open AI API key I'll add mine right now just like this fir crawl uncore key have that equal within parentheses your API key I'm going to grab mine copy that key paste it here and then just under that my open aior key equal my key where'd you go buddy there you are that's that's all we need for that file we'll hit control X Y enter to save type in LS you should see our file right actually no you won't see it it's hidden type in ls- and you'll see it right here that's what that dot does before a file it'll hide the file now we're going to create what's called a Docker file so we'll do Nano Docker file just like this and I'm going to copy and paste this config in this is essentially describing a Docker container we're about to build I will have all this config Below in a nice little blog post on blog. network.

com we h control XY enter to save and then one last thing we're going to create our Docker compose file type in Nano docker-compose. yml we'll paste this config in again this will be below crl X Y enter to save and then because we're doing environment variables a bit differently we will have to change one file type in LS where you are right now we're going to change the package. json file so we'll do Nano package.

json this won't be scary I promise it's going to be one little file right here the start TSX environment file blah blah blah blah we're just going to backspace this a bit to where it looks like this just like this hit control X Y enter to save and now we're ready to go actually all we have to do is run one command we'll type in Docker compose you may have to do a pseudo at the beginning of this I'll go ahead and do that right now just in case Docker compose run-- RM this will actually remove the container once we're done with it and then we'll call the container deep research Ready set go oh crap we got an error oh you know what I don't know what it is we messed up our package. json file let's jump back in there real quick package. json it's not proper Json we didn't put a comma you probably saw this didn't you we didn't put a comma after our start right here there we go proper Json control X Y enter to save hit the up Arrow to run that command one more time and come on it's working exciting what would I like to research hm let's research the same thing I would like to know what is better or what animal is better cats or dogs and then it we'll control the breadth of the research defaults to four we'll just do four and then depth so the first one was how wide second one's how deep what does that mean exactly I'm not sure I haven't I haven't done the breadth and the depth on this yet we're playing with it right now and it's creating the research plan it's going to ask actually ask me the questions that chat GPT just did earlier similar ones yeah that's pretty cool all of the above let's do data driven scientific sure okay and now it's researching coffee break while it does it now I'm not sure if I've exhausted my API key usage let me go look oh no I haven't I haven't done anything I haven't paid anything this is all free oh it's doing it this is so cool doing things on your own Hardware which this is not our own Hardware but it's open source and we're doing it in the command line it just feels cooler open AI keeps things very mysterious you don't really know what's happening here you do this the code is all there now I'm not sure if I mentioned what fir crawl was for if I did ah my rate limit was exceeded this is why I would probably want to run this locally which you can do that but I just found this like an hour ago so I can't do it right now check it out all the URLs it did some research we're about to see our results now it said it Sav it to report.

MD we'll look at that here in a second but I think yeah here's all the research right here in markdown right here in our German what was the conclusion I don't want to read all this you saw it here folks dogs are better take that suckers I am a dog person sorry cat lovers let's see let's see if chat GPT said the same thing it said dogs yes but check it out I mean the research is cool so obviously Chad gbt has a a prettier interface right we've got links right in here in line definitely pretty wordy but I think we got a similar output and we can control how deep and you know how wide this goes as well on the open source side what do you say we do one more thing by the way let me know how you feel about that conclusion do you agree you know it's hard not to agree it's it's AI it knows more it can it can now research way better than you can better than I can Let's do an age-old question and by the way to run this again we'll do the same command just like this which OS is better for it professionals Mac Linux or Windows spicy right let me ask the same question over here let's say system administration and network engineering and cyber security all of the above should the analysis be on cost licensing you know everything everything answer the same questions you know notice it's asking similar questions I mean we are both using chat gp03 mini High okay starting research I will say my command Line's a lot faster although I do reach my uh rate limits and I bet the research would be so much better if I didn't hit these rate limits it is so fun watching the thought process of these AIS also extremely scary but I don't know the more I play with AI which it feels like I've been playing with new things every single stinking day because of releasing things at the speed of light the more I play with it the more encouraged I am for us it people and Humanity in general it's going to make us so good at what we do and that's the question you have to ask yourself or this is the question you need to ask yourself every single day how can I use AI to help me do my job better that's how you're going to stay ahead in this oh I love me a good AI that takes a long time to answer feels good all right let's see what uh actually I'm going to show up the same time so video editor don't show it yet dude chat GPT is taking too long I have to go pee come on chat GPT it's taking so long I got a meeting with Jeremy Chara hurry up oh it's finally done okay and the conclusion I saw it coming you know what I agree with this because I do the same thing as an IT Pro aim to build competency in all three over your career you'll need all of them and I'm pretty sure this said the same thing the Deep research open source yeah here it is right here for it professionals ah I'm resizing it uh it's not a single OS decision it's implementing a hybrid strategy utilizing each system's strengths while utilizing rigorous platform agnostic security practices blah blah blah you heard it here folks Mac Windows Linux use them all all right that's the video let me know if you try this let me know what you think about the conclusions and um that's all I got I'll catch you guys later oh by the way this video is sponsored by no one but me and my coffee Network chuck.