How to Scrape ANY YouTube Video Transcript with n8n! (full workflow)

today I'm going to show you how I built this three-part workflow that allows me to grab YouTube transcripts put the entire transcript within an air table database and then I can actually later on vectorize that within a rag like system that allows me to talk to an agent about any YouTube video so I can pick out little parts of the video ask questions about the video do deep research with the video really whatever I want but the main part of this a aut a that I'm going to show you is just getting the YouTube transcript

into an air table database so that all I have to do is paste in a video URL so let me show you this in action let's say I'm watching a YouTube video by productive dude and he's building a Blog writing agent right here right let's say I really like this video I want to go in depth I want to learn more about it what he's talking about here how he's generating blog posts what I could do is I could copy this URL that's all I've got to do and then paste it in my form hit

submit and now within about 10 seconds this automation is going to run in the background this transcript workflow automation right here and it's going to add that YouTube transcript to my database just like that and it's going to label it as transcript complete so now in a matter of literally 10 seconds I have that entire 40 minute transcript in here from his YouTube video and now what I could do is I could even vectorize this later on today I'm going to show you how to do this and how to get a YouTube transcript in here

but I want to show you what what the future of this automation looks like now you could do competitor research it's even adding in the description here some hashtags that he used you can add in their Channel tags their video tags that they used so you could do multiple things with that but what you could also do is you could upload this transcript to a vector database and you could actually talk to this transcript so let me show you an example of that real quick in this automation it automatically adds that transcript to something called

a pine cone database so when I hit test workflow what this is going to do is search my table for new items and then add it to my Vector store as you can see it created 139 new records for me and then it updated my status automatically to vectorized so let me show you a quick example of how this might work maybe within the video that I just got in here I want to remember how long Carter said it took to produce a mediocre blog post I could ask a very specific question about a certain

part of any transcript within my database and keep in mind these are thousands and thousands of characters long and it has access to all of my transcripts that I use in here because of the way it's vectorized them it can pull certain parts as context so let me ask that question about mediocre blog post just so I can show you how granular this can get how long does it take to create mediocre blog posts without creating an agent for it and it should say four to five hours and it's going to go search my Vector

database it's going to use this query as a small sample in order to pull information from the videos so as you can see it pulls directly from what Carter said and keep in mind I mean I have like three to five uh 40 minute videos in here so it's pulling from a 40-minute transcript and then using that as context with my prompt so these Vector databases work like magic so today I'm going to show you how to build the YouTube transcript automation that can pull transcripts from YouTube into your Air table database and then you

can decide what you want to do with that transcript after if you want the full build we have it all within the AI foundations's Community but today I want to keep it simple now the biggest problem with most people is they get stuck at certain parts of automations they want to know how to build automations but they're kind of stuck in this thing I like to call the template handcuffs and they're kind of finding what they want on YouTube or they're just going to communities and buying oneoff agents but they don't know how they work

so Carter and I the person I was just using as an example my brother uh what we've done about that is we've created a community in order for you to learn AI from start to finish from literally the fundamentals all the way up to building these agent systems like I've just showed you today we have an n8n course with over 30 modules now within the classroom section we have all of the fundamentals so you can get up to speed and this course goes over literally everything you need to know about building Agents from the ground

up we have hours and hours of modules in here and we also have agent builds in here that you can copy and paste into your workflows so for example we have a deep research agent that does insane deep research for you we have blueprints that you can easily download so if you want to actually learn the knowledge it takes to build these agents I highly recommend joining us I'll leave a link in the description or the top pin comment and when and if you join make sure you introduce yourself we have over 8,000 comments on

the introduction thread we love getting to know you and um yeah just be ready for a warm welcome but with that being said let's jump into this automation so there's a couple of tools that you're going to need for this one you're going to need air table you need a spot to store all of your information air table is the best system for storing information you're going to need n8n obviously that's where we're going to be building this Automation and then you're going to need something called apify and this is for very cheap I mean

for a th000 videos it's going to cost 50s so create an apify account create an a table account and let's get building first we want to start an air table because what we need to do is actually create a database like the one you saw so what I'm going to do is hit create I'm going to start from scratch and here I can just name this YT DB maybe for YouTube database just like that and I can leave name but I do want to rename that name field to something else so what I could rename

it to instead of name is video name beautiful we can leave that single text and then I want to change the name of notes to description so that we can pull in the YouTube video description I want to duplicate that description field and since it's a long text that's going to work perfect for the transcript so I'm going to name that field transcript and hit save and then we need to add a couple of other fields let's add a field for the video URL so what I'm going to do on a sign or actually I

can just delete these fields just so I can show you how to do it from scratch but after you have video name description and transcript make sure description and transcript are long text with without enable Rich Text formatting and what you want to do is you want to hit this add button and where it says find a field type select URL you can type in url and then you can just name this field video URL then you can hit create field and after that what you want to do is you want to have a status

field but we're going to call this stage so under find a field type I'm going to type in status or single select and we want to select single select and then we can just name this stage and then we want to add some options in here so for option number one I'm going to make it green doesn't really matter what color you select I just want green and I'm just going to call this transcript complete and then I'm going to add another option make it purple and I'm going to call this one vectorized so we

know which transcripts are complete and which of those transcripts are actually vectorized for us so once that's done I can hit create field and now I'm just going to clean things up a little bit for our table I'm going to name this YouTube videos for our first view I'm going to rename this all transcripts and then what I'm going to do is in the very bottom leftand corner of the screen I'm going to select this form view right here and then I'm just going to hit create a form View and I can just label this

new YouTube transcript and I can create new view and here we want to create a form so I want to delete everything except video URL so I can just click into these and hit this little I icon on each one of the fields transcript we can remove cuz this is the only thing we want to have to input into this form in order for it to work so just like that we have a form and whenever you want to input a video URL into the form you're always going to have to hit this open form

button right here uh in order to submit that video URL so next we are good to go on creating an automation within our air table so this is going to allow us to send web hook data to our nadn once a new video URL is submitted so in automations and air table what I want to do is I want to add a trigger but before I do that let's add in a quick test field actually we've got three of them but I'm going to delete all these and then I'm just going to add in a

field and label it testing and then I can go create an automation because we're going to need to use that field a little bit later on so I'm going to select automation I'm going to hit add trigger and then I'm going to select when a form is submitted so when a form is submitted I can select the table of YouTube videos and then I want to select that new YouTube transcript form that we just created and from here I can just hit use suggested record it should say step test successful if you created that test

field like I just did that test table uh that test row excuse me next we need to hit add Advanced logic or action and we want to select update record and we want to where it says select a table do our YouTube videos table and for record ID we can hit this little blue plus button and select air table record ID then we want to choose a field and make that video URL and then add in with that blue plus button again the video URL now we can generate a preview that should test successfully and

the last thing we need to add in is a run script this is going to allow us to connect to our web hook usually when we build out agents we keep all of our resources within our build agents with naden course in the classroom so for example when we build out agents like this we usually have awesome resources in here that you can use so I'm going to copy this but I'll leave this one in the description for you and then I'm going to p paste it in in here what you want to do after

you paste that in is add some input variables so the first one I'm going to add is record ID and make sure you have this spelled exactly like I do you want to scroll down and select a table record ID and then video URL again scroll down and select video URL just like that so now on the air table side of things we're all good to go we're going to need to come back here and input our web hook URL right there but until then we're good to go so let's now go into our automation

side of things create a new workflow and again if you are part of the community you can do things like this I really like this so you can download the blueprint and then you can come in here import from file upload the blueprint and it's all set up for you and then you're ready to go pretty much but I'm going to show you how to build this thing now so that you can actually get some good skills what I can do is I can rename this workflow something like URL to transcript the first step of

this automation is going going to be a web hook so I'm going to search for that node web hook and this is what's going to start the workflow for HTTP method you want to select post and everything else should be good don't give away your web hook URL otherwise people other people will be able to contact it so what I'm going to do after I have web hook in is I'm actually going to grab this test URL when it's production you'll want to switch it but test I'm going to copy that you can just click

it to copy it head back to my automation under run script and air table edit that code and then where it says nadn web hook here I'm actually going to paste that in and now we can actually test this out if we want but I know it works I'm just going to get into building what we want to add after this web hook field is a set field so that we can set that URL so I'm going to to grab this set field right here and then I'm going to rename this just so I can

stay organized to URL and then I want to add a couple fields in here I want to add a URL field and then I want to also add a record ID field so I can uh update the record ID once it's been complete if I ever need to and now I'm going to need to send some data here in order to map these values so what I can do is I can hit test workflow and now it's waiting for me to test this out so what I could do is I could just hit finish editing

on this automation turn it on and now I can go to my form and actually just kind of run through this how I would want to so I'm going to go to YouTube so let's say I want to grab this uh video transcript again we're in a testing phase so this isn't actually going to pull me the transcript but just for testing purposes I'm going to grab this and then go paste it into that form make sure that this is waiting for an event make sure that you hit test workflow maybe I'll even hit save

up there and then I'm going to hit submit and this test data should send through to this web hook uh request just like that so now that that data sent through that means our form was hooked up correctly uh in our automation so now that that's successful what I can do is I can click in here and I can map all of the data that's being sent from the web hook so I can map record ID and video URL so I'm going to drag video URL under video URL right there record ID under record ID

and boom just like that we have these two Fields set next what we want to do is we want to set up our apify actor so in the description I'm going to have this YouTube scraper right here when you uh hit try for free you can go to the council and you can set this thing up the main thing that you're going to want to do is create an apify account add a card so that they can charge you the 50 cents per 1,000 videos pricing model and then you want to go to your settings

and you want to navigate to API and Integrations here you're going to be able to access your personal API token so you can create a new token give it a description if you want I'll just name this for test video delete so I remember to delete this and then I'll hit create and that will create a new key for you what you can do is save the that for later we're going to come back to that within your automation you want to click this plus button after URL and add an HTTP request here I have

a curl or a c URL import that you can use uh usually again this stuff is always in the community always have the good resources but I'm going to copy this this will be in the description for you and then I'm going to uh where it says import C URL I'm going to click that and then I'm going to paste it in and I'll hit import and then from here really all you have to do is paste in your API key and then also change this to using Json and add in another field which I'll

also have in the description for you so first let's focus on the API key I'm going to go to my apify account and grab that API key that I just created I'm going to come back to the Automation and I'm going to paste it in right there don't show that to anybody I'm going to delete it after this video and then for this body content type you want to change specific body to using Json and then you want to paste in the Json that I've left you in the description which I'm going to grab for

my community but I'm going to paste that in here and this should be all good to go this is a dynamic field at least it needs to be a dynamic field so what I want you to do is change this from fixed to expression and now this is dynamic for whatever URL comes in and now that we have all of this complete what I want to do is I want to test this but first let's go back and still use our test data but let's actually test this URL step to make sure that we get

an output right here so I'm going to hit test step we've got an output now I want to go back to the HTTP request and then I want to open up this and make sure that we have a YouTube video URL in the green over here as the result and then I want to hit test step if you've set it up just like me this should go through and this should work as you can see we get everything about the video the description the channel we get literally everything thumbnails which is really cool we get

everything so this is very good what we need to do next is set up a request to get a trans script so what I can do after this HTTP request right here is Select this and do another HTTP request and this is just going to be a get request and where it says URL we want to pull in a specific point of data and that specific data we need is all the way down at the bottom it's going to be under all of the Adaptive formats and what you're looking for is the caption tracks base

URL you can drag and drop that over in url and now when we test this step what you're going to notice is it actually pulls over the entire transcript R in this weird format so let's format that with a code node so I'm going to head up here and type in code and then I'm going to select this code node and then again in the description I'm going to have in one Google doc probably all of this information for you for free but I'm going to go back to my community where I keep all the

good stuff and then I'm going to drag copy and then paste in this code to this code node right here after I connect it to the HTTP request I'm going to delete all that out of here and then I'm going to paste in the new stuff and this is going to format it from looking like this big jumbled mess to being one cohesive transcript so when I run this code I'm just going to do a test run real quick as you're going to notice it cleans it up completely for us and now we can actually

just start appending everything back to the database now that we have this transcript so the next step is to merge everything so I'm going to select this merge tool and for number of inputs we want three and then I can drag this up here and the first merge that I want is going to be from the URL field so I'm going to drag the URL field to input one the next is going to be from this uh HTTP request the video details field I'm going to drag that to number two and then the last of

course from this code node all to input three so now I can just test run this to show you how it's pulling in as you can see it's giving me three items right here so we need to aggregate that into one so I'm going to type in Aggregate and in order to combine this and then I just want to select all item data into a single list I can test step and what that's going to do is completely create one list of all the data that I need and lastly we just need to append this

to our air table so these are some utility nodes if you don't know what utility nodes are then you should probably learn them again we have them in the course up here noting over coding we go over all of the node types in order to completely understand nadn but for now let's just append this back to our database so what we want to do is we want to select this and type in air table and we want to actually update a record and this is going to append the transcript to our air table record so

let's create a new air table credential if you're signed into air table you can go to this URL right here airtable.com slre tokens new you can give your token a name so I can call this YT transcript agent doesn't really matter if I spell it wrong delete so I know how to delete it and I'm going to add a scope the Scopes you want to add are data. records read data. records write and also schema. basis readed I'm going to add a base I'm going to scroll all the way down to the database that we

created for this video and then I'm going to select create token here you have another API key you don't want to show anybody I'm going to copy that hit done and then I want to go back to my URL the transcript workflow and paste in that access token and then hit save now that's connected to our database that we created which is very nice we can edit it within nadn now so I'm going to x out of there X out of there what we want to do next is we want to select that access token

that we just created and then we want to update a record so we've dragged the record ID of the page that pulled in from the URL all the way back here so we pulled in the record ID and we imported it all the way to the merge field right here so now I can update my record ID but first we want to choose a base and then our table if the table doesn't appear quickly change fields and then it will appear and what we need to do is where it says ID we need to drag

in our record ID under data and video name we need to drag in the title of the YouTube video description we can drag in the description transcript of course we can go drag in the transcript that transcript is going to be all the way down at the bottom under cleaned text just like that and then video URL we already have the video URL in there I can deselect that in stage I want transcript complete beautiful so now when we test this step what it should do is it should update the last field within our database

with all of that information so I'm going to refresh my air table to see if that worked beautiful now we have the transcript pulling in a description we have our title and we have the stage that it's currently at so now this automation is complete and I'm just going to delete all these test fields and try this out from start to finish I'm just going to grab my automating research with AI agents uh video URL right here so I'm going to copy that I'm going to select our form open it and then I'm going to

paste in that video URL while this event is listening I'm going to hit submit back in our aort table database it will have added that new YouTube URL and now it's just going to run through this automation get us all the data we need and then append that transcript and everything else back to my air table so when I go back to air table as you can see we have the entire thing in here with the transcript now if you want to take the next step you know you have the transcripts in here I've showed

you everything given you all the resources to complete this but if you want to be able to update this in a vector database and be able to talk to these transcripts then I recommend just joining the community we have the rest of the build in there and we even have the easy copy and paste system so in the YouTube transcript agent we have all the resources used in the video um and a lot more resources compared to what I've given you today so you can go watch the rest of this in these next two tutorials

now that you have this set up and I hope to see you within the community but with that being said I hope you enjoyed this video on how to scrape YouTube video URLs this is something that I have thought was has been very useful for me because when I want to do deep research on something when I want to talk to my transcript or when I want to recycle content and create more content for my YouTube videos this has been an excellent automation to use so again I hope you enjoyed this video let me know

in the comments what you think please drop a like And subscribe I would highly appreciate it if you want to become a member of AI foundations I will leave a link in the description or the top pinned comment in order to take you from the basics all the way up to building agents with AI we break everything down we make AI very easy to understand in our community so with that being said I hope you enjoyed and I'll see you in the next video