I experimented with the latest crop of AI tools to make a pretty impressive music video. Check it ou...
Video Transcript:
I want to make another music video that leverages the power of AI to do most of the work for me about 9 months ago I made this video called make a hit song and music video with AI for free and while the video did pretty well on my channel I wasn't super happy with the final result a lot of the technology that existed 9 months ago was cool for the time but things have come a long way since then we've got better music generators we've got better video generators so I wanted to revisit this concept and see how much better of a music video we could make and the bar is not very high because well here's the original video take a look let me tell you about this thing called AI it's smart it's quick it'll surely [Music] make it learned and adap it's a sight to be so yeah I mean if you want trip down memory lane go check out this video from 9 months ago and see how that video was made but the technology like I mentioned so much better now so let's try to make something a little bit more impressive so for creating the song itself you really have two good options for making really good highquality music you've got udio over at ud. com and you've gotso over ato. com I really like both of these tools I feel like udio is a little bit better at making music that's going to maybe trick the audience into thinking that it's real you can make songs with this listen back to it most people will probably be fooled that that was actually made with AI because it's very realistic music however I personally prefer the music that sunno makes it's sort of bizarre because a lot of times you can actually tell that it's AI you listen to the music and you can hear little elements you know you ask it to say things like AI it just go I and it will tend to sort of blend words together and things like that that are a dead giveaway that the song was generated with AI but I just feel like whenever I make songs with sunno I find myself sort of bobbing my head and being a little bit more impressed with the music of it yio better realism sunno just more enjoyable music I don't know why that's just my opinion on it and when it comes to the actual video portion of it back then AI generated video there wasn't a lot that we can use we had Runway and plasma Punk and deorum but nothing that generated higher quality more consistent AI generated video well since then we've got dream machine from Luma which makes some pretty dang impressive videos especially if you start with an image it's not very good at text to video yet from my experimentation but when you start with an image inside of dream machine and then you have it generate a video from that image it's actually pretty dang impressive and on that note the image generators have gotten gotten a lot better since that last video so when we want to make a video with dream machine our starting image that we're going to input is better now than what we would have gotten 9 months ago tools like Leonardo and mid journei and Dolly 3 have gotten a lot better so for this music video this time around I'm going to use sunno for the music now for the image generation portion of this video I'm actually going to use mid Journey right now for me it's always a tossup between Leonardo and mid Journey but I still think mid Journey edes out Leonardo a tiny bit on realism but if I was going for more of a cartoony or illustration I would probably go with Leonardo for this and before I go any deeper into this nobody's sponsoring this video and nobody has given me free access to their tools however you can generate up to 10 songs a day for free so you can do the music part for free with sunno with Luma dream machine once again I'm paying for it myself however dream machine does allow up to 30 Generations per month for free so this can also be done for free the video portion and then mid Journey same thing I'm a paying subscriber to Mid Journey now this is the only tool that I'm using that I'm going to show off during this video that you do need to pay for if you want to generate images in this way however there are so many free alternatives to generate images that you can then turn into videos you can use Leonardo you can use Bing image Creator which uses dolly three you can use idiogram there are plenty of free options available I'm just going to use mid Journey on this one because I really wanted to sort of test the limits of the realism that I can get but it's not required again that you use mid Journey for this process it's just the image generator tool that I'm choosing to use for this video all right now that we've gotten all of that out of the way let's get into making a song so I'm going to head over too.
com and I want a poppy dance song walking through the advanced ments in technology from the first computers all the way through generative Ai and into the Future Let's go ahead and click create and see what it comes up with let's take a peek at the lyrics for our first one here plugging in the first machine blips and bloops the early scene binary dreams so it actually did kind of work through the song in chronological order which is really cool so let's take a [Music] listen and blops the early scene binary dreams C so clean world's unfolding dat micro next so I really like that that might be the version I end up going with I like the lyrics that it shows and I want to make sure I use those same lyrics again so I'm going to go ahead and copy all of these lyrics over here we'll go ahead right click copy and then I'm going to switch over to custom mode inside ofso and under this lyrics I'm going to go ahead and just paste this in so it'll follow that same format of verse verse two chorus Bridge verse three Chorus for styles of music music let's go poppy Rock let's see if we can get it a little more rocking let's go ahead and do rock pop but also with synth let's see if we could get a poppy Rocky cynthy song with those lyrics here's our first [Music] version very very 80s let's go ahead and listen to the next one hell yeah inine [Music] ands yeah that's the one that's the one all right I like that version of the song I like the sort of Dancy cynthy poppy 80s rock like all of that Blended together just perfect especially for the subject material so now what I need to do is I need to start creating the scenes that are going to go with this music video so to start whenever there's no lyrics I want to have people sort of dancing around so to do that again I'm going to go ahead and use mid Journey so we'll come over to midj journey. com and you can use any image generator you want for this I'm just using mid Journey cuz I want to test see if I can get some realism into it so I'll do a colorful scene of a group of people dancing in the 1980s in a retro Club I want to go landscape for the size actually I'm going to stretch this out to 169 that's my preferred size I'm also going to switch to Raw mode instead of standard mode cuz raw tends to get you a little bit more realism and let's just see what we get out of that prompt there so it actually gave me more of like a cartoony style less realistic probably because I put the word retro in there maybe if I dropped the word retro it'll make it more realistic but I actually like some of this imagery so we might try some of this for our video I especially like these two on the right here so let's go ahead and click this one and we'll go ahead and download it I also liked this one so we'll go ahead and download this as well whether or not they make our video I'm not sure yet but doesn't hurt to play around with them now I'm going to go ahead and use the exact same prompt I'm just going to word it a little bit different to try to get some more realism out of it so a colorful scene of a group of people dancing in the 1980s I'll get rid of in a retro Club I think the Retro kind of tried to make it like a retro looking image and instead of a colorful scene let's do a colorful photo of a group of people dancing in the 1980s see if we can get that photo realism in there we got some interesting images here this one's looking a little more realistic but this girl looks like she's got a mustache here I don't know what's going on with her hands this girl has a disappearing arm this one's not too bad it just looks like there's like an extra leg in here I'll go ahead and download this one it might be usable we'll see I actually think this is one of the better photos but the problem is this girl's head's on backwards so that's not going to work very well and this one's not bad either but it's awfully close to being not safe for work right here so let's go ahead and do very but do very strong and make some other similar images but different so we got a few more variations this one here is not bad but I don't know what's going on with like this girl's face is turned this way but her arm is going this way like I don't know let's use it let's see what happens when we turn it into a video so now I've got four images that I downloaded let's turn these into videos real quick the way I'm going to do that is I'm going to jump over to Luma dream machine click on the little image button to upload an image and I'll upload our first little more illustration style image here and then what I've been doing which has been working pretty well is just using the exact same prompt so this one we used a colorful scene of a group of people dancing in the 198s in a retro Club if I just copy this text here and paste that as the same prompt it's actually been working out decent so let's go ahead and do that we'll grab the other image that we did on that first round once again paste in that same exact prompt we'll just toss all of these into Q it looks like it's moving pretty fast cuz this one already says it's dreaming but let's go ahead and pull in our other two images get those going we've got this Dance Image which used The Prompt a colorful photo I'll go ahead and grab this one we'll paste it in here but I'm going to get rid of a photo and just do a group of people dancing in the 1980s cuz it's obviously a video and not a photo all right so let's take a quick peek at what it generated with each of our videos here's our first generation not too bad not amazing but probably usable it just looks a little too morphe for my taste here's the next one that one's looking a lot better actually that's usable I think we've got our third one here yeah that looks good that's like perfect especially if I can line up the jumps to the beat of the song that would be awesome and here's the next one actually not looking too bad either her hand just disappeared there for a minute towards the end uh but I think that's going to be usable as well because most of these are only going to be on the screen for so quick that hopefully it just keeps on moving and you don't really notice some of that weirdness I mean we'll know cuz we're making the video but hopefully other people don't notice that so I've got some dancing scenes that I can jump over to when there's no actual lyrics going but I want to make some images that go along with the lyrics so plugging in the first machine blips and bloops the early scene let's just try an early retro computer and what I'm going to do with this one just to get more variations is I'm going to go ahead and do it with our style raw setting turned on then I'm going to go ahead and use this same text again and let's turn raw off and see how it changes our images so with style raw I really like this top right one and I really like this bottom left one so I'm going to grab both of these we'll download this image and we'll download this image here and then our normal styles are still generating here but we can see they're not too bad either actually I kind of like this bottom right one so let's go ahead and download this as well let's grab our prompt an early retro computer jump into dream machine and once again we're going to grab each of our images I'm going to use use the same prompt and I'll go ahead and do this with all of these images that I just generated here and here's the videos it generated from those images after I don't know roughly 3 minutes or so so we've got our computers it just kind of pans across and you can see like a retro office here here's another one where it's zooming in on this retro computer and then here's another one where it's panning across all of this retro equipment as well all very usable for the video so now what I'm going to do is I'm going to go through read through the lyrics do my best to generate an image in mid Journey that represents what's being sung about on the video then I'm going to take that image and I'm going to toss it in the dream machine and create a little short animation out of it this is going to take me probably a good hour to do not because it takes a lot of time to write the prompt for the image or because it takes a lot of time to you know take that image and put it into Luma Labs it's just all of that inference time the time it takes to generate the image the time it takes to generate the video from the image it's going to be a while before it's all ready so I'm going to go ahead and fast forward to the point where all of my images and videos are created and I'm going to show you how I'm going to edit them all together okay so that actually took me quite a while in fact I'm actually recording the second part the next day because of how long it took to generate all these images I definitely went a little overboard and generated way more images than I actually need you can see all the images I generated in mid Journey here I generated something like 200 300 images that I might potentially use for this music video which surprisingly didn't take that long the part that took the longest amount of time was downloading each image jumping over here to lum's dream machine putting the image and giving it a prompt I did this something like 120 times this is total Overkill I generated way more clips than absolutely necessary I just wanted to make sure I was going to have like the perfect clips for the song one thing I started doing to actually speed up the process was I went to Mid journey and originally I was entering prompts like a colorful photo of a group of people dancing in the 1980s and getting stuff like this an early retro computer binary code but then what I started to do after that was I literally just took the lyrics from the sunos song and plugged them into mid journey to see what mid Journey would generate with the lyrics so if I come up here you can I put from the past to the Future and then once I got the image I took this image pulled it into dream machine and this is the video clip it generated so AI generated the song lyrics it generated the beat to the song I actually took the song lyrics used those as the actual prompts to get the images and then took those images and turned them into videos again using that exact same prompt and got some pretty cool video footage that I think will go really well for a music video but I don't want to understate that this did take a while but I also went total Overkill and generated probably 10 times more videos that I'm actually going to need or use in this video also between the time I started recording this video and today when I'm finalizing this video Luma actually rolled out a new feature you can see here it says today we are releasing extended video so I went ahead and took some of my favorite prompts like this one of the people jumping around and dancing and actually extended it so now it's a 10-second video of people jumping around and dancing instead of just a 5-second so now I only have one last step to do and that's pull all this stuff together I've got all of these videos I've got our song here I'm going to go ahead and download the song So now I've got the MP3 version of the song and I've got all of these videos I went ahead and downloaded all the videos one by one off recording so you didn't have to sit there and watch me download them all and now I'm going to combine them all using Da Vinci resolve I like Da Vinci resolve cuz it's a free video editor it's one of the most power ful video editors and the fact that the most powerful video editor is available for anybody to use for free still blows my mind to this day it's really really good so I'm going to go ahead and boot up Da Vinci resolve here it also works on both PC and Mac so it doesn't matter which system you're on you should be able to use Da Vinci we'll create a new project pull our song into the project that we just created in sunno and now I'm going to pull our song onto the timeline now I have our song here on the timeline and I can listen back to it and then what I'm going to do in Da Vinci just to kind of keep it slightly cleaner is I'm going to create a new bin here and I'm going to call this bin Luma videos and then all of the videos that I just generated in Luma I'm going to pull it into this bin here again it's a lot it's Overkill I'm not going to actually end up using all of these I just wanted to make a ton of videos so I knew I could pull it all together Our Song is 3 minutes and 37 seconds I actually think I'm going to cut it off a little bit short it starts to kind of repeat itself towards the end and since this video is for demonstration purposes I don't think I need a full 3 and 1/2 minute music video now when it comes to editing a music video together the trick to getting a music video to look really good is to edit to the Beats so if I zoom in on my timeline here we can actually kind of see where the beats are I extend this a little bit larger we can see that each one of these spikes right here is the base hit so if the video is going to cut to a different scene we want those cuts to happen right on that hit for example if I'm going to take this video where they're all jumping around ideally I can time it so each little Hop Is Right on the beat let's go ahead and pull this in like this drop it on our timeline and this is what it looks like to start now it's not really timed perfectly and if I want it to be timed perfectly I can right click on my video clip here go to rtime controls and then I can actually speed up or slow down the video by dragging it out or dragging it in and ideally I can change the pace of the video so that it looks like the jumps are are happening right on the beat and so now you can see I slowed it down so it's 125% of the original length which is slightly slower than it was originally but if we watch it it kind of looks like they're bouncing to the beat now and when I do want it to cut to another clip what I'm going to do is make sure that that cut happens right on one of these changes and since the lyrics start off talking about retro machines I'm going to go through my video clips and find some of these little like retro machine videos that we've got we can go ahead and pull this one in right here I'll cut it to the beat so that clip will show for about 1 second and then we'll go to another clip that's somewhat similar and once again I'll cut it on the beat here's another retro computer clip cut it on the beat then I want to use this one last because it sort of Zooms in on the computer which will make a good transition into the next scene but because I still want it to sort of change scenes to the beat what I'm G to actually do is cut it at about 1 second right about there I'm going to do a cut and then I'm going to jump a little bit closer I'll cut to about right here so it's a little more zoomed in on the computer for the next shot cut this at the beat and so right on the beat you can see that it's sort of zoomed into the computer and then I want to go to some of this like binary like Matrix looking imagery I like this one cuz it sort of continues that zoom in effect so I'll plug this in here make this one go for about 1 second cut it right on the Beat Again move into this one that looks like a sort of similar style imagery we'll cut that on the beat pull in another one of these and you can see where this is going I just like to cut it to the beat so now I've got the first 12 seconds of the video kind of edited together and here's what it looks like so far in the first machine clips and boops the early [Music] and that's what we've got so far now this is probably going to take me a good 30 minutes or so to go and add all of my various Clips to the timeline so I'm going to go ahead and pause the video finish adding all these clips on and then show you the final product all right so I finally finished pulling it all together if I sort of zoom out on my timeline you can see I've got my cuts here my cuts are about every 1 second and then in certain areas where the pace of the song slows down I actually do my cuts every two seconds instead of every 1 second so the video itself is pretty fast-paced it's changing scenes every single second but I think it works really well with the pace of the song one thing I did while cutting it together was I had this intro that I started to use with everybody dancing and since a lot of this other video footage Just Feels So Random right we've got like old computer screens and then like Matrix style binary going on the screen people at computers you know stuff that's supposed to represent data waves and it's all kind of jumping around randomly so I decided to add these like dance Clips in every I don't know 20 to 30 seconds just to kind of give it this cohesiveness just to tie it all back together you can see a whole bunch of chops here this was me just trying to chop at every second so that it was easy for me to snap my Clips to the exact perfect time marker but then I got lazy and stopped doing it after that for a little bit and then at the end what I did this song was actually over 3 minutes but after about a minute and a half the song just kind of repeats itself so I did a fade out on the audio at the end and then I started to tie these dance Clips back in at the end to sort of bookend it with the same way that it started but I bring those back in every second so when you see the video you'll see that it sort of moves to the beat of the song as they're dancing and then the video finally wraps up this final shot was one of my favorite shots that Luma generated for me where there's this girl looking at a screen with flowers and then she kind of turns her head towards the camera at the end and then I end the video with her sort of looking to the side I just thought that was a cool way to end the video and that's it that's the whole video and I simply rendered the video out and now here's the world premiere of a song that I've settled on the title binary dreams PL in the first machine CPS and boops the early scene binary dreams Cod so clean worlds unfolding day streams microchips and next big Le silicone dreams they never sleep internet waks so deep bring wi the that we sweep from the past to the Future t on guide and our Tor wash the scen water on every be keep getting newer SM inside our hands virtual world we understand and across the land Minds create the now algorithm Take a Bow robots moving show us how future Tech will show from the past to the Future T our guide and our T wash the screen as water on every be keep getting Co [Music] and there it is a little bit better than the Country video that I showed you at the beginning and again quick recap I generated the song quickly with sun. com it generated the lyrics and the Beats for me behind it took me a couple tries to get a song that I really liked but I'm really happy with the way the song came out once I had of the lyrics I literally took every single sentence from the song and then plugged it into mid Journey here to get a few images for every line of the song I essentially let AI decide what this video should look like then once I had all the images I took each image plugged it into Luma dream machine and then gave it the exact same prompt binary dreams code so clean future Tech will surely wow and it generated these videos based on that sort of ambiguous is prompt and then finally I took the videos that I generated I took the Audio I generated I pulled it into Da Vinci resolved and then I edit it to the beat so that every scene change happens on the base hits and the final result was the video that you just saw and again most of what I did you can do for free you can generate a few songs for free with sunno when it comes to images you do need a paid account to generate with mid Journey but you can also do this in Leonardo idiogram B image creator stable diffusion there are so many other tools that you can also use to create the images and then once you have the images you can pull them into Luma dream machine which allows you to do up to 30 video Generations per month with a cap of about 10 per day on the free plan again none of this is sponsored I've paid for every single one of these tools that you're seeing myself in order to generate this video and I'm pretty happy with the result and when it comes to AI video generation we are just getting started this year is going to be crazy with new AI video tools coming out in fact the day that I'm finalizing this video Runway just announced gen 3 Alpha which is their sort of Sora competitor or Luma dream machine competitor it looks really really good if you check out some of these videos that they're demoing on the runway website now obviously when they put videos on the runway website they're most likely cherry-picked videos of some of the best Generations they got but this is going to be another really fun AI video tool that we're going to get to play with really really soon so expect a lot more tutorials and tips and deep Dives and explorations of these various AI video tools as they're made available for us CU I love playing with this stuff this is like my favorite thing to do this is the stuff that got me into AI in the first place my first AI videos were me making images with stable diffusion and mid journey and playing around with the text to song tools that were out and things like that so this type of stuff is what really really grabbed my atten attention and built my initial love for AI so excited that all of these tools are finally getting into our hands to play with and to show off and to make tutorials around so if you like stuff like this make sure you like this video and also make sure you subscribe to this channel because I'm going to be showing off a lot more tools keeping you in the loop with the AI news creating more tutorials and all of that fun nerdy stuff in the world of AI and make sure you also check out futur tools.
this is the website where I curate all of the cool AI tools that I come across it's updated almost every single day I also keep the news page up to date and there's a free newsletter so if you want the coolest tools and the most important AI news delivered directly to your inbox you can find all of that for free over at futur tools.