Runway Can Create Characters?! | An AI Animation Tutorial

in this video I'm going to show you how to create and animate an expressive character using AI in just a few minutes I'm excited to show you how to pull off this effect let's get to it before we get going I just want to give a special shout out to Timmy at Runway for creating the post that inspired this video tutorial we'll link below this video to his breakdown of how he put together a similar concept this workflow is going to be incredibly easy I recommend you following along with this video as we go so

let's kick things off the first thing that we're going to need is an image and there are different places that you can go to generate an AI image one place is the brand new image generator inside of Runway called frames to create images with frames just go to the generate image section inside a Runway and we'll paste in a prompt I'll say a 3D animated character of a man wearing a denim jacket standing in a village looking at the camera in clation and we'll go ahead and click generate okay let's take a look at our

four characters here you see it did a pretty good job now the problem is if you want it to have more of my likeness at least at this point using Runway frames you don't have the ability to basically pull a reference image or train a model you'd have to basically prompt back and forth so instead I'm going to use mid journey to create our image so I'm inside of mid journey and I will paste in the prompt and now we need to add in a reference image I already have a reference image of me pulled

up right here and make sure that you have that character icon selected because it will use that character for the reference image now we're going to type in just a short little code here that's-- CW space 0 so that means character weight of zero that will will allow the system to basically change the clothes and the haircut of the character into any style that makes sense for the prompt that you inputed I usually like turning down the character weight unless you are trying to get consistent clothing again and again and we'll also do --r space

and then a number to repeat the prompt a few times so I'm going to say R4 to repeat the prompt four times it just is completely up to you and when you're ready go ahead and click render now after a few seconds you can see we have a lot of really fun photos here but I think the one I like the most is this one and so all you have to do is go ahead and click download so now that we have our image it's time to deliver a performance so basically you are going to

act out the scene that you want to replicate now the video doesn't have to be super professional you can use a phone you can also use a computer webcam but it does help for the camera to be stationary or on a tripod I also recommend standing directly in front of a softbox like I am now or in front of a window just to create some flat lighting if you have really dramatic Shadows it can kind of mess up the camera track so here's a quick video performance that I recorded for this tutorial I'm super excited

to show you how to pull off this effect let's get to it okay not the best performance but you get the idea so now that we have our driving video and our image it's time to anim that image now I'm going to use Runway to animate the image but you totally could use cing Minx Sora Luma it's entirely up to you so to animate the video inside of Runway just go ahead down here and click generate video and in this little box here we're going to drag and drop our image and we'll paste in our

prompt again it's usually better to do tripod camera shot colon and then you can prompt for what you're looking for I'm also going to change the model from Turbo to 3 Alpha basically to get us the best result and we're going to change the duration to 10 seconds because we want the longest video possible and go ahead and click that generate button four to five times you're probably going to have to generate quite a few Clips before you get one that you're happy with and after a few seconds we've generated our video clips let's take

a look at number one we have our character it looks okay especially if you imagine that will be driving the performance but I really don't like the blinks I think those look kind of weird so not a great result there let's take a look at this one so this one's just very static we're not seeing those hand gestures again inside of our prompt here you can see we prompted in hand gestures we're not getting that here so that's not a great render let's take a look at this one oh man that one just kind of

falls apart and it looks like he lost some teeth there so uh really really kind of a creepy result there and let's take a look at this one okay yeah this one looks really good very expressive with the hands it looks like he's explaining things in a very natural way so I think this one actually did a really good job now I should note that inside of this one we have a spot where kind of looks like his hand goes up and he's holding like a lid to a cup so we'll probably have to cut

around that specific point but I think generally speaking it did a good job now the cool thing is inside of Runway they recently came out with the ability to upres this video clip to 4K directly inside of Runway which is pretty cool so we can click on this little arrow here and go to upscale to 4K just go ahead and select that and it will upscale your footage okay so we have our video clip here and I went ahead and removed that early part where it kind of distorted and had that thing that he was

holding inside of his hand and now we basically have two clips we have this clip here of this guy talking and it's animated and then we have the driving video footage of the performance that we're trying to put onto the animated character and to fuse both of these things together I'm going to use Runway act one so Runway act one is just over here on the left you can go to this little icon here of this guy in a square and go ahead and select act one so in the top box we need to add

in our performance so we will drag in the video footage of me talking to the camera and in the second box here we need to drag in our animated footage so we'll just drag and drop that into the box now I should note that if the driving character performance video is longer than the base footage that you're going to put the performance on you actually are going to end up doing like a weird Loop because it's going to try to fill in the Gap with the missing information so I definitely recommend trying to have a

video clip underneath that is longer than the driving video footage and if you are inside of Runway you can also go to the actions menu here and go to extend to extend your video clip it's not great at extending most of the time but sometimes you can get lucky and again with extending it's very similar to animating the first time you kind of have to try four or five times before you get the result that you're looking for okay so both of these video clips are ready to go let's go ahead and click generate so

let's take a look at our final result here I'm excited to show you how to pull off this effect let's get to it okay cool and like I said before you have the ability to upres to 4K and you can download it directly here from Runway which is super helpful now of course this is a very simple workflow but if you want to take your animations to the next level I highly recommend checking out our AI animation course at curious Refuge we've partnered up with storybook Studios to create an AI animation course that will show

you how to take the latest technology and integrate it into a professional AI workflow it's really fun I highly recommend checking it out now one quick tip from using this feature inside of Runway is to really try to get those driving videos to have expressive hand movements hand gestures and just those nonverbal Communications really help to sell the overall scene so I definitely recommend trying to prompt and generate driving footage that will give you those results okay thank you so much for watching this video of course you can always like like And subscribe here on

YouTube to get the latest AI tutorials and techniques directly here on the platform thank you so much we'll see you next time