Midjourney Runway Gen 3 Turbo Full Course (Create Lifelike Cinematic AI Videos)

the era of AI films has started cinematic Singularity the full democratization of film making looms on the horizon now with Runway Gen 3 Turbo you can ideate at the pace of your imagination lighting fast we've come a long way since I produced a cinematic AI video for the first time using Gen 2 we are coming closer to Super photo realism and excellent coherence couple of weeks ago I published a mini AI film just to push the bond and see what's possible with AI tools today in this video I will show you a deep dive tutorial

on how I created the mini AI film a blast from the past in the process of generating this mini film I tested and tried many different things and accumulated a lot of practical ideas that I will share with you now so you can save a lot of time prompting tips I will share are based on the official Runway Genry prompting guide published and recommended by the oneway team this video consists of three main parts covering the three steps of the process with a detailed deep dial I edit chapters for your convenience I'll show you how

to quickly do storyboarding for your AI film then transform this storyboard into ultra realistic cinematic shots with consistent style and character using M Journey then I will guide you on creating cinematic videos using runways image to video mode controlling camera movement for the best results and making the most out of Runway J tre's powerful image to video engine we'll also compare the output quality of Runway Genry Turbo with the Genry base model let's dive in I started my workflow by writing a rough draft of a synopsis for example I gave my character a name and

high level description this will help us in the storyboarding process I choose a character a Native American young man named ilas I also decided on the setting time location and weather conditions film is going to be in the US Wild West era set against a snowy backdrop in the Rocky Mountains region for this ideation phase you can utilize CET GPT cloud or any other large language model to generate scenes based on a brief synopsis draft however I didn't need to do this as I already had my own Creative Vision I wrote 16 scenes nothing detailed

just a few sentences of ideas and some high level scenes some shots I imagine for this mini film for example Native American young man ilas walks the forest to face bandits in snowy Forest next I will take these 16 sentences and feed into C gbt with the prompt help me storyboard below mini film synopsis with 16 to9 aspect ratio I highly recommend specifying the setting location and time because without this information Dolly might produce images with inconsistent time periods then cat GPT generates 16 scenes for me using Dolly I use cat GPT for storyboarding due

to its excellent character and style consistency given that we are in the low Fidelity phase we don't need to be overly concerned about image quality these images aren't intended for direct use in the film instead they provide a holistic overview of my mini synopsis additionally they can serve as valuable reference in images while crafting prompts for M Journey one final step before finalizing our low Fidelity phase from CAD GPT we want visual descriptions of each storyboard image using this prompt I want you to visually describe every photorealistic image in the storyboard one by one these

visual descriptions will come extremely handy now that I have some concept images in my hand I will create ultra realistic cinematic images for my AI video using the atomic prompting method on Mid Journey but you can also use flux or any other image generation tool depending on your preference this image from Del storyboard seemed perfect for setting the tone of the mini film we begin with style exploration for our images to achieve the desired immersion we need to establish a consistent T throughout the series of images on M Journey one of my previous videos I

created this image with dark cinematic tones that I particularly liked I decided to include these elements in my style reference on Mid Journey website using the style reference option or paperclip icon using the automic prompting method I crafted this prompt and attached these sty references along with my storyboard image as a image reference our goal here is to establish the right tone before generating more images the resulting shot beautifully combines the doy image with the style references and from this point on I will use this as a style reference for every new scene I will

generate now I'm transforming the first image in our storyboard from low Fidelity to High Fidelity in order to do that I will take full advantage of image prompting alongside with style reference on Mid Journey first I added this image as an image reference to my prompt box I then strengthen the image prompt with a detailed visual text description from CAD GPT you remember that we asked CET GPT to provide us some visual descriptions I specified the shop type on me Journey because this saves a lot of time while converting this image to video I added

my parameters aspect ratio style roll and Image Weight image weight is extremely important I picked 0.5 value as Image Weight to balance my image prompt with the text prompt to influence final image equally I'm sharing on the screen some reference values for your convenience if you want a balanced approach I would recommend you to use 0.5 as a value if you want to maximize your image reference influence then I would recommend you to use threee finally I'm including the previously generated image as a style reference using the paperclip icon here I'm hitting enter here's a

quick tip after starting image generation press up Arrow key to bring back the prompt this allows you to further refine it without copy pasting your prompt by this I transformed my image to this using exact same method I'm going to transform the second scene on my story Port from this image prompt to this one by one I'm creating my scenes and you can see I'm not using any complex prompt most of the job is done by image prompting and style reference now it's time to design our character I likeed this character depiction from the storyboard

images so I decided to make it more realistic using this mid Journey prompt I wrote cinematic close-up shot of a Native American young man inside his rated home he feels a deep sadness with things he saw in Wild West times I also attached style reference I ended up with this image the resulting character image was satisfactory but not ideal for a character reference due to its side profile perspective that's why I decided to continue refining the image further this shot provides a clearer view of the character's face making it an ideal reference for directing upcoming

scenes I achieved to this by using the image I generated as a character reference I added my style reference for style and colors finally I edit this image as an image reference just to help me out with Poe the face of the image looks off but it's not that important because we have our character reference here and all it is actually the post I supported my image reference with a prompt specifically asking for a full body shot now using this image as a character reference I can direct the upcoming scenes much more easily now I

continue with the next scenes from my storyboard take a look at this shot cinematic medium shot of a Native American young man holding a revolver and aiming before shooting in snowy forest in Wild West times in a snowy scene in Rocky Mountains region I attached to this prompt my character reference and my style reference here's another example cinematic bird ey view of a Native American young man seeing a group of cowboy bandits in distance in snowy forest in Wild West times Rocky Mountains region same thing again I use my character reference and I use my

star reference but one thing you will notice which is important I also started to use style weight and character weight let's explain mid Journey character reference weight to better understand how to direct a scene with character reference we need to simplify the decision- making process it comes to a very simple question what changes for my character in the next scene if our character is in a similar scene wearing same outfit same hairstyle it same day essentially then it makes sense to use a High character weight value like 100 to make it more challenging to modify

the character's appearance through text prompts due to the strong influence of reference image however if our character is in a different scene with a different outfit hairstyle weather conditions in a different day for example then it makes sense to use character weight as a zero this will allow for more flexibility in changing hairstyles and outfits through text prompts while preserving facial features because you will realize that if you keep the character rate at 100 sometimes M Journey just keeps bringing close-up shots which makes it difficult for us to direct the scene because our character needs

to be in different situations here's another example a cinematic Native American young man in a dark Coal Mine extremely dark scene in Wild West times I have again character reference and style reference another one a full body shot with the same character a shot from behind walking in the snow a close-up shot holding a dynamite this one our character is actually running in the dark Coal Mine the final shot with the same character reference and in this one I also use the image prompt because I had this shot in my storyboard and I decided to

use that and end it up with this shot another example same character over the shoulder shot and you will realize that low character weights causing our character to keep changing outfits again if you want to keep the character's outfit consistent then you can use character weight as a 100 as a value another example with a full body shot sometimes I use two images of the same character as a character reference this helps us to strengthen the message strengthen the prompt and it produces much better results but you need to be careful because images should look

similar otherwise it can be confusing for Mid Journey okay quick tip what if I want to use same image for both style and character reference Al together so instead of adding two images we use same image for two different functions the interface offers three image prompt types character reference style reference and regular image prompt which can be selected by clicking on the corresponding icons in order to use a combination of a reference types the trick is you need to hold shift and choose options that you want to use for this image by this we are

saying that we want to use this character's face in the image as a character reference and we also want the art style of the same image as a star reference including shot types in our prompts will be immensely beneficial for the image to video process it enhances creative control by providing a welldefined first or last frame since we already covered the first shot we can concentrate Solly on cinematography and camera movement on runway for example here I use closeup as a shot type right in the beginning of my prompt as a power token if you

go back to automic prompting while producing this kind of mini AI film using shot type at the beginning as a power token is extremely beneficial and helpful in the next example I also use a image reference from my storyboard just to give me a little bit of a creative canvas and I combined it with a well-defined shot type young men seen from behind finally I added my sty reference and finalize the prompt with a 0.5 Image Weight to not to have any istrative effect on the final shot you can see it looks quite realistic similar

example with the next shot from my storyboard here transforming into this realistic shot in combination with camera control which is wide angle shot of a group of Bandit cboys so we are fully taking advantage of using image prompts and also we are taking advantage of using shot types I use this image as an image reference in combination with the prompt high angle shot of maps of an abundant mine same way I'm fully taking advantage of the image reference again and transforming it into a realistic shot and here's another example this is storyboard shot and this

is the final realistic shot with a prompt cinematic photo massive explosion and blast inside of an old abandoned Coal Mine Wild West times massive destruction and debris flying around particles everywhere we we have a great starting point in our storyboard and a great final realistic end result note that you don't have to always use image prompting in order to create great realistic shots you can do it without any storyboard you can do it without any image reference or image prompting simply in this shot I described cinematic high angle shots a lot of footsteps in wildwest

times in a snowy scene in Rocky Mountains with the tall trees I only edit my style reference for the colors and the vibe rest is only text prompt and our Creative Vision I got similar result in this one I was hoping for bird's eye view shot to show the bandits I only had a star reference I tored down little bit my style reference because I was getting way too many close-up shots and I ended up with this it's not necessarily Bird's ey view shot it's more close to a high angle shot but I can work

with this so you don't have to always use image prompts in fact I will show you an easy way to create your Cinematic Universe very quickly here's the prompt cinematic photo in the US Wild West era set against a snowy backdrop in Rocky Mountains and in Brackets we are adding landscape people architecture The Prompt structure with brackets is called permutations or batch prompting on Mid Journey it allows you to create multiple image generations with a single prompt combining this prompt with the chos parameter c will ensure that it's going to be a variation among the

generated images increasing chaos leads to more unexpected and diverse results I also added my style reference here in order to ensure that my universe is consistent with the rest of the images I generated but as you can see when I hit enter it will create three jobs total 12 grids this one will consist of architecture this with some people and this with some landscape that we can use as a b-roll so chaos values range from 0 to 100 lower chaos values around 4 to8 at variety while maintaining prompt adherence extreme values like 50 to 100

images can deviate significantly from the original prompt so I would really recommend you to stay somewhere around 4 to8 if I like this landscape I can just come here quickly hit where I strong and generate more landscape images similar to the first one we like with a consistent [Music] style final step is hand over to Runway few tips for a smooth Handover process during the mid Journey image generation process you will create numerous images to save time like the ones you plan to use in your film this allows you to easily filter and find them

later liking your own images doesn't affect your personalization codes making it a safe way to bookmark images you can also utilize mid Journey folders to organize your generated images you can download entire folders or all images you generated in a specific day also by holding to shift you can choose specific ones and hit download this organization will be crucial for the next step which is going to be upscaling for upscaling you can use new MID Journey UPS scalers where you can find actually here it's by far the easiest workflow with a decent quality for beginners

and it's the most budget friendly option for this project I use magnific upscaler because I appreciate the additional realism and texture it brings however this is a paid tool so you don't have to use the same one just a tip if you're going to use magnific make sure to maximize the resemblance setting otherwise you might lose all the effort you put into maintaining character consistency [Music] after uploading my mid Journey image to Runway I will set the camera movement to direct the scene based on this first frame if you don't add a prompt it will

generate a simple effect like zooming in in most of the cases keep in mind that you can select this image as either the first or last frame if you choose it as the first frame Runway will start the video from this Frame and extend it according to your creative direction if you select it as a last frame then Runway will creatively generate what happened before it's like a time machine where you set either the starting point or the destination and in the end you will arrive to this Frame as the last and I can tell

you Gentry turbo is lighting fast I think it's fascinating to be able to generate a video faster than writing a sentence with this new model Runway team deserves a round of applause and here it's ready so Gentry Alpha Turbo is fast but how about quality of gentry turbo I didn't realize a massive drop in the quality in fact in some outputs I saw Gentry turbo did actually a better job especially when it comes to body coherence the the accuracy of details I realized occasionally some more face changes changes in the color and things like that

but it's not really drastic considering how fast it is the output quality is actually be very much satisfactory for me Runway team also recently added this prompt presets you can see it here this helps especially to beginners with controlling the camera in a much more comfortable way for example if we choose cinematic drawn it automatically adds a prompt structure here where you can use and take advantage of instead of writing your own prompts take a look at this for example a closeup portrait prompt structure a Croom into a closeup portrait shallow depth of field over

the shoulder shot you add your subject and then you have a prompt ready to use and it's very easy to apply it's simply one button according to official prompting guide here is the suggested prompting for runway for the best possible output this is particularly for text to video for image to video we don't need this element of establishing a scene in image to video instead of describing what's in the image focus on describing the camera movement character and scene movement you would like in the output so common mistake one if you are using image to

video you don't need to describe the image itself no need to reuse me Journey prompts on Runway by the way you can find this prompting guide in the description below so when it comes to image to video I formalized this prompt structure based on official guide and based on my testings we have three elements we have camera Technique we have camera f follows X format and we have an event happening in the scene you can pick elements and formulate them together with colon as a separation you can also use comma I don't think there's a

massive difference so simply you can use all of these elements in a single prompt or you can cherry pick and use the ones that you feel comfortable with so if you don't know specific camera technique names you don't need to worry about it then instead of that you can use the simple camera follows X format which is very simple to use for example here I described the camera moment which is static camera and I also addit event happening in the scene it's just a description of event the camera is static man pulls the trigger and

shoots with the revolver this is a very simple example of using two of the elements the camera movement and event and making a cinematic footage here's another example the camera very slowly moves up this is the first part and second part is event happening in the scene which is m is looking forward so even if I wouldn't write the second part I think this footage would be still fine just with camera movement being described next example static camera so I described the camera moment I added colon and after that I'm adding the event man walks

with his cowboy boots another one the camera is static men are alerted so the man on the behind doesn't seem so alerted but we have this shot here this can be used as it's kind of like hm what's happening there so as you can see it's very simple the camera movement and event do you do you have to always add the camera movement so in this example I didn't add the camera movement I only added the revolver is shooting left and I know this whole shooting thing and fire looks really unrealistic but my whole point

is even if I wouldn't describe the camera movement just by adding the event we can make this scene work so as I mentioned you can also build your prompt with camera technique and event together in this example I wrote handheld as a camera technique I added column sign and after that I'm describing the event which is camera slowly enters the ruined house while snow continues so this is kind of like a nice combination of camera technique the camera movement and event all together described in a single sentence here I'm starting actually with shot type at

the beginning which is low angle shot and then I edit the camera moment and it's almost like creating a storyboard I started with the first shot and I continued in my prompt what's going to be the next step so I had this nice smooth camera moment you can build your prompt with variety of different elements there is no one single source of Truth it really depending on the image you have and it depends on the complexity of movement you want so this one starts with the camera technique over the shoulder shot and then I also

o describe the camera slowly follows the man on the horse so I built a combination of two things the camera technique together with the camera follows format it's a very good shot next one is similar I have tracking shot as a camera technique separation colon sign and after that the camera follows the man generally I think this camera technique colon and the camera follows X format I think this works quite well with runway in this shot I used a camera technique called snor cam I think this keyword is great because problem with AI video is

most of the time you will realize that it's very static and it's very difficult in AI video to move characters from one point to another and show them they are walking or running without losing much of the coherence in the image think this is maybe not perfect but snor cam as a keyword I think works great especially if you want to show that the character is on the Move similar example here you can see we have a running character another shot again with snor Cam edit as a keyword to The Prompt again we have snor

Cam for moving our character in this shot again we have camera technique over the shoulder shot we have camera movement the camera is following the man and additional we have also in the end an event man looks back with fear so this is like combination of everything we learned and of course the the man's face looks a bit funny but you got the point that it's possible to use all of these things in combination when it comes to prompting image to video one key takeaway from this video can be if you are confused about this

whole camera technique camera follows X format or the event and everything simply just use camera follows X format simply describe the camera movement and Runway will do quite a good job you can see in this example I just said can camera is slowly zooming out and and it already produced a quite a good cinematic result when it comes to camera techniques in the prompting guide you will find quite lot of different camera techniques that Runway already recognizes and is good with so you can refer back to the prompting guides for some of these camera technique

keywords now let's take a look at some of the common problems with Runway and their Solutions potentially problem number one why everything is slow motion and frankly this is really the case I strongly feel we need some kind of setting when it comes to speed control but for now until we have this setting you can use keywords like normal speed or 30X speed or high speed in order to accelerate your footage it's not always accurate so you may need to accelerate it in the post production but potentially it can be a good solution another complaint

we hear quite a lot is about colors colors are changing from my original input image I observed this also quite a lot with Gen 3 Alpha turbo for this you can use keywords like muted colors or low contrast this will be helpful to maintain the consistency of colors actually Runway is really really good with depicting emotions so if you write things like smiling while talking deep concern worry sad shy angry Runway will do great job portraying these emotions the next thing I want to show you is runways lip sync feature in order to use that

after generating your image you can click on Lip Sync here after clicking it will take you to this UI so you can write any sentence here that your character will say it will also add lip sync for you based on the sentence you write you can also record your own audio or you can upload audio file here for this example I will keep it simple let's say Ruan is a dish served cold and we can change voice here we have quite lot of different voices let's say we want to have Jack I hit generate lip

sync revenge is a dish served cold I think we needed to choose a different voice for this character but I think you got the point the cool thing about Runway is you can actually add some hand gestures and it will make your character do that for example I wrote The Prompt camera is zooming in man makes the heart-shaped gesture let's hit generate and it's ready look at that this is really so cool that you can actually direct the scene like this and look at the Hard gesture okay let's start with addressing the elephant in the

room runways pricing yes the runway team updated their pricing options with the introduction of Genry Alpha turbo the new model is priced at five credits per second of a video which is half the cost of the original gen 3 Alpha that was priced at 10 credits per second in my experiments while building this mini film I had an average of 2.6 tries before I was satisfied with the footage this is an average number if it's a professional client work the number of Trials would double or triple so I appreciate price reduction but still credits run

out fast that's why for more affordability I recommend implementing a low resolution preview before generating the full resolution footage as an optional step we may need this at least until these models are a little bit cheaper this would help with better cost control and significantly improve user satisfaction the last thing I want to show you is fixed seed so fix seat on Runway creates visually similar Generations almost like style reference in mid Journey it's unchecked by default so if you paste the previous output seat number here it will then automatically check the box and it

will create similar style and moment to the original output hopefully this video was TR helpful for you to explore cinematic AI video making process if you felt a spark of inspiration show your support with a thumbs up and join our jungle Community by subscribing for more mind-blowing tutorials