Coding Adventure: Rendering Fluids

[Music] hello everyone and welcome to another episode of coding Adventures today we'll be waiting into the wonderful world of fluid rendering because about a year ago now more or less I worked on this little fluid simulation here which is particle based and so at the moment of course looks like balls we don't want lots of tiny little balls though we want a nice continuous surface and the idea that first comes to my mind for achieving this is matching cubes [Music] this is something I've used a lot in the past for creating terrain meshes out of

3D noise and so we should be able to reuse the code that I've already written to get something up and running pretty quickly here so to start with I've added this 3D texture to the simulation and at the end of each frame it just goes and samples the density of the fluid at regular intervals and writes those values into the texture which looks like this the resolution is perhaps a bit on the low at the moment but it does seem to be working at least now this is just a single slice through the middle of

the volume that we're looking at here but we can change the slice depth in this little viewer to see how it looks for other values so the matching cubes algorithm is just going to generate a triangle mesh along the boundary of this density field where it goes from being zero to non zero or we can specify exactly what value it should consider as being the boundary but since we've already implemented all of this in other episodes we have the enviably easy job now of just handing the density map over to this generated mesh function which

matches the cubes for us and we then just need to make sure that the mesh is attached to our renderer so that it shows up okay let's try that out and we can still see the particles here but let's maybe make them a more subtle gray color since they're no longer the stars of the show then we can enable the mesh renderer and run the simulation all right well that looks like it's working although I do think it'd be nice if the mesh was actually filled in along the sides that should be easy enough I

believe we can just jump into the Marchin cubes code quickly and there's a tiny helper function here for sampling the density map at a particular coordinate so if we detect that that coordinate is at the edge of the map we can simply lie about the density value there to tell the algorithm that we're at the boundary of the fluid all right let's see that goes and that is working nicely so I guess we should just try increasing the resolution at the moment it's 30 by 18 by 18 and I'm just going to double that and

reset the simulation so we can see it properly okay it's looking a lot more detailed already by the way I mentioned before how we can control this threshold value so just for fun let's play around with that quickly raising it to around 700 we can see that only the very dense IST regions are rendered and if we take this all the way down to around zero it will render pretty much everything which looks rather chunky the simulation is actually trying its best to keep the density the same everywhere but obviously it's not doing a perfect

job of it anyway I thought somewhere around 70 looked decent so let's put it back there and I want to try doubling the resolution once again so let's see how that goes and it's looking pretty good never mind as you can see the simulation has exploded this tends to happen when the frame rate drops too low and I have a pretty good idea of who to blame for it it's past me of course that guy is always making my life difficult this time he went to all the trouble of implementing matching cubes on the GPU

only to then waste who knows how many clock Cycles waiting to Lug the data back to the CPU in order to construct a mesh which then of course has to be shipped back to the graphics card again for rendering so I've been working on a new and improved rendering function to avoid all of this traffic basically after running the marching cubes we then assign the triangle buffer that it creates to a material with a custom Shader then we need to create these render arguments which store some information but most importantly the number of triangles with

that done we can then issue a draw call like this to avoid having to read any data back to the CPU so the idea is that on this draw material our custom Shader requests to know the index of the current vertex is working with and it can then manually read the position of that vertex from the buffer that we supplied okay this should be much more efficient than our old approach so let me switch over to our new version and of course it doesn't work at all actually strangely enough even the original version seems to

be broken now I think I've deeply offended the GPU with my questionable code all right after a bit of tinkering though I believe I have fixed the issue nope scratch that okay after a slightly deeper investigation I've just realized that how I had set things up with the marching cubes the vertices are not just positions but rather position and normal Vector pairs so I've adjusted the Shader here to account for that now and finally it works so we're able to render this at a reasonable resolution now it looks fairly smooth I'd say and I'm sure

there are still many optimizations we could make however I'm having some second thoughts as to whether this is really the approach I want to go with today because I'm not sure how to make this look like a volume of fluid rather than just the sort of shell around it I suppose that rracing the mesh would be the way to go but in order to have any hope of doing that in real time we'd have to construct an acceleration structure like the bvh I was experimenting with recently only we'd have to construct it every frame and

entirely on the GPU which makes things a bit trickier I actually have some papers bookmarked on this topic already but that's probably going to be an entire project on its own so it was fun to experiment with this but perhaps rather than generating a mesh with matching cubes we could try use the same technique I've experimented with in the past for volumetric effects such as clouds and atmosphere and draw the density field directly with Ray marching all right so I've set up a post-processing Shader now with a very simple Ray matching function here it starts

by calculating the direction we're looking through the current pixel and then tests for an inter section against the bounding box of the fluid simulation so that we know where to begin our journey next we have a little Loop to just much through the bounding box with a given step size and at each step we'll look up the density from our 3D map and multiply that value by the step size to get an estimate essentially for how much fluid we moved through on the step then summing all of those together gives us an overall estimate for

the total density along the viewray and let's see how that looks all right this is currently solid white but we can try decreasing this density multiply over here to get a more interesting result it is looking very sliced up at the moment though which means that our step size is probably way too high so I'll try reducing that as well and that is much better definitely lacking some color though and I think I'm going to just try working off of what I remember from the atmospheric rendering to start with it's probably not quite the right

approach to apply to something like water but I'm curious to see how it goes so back back in the code I've added a few lines to say that at each step we imagine some light from the sun reaching this point in the fluid and depending on the density here some amount of that light could scatter in towards the camera the exact amount is also determined by these scattering coefficients which we can tweak individually for red green and blue wavelengths of light technically it should depend on the angle as well but I'm going to ignore that

for this little experiment all right so that light that bounces towards the camera we'll have to travel through the fluid to reach it and along that path some of it will be absorbed or scattered away which we can model as an exponential decay here's a little visualization of how that transmittance which is to say the proportion of light that actually makes it through the medium will decrease as we raise our scattering or absorption coefficients okay so finally we just sum up all of this light that makes it to the camera and return that as the

result so let's head back to the simulation where we can now try tweaking our coefficients to get some color going and I'll just play around with these a bit here's a rather toxic looking fluid and here's one that looks pretty hot there is a rather big simplification being made at the moment though which is in saying that the light that reaches a point inside the fluid will always be pure white but we're not accounting for the fact that the light from the sun has to travel through the fluid to get you and some of it

will also be lost on that Journey so what we can do is just calculate how how much fluid exists between this point and the sun simply by marching through the volume like we've been doing here although with a larger step size to hopefully avoid igniting the graphics card once we have that value we can then just apply the same exponential decay to calculate the transmittance and that will then of course replace the constant white light that we were assuming before okay let's see how that looks so I'll start up the simulation and we are greeted

with a somewhat cloudy looking Cube here with some pretty pink fringes I think that's looking pretty weird but also kind of cool and I'm going to restart this quickly because I'm curious to see how it looks from below all right I want to try playing around with the colors again so I reset the coefficients and let's slow down time so that we can mess about a bit here and start Maybe by increasing the scattering of red light we you can see how the thin fringes of the fluid have turned somewhat reddish as more of that

wavelength scatters in towards the camera but higher scattering also means that more red light is lost of course and so when it has to travel further through the fluid we can see how we mostly end up with just a bit of blue and green shining through there anyway I'll continue playing around with these values a bit and I do think this looks quite interesting with a very ethereal Misty sort of feeling to it but it obviously doesn't really resemble a typical fluid Like Water for for instance if we want to start moving more in that

direction I suppose rather than having light scattering around all over the place we should focus more just on how it reflects and refracts so say we have some light traveling through the air and into a pool of water for reasons that I'd like to properly understand someday the light moves slow in water causing a little change in its direction which is of course what we call refraction we're not seeing that change at the moment though because I haven't implemented it yet for that we're to need to pull up Snell's law which tells us to consider

two angles here relative to the surface normal the angle of incidence and the unknown angle of refraction according to selus and others before him the sign of that unknown angle is equal to the sign of the incoming angle multiplied by how fast the light travels in water versus how fast it travels in air all right so I've just written up some very inefficient code for this here which takes in the direction of the light and the surface normal along with the two speed values and then converts the vectors into angles solves that equation to get

the angle of refraction and then converts that angle back into a direction if we now plug in the appropriate speed values roughly 299 million m/s for light traveling through air and about 225 million for light traveling in water we can see how the beam of light is now bent towards the normal as it's forced to slow down I ended up replacing these speeds with indices of refraction by the way which is just the ratio of light speed in a vacuum to its speed in a particular medium just because the numbers are a bit more friendly

to play with then also my horrible little refraction function was bothering me so I looked up a more elegant and efficient way of implementing it that works in three dimensions okay now we've seen light going from air to water but let's have a quick look at the other way around as well and in this case since the light speeds up when it exits the water we can see how the beam is bent away from the normal in fact at a certain point it will bend away so much that it exceeds 90° and the light will

no longer be refracted at all and instead be totally reflected Reflections are nice and easy we just need the angle of incidence to be the same as the angle of reflection but it does seem a bit bizarre to have this abrupt jump in the direction of the light and indeed we're still missing one piece of the puzzle which is the frel equations once again I don't understand the underlying physics of this stuff very well unfortunately but for now I've simply copied out the maths here which describes how surfaces become more reflective as you look at

them from more of a grazing angle the output is just a value between 0 and one with a zero meaning no light is reflected and one meaning that all the light is reflected so if we take one minus this value that would tell us how much of the light is refracted instead and using that we now no longer have that abrupt jump but rather a tasteful cross fade between the two directions all right so I have been working on this new version of the ray matching Shader here for a little while and this has a

loop for the number of refractions and Reflections that we want to resolve each iteration it starts by marching along until it finds the next surface which is to say the next boundary between air and fluid and we then reduce this transmittance value based on how much light is lost along that Journey next we calculate the directions of reflection and refraction as well as What proportion of light takes either of those paths and ideally we'd want to fully explore both paths but that could get pretty expensive and we saw that light anyway has a pretty strong

preference typically so what I've decided to do for today at least is to just choose the most interesting path by looking at how much lighter is going in that direction as well as how much fluid lies in that direction we can then do an approximation for the less interesting path so for example if we choose to follow the refracted Ray then we'll Now look up the light coming from the environment in the reflected direction using this light function here that light function takes in the position and Direction and just always returns plain white at the

moment but the idea is to later replace it with a simple environment that the fluid can receive light from anyway that incoming light is then multiplied by the transmittance along the reflected ray and added to our total received light with without testing for any more Reflections or refractions for the more interesting of the two paths though we simply set the ray up for the next iteration so that it can continue its journey and once the loop reaches its limit we can again just approximate the incoming light ignoring any further interactions finally we output the total

light of course although I've inverted it here just because I prefer the background to be black rather than white all right I'm very excited to try this out so let's give it a word and I crushed the GPU okay after rebooting and a few quick fixes we now have a rather strange looking Cub fluid on our hands let's see how it looks in motion that is not what I was hoping for at all I guess it's back to the bugging for me all right after a bit more poking around I found some issues with my

Surface finding function and my first attempt at fixing it actually made it a whole lot worse as you can see here but I think I might finally have got it working properly now at the moment I'm just taking a look at the surface normals to make sure that those all look correct and here's the normals after the first iteration of the loop and I think that's looking good we can also see the second and third iterations as well but at this point I'm mainly just enjoying the funky colors I can't tell what's going on anymore

I'd also like to quickly visualize which path we're actually taking on each iteration just to see if that's making sense so more green will mean we're looking at refractions while more blue means we're looking at Reflections all right on the first iteration we can see that it's almost entirely refractions since the reflections tend to just bounce off into empty space which is no fun except in places like the undersides of these droplets for instance where the light is reflected down into the fluid below okay let's take a look at the second iteration we can see

a lot more reflection in the mixia now since the light that goes into the fluid is reflecting back off the other side and we can see how at these grazing angles like along the floor the reflections are quite strong whereas the amount of light reflecting off the far end is very little since we're viewing that more headon all right It's tricky to know for sure but I get the feeling at least that the code is at last doing what I want it to do so I'll set it to just output the light again then we

can spawn in our little Cube once more and it is looking better this time I believe so let's run the [Music] simulation okay that is definitely a big Improvement let's have a look from another angle and I'll slow it down a touch so that we can see the details a bit [Music] better now I'm curious to try changing the refraction index of our fluid so let's bring it down so it's almost the same as air and then try try increasing it all right that's fun to play with and then I'd also like to mess around

with the colors a bit I think one thing that's bothering me slightly is the size of these droplets here in the middle I suspect it looked better if they were smaller so I'm going to try changing the density offset value hang on wrong direction we can't go too far because everything will just disappear of course but that's maybe now a little better at least okay okay so this is looking interesting already but as I mentioned before I'd like to make just a super simple test environment so that the fluid actually has something around it to

pick up some color from by environment I basically just mean a floor in a sky so I've started with the floor here just by recasting against a box and if we hit it then I thought it could be split into four colors based on the quadrant just so it's easier to see where our Reflections are coming from and to spice it up a bit I added a checkerboard pattern along with some random variation into the colors so over in the scene we can now pick our four quadrant colors quickly choose how dark the dark tiles

should be and also set the scale of the tiles here finally we can also choose how much we want to randomize the Hue and value of these tiles but let's keep that at least somewhat subtle then I also wanted to add in a little Cube that we could place inside the water to test the refractions and I made it cast a shadow on the floor by the way just by sending out a ray from the floor towards the sun to see if it hits a cube also we're now returning a sky color over here and

that's just using this very crude Sky function copied over from the first rate racing episode Let's test it out quickly so I'll swing around to face the sun which we can move about and that's all there is to it in the middle here of course you've probably already spotted our humble test Cube so I reckon this is efficient is a Bare Bones test scene and we now just have to update that light function here so that the fluid will receive its light from our environment one thing I'm not quite sure how to handle though is

what happens when light traveling through the fluid hits the floor or our test Cube for that matter I'm picturing these as diffuse materials so the light should be scattered in all directions but that doesn't seem super feasible in real time so I'm just breaking out of the loop here because I'm not sure what else to do anyway I also realized that the fluid should probably cast a shadow on the floor as well so I made another little addition to the environment function here which just tests how much fluid lies between the floor and the sun

all right let's try this out so I'll spawn in our fluid Cube and it looks a little underwhelming at the moment but that's just because the refraction count is at zero for some reason so let me bring in the settings and we can raise that up to one then let's try two and three and four seems to give very diminishing returns so we can probably stop there but that's looking quite nice I think so let's run the simulation now and see how that goes okay not too bad I'd say I want to try running this

again but this time I'll slow it down and we can zoom in and just take a bit of a closer look so un like our previous sort of Cloudy version this fluid doesn't have much color of its own it's really almost all coming from the environment which is definitely more successful in evoking a watery feeling anyway I'd like to try now grabbing that test cube of our and poking it into the water to do that I've just hopped over to the scene view now so that we can actually select stuff and I'll just drag that

Cube over here and elongate it somewhat and maybe angle it a little bit as well it's not set up to interact with the simulation at the moment I just want to basically confirm that the light is refracting properly so let's dip it in and we can indeed see that perplexing effect where the object comes out at a slightly different place than it went in I think this is working fine but I always find it a little bewildering thinking about which way things should bend so I thought i' quickly consult with reality just to make sure

that it checks out so I have a similar little setup on my desk here and and I'll just dip this Chopstick into the water and that does indeed line up with what we're seeing in the render so that's perfect oh and someone has just arrived on scene to drink my science experiment anyway one thing we haven't tried yet is going inside of the water but first just for a bit of variety I thought I'd change up the initial configuration of the fluid here and have it spawning in as two cubes on opposite ends of the

tank okay let's try that out wait what is this ply little puddle we've created I must have messed up how the particle quantities get divided up between the two spawn regions so let's try this again after a quick fix and that is looking more like it I'm going to freeze this for a moment though because I just want to zoom in on some droplets quickly to see how they look and I think it's neat how we can see a little upside down copy of the world in each of these we do also see quite a

lot of AR facts up close due to the limited step size but I'm getting sidetracked we wanted to go for a quick swim so I'll resume the simulation and then let's dive in no no retreat we are not welcome in this pool apparently okay so I've made some small tweaks to the code just trying random things basically to see if I can improve the performance a bit and let's see if we have more success this time around nope the water is still get gets very agitated when we approach it looks like my computer just can't

handle having the entire screen covered with fluid at least not without further optimizations let's take the easy way out for now though and just run it in a smaller window so it doesn't have to compute as many pixels and hopefully you'll be willing to let us in now okay at last we are successfully submerged and we can see some nice reflections of the underside of the water and a little along the edges as well then swimming over to the other end let's try looking up and there's a region above us where we're able to actually

see out through the water since the reflectance is very low there and we might as well pop our heads out over here I guess all right well it's been a lot of fun messing about with this Ray marching approach and I would definitely like to try and improve on it in the future I also think a non-real Time version where we can be more carefree and just Brute Force stuff instead of having to make approximations could be interesting too but for today there's one more technique I'm curious to try that is hopefully going to be

a little bit more friendly on performance the method I have in mind is a screen space technique that I first learned about from these GDC presentation slides it doesn't attempt to do fancy things like multiple refractions but I'm still very excited to see what kind of results we can obtain so rather than what we've been doing so far with constructing this whole 3D map of the fluid density this approach works much more simply by just drawing the fluid particles directly kind of like what we did in the first episode but rather than drawing each particle

as an honest to goodness spherical mesh with an absolutely exorbitant 128 triangles here it'll be much more performant to draw them as simple camera facing quads so I've been working on a billboard Shader to replace these little bows in there we start by getting the center of each quad from the position buffer which is supplied by the simulation of course and we also get the local vertex offset as determined by the quad mesh that we're drawing next we can find out which direction is upwards and rightwards from the point of view of the camera and

construct the world space position for the current vertex by taking the center point and then adding the two offsets along those camera axes so now if we run this we can see all these little quads flying about and no matter how we turn the camera they'll always turn to face us except that's not quite how I meant that all right well I got the maths a bit modeled up at first and on the second attempt it didn't quite work either I don't know what happened on my third attempt but the waves are now floating upside

down in the sky finally though on the fourth attempt the Stars aligned and the Billboards behaved sometimes easy things are hard but we can now move and rotate the camera around and see how they always align nicely with our view this is vastly more efficient for rendering than what I was doing before so here for example we can see a million particles being drawn to the screen and I'm sure it could handle many times that number but the simulation itself is of course the bottleneck here I did actually do a little bit of optimization work

on the simulation behind the scenes by the way so thanks to everyone for the helpful suggestions last time and I might make a short video about that in the future if you're interested in any case let's go into the fragment Shader of these particles now calculate the offset from the center of their texture coordinates and discard the current fragment if that length is greater than one that will transform the harsh quads into friendly little circles now currently these circles are colored based on their speed but the first step in this technique we're following is to

actually construct a depth map of the fluid so back in the Shader once more I'm now calculating the distance from the camera to the current fragment of the circle and returning that as the color instead and here's what that looks like this allows us to tell just by looking at the brightness of a pixel how far away the surface of the fluid is although one slight problem at the moment is that the surface looks rather holy an obvious way to get around this though is to just make the particles bigger and we probably will have

to do that but it does obviously make the fluid more Blobby looking and so one of the major steps in this technique is to try as much as possible to quite literally blur out these blocks okay so we need to experiment with blurring for a bit and to do that I've just set up a little test scene here with a rather grainy render of some dragons from my Ray tracing project now the simplest blur we could do is a box blur in which we just Loop over a bunch of pixels in a box around the

current pixel add up all their values and return the average it won't shock you too much I should imagine though to see that this has a bit of an unpleasantly boxy appearance so we could go with something a bit more tasteful like a gaussian blur the code for this is pretty much identical except that we use a waiting function to give more weight to samples close to the center and less at the edges how much less exactly depends on the value of this Sigma parameter in the weight calculation over here close to zero for example

we can see that only the middle pixel really has any influence so this effectively no blur at all but as we raise the value the pixels further away will be able to contribute as well although if we go too far the contributions will all even out and we'll basically be back to a boring box blur with this setup by the way if we change the size of the blur we'll always have to tweak the value of Sigma to account for that so it's more convenient to calculate it automatically from the blur size and some sort

of smoothness value like this where keeping the smoothness at one gives a pretty nice and consistent result and we can always tweak the smoothness if we're not happy trying this out on our Dragon image then we can see how it looks quite a bit better than the boxy blur both blurs run at about 3 frames a second though because doing all of those texture reads is quite expensive so thankfully there's a well-known trick we can use which is to Simply blur the image on just one axis so horizontally for example and then blur those blurred

values on the other axis this way we end end up reading far fewer pixels for the exact same result in this case we've gone from 3 frames per second to around 15 which is still pretty slow but hopefully we can get away with a smaller blur radius for our fluid all right so let's try it out we can take the fluid's depth map and then apply the blur to make it nice and smooth except the blur seems to be eating away at the edges which is not ideal that's probably due to the infinite depths where

no fluid is present so I'll just add a quick test to the code to ignore those values okay and now that the infinities are no longer feeding on the depths of the fluid this seems to be working much better but while the surface is nice and smooth now it is a bit concerning how fluid in the foreground such as these droplets here for example get kind of melted into the background let's nonetheless just forge ahead for now because what we really want from this blurry depth map is to construct a normal map so we can

figure out how light should reflect off the fluid this can be done with a simple finite difference method we just calculate the surface position at the current pixel using the depth map and then we do that again for a tiny horizontal step and a tiny vertical step the difference between those tells us how the surface is changing at the current point and the normal Vector will be perpendicular to that gradient so we can comput it using the cross product all right let's try it out and that is not looking terribly healthy I think maybe we

should just be returning zero here instead of discarding okay I'll try that again and crank up the blur it's looking okayish I guess but we might get a slightly more pleasing result if we treat the particles as spheres when we calculate the depths rather than flat circles so all the way back in the billboard Shader we can do a quick bit of maths to figure out the Zed value for a sphere given the X and y values and we just subtract that from the depth taking a look at that then we can see that indeed

our circles have inflated into spheres but they do seem to be constantly jostling for who gets to be drawn on top the problem is that I'm just outputting these depth values as a color which doesn't affect the actual depth buffer behind the scenes that's responsible for determining what gets drawn in front of what so we need to express our deep desire here to write to that buffer and we'll then be granted permission to set the value over here okay hopefully that will fix our problem and that is very much not fixed our problem I am

remembering now though that the engine's depth buffer is both nonlinear and also reversed on many platforms which is probably why this is turned out so strangely rumaging around online though I came across this little helper function for converting that linear depth into whatever exactly the engine is expecting so we can try this once again and and it looks like our little billboard spheres are working properly [Music] now however we do of course still have this problem where stuff in the foreground gets melted into the background due to the blurring which really messes with our normal

vectors so what is suggested in the slides I'm following is to add an additional waiting term to our gussian blur that considers the difference in depth between the pixels that it's blurring together let's experiment with this in our test scene quickly so I've rended out a depth map for this and we want to try to avoid having the different dragons all blending into one another so I've adapted our first unoptimized version of the gussian blur into this surface blur over here and this starts by looking at the depth at the current pixel and then for

every pixel in its neighborhood we calculate the depth difference that's then turned into a weight value here so that small depth differences result in a weight close to one meaning they'll be part of the blur while lar differences quickly go to zero so they won't affect the blur the total weight is then simply the product of the depth and gussan weights and we use that over here all right let's see how it goes well that looks pretty interesting but not quite what we're aiming for okay I've Tracked Down A minor mistake and I think this

should be working now so let's try increasing the depth factor and we can see how quite quickly the dragons have separated from one another while features inside of the dragons are still Blended together of course we could try cranking up the death factor a bunch more if we wanted and now the dragon's individual features are starting to become more distinct so this appears to be working well I'd say but it does have an unfortunate problem which is that our earlier optimization where we separate the two-dimensional blur into one-dimensional passes no longer gives a perfectly identical

result now out trusty slides do make note of the kinds of artifacts that occur from this dubious optimization but claim that they're not too noticeable once we've put everything together still I did some research and if it does end up bothering us there are a bunch of Alternatives that have been proposed such as curvature flow and the narrow range filter for example but let's just go with what we have for now so using this on our depth map results in a smooth out normal map that looks like this as we can see droplets in front

of the flow are no longer blending weirdly into the background so that's great news something that is a bit strange with our smoothing at the moment though is that the BL size is set to a fixed number of pixels and this means that stuff close to the camera will actually be smoothed less than stuff that's far away since pixels in the distance obviously cover more space in the actual world not to mention that the smoothing would also be affected by the screen resolution currently so what we can do instead is simply specify how many world

units we want the blur to cover along with the depth value from our depth map and that can be converted into the correct number of pixels that the blur should cover like this and looking at the result now we can see that this smothing is much more consistent at different depths okay let's test out this normal map with some simple shading so we can just take the dot product of the normal and the direction towards the light source and then to remove harsh Shadows we can remap this to be in the range 0 to one

and add inide extra environment light perhaps and here's how that's looking I think it's pretty decent so far but one thing I have noticed is that it looks a lot better from this sort of angle than it does from more of a grazing angle like this where we can see quite a lot of those artifacts showing up and this surface definitely looks more bubbly here than I'd prefer I think we should just roll with this for today but in the future I'd like to experiment with some of those other smoothing techniques and also maybe look

into anisotropic particle rendering which could help to reduce that surface bubbliness anyway for now let's start putting together our new fluid renderer so I've set up a post-processing Shader that reads in the fluid depth and first of all if it's not a valid depth then we can just draw the background for which I've pulled in the little test environment from our last experiment but otherwise we can use the depth to find where the vray hits the surface of the fluid look up the surface normal and use that to calculate the reflection and refraction vectors and

right now I'm just returning the reflected color over here so let's take a look at that all right next up is refractions but with our current approach there's no way of knowing how far the light has to travel through the fluid which is quite problematic so to fix that we're actually going to render all of the particles again but this time with an additive Shader that means that rather than getting the closest surface like before we're now adding up all of these semi-transparent particles to get a sense of the thickness of the fluid in other

words how much of it exists along each viewray of course since this is just rendered from the camera view not a full 3d texture like before we can't do multiple refractions but one refraction is certainly better than none also as the slides acknowledge drawing lots of overlapping transparent stuff is hardly the gpu's favorite task so it is recommended to render this at a reduced resolution and hope it doesn't cause unsightly ARS in any case in our Shader we can now look up the fluid thickness along the current viewray technically we actually care about the thickness

along the refracted Ray but we have to work with what we have so this then lets us very roughly approximate where the ray will exit the fluid and I've added a little parameter that we can just tweak until it hopefully looks good and now we can gather the environment light from the other side of the fluid we don't actually know how the light will refract on the other end since we don't know the normal Vector there so that's another limitation we have anyway the thickness value can also be used to estimate how much of the

refracted light will be lost due to absorption or scattering and so we're calculating here the remaining transmitted light as we've done before all right let's see how that looks so I'll try increasing the refraction multiplier first of all and let's have a look at this in motion I also want to try messing with the extinction coefficients so if we extinguish the blue light a bit we get this delightful yellow liquid and doing the same to Green turns it a reddish Hue finally if we turn up the red light Extinction we're getting more of a darkish

greeny blue which seems about right for some ocean water I suppose okay now that we've tested our Reflections and refractions separately let's just combine those colors together by blending between them based on that frel calculation that we implemented earlier and let's try it [Music] out all right a bit on the Blobby side as I've bomon before but I do think we're getting somewhere here let's perhaps add in some Shadows for this version as well that should be easy enough especially since our environment consists of literally just a flat floor so all we need to do

is render the thickness map again although not from the point of view of the light source and using an orthographic projection since we're saying that the Sun is far enough away that all the light rays hitting the fluid will be pretty much precisely parallel then when we're drawing the floor we can just take the current point on the floor and transform it through the shadow camera's View and projection matrices to finally calculate a texture coordinate that we can use to read the fluid thickness from the shadow map and the rest is just the same as

last time this did not work entirely as intended at first but after a bit longer than I care to admit I realized that I'd accidentally used the wrong camera on one half of the Matrix construction resulting in this nonsensical Shadow but after fixing that our shadow appears to be sensical now and it's looking very colorful this time around since we have some pretty high Extinction coefficients on the [Music] fluid I guess we should technically apply some shadowing to the C surface of the fluid itself not just the floor but I think I won't bother with

that for today what I'm more interested in right now is that we're getting some fairly big waves crashing about in our simulation but they look a little underwhelming I feel without that splash of white spray that one sees in the sea so I think a nice simple approach might be to just render the speed of the particles at the same time that we're rendering the depth and those values would then be smoothed out as well when we apply our surface blur which we can see over here my cunning plan is to then just make a

tiny tweak to the Shader such that the fastest moving regions of the fluid will be drawn in white so I've gone ahead and done that quickly and let's see how it looks all right that is unfortunately pretty bad so I've been searching about a bit and I came across this paper called unified spray foam and bubbles for particle-based fluids which sounds promising one of the things it talks about is how we can look at the velocity difference between nearby regions of fluid to determine where strong impacts or turbulence is taking place which can cause air

to become trapped and mix with the water forming white water and we'll want to spawn in extra particles for this here so that we're not limited by the resolution of the simulation itself to begin with then let's define a white water particle as having a position a velocity and a LIF time after which it will dissolve away and we'll want a buffer to hold all of those as well as a temporary buffer here to help with compaction in other words once a particle disappears we want to free up that spot in memory for a new

particle to spawn in but we want to make sure that all the empty spots are together at the end otherwise it will be a nightmare to keep track of them so here we'll need just two integers one to keep track of the number of particles alive or spawned in at the start of the frame and want to count how many survive to the next frame all right then for each fluid particle we can calculate a random seed based on its position in the current time for example and just as a test say there's a 0

1% chance of a white particle being spawned over here in which case we increment the number of active particles and assuming we haven't exceeded whatever maximum we choose to impose we can initialize the particle to just move upwards at a random speed then we can have a kernel for updating our white particles here just moving them based on their velocity and decreasing their remaining Lifetime and so long as that lifetime is above zero we'll copy the particle over to the compacted buffer of survivors we then just need one final kernel that simply copies the survivors

back into the main particle buffer and sets the number of active particles equal to the number that survived all right so trying this out it does appear to be working and by the way I've just used the same billboard technique for drawing these as we used for the fluid particles with one small adjustment that the scale decreases with the remaining lifetime all right with that up and running let's take a closer look at this equation from earlier so for each fluid particle we're going to Loop over all of its neighbors and sum up the magnitude

of their relative velocities so the more different the velocities are the higher this value will be but particles that are moving away from one another are not going to trap any air so we have this little Dot product here which scales the contribution Down based on how much the two particles are diverging all the way to zero if they're moving in complete opposite directions then finally each contribution is weighted by distance so that's one if the particles are squished right on top of each other decreasing linearly to zero when the distance reaches the radius of

influence defined in the simulation all right so I've translated this maths into Cod here quickly and I've just snuck it into the inner partical Loop of the pressure kernel since all the data we need is conveniently available there the idea then is that we'll take this final summed up value and remap it to be in the range 0 to one based on a pair of minimum and maximum parameters that we'll be able to tweak to our liking I'm doing that remapping over in a debug Shader for the fluid particles at the moment just so we

can easily play with the parameters and output the result to the red channel of the particle color here to see how it looks so so the particles are all completely red at the moment well pink rather I suppose since the base color is blue but let's try tweaking our Max parameter now and the minimum as well then let's focus on where this wave is about to come crashing down and we can see how it's correctly picked up on that strong impact there and also the sort of swirly vortexy region in here so I think that's

looking decent already but another factor that the paper considers is the kinetic energy of the fluid so essentially the squared velocity of each particle and I've quickly tweaked the debug Shader to show that in the green Channel now let me actually get rid of this constant blue color so we can focus on those values now what we're most interested in are the yellow regions where the two factors overlap because we're going to be multiplying them together to get a final spawn Factor telling us where the white particles should appear and let's actually visualize that final

spawn Factor here just is a grayscale value all right let's try actually spawning some particles in these regions now so I've moved the calculations from the debug Shader into the simulation and the spawn factor is also multiplied by a spawn rate parameter null as well as by the time step because we don't want the spawning to be affected by the frame rate one other little thing to help with that is to say that the number of particles we spawning is equal to the integer part of the spawn Factor factor with the remainder acting as the

probability of spawning an extra particle then the paper recommends randomly placing our white particles in a little cylinder around the current fluid particle with their velocities inherited from the fluid particle but jiggled a tiny bit by their position in the cylinder and here's my implementation of that there's not really anything interesting to say about it though so let's just go ahead and give it a well wow okay that is a lot of spray it is a bit difficult to tell if our spawning is working well or not if the particles just fly off like this

so we'll want them to be affected by gravity of course and to collide with the bounce of the simulation as well I suppose so here's some quick code to do exactly that and I also added a simple drag ter to just slow the spray down a bit in the air okay let's set this off again and it's suddenly looking more reasonable than our previous attempt so I feel like we're heading in the right direction what would be nice though is instead of just falling down to the bottom of the bounds if spray hitting the water

could then become either a bubble beneath the surface or a bit of foam floating on the surface so the paper recommends a nice easy approach for trying to classify bubbles foam and spray which is to just count up how many fluid particles are in the neighborhood of our white particle which we of course already have the infrastructure for doing at least some mod efficiently then if there are lots of fluid particles around the white particle is probably inside the fluid and should be treated as a bubble whereas if there are very few fluid particles nearby

then it's most likely flying through the air a spray finally if you're neither spray nor bubble what else could you be but foam now just so we can see what's going on as we tweak these thresholds I've stored the detected type in the particle data so that over in the particle Shader we can red in and draw Bubbles as red spray as green and the foam as blue all right let's try it out and oh jeez what have I done okay I just forgot to update the size and stride of the particle buffer after adding

that debug variable so the data was getting all muddled up with that fixed though we can now see our debug colors well it's all Bubbles at the moment so let's bring in our parameters to play with and let's maybe try increas ining the bubble threshold to start with meaning that more surrounding fluid is required in order to be classified as a bubble this is a little nebulous of course but I guess something like this is maybe okay let's then try increasing the spray threshold and we can see our spray popping out in green okay I

think that looks reasonable enough it's never going to be entirely perfect of course but for the most part the green spray seems to be in the air the blue foam seems to be touching the surface and the red bubbles will they're mostly hidden below the surface of course but the ones we can see at least are kind of in amongst the fluid particles so I think it's working fine all right we now just need to give each of these a different Behavior spray has gravity and air resistance already but both Bubbles and foam need to

flow along with the fluid and Bubbles in particular should of course rise to the surface so over where we're counting the number of nearby fluid particles we can at the same time add up all of their velocities just like so and then if we've decided we're dealing with foam at the moment we can let it be carried Along by the fluid simply by setting its velocity to the average of the surrounding fluid particles we'll also have the remaining lifetime atck down over here to simulate the foam dissolving over time for bubbles we can define a

buoyancy Force to counteract gravity and propel them upwards and we also have them accelerate to match their velocity with the surrounding fluid then lastly spray is just the same as before so let's run this simulation to see how this behaves okay not bad I think I would like to be able to see the bubbles properly though so I think what I might do is just turn off the fluid particle display for a moment here and let's try this again we can obviously observe the bubbles a bit better now and I'm no expert in bubble Behavior

or anything but this looks sensible enough to me I suppose I think there's just a few small tweaks I'd like to make overall firstly the spray feels like it's spraying perhaps too aggressively at the moment especially with the big impact near the start so I've just tried turning our linear drag term into a quadratic drag over here meaning it'll be slowed down with the square of its velocity and then also when the simulation starts up the velocities take a moment to settle from the initial spawn State since they might be too tightly packed together or

not quite tightly enough in this case it seems and I don't really want to spawn in spray or whatever from this so I'm just going to hackly fade in the spawn rate over the first half second or so all right let's try out these two changes quickly and is that looking better I'm honestly not sure but let's say yes so it feels like we made progress one last thing I want to try is that currently all of the foam dissolves at the same rate but the paper mentions how large collections of foam tend to be

more stable than smaller ones meaning they dissolve less quickly now actually detecting large clusters would be a bit of a pain and computational overhead so it's suggested to Simply scale the Lifetime by the current spawn Factor the idea being that if lots of particles are being spawned in one spot there's a decent chance that they'll end up in some sort of Clump together to try this out I've sent the spawn Factor value over to our particle Shader and we can then display particles that came from high spawn factors in green and from low spawn factors

in blue so what I'm hoping to see is hang on I did the thing again okay as I was saying I am hoping to see clusters of foam appearing in green and then the blue particles being more scattered about I suppose but is that actually happening it's looking kind of just random to me I'd say let's perhaps try setting the lifetimes based on the spawn Factor anyway and I'll do one version of the simulation where a high spawn factor means a long life and one where it's inverted and a low spawn factor means a long

life so that's playing out now over here at the top we have the normal version and on the bottom is the inverted case so we want to see clusters of green remaining at the top and more random scatterings of blue remaining at the bottom I don't really feel like we're seeing that so well at the moment though so I'm probably doing something wrong or maybe this simulation isn't large scale enough for this idea to really work but I'll need to experiment with that more in the future the paper also describes a way of detecting wave

crests and spawning more particles there so there's plenty still left to try but let's Pace ourselves a little and for today I just want to end by seeing how our current implementation of this looks when composited with the fluid rendering so I'm simply drawing the white particles as white particles again now and let's see how it goes [Music] okay I think it definitely looks much more dramatic now with the addition of the spray and foam so that's nice I also like how it gives one a better sense for the motion of the underlying fluid this

was about 300,000 fluid particles and 500,000 white particles by the way and I quickly want to try running this again but with those numbers both doubled and this is now really pushing the limits of what my computer can handle in real time but it does look a bit better I'd say so I'd say I'm pretty happy with the start we've made here today but there is certainly plenty room for improvement in all aspects of the rendering not to mention the simulation itself it is a lot of fun to work on this though so I'll definitely

be revisiting it soon enough to try and make some improvements and I of course very much welcome any all suggestions you might have on that front there's also other interesting effects such as cor sticks and godr for instance that I haven't even considered yet and actually on that note I wonder what happens if we try to go inside the fluid in this version so let's dive in and see all right that looks very not good we're running into the limits of the smoothing and just getting these giant reflective balls popping in and out of existence

so it' be good to find a way around that if possible I would still also like to experiment some more with the ray matching Shader or a new Ray traced solution perhaps because I think it'll also be fun to try and aim for something more realistic even if we have to give up the real-time aspect of the rendering for now though I think that that is all so thanks for watching and for all the other kinds of support that make these videos possible all right until next time cheers [Music] [Music] [Music] [Music] for