What Linear Algebra Is — Topic 1 of Machine Learning Foundations

hi though this is John crow welcome to my machine learning foundation series this is the first subject in the series it is intro to linear algebra this is an interactive primer on the theory and practice of tensor manipulation in Python and here is a wonderful illustration of my puppy oboe by illustrator Hagley Basson's so in this intro to linear algebra we are going to have three segments we're going to talk about data structures for algebra then we're going to cover common tensor operations with tons of hands-on examples and finally the final subject in this subject is matrix properties we're going to kick off segment 1 in his video data structures for algebra all of the topics in here are what linear algebra is a brief history of algebra tensors particular types of tensors including scalars vectors simple manipulation of vectors such as transposition ways of characterizing vectors like norms and special types of vectors like unit vectors more special vectors basic orthogonal north normal we're going to tackle building vectors in numpy and other types of tensors as well as matrices tensors more generally in tensor flow and pie Torche the two most popular automatic differentiation libraries in this first video we're going to talk about what linear algebra is and just do a single flight on a brief history of algebra you'll have to wait for future videos for the rest of the content so what is linear algebra well first let's talk about what algebra more generally is so algebra is arithmetic that includes any non numerical entities like X so here's a simple example if we have this equation 2x plus 5 equals 25 well we can subtract 5 from both sides of the equation and then that will leave us with 2x is equal to 20 we can then divide by 2 on both sides leaving us with 10 is our answer and you can confirm for yourself that X must be equal to 10 because you can plug it into the original equation and when X is equal to 10 2x plus 5 comes out to 25 it's the only solution to this equation so that's what algebra is now to take it into what linear algebra is a bit more it isn't linear algebra if it has an exponential term so for example an equation that has 2x squared plus 5 well that isn't linear it's a nonlinear transformation and then a square root that's also a nonlinear transformation so this equation is also not linear outgrow to give you a really nice and tidy definition of what linear algebra is we could say that it's solving four unknowns within a system of linear of equations so let's talk about that idea of a system of linear equations multiple linear equations were resolving four unknowns like X across several equations simultaneously so here's an example let's say a sheriff has a car that travels at 180 kilometers and out a bank robber has a slightly slower car it goes 150 kilometers an hour but that bank robber gets a 5-minute head start on the sheriff so how long does it take the sheriff to catch the robber and what distance will they have traveled at that point for simplicity let's ignore in celebration traffic etc and we'll just assume that they're traveling in a straight line in one direction so we can solve this problem graphically with a plot we can note here that 150 kilometers per hour corresponds to two point five kilometers per minute and a hundred eighty kilometers per hour corresponds to three kilometers per minute you can plot things out here and so we have a plot of time along the x-axis the horizontal axis in minutes and then we have a plot of distance in kilometers along the vertical axis you can see here there's a green line corresponding to our bank robber going two kilometers per minute so that's a particular slope and our sheriff is traveling at a faster speed so it has more slope a slope of three instead of two point five corresponding to three kilometers per minute it's bitter to the half kilometers per minute and then we have our 5-minute head start to the bank robber accounted for by the gap here along the x-axis so what we're trying to solve for is this crossover point where the sheriff catches the bank robber so what time is it how many minutes have elapsed at that point and what distance have they both traveled at that point so you can see that graphically you could come up with a solution alternatively we can solve the problem algebraically to represent the bank robber you can have this first equation which is distance D is equal to 2. 5 times time and then a second equation representing the speed of the sheriff however we also take into account that five minute head start so we have a penalty of five for the sheriff now both of these equations are equal to D so we can actually set this term here equal to this expression here so 2. 5 T is equal to 3 t minus 5 then multiply the three into the brackets to expand out this side of the equation and then we start trying to isolate T by moving 3t over to the other side of the equation here so we subtract 3t from both sides of the equation when we do that now 2.

5 t minus 3t gives us negative 0. 5 t is equal to negative 15 and so now we can divide both sides of the equation by negative 0. 5 and that isolates T completely giving us a time of 30 minutes so now we know how long it takes for the sheriff to catch the bank robbery to figure out distance this is really easy now that we've solved for T we can simply plug that 30 into the T in either equation so for the first equation 2.

5 times T 2. 5 times 30 is equal to 75 kilometers and then we can plug it into the second equation 2 so 30 minus 5 is 25 and 3 times 25 is 75 kilometers of course we get the same answer because it is a system of linear equations now that was a nice neat and tidy situation where we had one solution however there would have been no solution if the sheriff's car were the same speed as the bank robbers so if they both were traveling at exactly the same speed forever then there would be no solution to this problem on the other hand we could also have an infinite number of solutions if both the bank robber and the sheriff were traveling at the same speed and have the exact same starting time so the slope is the same the start time is the same now they overlap at every time point these are the only three options in linear algebra so you either have one solution no solutions or infinite solutions it is impossible for the lines to cross multiple times so this is a key part of us being linear algebra in these systems of equations in a given system of equations there could be many equations there could be many unknowns in each equation in the example that I just showed you there were two equations and there are two unknowns however let's consider another example here where we are building a model specifically this is something called a regression model which I'll get into a little bit of the detail here in case you're not already aware of it in this model here we are trying to predict Hesperus so we have for a given house we have its price that we're predicting Y and then we also have a number of features or variables that we are collecting to try to predict that house price and in a lot of cases the more features that you have the more accurately you're going to be able to predict what you're trying to predict the more relevant features so some of the relevant features here might be distance to school or number of bedrooms and so on I have M features here and so II C and M these examples of features this can be an integer the number 3 distance to school could be a value like 2.