CS50P - Lecture 5 - Unit Tests

so [Music] [Music] all right this is cs50s introduction to programming with python my name is david malin and this is our week on unit tests up until now we've been writing a lot of code and you might have been testing your code by running your program and passing in some sample inputs and running it again and passing in some sample inputs or you might have been waiting for us to test your code instead but it's actually much better practice to get into the habit sooner rather than later of testing your own code using code of

your own in fact whether you're writing a personal project or working in industry it's very common nowadays to not only write code to solve the problems that you want to solve but also to write a little extra code to test the code that you wrote and that's what we're going to focus on today writing our own tests so as to be all the more confident all the more certain that the problems we have been trying to solve are in fact solved correctly so let's rewind a few weeks now to a program we wrote a while

back namely to to calculate uh numbers and specifically we left off with this calculator on trying to compute the power of a number like x squared or where x might be 2 or 3 or some other number as well well let me go ahead and resurrect that file by going into my terminal window here and running again code of calculator dot pi and let me go ahead and pick up where we left off way back when by defining a main function here and then in my main function i did something like this i said x

equals int of input and i asked the user what's x question mark and then i immediately went ahead and printed out something like x squared is and then i passed in as a second argument to print the result of calling a function called square passing in that value x now of course i haven't yet implemented the square function so let's define that as well let me go down a couple of lines and define square and it takes an argument recall a parameter that at the time i called n for number so i'll do that again

though i could technically choose any name for this variable and i recall did this i returned n times n and there were multiple ways to do this the squaring a number is multiplying it by itself so i could also use other syntax here but this is what we ultimately settled on and then recall that i ultimately called main in order to kick off the process of running this program so just as a test manually let me go ahead and run python of calculator.pi and hit enter what's x let's start with 2. all right x squared

is 4 i think that's correct so let's run it again just for good measure python of calculator.pi let's type in 3 for x this time x squared is 9. and i think that's correct and i might be feeling pretty good at this point and i go off and submit my code to a course or i post it on the internet for others to use but i haven't really methodically tested this code and it's not necessarily the case that it works entirely in fact i haven't really considered a number of corner cases i went with some

pretty obvious numbers like two and three but what about 0 what about negative numbers what about any number of other infinite numbers well we're not going to test an infinite number of inputs to this because the program would never halt but we should test some representative inputs ultimately but before we do that let's get into the habit of making sure that maine isn't always called let's adopt this habit again of doing if underscore underscore name underscore underscore equals equals quote unquote underscore underscore main underscore underscore only then should we execute main and i'm doing this

now proactively because i want to make sure that when i import my square function perhaps from another library from another file treating it as though it's a library i want to make sure that main is not just automatically called itself now what do i want to do from here now that i've modified this program as follows well let's go ahead and write a completely different program whose sole purpose in life is to now test this program so i've got my actual calculator in calculator.pi i've readied myself to call main conditionally so that i can safely

import one or more things from this file in another file well what should that other file be well by convention i'm going to create a file that's called test underscore and then because the thing i'm testing is this calculator itself let's call this file testcalculator.pi that's going to give me a new tab in which i can write a brand new program whose purpose in life is now specifically to test that program but really that program specific functionality built into that program is the square function let's focus on testing that function all right so how do

i access that function in this program well recall that i can import a function from another file as though it's a library of my own a so-called module so i'm going to do this from calculator import square i could go ahead and just import square itself but then i would have to prefix my use of square recall by saying calculator dot everywhere and it's just a little cleaner to just import the one function and now let me go ahead and do this let me go ahead and define a function called test square this two is

a convention if you want to test a function called square your function for testing should be called test underscore square or alternatively you could do square underscore test but i'll adopt this convention here now what kind of tests can we do well i don't dislike the tests i ran earlier testing x equals 2 and x equals 3 but every time i want to test my program previously i would have to do that manually and that's going to get tedious it's not going to be easy for someone else to test it and if i'm actually working

in the real world it would be nice if i could automatically have my program tested again and again by having some automated process run my own code so let's do that and take the human ultimately out of the equation so how might i go about testing the square function that i've now imported per line one well in my test square function why do i do this if the result of calling square of 2 does not equal 4 why don't we go ahead and print an error message because i know that in the real world 2

squared should equal 4 so if square of 2 does not equal 4 there's a bug in my program there's a bug in my function i've made a mistake so let me go ahead and print something like that so i or someone else knows 2 squared was not 4 for instance so i could print out anything here what should i maybe next test well let's do more than one test let's say if the square of 3 does not equal 3 squared 9 then let's go ahead and print out that 3 squared was not 9. so i

haven't done any more testing than i did earlier but i've baked those two tests x equals 2 and x equals 3 into my own code here so i can now run those tests automatically if you will now it's not enough to just define a function called test square i actually if i want to run this function need to call it somehow and our convention for doing that is the same as always in this function here in this file too let me define main and main's sole purpose in life is going to be to test square

and now at the bottom of this file as before let me go ahead and adopt my convention of if underscore underscore name underscore underscore equals equals quote unquote underscore underscore main underscore underscore then go ahead and call main so a lot of this is just boilerplate like we've seen this before defining a main function and calling a function to kick off some process now adding the conditional at the bottom of the file to make sure i'm only conditionally calling main just in case i import anything from this file elsewhere so let's see let's go ahead

and test my code now let me go ahead and run test calculator with python and hit enter and nothing outputs nothing outputs but i think it's okay i think no output is good because look at my test square function i'm not printing anything if all seems well so let's let's demonstrate as much by going back to my calculator and let me break it let me introduce a bug maybe i didn't even get it right the first time maybe my code originally looked like this i wasn't thinking i forgot my squares and so i thought that

the square of a number is n plus n instead of n times n so a reasonable mistake to make perhaps arithmetically let me now go back to my test calculator which i'm not going to change but i am going to rerun it python of testcalculator.pi i'm going to cross my fingers here but for naught i'm going to see immediately that 3 squared was not 9. now what is it well let's see when your tests fail how can we put our finger on what's wrong well it's a little interesting that i completely broke my square function

and yet only one of these tests is failing it looks like this test lines 9 and 10 is fine because i'm not seeing that output but of course these two lines this test is failing because three squared is not nine when i'm using plus so just to be clear here why is my function only partially broken just to be clear why am i seeing only one error instead of two even though the square function is now mathematically broken because two plus two is four yeah i mean it's as simple as that i just got lucky

that two plus two is the same thing as two times two so this is one of those corner cases and this is why it's good to be in the habit of not just testing one thing but test several and make sure you're covering your bases so to speak so i got lucky here and that explains why i'm seeing only one error even though the function itself is flawed but let me propose that there's another way we could do this because honestly if i extrapolate from this simple example running not just two tests but three or

four or 10 or 20 tests you can imagine that my god the code is going to get so much more complicated than the function itself i mean already look in calculator.pi the function in question is two lines long and yet in test calculator the code in question is five lines long like i've written more code to test my code than i actually wrote original code so the fewer lines of code we can write when testing code i think the more likely you and i are to do it because it's going to be literally a little

less work and just fewer opportunities for mistakes so what's another approach i can take here well it turns out in python there is another keyword that we haven't yet used which is this here assert a cert is a keyword in python and some other languages as well that allow you to do exactly that as in english to assert that something is true to sort of boldly claim that something is true and if it is nothing's going to happen no errors are going to appear on the screen but if you assert something in python and it

is not true that is the thing you're in asserting a boolean expression is false you're actually going to see some kind of error on the screen so let's go ahead and try this new keyword as follows let me go back to my code here and just to make it a little simpler let me propose that i i use this new keyword as follows let me simply assert that the square of two should equal four so i've changed my logic instead of checking for not equals i'm now asserting very loudly that it should equal four and

then on one additional line let me do the other test assert that the square of three equals equals nine and that's it no if no indented print i'm just going to assert more simply these two things that i want to be true well let me go ahead now with calculator.pi still broken i'm still using plus accidentally instead of multiplication let me go ahead now and run python of testcalculator.pi crossing my fingers as always but it's not going to go well this time a whole lot of errors seem to appear on the screen and if i

scroll up here for this traceback we'll see that the thing that failed was this line here a cert square of 3 equals equals 9. now unfortunately when you're using the assert keyword it's not terribly user friendly it shows you the files and the line numbers involved but it does show you the specific line of code that failed the assertion that failed so to speak it's now kind of up to you and me to infer from this wait a minute why is the square of three not equal to nine so it's not super user friendly but

honestly it was like half as much code for me to write it's just two lines instead of those previous four but notice this little remnant down here this was an assertion error and we have seen errors before we've seen errors before when we've made other mistakes in our code and in the past what was our solution for catching those errors how do we catch errors that seem to resemble this even though we've not seen this one before um yeah in python we can use the try and accept keywords to try to do something optimistically except

if something goes wrong do something else instead so this is a step forward and that i can at least catch this error but it's going to be perhaps a step backward and that i'm going to end up writing i'll admit in advance a little more code instead so let me go ahead and try this let me go back into my code here and instead of just asserting blindly let me go ahead as tolu proposed and try to do this first assertion except if there is an assertion error like we saw a moment ago then go

ahead and print out something more user-friendly that explains what actually failed 2 squared is was not 4 and let me go ahead similarly and try to assert that the square of 3 equals 9 except if there's an assertion error there in which case i'm going to print out more user-friendly three squared was not nine so i've taken a step forward but also a step back because now i have more code but i have at least introduced assertions and exceptions in a manner consistent with how we've seen in the past when something goes wrong you actually

see an exception raised well let me go ahead and run this version of the program now instead python of testcalculator.pi crossing my fingers all right it still failed because i'm seeing output but we're back to at least user-friendly output so that's at least progress in some way here but it's again more code than might have been ideal and in fact if we continue this further what if we actually want to add additional test cases here as well well it seems like we might end up writing way more code than would be ideal for instance i'm

testing two and three now i should probably test some negative numbers as well so why don't i go ahead and add in for instance let me go ahead and copy and paste this let me try to assert that the square of negative 2 equals equals 4 which should be the case mathematically and if not let me go ahead and change this to say negative 2 squared was not 4 and you know what let me go ahead and copy paste this again test another negative number just for good measure let's test the square of negative 3

which should equal 9 but if it doesn't let's go ahead and say that negative 3 squared was not 9 and just to think aloud here what might be another good value to test i've tried two i've tried three i've tried negative two i've tried negative three i can't try in infinite numbers but there's at least something that's a little different in between those values let's try zero zero is an interesting case too just in case something might be wrong and y zero i'm just going with instincts here right odds are positive numbers are generally going

to behave the same negative numbers might generally behave the same 0 might be a little anomalous there's no science to it necessarily but rather considering for yourself based on your own experience like what are the potential corner cases based on the function you're trying to test i'm trying to test something mathematical so i want to test representative values so let me go ahead and paste in one more try except block let's assert that the square of zero should equal zero and if not i'll say something explanatory like zero squared was not zero now if i

go ahead and run this python of testcalculator.pi and hit enter now i see multiple errors and this is interesting it's a bit of a clue because notice that some but not all of these assertions are failing the one for two squared is apparently okay as we noted earlier recall that two squared happens to be two plus two so that bug doesn't really throw off our test but it's a good thing we tested for three it's a good thing we tested for negative two and negative three because all of those tests caught this error the zero

test did not notice because zero squared is of course zero but zero plus zero is zero so we're getting lucky or unlucky there depending on how you you view the glass is half full or half empty here we at least by way of having multiple tests caught this mistake somehow so it would be nice though if we weren't writing so much darn code here right because notice what i've done i have try accept try except i have all of these assertions i have a main function i have this if conditional at the bottom of my

file i mean honestly who's going to want to write 31 lines of code now just to test a two-line function right no one's going to write test code like this if we're all writing so much more code to do the actual testing so people have solved this problem if you are in the habit of testing your code a lot or wanting to if i'm in the habit of wanting to test my code a lot if everyone else in the real world is in this habit of wanting to test their code why don't we create tools

that make it a little easier to do so and in fact there is a mechanism for doing this whereby we can use a tool that's popularly called pi test so pi test is a third party program that you can download and install that will automate the testing of your code so long as you write the tests but what's nice about this library and others like it is that it adopts some conventions so that you don't have to write as many lines of code yourself manually they do some of that automatically for you now this is

a third-party library there's other libraries for unit tests so to speak that is testing units of your code some of them come with python itself we're proposing that we look at pi test today because it's actually a little simpler than the unit testing frameworks that come with python itself and what do we mean by unit testing unit testing is just a formal way of describing testing individual units of your program what are those individual units they're typically functions so unit tests are typically tests for functions that you have written now what does this mean in

practice here well let me go back to my vs code here and let me propose that we simplify my test calculator significantly i'm going to go ahead and delete all of these tests which were which were accumulating to like 31 lines of code and let's see if we can distill the tests to their essence using pi test from my same calculator program let me still import square so i do still need that line of code so that i can test that specific function now i'm going to go ahead and define a function just like i

did before as follows i'm going to define a function called test square again by convention test underscore and the name of the function you want to test though it doesn't have to be that way and now i'm going to go ahead and make a few assertions i'm going to assert that the square of 2 should equal 4. i'm going to assert that the square of 3 should equal 9. i'm going to assert that the square of negative 2 should equal 4 and i'm going to insert that the square of negative 3 should equal 9 and

lastly for now i'm going to assert that the square of zero should equal zero so i'm still using the assert keyword as i introduced earlier and even though it was a little tedious to type those i mean it's only eight lines of code now and they're so easy to type it's not trying except in all of this wouldn't it be nice if something else someone else handled the try the accept the printing all of the standardization of actually running these tests and that's where indeed pi test comes into play per the documentation for pi test

which can itself be installed with pip install pi test which we've used to install other libraries in the past you can look at the documentation here for all of its formal usage but fortunately pi test is pretty user friendly as testing frameworks go and it actually allows us to dive right in by just running pi test on the code that we've written so if i go back to vs code here and look at my testcalculator.pi which notice has no main function anymore it has no conditional it has no tries it has no accepts it has

no prints it just has my few assertions pi test and other libraries like it are going to automate the process of running these tests for me and informing me on the screen whether or not any of those tests failed so let me go ahead and do this i'm going to go ahead and increase the size of my terminal window for a moment just so we can see more on the screen and i'm going to run not python as i've been doing i'm going to run pi test which again is this third party tool for running

tests in your code i'm going to run pi test of test underscore calculator so that same file i'm going to cross my fingers as always and hit enter and we'll see that ah something has failed now admittedly even though i do think you'll find that pi test is relatively simple to use its output at least at first glance is not necessarily super user friendly so what are we seeing here we'll notice at the very top of my window is the command that i ran after my prompt right below that is a single f in red

which means fail so not very encouraging i tried really hard here but fail is my grade on this program but let's see exactly what happened well if i look at this excerpt here under failures you'll see that test square is the function that failed all right that makes sense because that's the only one i wrote and you'll see here somewhat uh arcane output describing what the error was so what you're seeing here is the first line of output equals equals 4 which is fine there's no red error message below that so that one's okay but

this line of code here assert that square of 3 equals equals 9 pi test did not like that assertion because it didn't end up being true in fact per the red e at the start of this line you'll see that i'm effectively trying to assert that 6 equals equals 9. now where did the 6 come from okay wait a minute if my test involves this notice that where 6 equals square of 3 this is saying that because i've called square passing in a value of three it turns out its return value is six and of

course mathematically six does not equal equal nine so that's why this is failing now pi test is not as user-friendly as telling you exactly why the bug is there or how to fix it this is really just a clue to you what must be wrong what you're seeing here is a clue that the first test passed because there's no red error below that line of code but this test failed somehow or other your square function is returning six when passed in three instead of nine so at this point you sort of put your detective hat

on you go back to your actual code and you think about in calculator to pi how in the world is line seven of my square function returning six instead of nine and at this point odds are the light bulb would go off above your head proverbially and you would see oh i'm using addition instead of multiplication but what pi test has done for us is automate the process of at least pointing out that error for us and if i now go in and fix this let me go ahead and the light bulb has gone off

i change the plus to a to multiply now i'm going to go ahead and after clearing my screen i'm going to run not python but pi test of testcalculator.pi crossing my fingers again and now it's green and i see just a dot which indicates that my one and only test passed i'm good a hundred percent success with my test now after fixing that bug all right let me pause here and see if there's any questions so my question is what if a user instead of because we are taking input from the user what if the

user is somewhat malicious and types in a string instead of an integer or maybe he types in a float or some other data type yeah so what if the user like we've seen in past examples types in cat instead of a number when we're expecting an integer how do we test for something like that at the moment i'm admittedly not testing user input if i go back to my code here notice that my calculator function of course has the square function that we keep testing and retesting but notice that all of the user input is

currently relegated to my main function and admittedly as of now i am not testing my main function so there could be one of those bugs and in fact there would be because if the user types in a string like cat instead of an integer like two or three then line two recall would actually raise a value error exception so we've seen that before so when it comes to testing your code this is actually a good reason for having multiple functions in your program rather than putting all of your logic in just the file itself rather

than putting all of the logic in just main it's actually really good really helpful practice to break your ideas up into smaller bite-sized functions that themselves are testable and what do i mean here square is perfectly testable why because it takes as input a parameter called n and it returns as output an integer which is going to be the square thereof hopefully it has a well-defined input and a well-defined output it is therefore completely within your control in your test program to pass in those values now i will say if you want to test whether

square behaves properly when past something like a string like quote unquote cat we could absolutely do something like this assert that the square of quote unquote cat it's not going to equal something you can actually using different syntax assert that a specific exception will be raised so if we were actually going to go back into our square function improve it and deliberately raise an exception we could test for that too but for now i'm deliberately only testing the square function i'm not testing for specific user input but that's another problem to be solved other questions

now on unit tests do you use the you need to test to test a code for the cs50 check so check 50 is similar in spirit check 50 is a tool that we cs50 wrote that is essentially doing something like pi test for the evaluation of students code it is similar in spirit but think of check 50 as being an alternative to pi test if you will but it works a little bit differently but same idea pi test and unit testing more generally is a technique that is independent of cs50 and is something that you

can and should be doing on your own code both in or outside of this class how about one other question here on on on our uh unit tests my question is instead of writing 4 times like ss square of 2 is equal to 4 instead of that can we write if is it i is equals to in square brackets the numbers we want instead of writing four lines a really good question absolutely right now if i go back to testcalculator.pi it's indeed pretty manual i mean it took me a while to say and to type

out those several lines and you could imagine writing some kind of loop to just assert in a loop that this equals that that this equals that and so forth using a list or using maybe a list or a dictionary or some structure like that so yes you can absolutely automate some of these tests by not just doing the same thing again and again you can still use all of the syntax of python to do loops but generally speaking your tests should be pretty simple and in fact let me propose that we improve upon even this

design further because at the moment what's not really ideal when i run all of this uh when i run all of these tests when my function is buggy is notice the output that i got let me reintroduce that same bug by changing my multiplication back to addition let me increase the size of my terminal window again and let me run pi test again of testcalculator.pi so this is the version of my code now that has the bug again so i'm going to see that big massive failure where this failure has been displayed to me but

this is not as helpful as it could be right because i have all of those other tests in my code recall that i had what one two three four five separate tests and i'm only seeing the output of the first now why is that well if we go back to my code here you'll see that the first assertion that's failing namely this one here that assert of square of three equals equals nine the other tests aren't even getting run and that's not a big deal in the sense that my code is buggy so one or

more of them are probably going to fail anyway but wouldn't it be nice to know which of them are going to fail and in fact it's ideal to run as many tests all at once as possible to give you as many clues as possible to finding your bug so let me propose that we improve the design of my testing code now still using pi test as follows instead of having one big function called test square that tests the entire function itself with so many different inputs let's break down my tests into different categories and here

too there's no one right way to do this but my mind is thinking that i should maybe test positive numbers separately test negative numbers separately and test zero separately i could think of other ways i could test even numbers i could test odd numbers or maybe some other pattern altogether but separating this big test into multiple tests is probably going to yield more clues for me when something goes wrong so let me do this let me go ahead and rename this function to test positive initially and let me include in that function only those first

two tests let me then create another function here called test negative and in this function let me test only negative two and negative three then down here let me do one more def of test zero and i'll just run one test in there so i have the same assertions the same five but i've now divided them up among three separate functions what's nice about pi test and other unit testing frameworks is that all three of these test functions will be run automatically even if one of them fails the others will be attempted that means that

if one or two or three of them fail i'll have one or two or three clues now for helping me find that mistake so let me go ahead and again increase the size of my terminal window just so we can see more on the screen my calculator still has the bug using addition instead of multiplication let me go ahead and run not python but again pi test of testcalculator.pi crossing my fingers is always and now oh my god there's even more errors on the screen but this in itself is more helpful let's work through them

from top to bottom so under failures here in all caps which i know is not very encouraging to see failure when you're just trying to solve a problem but that's what these frameworks do under failures the first function that failed is test positive but here too we see the same clue as before the first one 2 the square of 2 equals equals 4. that one's fine it's not airing with any red errors but the next one is failing so i know that square is broken when i pass in 3. all right what about down here

it looks like unfortunately my test negative function is failing 2y well when i pass in oh this is interesting here now negative 2 doesn't even work so i got lucky with positive 2 but negative 2 isn't working so that's a bit of a clue but in total only two tests failed so notice at the very bottom this summary two failed and one passed what's the other one what was the third one test zero so test zero is passing these two are failing and so that kind of leads me logically mathematically if you will to the

source of the bug and just to be clear too if you have a lot of tests this little one line output is helpful even though also a bit discouraging fail fail and dot means pass so there the three tests just depicted graphically a little bit differently well let me rewind now and go back into calculator.pi let's fix that bug because let's suppose that i've deduced okay i'm using addition i should have been using multiplication all this time let me now after fixing the bug yet again let me go back to my big terminal let me

run pi test of test calculator up high hitting enter crossing my fingers now and dot dot dot means all is well 100 of my tests passed all three of them so now i'm good it doesn't necessarily mean that my code is 100 correct but it does mean that it has passed 100 of my current tests and so it would probably behoove us to think a little harder about maybe we should test bigger numbers maybe we should test even smaller numbers maybe we should test strings or something else the onus is ultimately on you to decide

what you're going to test but in the real world you're going to be very unhappy with yourself or someone else maybe your boss is going to be very unhappy with you if you did not catch a bug in your code which you could have caught had you just written a test to try that kind of input all right let me pause again and see if there's any questions now on unit testings with pi test um so if you wanted to test like someone suggested before user input as well as testing your function do you do

that within the same file or do you make separate files for different types of tests really good question you could absolutely make separate files to test different types of things or if you don't have that many you can keep them all in the same file at the moment i've been storing all of my tests in one file for convenience and there's not terribly many of them but we'll take a look in a bit in an example that allows me to put them into a folder and even run pi test on a whole folder of tests

as well so that's possible other questions on unit testing so i've got two questions um so a couple while ago you just used an exception called assassin i'm not sure what was oh yeah a certain error um what exactly does that particular error catch and my second question is does the assert keyword um stand out to the compiler and exactly tell them to insert this particular um line of code indeed the assert keyword we're seeing and the assertion error we saw earlier are intertwined so when you use assert and the assertion fails because whatever boolean

expression you're using is not true it's false an assertion error by definition of python will be raised so those two work in conjunction those errors those assertion errors are still being raised by my code here when any of these lines of code fail however pi test this third party library is handling the process of catching those exceptions automatically for me so as to give me the standard output so we started today's story by really implementing unit testing myself i wrote all of the code myself i wrote main i did my conditional i did try and

accept honestly it's going to get incredibly painful to write tests long term if you and i have to write that much code every time especially when our function is this small so pi test and unit testing frameworks like it just automates so much of that essentially pi test adds the try the accept the if the prints for you so you can just focus on the essence of the tests which really are these inputs and outputs how about time for one other question here on unit testing as well sir uh when we uh enter minus six

or minus five uh the uh square or square uh square root of that number comes up but when we put a 6.6 or 5.6 something like that integer then uh a line shows error so what's happening there so uh if i'm deliberately testing integers right now in large part because i only want pow to operate on integers and that might be conveyed in python's documentation or my own documentation for that function if we were to pass in something else like a float it turns out that floating point values in python in other languages are actually

very hard if not impossible to represent 100 precisely and so if you are trying to compare it against some other value they might be slight rounding errors as a result i'm just inferring from what you've described but i'm very deliberately now testing this function with only the inputs that i would expect it might indeed throw other errors if other inputs are passed all right allow me to propose that we consider what should happen if square isn't actually past a number for instance if i go back to calculator.pi and suppose that i or perhaps someone else

using my square function simply forgets to convert the return value of input from a stir to an int as by modifying line two here well now something's definitely going to go wrong if i type in a stir instead of what appears to be an in for instance if i clear my terminal here run python of calculator.pi and hit enter let's type in cat as our value for x and of course this raises now a type error y can't multiply sequence by non-int of type stir what does that mean well you can't do cat times cat

because indeed square is expecting that n will be some number but that doesn't necessarily mean that square itself is buggy but this does mean that if i expect a type error to be raised let's test for that too so that i know the behavior indeed works as expected so let me go back to testcalculator.pi and let me go and add a fourth test down here how about define test underscore and i'll call this tester because i'm going to specifically and deliberately pass in a stir for testing and i want to in spirit assert that passing

in something like cat to square will raise a type error but we don't use the assert keyword for that rather we need this let me go to the top of this file and let me additionally import the pi test library itself because it turns out there's a function in that library called raises that allows me to express that i expect an exception to be raised and i can express that as follows with pi test dot raises and then in parentheses i can pass in the type of exception i expect which is going to be a

type error in this case and now when do i expect that type error to be raised whenever i do something like calling square and passing in not a number but something like cat so now if i go back to my terminal window run pi test of testcalculator.pi this time having four tests i should see that all four now are successful all right let's now consider how we could test code that doesn't just expect numbers as input but actually strings and let me rewind us in time here in vs code to that very first program we

wrote a few different versions of in hello.pi that ultimately looked a little something like this i had a main function that prompted the user for the value of a variable by asking them what's your name question mark and then we went ahead and did something like hello open paren name passing that user's name into a function called hello now that function hello recall ultimately looked like this we defined hello as taking a parameter called 2 the default value of which was world and that function very simply printed hello followed by a comma and then whatever

the name that had been passed in and then we ultimately called main but for now onward i'm going to always add this if conditional if name equals equals underscore underscore main then and only then do i want to call main so that's essentially what this program looked like in its last incarnation how do we go about testing it well here again too i'm not going to test the user's input per se in may and i'm going to focus really on the module the module of code here that's of interest which is the hello function itself

how can i go about testing the hello function well unfortunately even if i start by doing something like code of test hello dot pi let me go about and start writing a test program i could import from my hello program a function called hello so a bit strange to see from hello import hello but notice that on this line here i'm importing from the module that is the file called hello.pi the function called hello and how do i go about testing this well if i have a function like define test argument like this well let

me see this so if i were to find a function like define test hello what could i do well i could call hello with quote unquote say david and then check if it equals what hello comma david so would this work this approach here if i've written a test called test hello that calls hello with an argument of david and then tests its return value just like we've done for our calculator with this work as written and let me go back to in just a moment the version of hello that we're testing so you can

see that function hello here's the test here is the actual code would this test now work any thoughts uh i think the problem is that in the first version in hello dot pi you're using the toe argument that you first declare when you declare the function instead of using the name okay i that is actually uh not a bug here so let me stipulate that in hello.pi this code actually does work as intended and let me go ahead and test it manually just to demonstrate as much let me run python of hello.pi typing in as

my name dav id and i see in fact that it says hello david if though i were to change this program and get rid of the name argument get rid of the name variable and just call hello again running python if hello.pi this time i'm not even prompted because i got rid of my input call but it does pre-behave as i expect it does say hello world so let me stipulate that this code in its current form is actually correct but my test is not going to work as i'd hoped and there's a subtle difference

between hello versus my pow function that expl versus my there's a subtle difference between my hello function and my square function that explains why might this test not work as intended uh it because it's not returning a value yeah exactly recall our discussion early on about functions functions can either return a value like my square function hands you back the square of some value or they can have side effects sort of visual artifacts that might happen on the screen like printing something out on the screen and by definition that's how print works notice that hello

it is short but it's implemented ultimately using the print function which does not return a value as i'm using it here it instead has this side effect of printing something onto the screen so it is not correct in my test function to check if the return value of hello equals equals hello david because again hello is not returning anything it's printing something that side effect but notice literally it has no return keyword unlike my square function which did so here's an opportunity too to perhaps change how i go about implementing my actual functions it turns

out that as your programs get more and more sophisticated more and more complicated it tends to be best practice not to have side effects if you can avoid it especially if you want your code to be testable and in fact i'm going to propose that we change my hello program to now work as follows let me go ahead and change this function to not print hello and then that name let me go ahead and literally return maybe an f string which will clean this up a little bit hello comma two close quote at the end

so my syntax here is just the familiar f string or format string it's going to return hello world or hello david or hello whoever's name is passed in is that argument but i'm returning it now i'm not printing it out so what needs to change up here well i could do something like this i could say something like output equals hello and then print output in my main function or i can simplify that because i don't really need that variable i could instead just do this i could still call hello but i could immediately print

out the result and this version of my hello program now is actually more testable why because these assert statements that we're using and we've seen thus far for our tests are really designed to test arguments into functions and return values they're from not testing side effects so if you're doing equals equals you're looking for a return value something that's handed back from the function so that's fine if i modify the design of my program now not to just print hello but to return the string the sentence the phrase that i want to construct i can

leave it to the caller that is the function who's using this hello function to handle the actual printing now what does this mean in my code well it means now if my hello.pi looks like this and hello is indeed returning a value in my test to low function i can test it exactly like this so let me go ahead and run pi test of test hello.pi crossing my fingers as always and voila one passed so i passed this test because apparently the return value of hello does indeed equal hello comma david well let's test the

other scenario what if i pat call hello without any arguments let's assert that calling hello with nothing in those parentheses similarly equals hello comma but world the default value let me now go ahead and run pi test of test hello dot pi and that too passes entirely but there too suppose that i had made some mistakes suppose that there were a bug in my code it might not be best practice to combine multiple tests in this one function so let's make it more clear what might pass or fail let's call the first function test the

default to this function and let's only include this first line of code and then let's go ahead and define another function like test argument to test this other line of code here so now i have two different tests each of which is testing something a little fundamentally different so now when i run my code it's still not broken if i run pi test of test hello dot pi enter i've now passed two tests and that's just as good as before but if i did have a bug having two tests instead of one would indeed give

me perhaps a bit more of a hint as to what's wrong questions now on this testing of return values when these return values are now strings instead of integers and why we've done this so my question is about a function inside the function can we test that too for recursion we we haven't seen if you have if you have a recursive function which we've not discussed in this class yes you can absolutely test those too uh by simply calling them exactly in this way recursion does not affect this process all right how about one more

question here on unit test before we look at one final example when testing our uh arguments like uh can we use something like a loops or inside of assets or for the values absolutely you can absolutely use a loop to test multiple values in this case for instance i could do something like this i could say for name in the following list of hermione uh say harry and ron i could then within this loop assert that hello of that given name equals equals say the format string of hello comma name and then run all of

these here at once by running again pi test of test hello.pi it's still going to be just one test within that function but if there's something interesting about those several strings that makes it compelling to test all of them you can absolutely automate the tests in that way with that said each of your tests should ideally be pretty simple and pretty small why because you don't want to write so much code so much complicated code that your tests might be flawed what we don't want to have to do is write tests for our tests and

test for our tests for our test because it would never end so keeping tests nice and simple is really the goal so that a reasonable human yourself included can eyeball them and just claim yeah that is correct we don't need tests for our tests all right how about one other feature suppose that we don't have just one test but many different tests instead and we want to start to organize those tests into multiple files and even a folder well pi test and other frameworks support that paradigm as well in fact let me go ahead and

test hello.pi using a folder of tests with technically just one test but it'll be representative of having even more in that folder i'm going to go ahead and create a new folder called test using makeder at my command line and then within that file i'm within that and then within that folder i'm going to go ahead and create a file called test hello dot pi within this file meanwhile i'm going to test the same things i'm going to go ahead and from hello import hello and i'm going to go ahead and define a function like

test default that simply tests the scenario where hello with no arguments returns hello comma world and i'm going to have that other function where i test that an argument is passed and in this case i'll choose an argument like asserting that hello quote unquote david equals indeed hello comma not world but david so in this case i've just recreated the same test as earlier but they're in a file now in a folder called test well pi test allows me to run these here too but to do so i actually need to create one other file

within my test directory i need to create a file called underscore underscore init underscore underscore dot pi which has the effect even if this file is empty of telling python to treat that folder as not just a module but a package so to speak a package is a python module or multiple modules that are organized inside of a folder and this file underscore underscore in it underscore underscore pi is just a visual indicator to python that indeed it should treat that folder as a package if i had more code in this folder i could do

even more things with this file but for now it's just a clue that it's indeed meant to be a package and not just a module or file alone what i can now do in closing is run pi test not even on that specific file but on a whole folder of tests so if i run pi test of test where the test is the name of that folder pi test will automatically search through that folder looking for all possible tests granted there's just those two in this one file but when i run it now with enter

i'll still pass those tests i'll still get a hundred percent and i now have a mechanism ultimately for testing my own code so whether you're writing functions that return integers or something else functions that have side effects that could be rewritten as functions that return values you now have a mechanism to not just wait for want someone like us to test your code and not just test your code manually again and again which might get tedious and you might make mistakes by not including some possible inputs we now have an automated mechanism for testing one's

own code that's going to be even more powerful when you start collaborating with others so that you can write tests that ensure that if they make a change to the same code they haven't broken the code that you've written all right that's it for this week we'll see you next time