Software Testing Explained: How QA is Done Today

[Applause] on January 1st of mm the world almost ended well it did in some places a nuclear station in Japan sounded an alarm the Pentagon satellites stopped processing information and all over the world people became 100 years older digitally at least sometime in the 80s computer engineers discovered that software had a drawback it recorded only the last two digits of the date so the transition from 1999 to 2000 wouldn't make sense to it the outcome could be unpredictable from planes falling from the skies to life-support systems failing to prisons releasing criminals the apocalyptic event was

called the year 2000 or y2k problem as cult leaders were preaching about the world's end marketers were promoting survival kits and ordinary people were building bunkers millions of dollars were spent on programmers hoping they could fix the issue and they did most of us didn't notice any chaos those countries that decided to ignore the issue didn't suffer any significant damage either but all the engineering effort used for y2k wasn't for nothing the bug had one big benefit software testing became very popular software testing basics how QA is done today [Music] to understand how we arrived

where we are today let's travel a few decades back historically testing hadn't been a priority probably the only place where was taken seriously in the old days was NASA Project Mercury in 1958 signified the beginning of the space race in its 5 years of existence Mercury launched America's first satellite and sent Alan Shepard into space Gerry Weinberg was leading software operations for the project and didn't have the typical attitude to testing in a sense he was way ahead of his time so we formed as far as I know that the first real professional testing group

and existed up to that time programmers were expected well first of all they were expected to write programs that didn't have flaws on them Gerry's testing crew set the standard an industry caught up but not in the way it was intended and so that was picked up by lots and lots of managers all over the world and it made it hire people who they didn't feel were qualified to be programmers they can hire missed testers in the words of a computer science pioneer Edsger Dijkstra the industry scale too quickly at the dawn of software development

back in the 40s and 50s software was built by those who would end up using it and on the machines that would run the program but that was before engineering was commercialized in the 1960s IBM introduced system/360 a legendary mainframe computer built specifically to be modular and compatible with any task the cost and time it took to build or just as legendary around a thousand employees worked on the system for 10 years and the initial budget of 25 million dollars was raised to 5 billion programs took ages to build companies hired excessive numbers of Engineers

to speed up development but the pool of qualified programmers was limited and soon they ended up with low-quality bug-infested software this was also the reason why 2k happened in the first place people were still using legacy software developed decades ago by people were unconcerned about the distant future solving the crisis was on the minds of developers and researchers until the 90s in his famous article no silver bullet IBM system/360 architect Fred Brooks lamented how the growing complexity of computer hardware was disproportionate to engineering practices not only are there no silver bullets now in view the

very nature of software makes it unlikely that there will be any no inventions that will do for software productivity reliability and simplicity with electronics transistors and large scale integration did for computer hardware we cannot expect ever to see twofold gains every two years it was like software development had nowhere to grow turns out he was wrong in 2001 seventeen software development leaders created the manifesto for agile software development this new approach was a direct response to the traditional method known as waterfall that migrated from the old manufacturing practices and waterfall you work in stages that

strictly follow one another which is intuitive but risky if or more correctly when a tester finds a bug in the code design or even product requirements the project has to start all over again this also means that the engineering and testing teams don't work closely together agile is 12 principles all come down to a few core ideas embrace change stick to the shorter cycle and collaborate more yes open space offices likely became a trend due to agile recommendations testers are now the required members of the engineering team who maintain the product quality from the very

start now feedback exchanges instant and bug fixes happen at the time of coding also automated tests are way more widespread testing became more than a routine task of sorting through the code a new process deserved the title of Quality Assurance which also covers planning monitoring and control here's how agile teams are doing software testing today planning is something considered anti agile here's an approximate list of things that should be recorded in a test plan a document dictating the testing strategy for the product but agile test leaders are looking into simpler alternatives and test plans were

one of the things that were really annoying me about about Google they annoyed me at Microsoft who they've annoyed me they've always annoyed me this is James Whittaker former engineering director at Google and Microsoft and author of how Google tests software he came up with a hugely popular technique for writing test documentation called the ten minute test plan James defined three core elements that a test plan needs first are the attributes these are the adjectives describing the main purpose of testing what is testing supposed to check then the components what are we testing these are

the nouns describing pieces of code or features finally we document capabilities here you use verbs to explain what a user should be able to do then we never built a test plan in ten minutes but we did do we did discover the things that are absolutely important in a test plan and we threw away the rest even the minimalistic version of a test plan will be enough to start a second stage which is writing test cases and test scripts a test case is a detailed description of the task that would allow you to perform the

test and then determine if the program passed or failed say your task is to check what happens when a logged user tries to book a hotel room the test case will have information about the steps you'll need to take conditions that must be followed before you do the test and the expected outcome this would actually be an example of functional testing a type of test performed when we want to check how the software works in terms of features there are a few types of tests to know about functional testing usually has an input what to

do and an output what we should expect this testing can be done even by people who have minimal knowledge about how the system works from the inside which is why it's also called blackbox testing what else could you test using a black box approach usability for example you don't have to read code to understand if the system is easy to use there's also use case testing when you have to check if the software will be used as intended this would verify the work of UX designers blackbox testing will also be enough to check the system

stability and performance basically the conditions under which it would or wouldn't crash but often you need to test the quality of the code itself and its security how its smallest parts work and how these parts work together this is called white box testing and it's either done by programmers who develop this particular software or test engineers who know how to code not all tests need written cases sometimes tests are done sporadically to save time or simply to see what will happen if we improvise this is called ad hoc testing like a fire drill obviously this

should be done by a human tester because the machine can't improvise like a human but what can a machine do the hits and misses of automated testing before the agile manifesto emerged test automation was a badly underused technique but it makes perfect sense in the logic of continuous iterative improvement you write a test script once and run it simultaneously on as many devices or browsers as you want as many times as you want and for as long as you can leave your computer on here's an example of an automated test checking system response to invalid

logins and here are the reports it made using it however can be challenging only highly skilled testers can write test scripts so project managers have to be smart when distributing testing efforts while a machine can't be trusted with all tests in some situations it's a real lifesaver in regression testing for example regression testing aims to check how the system works after changes it's ineffective and totally unfair to make people perform dozens of the same tests every time you want to ship an update automation testing is also one of the pillars of continuous delivery the new

stage in the evolution of agile devops practices are also dependent on continuous testing embracing the long term thinking there's a monumental mechanical clock being built in the Sandy habló mountain range of West Texas right now sponsored by Jeff Bezos and designed by American inventor danny hills the so called 10,000 year clock will chime every thousand years as a symbol of long-term thinking and optimism while making calculations for the clock the design team was met with some software limitations Microsoft Excel didn't recognize the five digit number as the date among computer scientists the problem is called

the Y 10k bug it's been an issue for many long term analyses like the ones concerning nuclear waste if people living in the deck of millennium will still be using Windows 10 this may cause a great problem but most likely people won't have to worry about that test automation is the fastest growing area in QA today and a long term investment for a software project if we managed to replace most routine testing tasks with machines we can allocate more resources to making software more user-friendly or accessible or simply satisfying we can also ensure quality in

earlier stages and go beyond classic QA towards more analytical approaches like quality engineering if there's anything we know about testing is that the progress matters not perfection what shape will software testing take in the coming years is complete automation even possible let us know your thoughts in the comments below [Music]