Testing – The Renegade Coder

Chances are, if you're reading this article, you've written some Python code and you're wondering how to automate the testing process. Lucky for you, this article covers the concept of unit testing in Python. Concept Overview In the world of software development, testing is probably one of the few main pillars. After all, we need to be confident that our software works in a variety of circumstances. One way of testing software is to perform what is known as unit testing. Unit testing refers to the process of surfacing bugs in small "units" of code that can be easily isolated from a broader system. Common examples of units include functions and constructors. Of course, units might also refer to classes or modules. The idea being that the units are independent of the broader software system. In Python, unit testing often describes the process of testing functions, and this can be done with a pair of standard libraries: doctest and unittest. As the names imply, one tests code through documentation and the other tests code in a more traditional way. Let's take a look at both! To start, here's a simple function that recursively computes the Fibonacci sequence: def fib(term: int) -> int: if term in (1, 2): return 1 else: return fib(term - 1) + fib(term - 2) Using doctest, we can verify that our function works as follows: def fib(term: int) -> int: """ >>> fib(1) 1 >> fib(2) 1 >>> fib(3) 2 >>> fib(10) 55 """ if term in (1, 2): return 1 else: return fib(term - 1) + fib(term - 2) If you load this up into a tool like IDLE, then the following will run the tests and let you know if anything failed: >>> import doctest >>> doctest.testmod() TestResults(failed=0, attempted=3) Doctest works just like IDLE. It feeds your code into a REPL and compares the text result to the text you provided. If the texts match, then the code passes the test. On the other hand, the unittest library provides unit testing utilities that are more akin to traditional unit testing. You will need to create a new module which imports unittest and write a function for each test case: import unittest class TestFibFunction(unittest.TestCase): def test_first_term(self): self.assertEqual(fib(1), 1) def test_second_term(self): self.assertEqual(fib(2), 1) def test_tenth_term(self): self.assertEqual(fib(10), 55) if __name__ == '__main__': unittest.main() Naturally, if you run this file, you get the expected output. ... ---------------------------------------------------------------------- Ran 3 tests in 0.066s OK With these two standard libraries, you know just about all you need to know to start writing unit tests. In the rest of this article, we'll look at some questions you might still have. What Are the Benefits of Unit Testing? Surprisingly, unit testing is still a highly debated topic in the tech space. Not everyone does it. In fact, when I first started teaching in grad school, another grad student told me he thought unit testing was "antiquated." Apparently, he had been working at a company, which he claimed made use of "telemetry" instead. I am of the opinion that unit testing should still be employed in the vast majority of cases as a baseline for eliminating bugs in production software. In combination with code reviews and continuous integration, unit testing rounds out a pretty comprehensive set of techniques for ensuring code quality. One of the biggest perks of unit testing to me is continuous integration, especially as someone who maintains a variety of projects. With unit tests, I can quickly jump back into an old project, make some changes, and make a commit (i.e., save my code). When I push that commit to GitHub, the continuous integration system will run the tests for me, and I don't ever have to worry about the old "it worked on my machine" issue. That said, don't take my word for it. Do a quick search for "is unit testing bad?" in your favorite search engine. I found quite a few debates that raised a lot of cool points from "(paraphrasing) unit testing isn't useful in web contexts" to "(paraphrasing) you'll never make the same bug twice." What Are Good Unit Test Cases? While I've previously written about testing, it doesn't hurt to share my thoughts again. The first thing you should know is that there is no way to write perfect unit tests. The goal of unit testing is to surface bugs, not prove correctness. As a result, I can only suggest good testing strategies. In one of the software courses I teach, we tell students that there are three classes of unit tests you should write: routine, challenge, and boundary. Personally, I never really liked this framing, but the idea still stands. You should write a test case that targets what you would expect a user to do with your function (i.e., routine). You should also write a test case that targets the limits of what your function is designed to do (i.e., boundary). Then, you should probably write a test case or two to cover complex scenarios (i.e., challenge). Personally, I subscribe to a testing mnemonic that I learned in my first coding class: zero, one, many, first, middle, last. The first triplet refers to writing three unit tests for scalar data: one for the smallest input(s), one for the next smallest input(s), and one for some arbitrary input. The second triplet refers to writing three unit tests for sequential data: one for the leftmost element, one for an arbitrary middle element, and one for the rightmost element. With that said, your experience with testing will tell you what are good tests to write. However, I might suggest a few other things to keep in mind. For example, there are two different forms of testing known as white-box testing and black-box testing (well, more like three if you consider "gray-box testing"). White-box testing refers to writing tests that specifically target the implementation details while black-box testing is implementation agnostic. In my opinion, black-box testing is the way to go. Otherwise, you end up writing tests for coverage (i.e., testing that covers every line in the codebase) rather than behavior. The result is a bunch of test cases that are quickly outdated when the implementation details change. In nightmare scenarios, the tests might even rely on internal variables that are deleted or changed, causing many tests to break. In the past, I had written about how it might be okay to test private methods, just to be confident they work. I realize now that the desire to test every little detail in the code was really a reflection of my bad debugging skills. The tests were a crutch. Since then, I'm much more in favor of black-box testing to save the headache of brittle tests. Are There Alternative Unit Testing Libraries for Python? Absolutely! I'm sure there are dozens of third-party testing libraries. However, I have only personally used one other testing library: pytest. It's a handy little third-party library that makes writing tests a lot easier. You make use of the built-in assert statements, and pytest treats those as test cases. For example, the unittest code above could be written just as easily as follows: def test_first_term(): assert fib(1) == 1 def test_second_term(): assert fib(2) == 1 def test_tenth_term(): assert fib(10) == 55 Then, it's a matter of running pytest from the command line. Are There Other Kinds of Testing? As it turns out, unit testing is not the only way to test our code. Sometimes, we want to know how well our code scales. That's called performance testing. Likewise, in larger projects, there are a lot of moving parts. In those cases, when we want to check if two subsystems work together, we call that integration testing. In the case of integration testing, one subsystem might be a data processing application while another is a database. The purpose of integration testing is to show that the data processing application can handle a real database, not just fake data that we tend to use in unit tests (if tested at all). When all the subsystems are put together, we call that system testing. Typically, the purpose of system testing is to replicate the production environment as much as possible from the client's point of view. Putting it all together, we might imagine an app on a phone. The subsystems of this app might include some code for a user interface, some other code for managing a database, yet more code for hitting a couple APIs, etc. Each of these could be unit tested separately. Then, we might check if the UI can be populated with data from the database through some integration tests. Similarly, we might verify that the UI can be updated from a variety of API calls through some other integration tests. Finally, we might test the complete system by using a real database and real API keys to see if the application works. What Other Unit Testing Techniques Should I Know About? While writing tests that verify the behavior of a function will get you most of the way there, most unit testing libraries support a variety of addition features, such as: Executing boilerplate code before and after every test case, such as initializing some variables Executing boilerplate code before and after an entire test suite, such as initializing a database connection Executing a single test case on a parameterized list of inputs (i.e., parameterized testing) Executing tests with fake data for complex data types like databases (i.e., mocking) In general, I would consider these intermediate or advanced features of unit testing, but they come stock with almost every unit testing framework. An Absolute Unit Unit testing is a technique you can and probably should employ in your Python projects. You will, of course, need to follow other solid practices, like only exposing functions that you intend for a client to use. Otherwise, you won't be able to take full advantage of the benefits of unit testing. That said, even if you don't explicitly unit test your code using one of the many Python frameworks, you most certainly test your code in some way. I'm sure you run your code from time to time. That's a way of unit testing (i.e., verifying the behavior of your code). If you do that already, why not automate the process a little bit? The last thing I'll say is that my students cheer when I tell them I won't be assessing their testing. They cheer because they see testing as a laborious task that they don't want to do. It's like keeping your room clean; it's a chore. However, when those same students fall back on their old ways of testing (i.e., running the code a few times by hand), they tend to miss unit testing, and they often write buggier code. I don't know what the moral of the story is, but I stand by unit testing as yet another helpful tool in your software development toolkit. With that said, let's call it here. If you want to read more like this, check out some of the following links: The Haters Guide to Python How to Version Your Python Projects for Pip Poetry Is The Best Way to Manage Your Python Projects Likewise, here are some resources to help you learn Python (#ad): Finally, check out my list of ways to grow the site. Over there, you'll find links to our Discord, Patreon, and Newsletter. Otherwise, take care!