Performance – The Renegade Coder

As this series grows, I often wonder if I'll ever run out of ideas. Wouldn't it be nice to just empty the list and start over? Oh wait, no! That's the topic of today's article: how to empty a list in Python. If you're short on time, there are two main ways I recommend emptying a list. First, you can overwrite the existing list with a new one as follows: `my_list = []`py. Alternatively, you can make use of the `clear()`py method: `my_list.clear()`py. Otherwise, there are a lot of other ways to get it done that might work best for you given your specific context. With that said, let's go ahead and talk about today's problem. Video Summary https://www.youtube.com/watch?v=vf1EVGCLIFc&t=10s As always, I like to condense all of the content below into a YouTube video. If you're into that kind of thing, take a look! I find it helpful to see all of the code below running, so you know it works. Problem Description Throughout this series, I've written a lot about lists. Partially this is because I've run into a lot of challenges with lists over the years, but it's also because people search up questions about lists constantly. As someone who has been trying to grow my site, I keep tabs on the types of Python searches folks make, and I try to cater my content to that. This one of those problems. To provide some context to this problem, let's image that you are building up a list of anime to watch. For instance, you're really interested in the Winter 2021 anime which featured shows like Attack on Titan, Beastars, and Re:Zero. The purpose of this list is to track all of the anime that you've seen. In other words, once you've finished the latest season of Beastars, you add the anime to your anime list as follows: anime = [] anime.append("Beastars") Eventually, you fill this list up with all the anime you've seen this season. For me, that might look like this: anime = [ "Promised Neverland", "Attack on Titan", "Re:Zero", "Beastars" ] With as awesome as Winter 2021 has been to us for anime, we have to move on as Spring 2021 is already here! That said, you don't want to mix up the shows you saw last season with the shows you're watching this season, so you decide to empty the list and start over. The question becomes: how do you do this? Fortunately, that's what we'll be tackling in the remainder of this article. Solutions With the problem description out of the way, let's get to the various ways that we can actually empty a list. To be honest, I was a bit surprised with how long this list ended up being. That's not to say that they're all good options, but I love to be as exhaustive as possible. At any rate, let's get to it! Empty a List Manually As with any article in this series, I like to take some time to "brute force" a solution. After all, it's helpful to think about how to approach these types of problems without memorizing the "best" solution. In other words, this is the type of solution you might come up with if you didn't know the proper idiom. So, how might we empty a list manually? Well, if we create a list, we can populate it using the `append()` method: anime = [] anime.append("My Hero Academia") anime.append("Your Lie in April") If `append()` adds items to a list, then surely there is a method that can remove items. In Python, there are actually two ways to remove items. First, we could use `remove()` which takes a value and removes the first instance of that value from the list. That would look something like this: anime.remove("My Hero Academia") anime.remove("Your Lie in April") Of course, to be able to remove these items, we'd need to know what they are. A simpler approach would be to use `pop()` which removes the last item from the list: anime.pop() anime.pop() Then given any list, we could clear it as follows: while anime: anime.pop() We can be sure this works because the loop condition only breaks when the list is empty (more on that here). In other words, this loop will continue removing items from the list until the list is empty. That said, this is by no means the way lists should be emptied. There are better alternatives that we'll cover in the sections below. Empty a List by Assignment Another way to empty a list is to replace the current list with a new one. You can do this using the assignment operator. For example, if we create a list and populate it, we can start fresh by reassigning our list: anime = [] anime.append("Tokyo Ghoul") anime.append("One Punch Man") anime = [] This works because variables are just references in Python. As a result, we can point our variable to some list, populate that list, and move the pointer to a new list without a batting an eye. Assuming no one else is referencing our original list, it will be cleaned up by the garbage collector. I personally use this solution because it's often the most practical in a loop. For example, it's common to have a set of nested loops where the outer loop initializes a list for the inner loop. Each time the outer loop is executed, the list is reinitialized to an empty list. That said, there are other practical options. And as always, we'll take a look at how they compare in terms of performance. Empty a List Using `clear()` If you prefer to use list methods, there is a really quick way to empty a list: `clear()`. Basically, this method can be called at any time on a list to empty its contents: anime = [] anime.append("Re:Zero") anime.append("Beastars") anime.clear() One reason you might prefer to use the `clear()` method over the alternatives is for readability. Though more often, it's because you want to clear the list for all references. For example, if for some reason you had a couple aliases for this list, `clear()` would empty the list for all of the references: anime = [] tv = anime anime.append("Re:Zero") anime.append("Beastars") anime.clear() # clears both tv and anime If, however, we were to reinitialize one of the lists, the original list would remain on the other reference: anime = [] tv = anime anime.append("Re:Zero") anime.append("Beastars") anime = [] # tv still stores ["Re:Zero", "Beastars"] That said, aliasing is generally considered bad practice, so this is an issue that should rarely come up. Empty a List Using `del`py Python is a bit weird in that it has keywords like `print` and `del`py which perform functionality. This is less common in Python 3+ as the trend has been to move toward a more consistent style (e.g., `print` became `print()`), but `del`py is still hanging around. It's purpose is to remove variables from the namespace: anime = [] anime.append("Madoka Magica") anime.append("Clannad") del anime `del`py is slightly different from the previous solutions because you cannot readily use the variable after deletion; you have to reinitialize the list: anime = [] anime.append("Madoka Magica") anime.append("Clannad") del anime anime.append("The Promised Neverland") # ERROR If, however, you need the variable, you can delete the contents using a slice: anime = [] anime.append("Madoka Magica") anime.append("Clannad") del anime[:] The difference here is that variable name is still intact, but the list is now empty. As a result, you can go back to using the list as needed. Empty a List Using Slice Assignment Another interesting way of clearing a list is to take advantage of slice assignment. This is a feature that got my attention a few months back because I had never seen it before. That caused me to write a whole article about how cool the feature, among others, was. As it turns out, it's a feature that keeps on giving because we can also use it to empty a list: anime = [] anime.append("Mirai Nikki") anime.append("Girl's Last Tour") anime[:] = [] Now, this is by no means a readable solution, but I think it's pretty! That said, I wouldn't expect to see it in production code. Same goes for our next solution. Empty a List Using "Multiplication" One of the things we can do with a list is multiply it by a scalar. For example, if we wanted to take a list that has one item and turn it into a list with ten items, we could do that using the multiplication operator: anime = ["Seven Deadly Sins"] anime * 3 # Creates a new list: ["Seven Deadly Sins", "Seven Deadly Sins", "Seven Deadly Sins"] Like with multiplication, we can actually leverage this scaling technique to empty the list: anime = ["Seven Deadly Sins"] anime * 0 # Creates a new list: [] Because this creates a new list, we need to save the results in the original variable: anime = ["Seven Deadly Sins"] anime = anime * 0 # Overwrites anime with [] Or, you might prefer the slightly cleaner version: anime = ["Seven Deadly Sins"] anime *= 0 # Overwrites anime with [] That said, this solution comes with the same issues as the assignment solution. If we have aliases of our list, the other references will not be changed. However, if we avoid aliasing, there shouldn't be any problems. Performance Given the various solutions above, it's time to determine how they compare in terms of performance. Usually, I would do this on the command line, but I figured we could celebrate 40 articles in this series by putting together a nice test bench for these types of articles. Here's what that test bench looked like at the time of writing. Feel free to check out the official repo for the latest source code: import timeit import seaborn as sns import pandas as pd import matplotlib.pyplot as plt import copy def test_bench(funcs: list, test_data: dict): results = _test_performance(funcs, test_data) _show_results(results) def _test_performance(funcs: list, test_data: dict) -> pd.DataFrame: num_tests = len(funcs) * len(test_data) print(f"> Collecting {num_tests} test(s)") results = [] i = 0 for name, test in test_data.items(): for func in funcs: print(f"\t> Test Progress: {i + 1} / {num_tests}") print(f"\t\t> Function Name: {func.__name__}") print(f"\t\t> Test Name: {name}") performance = min(timeit.repeat(lambda: func(*test))) results.append([func.__name__, name, performance]) i += 1 print(f"> Testing Complete") return pd.DataFrame(results, columns=["Function", "Input", "Performance"]) def _show_results(results: pd.DataFrame): print(results.to_string()) sns.set_theme() with sns.plotting_context("paper", font_scale=1.5): sns.catplot( x="Input", y="Performance", hue="Function", kind="bar", data=pd.DataFrame(results), legend=False, height=8, aspect=2 ) plt.title("How to Python: Function Performance Comparison", fontsize=16) plt.legend( bbox_to_anchor=(1.05, 1), loc=2, title="Functions", fontsize='12', title_fontsize='12' ) plt.tight_layout() plt.show() I won't bother digging through this right now; that's a topic for a different time. That said, I wanted to drop this here for anyone interested in how I setup my performance testing. Also, if you don't want to learn about my specific test bench, you're welcome to check out my article on Python performance testing. With that out of the way, let's talk about how I tested the performance of the solutions. For each solution, I created a function: def control(anime): anime = anime.copy() def empty_list_by_hand(anime): anime = anime.copy() while anime: anime.pop() def empty_list_by_assignment(anime): anime = anime.copy() anime = [] # Wouldn't actually work as a function def empty_list_by_clear(anime): anime = anime.copy() anime.clear() def empty_list_by_del(anime): anime = anime.copy() del anime[:] def empty_list_by_slice_assignment(anime): anime = anime.copy() anime[:] = [] def empty_list_by_multiplication(anime): anime = anime.copy() anime *= 0 # Also, would not work as a function Because these functions will be executed many times with the same list, I made sure that each function took a copy of that list first. This is somewhat annoying because copying the input list is an overhead that has to be executed every time the function is run. In other words, we are unable to test the performance of emptying a list directly. As a result, I made a control function which should give us a rough idea of how much of each runtime can be attributed to the copying. Of course, that still means that whatever results we get back will be slightly misleading. For example, if we were to test each function with progressively larger inputs, we might conclude that all of the solutions are linear—since copying is an O(N) operation (more on Big O, if you're interested). Regardless, I think this is good enough for our purposes. In the future, I'll try to avoid writing functions with side effects. At any rate, the next step is to setup the test bench. To do that, we can import the `test_bench()`py function. At which point, we can call it with a list of functions as well as a dictionary of possible inputs. For this, I chose four lists: an empty, a list containing one item, a small list containing a handful of items, and a significantly larger list. The purpose of choosing different sized lists is to see how the various solutions scale. Here's what the testing looks like in full: from test_bench import test_bench def control(anime): anime = anime.copy() def empty_list_by_hand(anime): anime = anime.copy() while anime: anime.pop() def empty_list_by_assignment(anime): anime = anime.copy() anime = [] # Wouldn't actually work as a function def empty_list_by_clear(anime): anime = anime.copy() anime.clear() def empty_list_by_del(anime): anime = anime.copy() del anime[:] def empty_list_by_slice_assignment(anime): anime = anime.copy() anime[:] = [] def empty_list_by_multiplication(anime): anime = anime.copy() anime *= 0 # Also, would not work as a function test_bench( [ control, empty_list_by_hand, empty_list_by_assignment, empty_list_by_clear, empty_list_by_del, empty_list_by_slice_assignment, empty_list_by_multiplication ], { "Empty List": [[]], "One Item List": [["Your Lie in April"]], "Small List": [["My Hero Academia", "Attack on Titan", "Steins;Gate"]], "Large List": [["One Punch Man"] * 100] } ) When executed, we get this lovely table containing the results of all 28 tests. IndexFunctionInputPerformance0controlEmpty List0.1849221empty_list_by_handEmpty List0.1972292empty_list_by_assignmentEmpty List0.2025883empty_list_by_clearEmpty List0.2128254empty_list_by_delEmpty List0.2362255empty_list_by_slice_assignmentEmpty List0.2546236empty_list_by_multiplicationEmpty List0.2237317controlOne Item List0.1959538empty_list_by_handOne Item List0.2523359empty_list_by_assignmentOne Item List0.21359710empty_list_by_clearOne Item List0.22260711empty_list_by_delOne Item List0.25450312empty_list_by_slice_assignmentOne Item List0.27148313empty_list_by_multiplicationOne Item List0.23923514controlSmall List0.19984415empty_list_by_handSmall List0.43178116empty_list_by_assignmentSmall List0.21717017empty_list_by_clearSmall List0.22726218empty_list_by_delSmall List0.26060019empty_list_by_slice_assignmentSmall List0.27769920empty_list_by_multiplicationSmall List0.24075721controlLarge List0.57188122empty_list_by_handLarge List6.05976323empty_list_by_assignmentLarge List0.61760924empty_list_by_clearLarge List0.66917225empty_list_by_delLarge List0.68441926empty_list_by_slice_assignmentLarge List0.68203627empty_list_by_multiplicationLarge List0.638110 Now, the cool thing about this test bench program is that we get this wonderful data visualization as well! Hopefully, it's clear that all of the solutions run at roughly the same speed—except one, of course. That giant orange bar is the loop we wrote to clear out the list. Beyond that, it looks like assignment is consistently the fastest way to "clear" a list, with multiplication right behind that. If I had to pick a solution based on these metrics, I'd go with assignment. That said, the `clear()`py method is probably a solid alternative. Feel free to use the solution that makes the most sense to you (and seems the most pythonic). Challenge With all that out of the way, let's talk today's challenge! Usually, when I make an article about lists, I try to scale the idea for the challenge. For example, if you can empty a single list, could you empty a nested list. Certainly, that would be a good challenge, but it's bit overdone in this series. As a result, I figured I'd try to come up with something unique. Today's challenge will be to write a function that empties a list only under certain conditions. Since we're dealing with anime, the conditions will be as follows: Empty the input list if it meets any of the following conditions:Contains any duplicate showsContains more than 17 showsContains shows that feature the word "Your" in the title Here's what that might look like for some sample input: to_empty_or_not_to_empty(["Haruhi", "Haruhi"]) # Empty! to_empty_or_not_to_empty(["Nagatoro"] * 17) # Empty! to_empty_or_not_to_empty(["Your Lie in April"]) # Empty! to_empty_or_not_to_empty(["Steins;Gate", "My Hero Academia", "A Place Further Than the Universe"]) # Do NOT empty! As always, I'll share my solution below and over on Twitter using #RenegadePython. Feel free to do the same! https://twitter.com/RenegadeCoder94/status/1384708141305171970?s=20 If you have questions about this challenge, feel free to hit me up on Discord. A Little Recap As with every article in this series, I like to close out with a list of all the solutions: anime = [ "Promised Neverland", "Attack on Titan", "Re:Zero", "Beastars" ] # Empty by hand while anime: anime.pop() # Empty by reassignment anime = [] # Empty by clear method anime.clear() # Empty by del keyword del anime[:] # Empty by slice assignment anime[:] = [] # Empty by multiplication anime *= 0 As always, if you liked this article, there's plenty more like it below. You can check out the next article in the series or read something totally different. Anything helps! If you'd like to take your support a step further, head over to my list of ways to grow the community. There you'll get access to Discord, Patreon, and YouTube. Likewise, here are some Python resources from the folks on Amazon (#ad): Otherwise, take care! See you next time.