The Renegade Coder – Code First. Ask Questions Later.

One thing I've noticed as I continue to write these Python articles is that a lot of problems seem to be universal. For instance, this article covers a question I've seen a lot of folks ask: how do you iterate over multiple lists at the same time in Python? In fact, I've even asked this question myself, so I decided to document a few solutions to it. Luckily, looping over parallel lists is common enough that Python includes a function, zip(), which does most of the heavy lifting for us. In particular, we can use it as a part of a for loop to effectively transpose a set of lists as follows: for a, b, c in zip(a_list, b_list, c_list): pass. In this example, a, b, and c store the items from the three lists at the same index. Of course, if you're interested in more details about this solution, make sure to keep reading. After all, the remainder of this article includes challenges and performance metrics. Otherwise, I'd appreciate it if you ran over to my list of ways to support the site, so I can keep providing this sort of content for free. Problem Description When it comes to working with data in Python, there are always challenges. For example, I've written extensively about different scenarios that might come up when working with lists and dictionaries. As it turns out, this article is no different. Specifically, our topic today is iterating over a few lists in parallel. For instance, we might have many rows and/or columns of data that we want to analyze. For fun, we'll be working with Pokemon data: pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] levels = [16, 11, 9, 12] For simplicity, I've created three lists of the same length. Of course, there's nothing really stopping us from working with lists of different lengths. It's just a bit more messy. If length matters, we'll make a note of it in each solution below. That said, the goal of this article is to learn how to loop over these lists. In other words, how do we get Pikachu's level and type given the three lists? Well, if we assume that Pikachu's information is at the same index in each list, we just need to know Pikachu's index: pokemon[0] # returns 'pikachu' types[0] # returns 'electric' levels[0] # returns 16 Of course, if we need the information for all of the Pokemon, how would we do that? Fortunately, that's the topic of this article. Let's get started! Solutions In this section, we'll take a look at a few ways to loop over a collection of lists. To start, we'll look at a brute force solution for two lists. Then, we'll try to refine that solution until we get to something a bit more practical. If you're interested in jumping straight to the preferred solution, see the zip() solution below. Looping Over Two Lists Using While Loop When it comes to this kind of problem, my gut is to try to write my own solution using some of the core syntax in Python. For example, if we want to loop over a few lists simultaneously, we can do that with a classic while loop: pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] index = 0 while index < len(pokemon) and index < len(types): curr_pokemon = pokemon[index] curr_type = types[index] # Do something with these variables index += 1 Here, we create a counter called index which stores zero. Then, we loop over both lists using a while loop. Specifically, the while loop only breaks if index grows to be as large as the length of one of the lists. While inside the loop, we store our information in variables and increment index. With a solution like this, we can loop until the index is equal to the length of the smaller list. Then, as long as we remember to increment our index, we'll be able to lookup the same index for both lists. Of course, the drawback here is that we can't really handle more than two lists without changing our loop condition. Luckily, we can take advantage of the all() method in the section. Looping Over Multiple Lists Using While Loop In the previous solution, we were really restricted to the number of lists we could loop over at any given time. As it turns out, that restriction was imposed on us by the loop condition. In other words, if we can find a way to make the loop condition more dynamic, we might be able to extend the previous solution for multiple lists. Luckily, there's a function that comes in handy here. It's called all(), and it allows us to check a condition against a collection of items. For example, we could change our loop condition as follows: pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] index = 0 while all(index < len(row) for row in [pokemon, types]): curr_pokemon = pokemon[index] curr_type = types[index] # Do something with these variables index += 1 Now, the first thing that should jump right out to us is that this doesn't exactly simplify our loop condition—at least for two lists. However, if our lists were already in some nested form, this structure could come in handy: pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] levels = [16, 11, 9, 12] poke_info = [pokemon, types, levels] index = 0 while all(index < len(row) for row in poke_info): curr_pokemon = pokemon[index] curr_type = types[index] curr_level = levels[index] # Do something with these variables index += 1 With a structure like this, the loop condition never has to change. All we have to do is populate our main list before the loop. That said, there are definitely simpler ways to loop over multiple lists. In fact, we haven't even tried to make use of Python's for loop yet which would eliminate the need for indices altogether. Luckily, we have a solution just for that in the next section. Looping Over Multiple Lists Using Zip In the previous two solutions, we largely tried to write a solution to this problem using the core syntax of the language (with a little help from all()). Now, we're going to take advantage of another function, zip(), which will remove the need to track indices altogether: pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] levels = [16, 11, 9, 12] for poke, level in zip(pokemon, levels): # Do something with these variables Not only does this solution remove the need to track indices, but we also don't have to worry about storing variables per list. Instead, the variables are directly unpacked in the loop structure. In fact, we used this exact structure when we talked about performing an element-wise sum of two lists, and I'd argue it's the best solution here as well. That said, even this solution has some drawbacks. For instance, the zip() function doesn't scale well, at least visually. If we wanted to reintroduce a third list, we'd have to rewrite the loop: pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] levels = [16, 11, 9, 12] for poke, t, level in zip(pokemon, types, levels): # Do something with these variables That said, we can simplify this a bit by pulling the call to zip() out of the loop: pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] levels = [16, 11, 9, 12] poke_info = zip(pokemon, types, levels) for poke, t, level in poke_info: # Do something with these variables Alternatively, if we had a nested list already, we could unpack that list in the call to zip(): pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] levels = [16, 11, 9, 12] poke_info = [pokemon, types, levels] for poke, t, level in zip(*poke_info): # Do something with these variables Unfortunately, neither of these options really does anything for the process of unpacking each sublist. That said, we could probably maintain the loop structure if we chose to defer unpacking to the inside of the loop: pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] levels = [16, 11, 9, 12] poke_info = [pokemon, types, levels] for sublist in zip(*poke_info): poke, t, level = sublist # Do something with these variables In any case, I'm not sure there is really any way to simplify this structure any further. It's up to you to decide how you want to structure your solution. I'm most partial to the initial use of zip(), but I can see how that would become cumbersome with more than a few lists. That's why I shared some of these other options. Before we move on to performance, I should probably mention that zip() will silently truncate any lists that are bigger than the smallest list being zipped. In other words, if for some reason we had more Pokemon than types (which would definitely be an error), we'd lose all Pokemon up to the length of the list of types. With that out of the way, let's talk performance! Performance If you've never seen one of my articles before, the performance section is where I tend to take the solutions above and compare them using the timeit library. To learn more about this process, I recommend checking out my performance testing article first. Otherwise, let's start by storing our solutions in strings: setup = """ pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] levels = [16, 11, 9, 12] """ while_loop = """ index = 0 while index < len(pokemon) and index < len(types): curr_pokemon = pokemon[index] curr_type = types[index] # Do something with these variables index += 1 """ while_all_loop = """ index = 0 while all(index >> import timeit >>> min(timeit.repeat(setup=setup, stmt=while_loop)) 1.0207987000003413 >>> min(timeit.repeat(setup=setup, stmt=while_all_loop)) 3.0656588000001648 >>> min(timeit.repeat(setup=setup, stmt=zip_loop)) 0.33662829999957466 To be honest, I was quite surprised by these times. It seems the all() function really slows things down. Also, zip() seems to be pretty fast! To be sure, I ran this again for three lists rather than two: setup = """ pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] levels = [16, 11, 9, 12] """ while_loop = """ index = 0 while index < len(pokemon) and index < len(types) and index < len(levels): curr_pokemon = pokemon[index] curr_type = types[index] curr_level = levels[index] # Do something with these variables index += 1 """ while_all_loop = """ index = 0 while all(index >> min(timeit.repeat(setup=setup, stmt=while_loop)) 1.4052231000005122 >>> min(timeit.repeat(setup=setup, stmt=while_all_loop)) 3.614894300000742 >>> min(timeit.repeat(setup=setup, stmt=zip_loop)) 0.39481680000062624 With the additional list, I don't really see much of a difference. All three solutions seem to be growing slower at about the same rate. Although, the zip() solution is clearly the fastest. If I had the time, I'd try testing these solutions with longer lists, more lists, and different data types. For reference, I ran these solutions on my desktop running Windows 10 and Python 3.8.2. Feel free to run these tests and let me know what you find! Otherwise, we're going to move into the challenge section now. Challenge As with many of these articles, I like to keep things interesting by offering up a bit of a challenge. Since we talked about looping over lists today, I figured we could do something to take it a step further. Given the same Pokemon related data from above, write a program that does some simple analyses. For example, can you figure out which Pokemon has the highest level? How about the lowest level? If you want to go the extra mile, you could even try sorting these lists by level or type. Really, the skies the limit! I'm just interested in seeing if some of the solutions from this article are applicable, or if there are easier ways to do some data analysis. To kick things off, here's my crack at the challenge: https://twitter.com/RenegadeCoder94/status/1276967356997218305?s=20 As you can see, I decided to leverage the zip() solution to write a simple "Next Pokemon" algorithm. In other words, if one of our Pokemon faints, we can call this function to retrieve the next strongest (and healthiest) Pokemon by level. If you want to jump in on this challenge, head on over to Twitter and use the hashtag #RenegadePython. Of course, if you're not the social media type, you can always drop a solution in the GitHub repo. Then, I can always share your solution on your behalf (with credit of course). Otherwise, that's it for today! In the next section, we'll review all of the solutions in this article, and I'll share my usual request for support. A Little Recap As promised, here's a quick recap of all the solutions we covered in this article: pokemon = ['pikachu', 'charmander', 'squirtle', 'bulbasaur'] types = ['electric', 'fire', 'water', 'grass'] levels = [16, 11, 9, 12] # Brute force while loop solution index = 0 while index < len(pokemon) and index < len(types) and index < len(levels): curr_pokemon = pokemon[index] curr_type = types[index] curr_level = levels[index] # Do something with these variables index += 1 # Brute force + abstraction solution index = 0 while all(index < len(row) for row in [pokemon, types, levels]): curr_pokemon = pokemon[index] curr_type = types[index] curr_level = levels[index] # Do something with these variables index += 1 # For loop + zip() solution **preferred** for poke, t, level in zip(pokemon, types, levels): # Do something with these variables pass If you liked this article, and you'd like to see more like it, this is part of a growing series of articles called How to Python. You can get a feel for the types of articles in it by checking out this list of solutions to everyday problems. To get you started, here are some of my favorites in the series: How to Format a String in Python How to Merge Two Dictionaries in Python Likewise, you can help support the site by checking out this list of ways to grow The Renegade Coder. It includes fun stuff like my YouTube channel and my Patreon. In addition, here are some Python reasons on Amazon (ad): Otherwise, thanks for stopping by! I really appreciate you taking some time to check out the site, and I hope you'll swing by again soon.

Welcome to The Renegade Coder, a site for people interested in Python programming and college teaching! Join me on Patreon for even more content.

Latest Articles