Articles – Page 101 – The Renegade Coder

Welcome back to another edition of the How to Python series. This time I want to sum elements of two lists in Python. I got the inspiration for this topic while trying to do just this at work the other day. In short, one of the best ways to sum elements of two lists in Python is to use a list comprehension in conjunction with the addition operator. For example, we could perform an element-wise sum of two lists as follows: [x + y for x, y in zip(list_a, list_b)]. But, as always, we'll take a look at other options. Video Summary https://youtu.be/-ueWDzP88eQ After a year and a half, I finally got around to making a video summary for this article. If you're into that sort of thing, check it out. Of course, it's usually quicker just to read the article, but you're welcome to head on over to YouTube and give it a like. Problem Introduction Recently, I ran into a problem where a library wasn't working exactly how I wanted, so I had to hack together the results to make my life a bit easier. In this scenario, I had a connection library which I was using to list all the available devices. However, the list functionality short circuited for certain types of connections, so it never actually listed everything. As a result, I was forced to run the function twice: once for USB and again for Ethernet. The results of this list function returned a list that looked something like the following: # [size, [types], [interfaces], [serial numbers], [IP addresses]] [2, [7, 7], [1, 2], [2314567, 8374163], [0, 84302738]] However, I was stuck retrieving the lists separately: [1, [7], [2], [8374163], [84302738]] [1, [7], [1], [2314567], [0]] Naturally, I wanted to be able to merge the two lists back into what I was expecting initially. However, I wasn't totally sure how I was going to do that. So, let's take a look at some possible solutions. Solutions When it comes to merging two lists, there are several possible ways to do it: Merge the two lists by hand using the addition operator Sum the elements of both lists using a list comprehension Sum the elements of both lists using the map function Feel free to jump straight to the solution for you or browse as long as you need. Merge Two Lists by Hand At first, we might try just merging the two lists by hand. In other words, we might try the following: ethernet_devices = [1, [7], [2], [8374163], [84302738]] usb_devices = [1, [7], [1], [2314567], [0]] all_devices = [ ethernet_devices[0] + usb_devices[0], ethernet_devices[1] + usb_devices[1], ethernet_devices[2] + usb_devices[2], ethernet_devices[3] + usb_devices[3], ethernet_devices[4] + usb_devices[4] ] Now, that solution is hardly elegant, but it gets the job done. After all, we are taking advantage of the fact that lists can be merged using the same operator as addition. In other words, [5] + [7] yields [5, 7]. Why not take advantage of this property in a loop? all_devices = [] for i in len(ethernet_devices): all_devices.append(ethernet_devices[i] + usb_devices[i]) As you can see, we can sum each pair of elements using a loop. Of course, if we're able to generate a list using a loop, we might be able to use a list comprehension! Sum Elements of Two Lists with a Comprehension Just like the last lesson on inverting dictionaries in Python, we can take advantage of comprehensions to dramatically simplify this problem. Let's take a look: ethernet_devices = [1, [7], [2], [8374163], [84302738]] usb_devices = [1, [7], [1], [2314567], [0]] all_devices = [x + y for x, y in zip(ethernet_devices, usb_devices)] Now we're talking! That's five lines of tedious mapping compressed down into a simple list comprehension. If this is your first time seeing the list comprehension, I recommend checking out my article on how to write a list comprehension. That said, to summarize, the comprehension works by looping over the pairs of tuples generated by zip(). For each component (x and y) in the tuples, we sum them and add them to the newly generated list. How cool is that?! But wait, it gets better: all_devices = [sum(pair) for pair in zip(ethernet_devices, usb_devices)] Or, does it? Thanks to our fried, rhymes, we'll notice that this solution doesn't actually work for our situation. While it does a great job summing integers in an iterable, it crashes when trying to merge two sublists with the following error: Traceback (most recent call last): File " ", line 1, in all_devices = [sum(pair) for pair in zip(ethernet_devices, usb_devices)] File " ", line 1, in all_devices = [sum(pair) for pair in zip(ethernet_devices, usb_devices)] TypeError: unsupported operand type(s) for +: 'int' and 'list' That said, you may find it useful for merging two lists of strictly integers. Sum Elements of Two Lists with a Mapping At this point, we've basically answered our question. However, there is another solution which requires arguably even less code. Check it out: ethernet_devices = [1, [7], [2], [8374163], [84302738]] usb_devices = [1, [7], [1], [2314567], [0]] import operator all_devices = list(map(operator.add, ethernet_devices, usb_devices)) Of course, we also need to add a dependency which does make this solution a bit less attractive. In fact, if we were going to go down that road, we could easily leverage numpy: ethernet_devices = [1, [7], [2], [8374163], [84302738]] usb_devices = [1, [7], [1], [2314567], [0]] import numpy as np all_devices = np.add(ethernet_devices, usb_devices) However, numpy is a massive library, so we should probably stick with our list comprehensions. Performance As always, I like to take a brief look at performance since it may or may not influence your decision. To do that, we have to setup our solutions in some strings: import timeit setup = """ import operator ethernet_devices = [1, [7], [2], [8374163], [84302738]] usb_devices = [1, [7], [1], [2314567], [0]] """ brute_force = """ all_devices = [ ethernet_devices[0] + usb_devices[0], ethernet_devices[1] + usb_devices[1], ethernet_devices[2] + usb_devices[2], ethernet_devices[3] + usb_devices[3], ethernet_devices[4] + usb_devices[4] ] """ comprehension = """ all_devices = [x + y for x, y in zip(ethernet_devices, usb_devices)] """ maps = """ all_devices = list(map(operator.add, ethernet_devices, usb_devices)) """ Unfortunately, I wasn't able to perform the numpy test as I didn't have the package. Feel free to share your own metrics in the comments. At any rate, all we need to do now is run the timeit commands as follows: >>> min(timeit.repeat(stmt=brute_force, setup=setup, repeat=10)) 0.5299746000000027 >>> min(timeit.repeat(stmt=comprehension, setup=setup, repeat=10)) 0.7775744999999858 >>> min(timeit.repeat(stmt=maps, setup=setup, repeat=10)) 0.754784299999983 Here, we've chosen to repeat the test ten times before choosing a lower bound execution time. As it turns out, the brute force method is the fastest option. Obviously, it isn't ideal—or even scalable—but it is fast. For reference, all tests were completed on Windows 10 with Python 3.7.3. Challenge Now that you know to perform an element-wise sum of two lists in Python, I have a challenge for you! Write some Python code that can sum any number of lists assuming they are the same length. For example: ethernet_devices = [1, [7], [2], [8374163], [84302738]] usb_devices = [1, [7], [1], [2314567], [0]] wireless_devices = [1, [7], [4], [4315432], [21030132]] merge(ethernet_devices, usb_devices, wireless_devices) In this example, we merge three lists instead of two. As a result, the expected output should be as follows: [ 3, [7, 7, 7], [2, 1, 4], [8374163, 2314567, 4315432], [84302738, 0, 21030132] ] Naturally, your solution should be able to merge any number of lists—not just 3. When you're ready, share your solution on Twitter using the #RenegadePython hashtag. If I see it, I'll give it a share! In the meantime, here's my crack at the problem which uses a series of list comprehensions: https://twitter.com/RenegadeCoder94/status/1251952416205033473?s=20 Are there any better ways to get this working? A Little Recap Using the handful of methods above, we should be able to sum elements of two lists. ethernet_devices = [1, [7], [2], [8374163], [84302738]] usb_devices = [1, [7], [1], [2314567], [0]] # The long way all_devices = [ ethernet_devices[0] + usb_devices[0], ethernet_devices[1] + usb_devices[1], ethernet_devices[2] + usb_devices[2], ethernet_devices[3] + usb_devices[3], ethernet_devices[4] + usb_devices[4] ] # Some comprehension magic all_devices = [x + y for x, y in zip(ethernet_devices, usb_devices)] # Let's use maps import operator all_devices = list(map(operator.add, ethernet_devices, usb_devices)) # We can't forget our favorite computation library import numpy as np all_devices = np.add(ethernet_devices, usb_devices) As we can see, there are a lot of ways to run an element-wise sum of two lists. Take your pick. As always, thanks for stopping by! If you liked this article, I have a massive list of code snippets just like this for your perusal. If you're interested in learning more about Python, consider subscribing to The Renegade Coder—or at least hop on our mailing list, so you'll never miss another article. Tune in next time to learn how to check if a file exists in Python. While you're here, you might be interested in some these other Python articles: How to Automate Your GitHub Wiki How I Automated My Grading Responsibilities Reverse a String in Python Once again, thanks for stopping by. I appreciate it!