Articles – Page 13 – The Renegade Coder

I'm kicking off a new Python series that covers core programming concepts. This is a bit different from my usual problem solving content, so it should be a lot of fun! Today, we're kicking off the series with an explanation of what an iterable actually is in Python. Concept Overview Broadly, an iterable is a type of data structure that can be looped over. I say broadly because I don't believe Python has a strong definition for iterable. The closest you will get is from the glossary, which defines an iterable as: An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any classes you define with an __iter__() method or with a __getitem__() method that implements sequence semantics. Iterables can be used in a for loop and in many other places where a sequence is needed (zip(), map(), …). When an iterable object is passed as an argument to the built-in function iter(), it returns an iterator for the object. This iterator is good for one pass over the set of values. When using iterables, it is usually not necessary to call iter() or deal with iterator objects yourself. The for statement does that automatically for you, creating a temporary unnamed variable to hold the iterator for the duration of the loop. See also iterator, sequence, and generator. Source: Python 3 Glossary To me, the definition above is a bit more descriptive than prescriptive. In other words, it sort of tells us what kinds of data structures would count as iterables, but it doesn't give us any strict rules for creating one. As a result, there aren't really any strong claims I can make about what you can do with an iterable because there are different kinds of iterables. If you've used any other mainstream programming languages (e.g., Java), you might find the lack of a strong definition to be a bit jarring. After all, an iterable would typically be a data structure (i.e., an object) which inherits iterable properties from something like an interface. Therefore, you would know if a data structure was iterable through a type check (e.g., is this data structure an instance of "iterable"?). In contrast, as with many things in Python, you sort of have test an object for its limits. For instance, if you want to know if an object is iterable, you have to call iter() on it. Alternatively, the more Pythonic approach might be to just try iterating over it using a for loop, which the Python docs call EAFP: Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many try and except statements. The technique contrasts with the LBYL style common to many other languages such as C. Source: Python 3 Glossary Ultimately, we'll take a look at the definition provided in more detail and try to explore iterables of various types. Examples of Common Iterable Data Types Despite the lack of a solid definition for iterables, most of the data structures you encounter will be iterable to some extent. Perhaps the most obvious example is the list, which you can loop over easily: dice_rolls = [7, 6, 7, 10, 8, 7] for roll in dice_rolls: print(roll) # prints: # 7 # 6 # 7 # 10 # 8 # 7 You can also loop over a string to look at individual characters: dna_sequence = "CTCTAAA" for base in dna_sequence: print(base) # prints: # C # T # C # T # A # A # A Unfortunately, things start to get a little tricky with a dictionary. If you try to iterate over them directly, you'll only see the keys: grades = {"math": "B-", "art": "C", "science": "A+"} for subject in grades: print(subject) # prints: # math # art # science If you want to iterate over the actual pairs (as tuples), you have to make use of the items() method: for pair in grades.items(): print(pair) # prints: # ('math', 'B-') # ('art', 'C') # ('science', 'A+') More often than not, however, you're going to want to take advantage of iterable unpacking anyway: for subject, grade in grades.items(): print(subject, grade) # prints: # math B- # art C # science A+ In any case, you can probably imagine that most built-in data types are iterable including strings, lists, tuples, and dictionaries. Up next, we'll look at some built-in functions that take advantage of this trait. Built-in Functions That Accept Iterables Some of the very first functions that come to mind for me when I think of iterables are min() and max(). These methods take an iterable and return the smallest and biggest items in them, respectively. Here's what that looks like on a list: dice_rolls = [7, 6, 7, 10, 8, 7] min(dice_rolls) # returns 6 max(dice_rolls) # returns 10 Then, there are functions like zip(), which take a series of iterables and combine them. Of course, zip() is a lazy function, so we have to convert the result back to a list to see our items: subjects = ["math", "art", "science"] grades = ["B-", "C", "A+"] list(zip(subjects, grades)) # returns [('math', 'B-'), ('art', 'C'), ('science', 'A+')] There are even interesting boolean functions like all() which can be used to determine if everything in an iterable is truthy: truthy_values = [5, "hello", True] all(truthy_values) # returns True If you get bored, I'd recommend browsing the built-in functions list. There are several more functions that take iterables, such as map(), any(), and enumerate(). Making a Custom Iterable At this point, we have a rough idea of what an iterable is in Python. We also know of a few examples of common iterables such as lists and strings, and we know a few built-in functions that take iterables. Now, I want to take a moment to craft a couple iterable objects by following the definition provided in the Python docs and play with them a bit (i.e., use them in loops, pass them to built-in functions, etc.). Creating an Iterable Using __iter__() Keeping with the dice roll theme, I made a simple object that let's you roll a pair of dice and stores every roll in your history: import random class DiceRoll: def __init__(self): self.history = [] def roll(self): total = random.randint(1,6) + random.randint(1,6) self.history.append(total) return total def __iter__(self): return iter(self.history) To make make this object iterable, you'll notice that I have implemented the __iter__() special method. The idea being that the underlying list is iterable, so I just return its iterator. Now, we can roll the dice a few times and report our history: test = DiceRoll() test.roll() # returned 6 test.roll() # returned 11 test.roll() # returned 5 for roll in test: print(roll) # printed: # 6 # 11 # 5 What's cool is that we can start taking advantage of its iterable status by passing it to a variety of built-in methods: min(test) # returned 5 max(test) # returned 11 any(test) # returned True Ultimately, it looks like we've created an iterable. Let's try another way. Creating an Iterable Using __getitem__() According to the docs, we can also create an iterable by implementing the __getitem__() special method, which allows us to index our data structure. Fortunately, that's an easy change: import random class DiceRoll: def __init__(self): self.history = [] def roll(self): total = random.randint(1,6) + random.randint(1,6) self.history.append(total) return total def __getitem__(self, key): return self.history[key] Instead of making use of list's iterator, we make use of the ability to index a list. The results are virtually identical: test = DiceRoll() test.roll() # returned 10 test.roll() # returned 3 test.roll() # returned 4 for roll in test: print(roll) # printed # 10 # 3 # 4 min(test) # returned 3 max(test) # returned 10 any(test) # returned True So, it seemingly doesn't make a difference which one you implement. Though, I imagine the context matters. For example, you might only implement __getitem__() if it made sense for you to be able to index your data structure (i.e., it's a sequence-like data structure). Why Do Both Strategies Work? Previously, we looked at two different ways to create an iterable data type in Python. In the first example, we looked at an object which implements the __iter__() special method. In contrast, the second example creates an iterable using the __getitem__() special method. While I didn't go into detail around each of these special methods, they work quite differently. One relies on a separate one-time-use iterator object, and the other relies on sequence semantics. This begs the question: how are both supported so seamlessly? I was genuinely curious about this, so I dug into the docs a bit. First, the reason the iterator solution works is somewhat obvious. The concept of an iterator is not new; many programming languages support them. In short, they're an object with a "next" method which you use to get the next item in the collection. Some languages have the additional requirement of a boolean method to check if there is anything next, but as previously mentioned, Python subscribes to the EAFP principle. As a result, "next" will eventually crash, and that's when we know to stop looping. In contrast, when you implement indexing instead of an iterator, the iterable then assumes a counting mechanism for iterating over the data structure. Normally, you might need the length of the data structure to know when to stop, but again we don't really want to be checking things first. Instead, the loop will count up until it gets an IndexError, at which point the loop terminates. While the logic is a bit different in both cases, the design is fundamentally the same: we iterate until we encounter an error. Iterables Are Only the Beginning With this being a new series, I'm expecting to build on existing concepts. For example, while we now know what an iterable is, we'll need to explore related terms like iterable unpacking and indexing. Because concepts are all very much connected, I'm sure there will be a lot of unpacking (pun absolutely intended) going forward. Until then, however, why not explore some of my other articles: How to Swap Variables in Python: Temporary Variables and Iterable Unpacking How to Get the Last Item of a List in Python: Iterable Unpacking and More Obfuscation Techniques: Shadowing Built-in Functions Alternatively, here are some resources for helping you learn Python (#ad): Finally, you can take your support even further by helping me grow the site. Otherwise, take care!