The Python Concept Map – Page 3

As I continue to write about Python, I find myself wanting some sort of place to direct my readers to learn some of the more fundamental programming concepts. For instance, I write a lot about Python dictionaries, but I don't actually explain what a dictionary is to the level of detail that some of my readers might like. As a result, thought it would be fun to put together an in-depth article which answers the question: what is a dictionary in Python? A dictionary is a list-like data structure where the indices are replaced by keys. In other words, rather than looking up values by their numerical index, a dictionary allows us to use immutable data types as keys. As a result, keys like strings, integers, and tuples can be mapped to any type of data we want. In the remainder of this article, we'll take a look at dictionaries in more detail. Specifically, we'll take some time to look at the structure of a dictionary. Then, we'll look at dictionary syntax and methods. After that, we'll look at few scenarios where a dictionary might be useful, and we'll close things out with some frequently asked questions. Concept Overview In Python, a dictionary is a data structure that stores pairs of items. Specifically, each pair consists of two items: a key and a value. In this context, a key is an immutable data type which serves as the access point for its associated value. For example, in a language dictionary—the kind that stores terms and definitions—the terms would be keys and the definitions would be values: Appear: come into sight; become visible or noticeable, typically without visible agent or apparent cause. Play: engage in activity for enjoyment and recreation rather than a serious or practical purpose. Trill: a quavering or vibratory sound, especially a rapid alternation of sung or played notes. Given this structure, it should be clear that language dictionaries operate in one direction. In other words, we can only look up definitions given a term. It's much more difficult to look up a word given a definition. As it turns out, Python dictionaries operate in roughly the same way. In fact, we can create a Python dictionary which looks exactly like the language dictionary example from above: terms_and_defs = { 'appear': 'come into sight; become visible or noticeable, typically without visible agent or apparent cause.', 'play': 'engage in activity for enjoyment and recreation rather than a serious or practical purpose.', 'trill': 'a quavering or vibratory sound, especially a rapid alternation of sung or played notes.' } Now, we have a language dictionary which we can use to lookup definitions. In the next section, we'll take a look at a few of the ways we can work with Python dictionaries. Python Dictionary Syntax Now that we're comfortable with the concept of a dictionary, let's take a look at a few things we can do with them. In particular, we'll take some time in this section to focus on the syntax for initializing a dictionary as well as adding, removing, and looking up values. Initializing a Dictionary In the previous section, we defined a Python dictionary using the following syntax: terms_and_defs = { 'appear': 'come into sight; become visible or noticeable, typically without visible agent or apparent cause.', 'play': 'engage in activity for enjoyment and recreation rather than a serious or practical purpose.', 'trill': 'a quavering or vibratory sound, especially a rapid alternation of sung or played notes.' } Here, we created a dictionary called terms_and_defs using the usual variable initialization syntax. In other words, we created a variable, terms_and_defs, and assigned it some value using the assignment operator, =. If we take a peek at the right side of the assignment operator, we'll see where the magic happens. Specifically, we create a dictionary using curly brackets, {}. To define items in this dictionary, we use the following syntax: key: value Then, to separate each item, we use commas: key_1: value_1, key_2: value_2, ... key_n, value_n And, that's it! We have a dictionary. Of course, now that we have a dictionary. What else can we do? In the next section, we'll learn how to use keys to lookup values. Looking up a Value in a Dictionary Now that we have a dictionary, we can do a lot of fun things with it. For example, we can try looking up definitions given one of the following terms: appear, play, and trill. Here's what that would look like: terms_and_defs['appear'] # returns defintion In order to access a value in a dictionary, we need to use a syntax that is similar to the list indexing syntax. In other words, we take the variable which stores the dictionary and pass a key to it using square brackets: dictionary[key] In our example, we're working with terms as keys, so we can pass any term we want directly. For instance, we used the term 'appear' which would cause the dictionary to return the following definition: come into sight; become visible or noticeable, typically without visible agent or apparent cause. Now, this is great because we can lookup any definition we want, right? Well, it's not that simple. For example, the following call is invalid: terms_and_defs['sword'] In particular, we'll get an KeyError like the one that follows: Traceback (most recent call last): File " ", line 1, in terms_and_defs["sword"] KeyError: 'sword' Basically, this means that the key doesn't exist. Of course, if we look back at our original dictionary, we'll notice that "sword" was never defined. As a result, we get this error. Later, we'll talk about how to deal with this issue. For now, be aware that dictionaries won't let us use keys that were never defined. Updating an Item in a Dictionary One of the nice things about dictionaries in Python is that they're mutable. In other words, once we've created one, there's nothing stopping us from updating values. To do that, we'll need to take advantage of the lookup syntax from the previous section: terms_and_defs['appear'] # returns defintion Here, we've returned the definition. If we want to overwrite the current definition, we can reintroduce our old friend the assignment operator, =: terms_and_defs['appear'] = "seem; give the impression of being." Now, the dictionary will look like this: terms_and_defs = { 'appear': 'seem; give the impression of being.', 'play': 'engage in activity for enjoyment and recreation rather than a serious or practical purpose.', 'trill': 'a quavering or vibratory sound, especially a rapid alternation of sung or played notes.' } In other words, we've replaced the definition of the word "appear". Naturally, being able to update values in a dictionary is pretty handy. Later on this article, we'll talk about a few places this might be useful. Adding an Item to a Dictionary While updating values in a dictionary is great, we can actually take things a step further by adding completely new items. Fortunately, there's no new syntax to learn here. The update syntax works for adding items as well: terms_and_defs['materialize'] = "become actual fact; happen." After executing, the dictionary will look something like this: terms_and_defs = { 'appear': 'seem; give the impression of being.', 'play': 'engage in activity for enjoyment and recreation rather than a serious or practical purpose.', 'trill': 'a quavering or vibratory sound, especially a rapid alternation of sung or played notes.', 'materialize': 'become actual fact; happen.' } Now is probably a good time to mention that dictionaries are unordered—at least until Python 3.7. In other words, there's no way to guarantee the order of items in the example above. That said, if you are using a more recent version of Python, you may be able to depend on insertion ordering. At any rate, that's how we add a new item to a dictionary. In the next section, we'll take a look at how to do the opposite: remove an item from a dictionary. Removing an Item From a Dictionary Perhaps the last thing worth talking about in terms of syntax is removing an item from a dictionary. To do that, we'll need to take advantage of the del keyword in Python. In this case, we'll use it to remove the 'materialize' term from the previous section: del terms_and_defs['materialize'] Now, if we take a peek at the dictionary, we'll see that the item is gone: terms_and_defs = { 'appear': 'seem; give the impression of being.', 'play': 'engage in activity for enjoyment and recreation rather than a serious or practical purpose.', 'trill': 'a quavering or vibratory sound, especially a rapid alternation of sung or played notes.' } Of course, I should probably warn you that deleting items like comes with some risks. For example, if we were to run that same command again, we'd get the following error: >>> del terms_and_defs['materialize'] Traceback (most recent call last): File " ", line 1, in del terms_and_defs['materialize'] KeyError: 'materialize' Much like the lookup warning, if we try to delete a key that doesn't exists, we'll get a KeyError. To account for that, we'll need to check if the key exists first. Alternatively, we can make use of a helpful method, pop(). In the next section, we'll look at methods like this one. Python Dictionary Methods While dictionaries come with their own syntax in Python, the fun doesn't stop there. In fact, dictionaries are objects, so they support several methods. In this section, we'll break down some of the more important methods. The get() Method Previously, we had talked about how to lookup a value in dictionary. As it turns out, dictionaries have a method that does the same thing called get(): terms_and_defs.get('appear') # returns the definition for 'appear' One of the nice things about get() is that it won't fail if the key doesn't exist. Instead, it'll return None. Naturally, this totally eliminates the need to check if a key exists. To make life even better, we can use the get() method to return a default value if the key doesn't exist. For example, let's say we wanted to look up the definition for the word 'extra'. It's possible that the word doesn't exist in our dictionary, so we can set a default definition using get(): terms_and_defs.get('extra', 'added to an existing or usual amount or number.') Then, if for some reason the term doesn't exist, we'll still have a definition to use. Personally, I find this method pretty handy because it eliminates a lot of the boilerplate associated with verifying key existence. That said, you definitely want to be cautious of the fact that this method always returns a value. In other words, it's not a good method to use for checking the existence of a key—especially if values in the dictionary can be None. At any rate, I think that's enough chatting about get(), let's move on to pop(). The pop() Method Earlier in this article, we talked about deleting keys from a dictionary using the del keyword, and I hinted at a method, pop(), that could be used to accomplish the same goal. As it turns out, pop() is the way to go for deleting keys. To use pop(), we'll just need to pass the key we want to delete: terms_and_defs.pop('appear') If we were to run this command, we'd find that our dictionary would be missing one of our keys: >>> terms_and_defs.pop('appear') 'seem; give the impression of being.' >>> terms_and_defs {'play': 'engage in activity for enjoyment and recreation rather than a serious or practical purpose.', 'trill': 'a quavering or vibratory sound, especially a rapid alternation of sung or played notes.'} Unfortunately, unlike get(), pop() doesn't handle missing keys right out of the box. For example, if we tried removing a missing key like "mic", we'd find ourselves with yet another KeyError: >>> terms_and_defs.pop('mic') Traceback (most recent call last): File " ", line 1, in terms_and_defs.pop('mic') KeyError: 'mic' Luckily, pop() let's us specify a default value which can be used if the key is not found: >>> terms_and_defs.pop('mic', 'a microphone.') 'a microphone.' Now, we no longer have to deal with that pesky check for a key. One thing I should note is that pop() returns whatever value was removed from the dictionary (or the default value). If for some reason you need that value, you don't have to retrieve it first. pop() has you covered! At this point, we've covered all of the methods that mimic some of the built-in syntax. However, there is no method for adding items to the dictionary (except update(), kinda), so we'll move on to some of the big picture methods like keys() and values(). The keys(), values(), and items() Methods Much of the core syntax around dictionaries is dedicated to working with items individually. Fortunately, dictionaries come with a handful of methods that can be used to explore larger portions of a dictionary. For example, there are a class of methods which can be used to get different sections of a dictionary as "lists" (more on that later). One of those methods is the keys() method which can be used to retrieve the list of keys from a dictionary. At this point, our dictionary is pretty messy, but we can still retrieve the set of keys as follows: >>> terms_and_defs.keys() dict_keys(['play', 'trill']) Unfortunately, the object we get back from keys() isn't super easy to use—though it saves computational resources. In particular, it's a dictionary view, so we won't get the same sort of functionality we would normally get from a list. If for some reason we wanted a list of keys, we can wrap the result in the list constructor: >>> list(terms_and_defs.keys()) ['play', 'trill'] As it turns out, there are actually two other methods like this: one to retrieve a "list" of values and another to retrieve a "list" of items. For example, if we wanted to look at the values in a dictionary, we could do that with the values() method: >>> terms_and_defs.values() dict_values(['engage in activity for enjoyment and recreation rather than a serious or practical purpose.', 'a quavering or vibratory sound, especially a rapid alternation of sung or played notes.']) Likewise, we could get an iterable version of the dictionary using the items() method: >>> terms_and_defs.items() dict_items([('play', 'engage in activity for enjoyment and recreation rather than a serious or practical purpose.'), ('trill', 'a quavering or vibratory sound, especially a rapid alternation of sung or played notes.')]) Again, all three of these methods return a dictionary view which has its own set of functionality. If you need to common functionalities like searching, sorting, and filtering, it might be helpful to convert the view to a list using the list constructor. That said, that's it for the dictionary view methods! In addition to these methods, there are few others that we won't dig into like clear(), copy(), popitem(), and setdefault(). I don't find myself using these methods very often, so I won't waste your time. However, I recommend familiarizing yourself with them when you get the chance. Now that we've had some time to look at syntax and methods, it's time to talk about when dictionaries are actually useful. When to Use a Dictionary in Python One of the great things about Python is that there are a ton of data structures built right into the language. Unfortunately, the downside of having options is deciding what's best to use and when. This decision gets more complicated as you consider paradigms and idioms in a language. That said, I'll do my best to try to summarize when I think using a dictionary is the best option. In computer science, there are probably thousands of different data structures. Fortunately, most of them fall can be reduced into a handful of classes. For example, lists and tuples generally fall into the sequence category. Meanwhile, there are several other categories including stacks, trees, sets, and graphs. As it turns out, our friend, the dictionary, falls into the associative array category. So, what does this mean in terms of use? Well, in the following subsections, we'll look at a few scenarios where a dictionary might be useful. Managing Data (Spreadsheets) As you can probably imagine, certain classes of data structures tend to suit certain types of problems. For example, sequences (e.g. lists, tuples, etc.) are most useful when we have data that can be placed in a series like costs of products or names of hockey players. Unfortunately, sequences fail to recognize relationships in data. For example, if we wanted to track hockey player names AND cap hits, we'd need two parallel sequences or a sequence of sequences: players = ["Crosby", "Ovechkin"] cap_hit = [8.7, 9.5] players_and_cap_hit = [ ["Crosby", 8.7], ["Ovechkin", 9.5] ] Sadly, both solutions have some downsides. For instance, the parallel sequence solution requires heavy maintenance. If a change in order is made to one sequence, that same change needs to be made in the second sequence. Meanwhile, the sequence of sequences solution isn't much better. While the inner sequences are easy to maintain, there still needs to be some association between the indices, so we know that index zero refers to the player names and so on. Luckily, there is a data structure for this exact scenario: the associative array (i.e. dictionaries). The associative array allows us to label those indices with human readable names, so we don't have to maintain some sort of lookup table. For example, I might take the sequence in sequence solution and replace the inner sequences with associative arrays. That way, the inner associative arrays account for data that is all associated (i.e. player name, cap hit, years in NHL, etc.). Meanwhile, the outer sequence allows us to perceive each data unit as a portion of a series: player_info = [ {"name": "Crosby", "cap_hit": 8.7}, {"name": "Ovechkin", "cap_hit": 9.5} ] Now, we probably won't ever find ourselves defining data like this in Python. In all likelihood, we'll be pulling data into a program from some outside source. For example, we might be trying to read in a spreadsheet. Organizing data in this form will save us some headaches in the future. Detecting Duplicates (Counting) While dictionaries are great for managing large sets of data, there also good for detecting duplicates in a data set. For example, we might want to know if a list of numbers contains the same number more than once: nums = [6, 4, 5, 6, 1, 3] One way to do that is to count the appearances of every number using a dictionary: counts = {} for num in nums: if num in counts: counts[num] += 1 else: counts[num] = 1 In this case, we'll end up with the following dictionary: { 6: 2, 4: 1, 5: 1, 1: 1, 3: 1 } Since 6 appears more than once, we know we have duplicates. Of course, counting alone is a pretty good reason to use a dictionary. After all, counts can be used in a ton of applications like generating a histogram. And, as you can see, it's a pretty easy algorithm to implement. If a key exists, we add one to it. Otherwise, we create a new entry. Looking Up Data Quickly (Search) All that said, perhaps the best reason to use a dictionary is because you have a need for speed. One of the biggest advantages of using a dictionary is that the hashing algorithm for lookup takes constant time (i.e. O(1)). In other words, if you need to search for values often, a dictionary is likely the quickest way to do it. Let's use our terms_and_defs example once again: terms_and_defs = { 'appear': 'seem; give the impression of being.', 'play': 'engage in activity for enjoyment and recreation rather than a serious or practical purpose.', 'trill': 'a quavering or vibratory sound, especially a rapid alternation of sung or played notes.' } Here, we have a dictionary which acts like a language dictionary. In other words, it maps terms to definitions. If for some reason we need the definition of a word, we don't have to perform a search. We can just key into the dictionary directly using the name of the term. If we were to store the terms as a list of tuples instead, we'd find the search process to be a lot more painful: terms_and_defs = [ ('appear', 'seem; give the impression of being.'), ('play', 'engage in activity for enjoyment and recreation rather than a serious or practical purpose.'), ('trill', 'a quavering or vibratory sound, especially a rapid alternation of sung or played notes.') ] appear_def = None search_term = "appear" for term in terms_and_defs: if term[0] == search_term: appear_def = term[1] break I won't bother to compare the performance of these two solutions, so you'll have to take my word for it: this solution is slow! That said, you're welcome to compare them yourself using any performance testing tool you'd like. Making Simple Objects (Data Classes) Finally, the last reason I can think of to use a dictionary is to avoid creating a C-style struct. In Python, I believe it's generally discouraged to have a class that just holds data—of course, Python 3.7 does include the @dataclass decorator, but that's a different story. Instead, you should opt for a dictionary. As someone who came from Java, I was always tempted to create data classes, so I could have simple objects to track things like hockey player information. Of course, I never intended for those objects to perform an actions. As a result, I'd end up with a class that looked something like this: class HockeyPlayer: def __init__(self, name, cap_hit): self.name = name self.cap_hit = cap_hit Then, we could create players as follows: crosby = HockeyPlayer("Crosby", 8.7) ovechkin = HockeyPlayer("Ovechkin", 9.5) Now, there's nothing inherently wrong with this. In fact, this allows us to document our class and track changes to it over time. That said, this tends to be a bit overkill in the simple case. After all, it's probably simpler to make use of a dictionary instead: crosby = {"name": "Crosby", "cap_hit": 8.7} ovechkin = {"name": "Ovechkin", "cap_hit": 9.5} Of course, this debate tends to get a bit heated, so I'll defer to several posts which cover a lot of the debate in detail: Stack Overflow Reddit That said, this definitely something to consider. At any rate, that's about all I could come up with in terms of reasons for using a dictionary. In the next section, we'll talk about some of the common questions people have when using a dictionary. Spoiler alert: it's mainly a section to plug some of my other content. Python Dictionary FAQs At this point in the article, we've talked about what a dictionary is, how to go about creating and using one, and when exactly to use one. Now, I want to talk about some of the questions you might still have like how to loop over a dictionary and how to merge dictionaries. How to Loop Over a Dictionary Up until now, we've spent most of this article talking about how to setup a dictionary. Of course, once we have one, we will probably want to do some more interesting things with it. For instance, we might be interested in looping over different parts of a dictionary—like searching for a specific value. To do that, we can take advantage of the three methods we've talked about previously: keys(), values(), and items(). Then, it's just a matter of iterating over those views: my_dict = {} # define some dictionary for key in my_dict.keys(): pass # do something my_dict = {} # define some dictionary for value in my_dict.values(): pass # do something my_dict = {} # define some dictionary for key, value in my_dict.items(): pass # do something In other words, we can loop over each key using the keys() method, over each value using the values() method, and over each key and values using the items() method. Knowing how to use these three methods is enough to do a lot of interesting things with dictionaries. If you're interested in learning about the various looping mechanisms for dictionaries, check out my article which covers even more details including performance metrics and a challenge. How to Merge Two Dictionaries Throughout this article, we talked a lot about dictionaries as if we were only working with one at a time. But, what if we had more than one dictionary? Even better, what if we needed to combine those dictionaries? As it turns out, merging dictionaries is a common problem! In fact, it's so common that Python 3.9 will likely include a dictionary union operator just for this purpose. Until then, however, we'll have to do things the old fashioned way. In general, there are a lot of ways to merge a dictionary. Unfortunately, none of them are really intuitive. For example, even the quickest solution is pretty convoluted: num_cats = {"cats": 2} num_dogs = {"dogs": 3} num_cats_and_dogs = {**num_cats, **num_dogs} This takes advantage of Python's dictionary unpacking syntax to build a completely new dictionary from scratch. Sadly, this solution is restricted to folks who can use Python 3.5 and up. If you're using an older version of Python, you'll have to reach for update() or a dictionary comprehension. If you're interested in learning more about dictionary merging, I have another article which covers all the details including performance metrics and a little challenge. How to Invert a Dictionary https://youtu.be/lN5qX73H2Bc While working through dictionary syntax, there was one question we never asked: what if our data needs to change form? For instance, what if we actually needed to flip the keys and values. As it turns out, a lot of folks want to know how to do just that. Unfortunately, this problem isn't trivial. After all, values aren't held to the same standards as keys, so it's possible that some values can't be keys (e.g. lists). Likewise, there are other issues like handling clashes when a value appears multiple times. That said, if we ignore these kind of issues, we can get a pretty rough inversion as follows: my_dict = {} # Add items as needed my_inverted_dict = dict(map(reversed, my_dict.items())) This will create a new dictionary where the keys and values are swapped. Of course, be aware that a solution like this doesn't account for the issues I outlined above. For example, duplicate values will clash, and only one of the keys will remain. If you're interested in learning more about dictionary inversion, feel free to check out the YouTube video above or head on over to the original article. In either case, you get a ton details including performance metrics and challenges. How to Perform a Reverse Dictionary Lookup In the previous section, we talked about completely inverting a dictionary. However, if the dictionary is too big (or it doesn't fit the requirements for inversion), it might not make sense to invert the whole dictionary. Instead, we might choose to look up a single key given a value. Unfortunately, there isn't a clean way to do this. Beyond writing a loop to search for the value, the best solution is probably to use a generator expression: my_dict = {} # Populate as needed value_to_find = "" # Pick a value to find key = next(key for key, value in my_dict.items() if value == value_to_find) Here, we use a generator expression to produce an iterator of values that match the value we're trying to find. Then, we take advantage of the next() function to retrieve the first value in that iterator. Naturally, a solution like this is only going to return a single matching key. If you want something to return all matching keys, the generator expression can be used in a loop: generator = (key for key, value in my_dict.items() if value == value_to_find) keys = [] for key in generator: keys.append(key) Of course, at that point, it probably makes more sense to use a list comprehension to generate the list of keys outright. To each their own, I suppose. At any rate, if you're interested in learning more about reverse dictionary lookups, there's an article for that. As always, it includes additional information like performance metrics and challenges. Summary To summarize, a dictionary is a data structure where keys are mapped to values. This structure has a lot of perks including the ability to lookup values quickly by key rather than index. As a result, dictionaries lend themselves best to applications where their strengths can shine like managing data from a spreadsheet or making simple objects. Naturally, this article couldn't exist if I had already done a ton of writing on the subject, so here are some relevant articles: How to Loop Over a Dictionary in Python How to Perform a Reverse Dictionary Lookup in Python How to Merge Two Dictionaries in Python Likewise, if you liked this article, I'd appreciate it if you showed your support. To make that easier, I've created a list of ways you can do just that. In particular, you'll find links in that article to my Patreon and YouTube channel. Thanks for your help! In addition, you may find value in the following Python resources on Amazon (ad): Otherwise, thanks for taking some time to check out the site. I appreciate the support!