Python – Page 19 – The Renegade Coder

When it comes to working with data structures, one question constantly emerges: how do I find out if the data I want actually exists? Well, in this article, we're going to answer that question for dictionaries in Python. Specifically, we'll be learning how to check if a key exists in a dictionary. Of course, I won't bury the lede here. Perhaps the best way to check if a key exists in a dictionary is to use the in keyword. For example, we might take some dictionary (e.g., my_dict) and check if it contains some key (e.g., "peach") as follows: if "peach" in my_dict: pass. If that's all you needed, I'd appreciate it if you took a moment to show this site some support. For instance, you can check out my list of ways to help grow The Renegade Coder which includes links to my YouTube channel and Patreon. Otherwise, keep reading to learn about a few other options including try/except and the get() method. Likewise, we'll talk performance, and I'll even ask you to complete a little social media challenge. Problem Description In Python, a dictionary is a map-like data structure. In other words, it allows us to map pairs of values—much like a dictionary of words and definitions. In fact, the following terms and definitions could be stored in a Python dictionary: Fire: combustion or burning, in which substances combine chemically with oxygen from the air and typically give out bright light, heat, and smoke. Wood: the hard fibrous material that forms the main substance of the trunk or branches of a tree or shrub, used for fuel or timber. Glass: a hard, brittle substance, typically transparent or translucent, made by fusing sand with soda, lime, and sometimes other ingredients and cooling rapidly. It is used to make windows, drinking containers, and other articles. To convert these terms and definitions into a Python dictionary, we can use the curly brace syntax: my_dict = { "fire": "combustion or burning, in which substances combine chemically with oxygen from the air and typically give out bright light, heat, and smoke.", "wood": "the hard fibrous material that forms the main substance of the trunk or branches of a tree or shrub, used for fuel or timber.", "glass": "a hard, brittle substance, typically transparent or translucent, made by fusing sand with soda, lime, and sometimes other ingredients and cooling rapidly. It is used to make windows, drinking containers, and other articles." } Of course, language dictionaries often store multiple definitions. Naturally, we can do the same thing with python dictionaries. All we have to do is convert our string definitions to lists of definitions: my_dict = { "fire": [ "combustion or burning, in which substances combine chemically with oxygen from the air and typically give out bright light, heat, and smoke." ], "wood": [ "the hard fibrous material that forms the main substance of the trunk or branches of a tree or shrub, used for fuel or timber." ], "glass": [ "a hard, brittle substance, typically transparent or translucent, made by fusing sand with soda, lime, and sometimes other ingredients and cooling rapidly. It is used to make windows, drinking containers, and other articles.", "a drinking container made from glass." ] } The reason we're able to do this is because Python dictionaries store information in pairs. In this case, each term is known as a key which we use to retrieve a definition (aka a value). In other words, key-value pairs are the building blocks of Python dictionaries. Now, you're likely here because you already knew all that. In fact, you're probably already familiar with this structure. However, the problem you've run into is that sometimes the key you're looking for doesn't exist. This is a pretty common problem. For instance, what if we tried to search for the word "shrub" in our current dictionary? Unfortunately, the program would throw an exception: >>> my_dict["shrub"] Traceback (most recent call last): File " ", line 1, in my_dict["shrub"] KeyError: 'shrub' In other words, the key doesn't exist! So, how can we avoid this? That's the topic of this article. Solutions As with many of the articles in this series, I like to share a list of potential solutions. However, not all of the solutions are totally practical. For example, I tend to share a "brute force" solution first, so we can get a deeper understanding of the problem. That said, Python is user-friendly, so there are often more practical solutions. For instance, in this article, we'll try to search through the dictionary for a matching key first. Then, we'll take a look at a few more straightforward solutions including try/except, the in keyword, and even the get() method. Not everyone appreciates learning the hard way first, so you're welcome to jump ahead. I recommend going straight to the section about the special keyword. Otherwise, let's dig in! Check If a Key Exists by Search If you've been following along, you know I recently wrote about looping over a dictionary. Well, as it turns out, looping could actually be useful here. All we'd have to do is search for the key that matches the one we want to find: term = "shrub" my_dict = {} for key in my_dict: if key == term: print(f"{term} exists in the dictionary!") Naturally, given the dictionary we've provided, this code snippet won't appear to do anything. However, if we swap out this dictionary for one that actually includes the definition for the word "shrub", we'd be in business: term = "shrub" my_dict = { "shrub": "a woody plant which is smaller than a tree and has several main stems arising at or near the ground." } for key in my_dict: if key == term: print(f"{term} exists in the dictionary!") Now that we know this works, let's go ahead and clean things up with a function: def key_exists(my_dict: dict, term: str): for key in my_dict: if key == term: return True return False With a function like this, we could easily check if some term exists by calling key_exists() as a part of some conditional: term = "shrub" my_dict = { "shrub": "a woody plant which is smaller than a tree and has several main stems arising at or near the ground." } if key_exists(my_dict, term): print(f"{term} exists in the dictionary!") Of course, writing our own search algorithm is a bit excessive. Luckily, there are better solutions to follow. Check If a Key Exists Using Try/Except Rather than writing our own search algorithm, we could opt for a much lazier approach. For instance, what if we just let the key fail as we did in the problem description? We could totally do that! We just have to add a bit of boilerplate: term = "shrub" my_dict = {} try: definition = my_dict[term] except KeyError: print(f"{term} does NOT exist in the dictionary!") Here, we've decided to wrap the part where we request a value from the dictionary in a try block. Basically, this allows us to throw any key at the dictionary without worrying about the program crashing. In fact, we know the program won't crash because of the except clause. Specifically, we plan for the program to occasionally throw a KeyError. When this happens, we catch the error and display some error message to the user. Now, if we decide to look up the word "shrub" in our dictionary, we'll get a nice little message that tells us the word doesn't exist. Of course, even this solution is a bit painful. No one wants to write this sort of boilerplate every time they have to access the dictionary. Luckily, there's an even cleaner solution in the next section. Check If a Key Exists Using the in Keyword One of the reasons I love Python is that it always seems to have everyday tasks built right into the language. For instance, we don't actually need to search for the key at all. We can make use of the in keyword instead: term = "shrub" my_dict = {} if term in my_dict: print(f"{term} exists in the dictionary!") In this example, the term "shrub" is checked against the dictionary. Since that term isn't in the dictionary, the program will do nothing. However, if that term were in the dictionary, we'd get a nice message telling us that the key exists. No longer do we need to write our own search algorithms or play around with try/except. Instead, we can leverage the in keyword to check if a key exists in the dictionary directly. Of course, while this is convenient, Python actually has another option that simplifies the value retrieval process as well. We'll take a look at that in the next section. Check If a Key Exists Using the get() Method Up to this point, we've really only concerned ourselves with checking if a key exists in a dictionary. However, in reality, we're probably going through all this hassle because we intend to retrieve the value if the key exists. For instance, here's what that might look like using the in keyword: term = "shrub" my_dict = {} definition = None if term in my_dict: definition = my_dict[term] Here, we've decided that the definition should be None if the term doesn't exist. Otherwise, we overwrite it if it does. While this is great, I tend to opt for the get() method instead. After all, it basically compresses those last three lines into a single line: term = "shrub" my_dict = {} definition = my_dict.get(term) And, that's it! No need to check if the key exists. That's baked directly into the method. If the key doesn't exist, definition will store None. What makes this method so nice is the fact that we can define default values. For example, we might decide that definition should never be None. Instead, it should store an empty string if the key doesn't exist. That said, I will warn you that there are drawbacks to using get(). For instance, I use the method a lot in my Image Titler—the tool that generates my featured images at the top of each article. If you dig through the code, you'll notice (as of June 2020, at least) that I use a dictionary to store image settings. In many cases, I set unused settings to None rather than some default value. Using None as a value has consequences when I use the get() method. After all, how can I tell the difference between a missing key and a missing value? I can't! So, doing something like the following can be a source of bugs: term = "shrub" my_dict = {} if my_dict.get(term): # Do something assuming the value is not falsy (oof!) If for some reason, the key exists but the value returned is falsy—meaning the if statement interprets the value as false (e.g. 0, None, [], etc.)—the condition will not execute. For me, this can be a great source of bugs, so I recommend exercising some caution with the use of get(). That said, I still use it extensively. With all that said, that's all the solutions I could come up with. At this point, we'll take a look at how each solution compares in terms of performance. Performance Now that we have four solutions ready to go, let's try measuring their performance. To do that, we'll use the timeit library. If you're not familiar, you can learn all about it in my performance testing article. Otherwise, we'll learn as we go! First, we'll want to collect all of our solutions in strings. However, it's important that all solutions are doing the same thing. Otherwise, the test won't make any sense. As a result, I'll have each solution store the resulting value if it exists: search = """ if key_exists(my_dict, term): definition = my_dict[term] """ exception = """ try: definition = my_dict[term] except KeyError: pass """ keyword = """ if term in my_dict: definition = my_dict[term] """ method = """ definition = my_dict.get(term) """ In addition, we'll want some sort of setup string which holds import setup information like necessary libraries, functions, and variables: setup = """ term = "shrub" my_dict = {} definition = None def key_exists(my_dict: dict, term: str): for key in my_dict: if key == term: return True return False """ Now, it's just a matter of important timeit and running our tests: >>> import timeit >>> min(timeit.repeat(setup=setup, stmt=search)) 0.1179294000000013 >>> min(timeit.repeat(setup=setup, stmt=exception)) 0.22074170000000493 >>> min(timeit.repeat(setup=setup, stmt=keyword)) 0.021504300000003695 >>> min(timeit.repeat(setup=setup, stmt=method)) 0.05840359999999123 With our results, it's quick to see which solution is the fastest. However, it's important to note the context. Here, we used an empty dictionary which didn't contain the key. The results may be different if we supply a different setup string (i.e., one that contains the key): setup = """ term = "shrub" my_dict = { "shrub": "a woody plant which is smaller than a tree and has several main stems arising at or near the ground." } definition = None def key_exists(my_dict: dict, term: str): for key in my_dict: if key == term: return True return False """ Now when we rerun our tests, we'll find a very different result: >>> min(timeit.repeat(setup=setup, stmt=search)) 0.17445049999997764 >>> min(timeit.repeat(setup=setup, stmt=exception)) 0.036218700000006265 >>> min(timeit.repeat(setup=setup, stmt=keyword)) 0.04437409999999886 >>> min(timeit.repeat(setup=setup, stmt=method)) 0.05888250000000994 Suddenly, the try/except solution is the fastest! Care to guess why? It's because it totally skips over any checking. So, what does this really mean? Well, if you had a dictionary which wasn't expected to have many "misses" (i.e., terms that don't exist), the try/except solution would probably be the way to go. Otherwise, I'd definitely recommend the in keyword. Of course, before we go crowning a victor, it's probably a good idea to expand our dictionary a bit. In the table below, you'll find performance metrics trimmed to 4 decimals places for each solution under different circumstances (i.e., hit vs. miss and size of dictionary): AlgorithmEmpty1 (Hit)10 (Miss)10 (Hit)100 (Miss)100 (Hit)Search0.11790.17440.44140.39582.82020.4021Exception0.22070.03620.22580.04060.23080.0409Keyword0.02150.04430.02430.05340.02640.0536Method0.05840.05880.06060.06450.06450.0651Table 1: Timings, in seconds, of a handful of key existence algorithms. In hindsight, it might have been easier to look at the "misses" in one group and the "hits" in a separate group, but WordPress tables don't exactly make it easy to move columns. At any rate, it seems that all three built-in solutions seem to run in constant time (i.e., O(1)) which is nice. As a result, my original comment still stands: stick with the in keyword unless you know you won't have many "misses". That said, take these metrics with a grain of salt. I ran them on my Windows 10 desktop which happened to be running Python 3.7.3. Your results may vary. Challenge Now that we know how to check if a key is in a dictionary, I have a little #RenegadePython challenge for you all. However, we'll need some background information. Despite talking an awful lot about dictionaries in this article, we neglected to talk about what makes a key valid. As it turns out, there are basically two main rules: No duplicate keys are allowed Keys must be immutable (e.g. int, float, tuple, etc.) Unfortunately, these restrictions still leave us open to nasty issues. For example, let's imagine a dictionary where all the keys are words—just like our terms and definitions example. What is stopping us from including the same term multiple times? After all, the dictionary has no idea that "shrub" and "Shrub" are the exact same word. To deal with this issue, we'd probably want to implement some form of case-insensitive dictionary. That way, terms that we would consider the same—ignoring proper nouns and other inconsistencies—wouldn't have separate entries in the dictionary. However, for this challenge, creating a case-insensitive dictionary is a bit overkill. Instead, I'll ask you to use your new knowledge to write a function which performs a case-insensitive search of a dictionary of terms and definitions given some user-supplied term. For example, the user might be interested in the definition of the word "shrub", but they spell it in one of the following ways: "shrub" "Shrub" "ShrUb" "shruB" To keep things simple, we'll assume that all keys in the dictionary are lowercase (although, you're welcome to take on a more complicated form of the problem). How would you go about performing that search? When you have a solution, head on over to Twitter and give it a share! For instance, here's my solution: https://twitter.com/RenegadeCoder94/status/1270232431325478915?s=20 I'm excited to see what you come up with! A Little Recap As always, here's each solution we explored in the article: term = "shrub" my_dict = { "shrub": "a woody plant which is smaller than a tree and has several main stems arising at or near the ground." } # Checks if a key exists using a homemade function def key_exists(my_dict: dict, term: str): for key in my_dict: if key == term: return True return False if key_exists(my_dict, term): definition = my_dict[term] # Checks if a key exists using try/except try: definition = my_dict[term] except KeyError: pass # Checks if a key exists using the "in" keyword *preferred* if term in my_dict: definition = my_dict[term] # Checks if a key exists using the get() method definition = my_dict.get(term) And with that, we're all finished for today. Somehow I ended up writing over 2,500 words for this! Hopefully, this helps some folks. Feel free to reach out and let me know. In the meantime, here's the usual end-of-article pitch. If you like this sort of thing, I can always use more support. Head on over to my article of ways to grow The Renegade Coder. It includes links to resources like my newsletter and YouTube channel. In addition, here are a few related articles: How to Perform a Reverse Dictionary Lookup in Python How to Merge Two Dictionaries in Python Likewise, here are some resources from Amazon (ad): Likewise, special thanks to Jamie Moore for becoming my 6th Patron! If you'd like to have your name listed in an article like this, head on over to my Patreon and pick a tier. In addition to this cool shout out, you'll also get your name listed on my Wall of Fame. Finally, thanks for taking some time to check out my site. Hopefully, you found what you needed, and you'll come back soon!