Once again, welcome to the How to Python series. In this collection, we explore programming problems that have quick solutions in Python. In this edition, we explore a few ways to check if a file exists in Python, so let’s dive in!
Table of Contents
Problem Introduction
Recently, I was looking for a way to persist some user settings of an app to a file. For instance, I wanted to save options like font size, background color, etc. Naturally, I settled on a simple mapping format like YAML, but I could have just as easily used a CSV which we already know how to parse.
Unfortunately, the configuration file wouldn’t be backwards compatible without some support in the software. In other words, if the configuration file didn’t exist, I would need to assume some original preset values. Otherwise, I would pull the data from the configuration file.
Fortunately, I did my research and came up with a solution. The plan was to check to see if the configuration file existed. If it did, the program would read from it and populate the necessary fields. Otherwise, the program would lean back on some arbitrary preset values.
To do that though, I had to find a way to verify the existence of a file. As it turns out, there are plenty of ways to do that in Python.
Solutions
If we’re looking to check if a file exists, there are a few solutions:
- Check if a file exists with a
try/exceptblock (Python 2+) - Check if a file exists using
os.path(Python 2+) - Check if a file exists using the
Pathobject (Python 3.4+)
Of course, it’s up to us to determine which solution is the best for us!
Check if a File Exists with a Try Block
Up first on the list is a simple try-except block
. In this scenario, we would attempt to open our file in the try block. If the file fails to open, we run the preset values. For example:
try:
with open('/path/to/file', 'r') as fh:
# Load configuration file values
except FileNotFoundError:
# Keep preset values
In this example, we try to open the file using a with statement. If successful, we can load our configuration file values. Otherwise, the open command will throw an error that we catch in the except block. At that point, we keep our preset values and move on.
To be honest, this solution is perhaps the simplest and most robust, but the FileNotFoundError is an update from Python 3. You’ll have more trouble with catching an IOError in Python 2.
Also, it’s worth noting that we use a with statement in this example. As an alternative, we can open the file directly. We just have to remember to close it when we’re done:
Check if a File Exists with OS Path
Another option is to skip error handling altogether and directly verify that the path exists. For example:
import os
exists = os.path.isfile('/path/to/file')
if exists:
# Load configuration file values
else:
# Keep presets
Of course, the drawback here is the race condition from line 2 to line 4. If for some reason the configuration file gets deleted between line 2 and line 4, then the script will crash. If that’s not a risk in your application, then this solution is great.
That said, I’ll emphasize this issue again because I did get a bit of pushback on it in the past:
Checking if a file exists and opening that file are two different operations in this example. They would need to be one atomic operation to avoid a race condition.
Check if a File Exists with a Path Object
If you’re obsessed with object-oriented programming like me, then maybe this solution is for you. As of Python 3.4, we can wrap our file reference in an object which brings along a host of new functionality. For example:
from pathlib import Path
config = Path('/path/to/file')
if config.is_file():
# Load configuration file values
else:
# Keep presets
In addition, this new object representation allows us to use our original try-except block:
try:
absolute_path = config.resolve()
# Load configuration file values
except FileNotFoundError:
# Keep presets
Of course, we may not need all this functionality. After all, if reading the contents is the goal, then the first option is probably the best.
Performance
While we’ve already shared all the solutions, it may be important for our application to look at performance as well. To do that, we’ll leverage the timeit library. First, however, we need to generate a few strings:
setup = """
import os
from pathlib import Path
"""
try_except = """
try:
with open('/path/to/file', 'r') as fh:
pass
except FileNotFoundError:
pass
"""
os_isfile = """
exists = os.path.isfile('/path/to/file')
"""
path_lib = """
config = Path('/path/to/file')
if config.is_file():
pass
"""
With the strings ready to go, we’ll run this test twice: once where the file exists and again when it doesn’t, respectively.
>>> timeit.timeit(stmt=try_except, setup=setup) 25.758140300000036 >>> timeit.timeit(stmt=os_isfile, setup=setup) 23.302945200000067 >>> timeit.timeit(stmt=path_lib, setup=setup) 36.851380800000015
Normally, we would use the repeat function to try to calculate some sort of lower bound for each function, but it was just way too slow. Feel free to try it and share the results.
For the existing file tests, we’ll have to change the paths in each of the strings above, so they include an existing file. As a result, some of these solutions are significantly slower:
timeit.timeit(stmt=try_except, setup=setup) 220.5547474 >>> timeit.timeit(stmt=os_isfile, setup=setup) 194.13558469999975 >>> timeit.timeit(stmt=path_lib, setup=setup) 208.86859360000017
Here, we can see all of the solutions are quite a bit slower when dealing with an existing file. That said, it seems the os solution is the fastest in both circumstances. Of course, it does have the race condition drawback, so be sure to take that into account when choosing one of these methods.
For reference, all tests were completed using Windows 10 and Python 3.7.3. If you want to learn more about Python performance testing, check out my article titled How to Performance Test Python Code.
A Little Recap
Using the methods above, we have several options to check if a file exists in Python:
# Brute force with a try-except block
try:
with open('/path/to/file', 'r') as fh:
pass
except FileNotFoundError:
pass
# Leverage the OS package
import os
exists = os.path.isfile('/path/to/file')
# Wrap the path in an object for enhanced functionality
from pathlib import Path
config = Path('/path/to/file')
if config.is_file():
pass
For the purposes of this tutorial, we were only interested in files. However, these solutions can be adapted to verify the existences of directories and symbolic links, so don’t be afraid to play around. That’s the beauty of Python!
At any rate, thanks for taking the time to check out this article. If it’s your first time here and you found this article helpful, why not subscribe to The Renegade Coder? Alternatively, you can always hop on the mailing list and decide to become a member at a later time.
In the meantime, you may benefit from my list of Python Code Snippets for Everyday Problems.
If you’re not convinced, check out some the following related posts:
- How to Parse a Spreadsheet in Python
- Rock Paper Scissors Using Modular Arithmetic
- Make Featured Images Just Like The Renegade Coder
Finally, feel free to browse the following related books:
- Learn Python 3 The Hard Way by Zed A. Shaw
- Python for Kids: A Playful Introduction to Programming by Jason R. Briggs
See you next time!
Recent Posts
Teaching at the collegiate level is a wonderful experience, but it's not always clear what's involved or how you get there. As a result, I figured I'd take a moment today to dump all my knowledge for...
It's been a weird week. I'm at the end of my degree program, but it's hard to celebrate. Let's talk about it.