Poetry Is The Best Way to Manage Your Python Projects

Recently, I shifted over to developing Python packages with Poetry, and I swear I will never look back. Seriously, I never praise anything to this extent, except maybe Python itself. Perhaps, you’ll find some love for it too.

Table of Contents

Python Project Management

Not long ago, I wrote a nice little article on how you could go about versioning your Python projects. While writing that article, I was digging around GitHub repos to find ways that other folks version their projects. Very quickly, I realized I had become a boomer: no one was using the setup.py files anymore.

Perhaps you already knew that the Python community is beginning to move on to the pyproject.toml standard. In fact, there’s an entire PEP about it. But I only found out because the pip documentation lists setup.py as the legacy solution:

The interface documented here is retained currently solely for legacy purposes, until the migration to pyproject.toml-based builds can be completed.

Pip Documentation, May 2023

Generally, I would assume these sort of migrations would be a nightmare, but I found the new file format to be almost seamless. Though, I suppose I have one tool to thank for that: Poetry. Today, I’m going to do a bit of free promotion for the folks at Poetry. It really is that good.

The Old Workflow

If you’re not familiar with Poetry, I would describe it as a package and dependency management tool. In a lot of ways, it’s like pip + venv + setuptools + twine, but it’s significantly more powerful. Let me walk you through it!

Prior to Poetry, I would have to handle a lot of the struggles of using Python myself. For example, a piece of advice you might get from intermediate and expert Python users is to use a virtual environment to contain your dependencies. This usually involves creating a folder to contain your project and then running the following commands (on Windows):

python -m venv .venv
.venv\Scripts\activate.bat

These commands create a closed off environment, so you can install exactly what you need. Therefore, a typical follow-up command might be to install all of the necessary dependencies from a requirements.txt file:

pip install -r requirements.txt

Assuming all goes well, then (and only then) can you start working on your project. From there, if you wanted to actually publish your code as its own package, you’d have to define a setup.py file—which makes use of setuptools—that contains all your dependency constraints, among other things. If I recall correctly, the command for actually building your package is as follows:

python setup.py build

With the package built, we can then deploy the result using twine:

twine upload build/*

Now, there are of course 1,000 steps in between like testing, documentation, etc. And I love Python, so I was always willing to put up with this workflow. After all, you can and should automate most of this in your continuous integration pipeline. However, Poetry reduces our pain significantly.

The Poetry Workflow

With poetry, we can begin deleting a handful of files. For example, the setup.py file is completely replaced by the pyproject.toml file. The same can be said for any requirements.txt files. Similarly, we have no need for venv, pip, setuptools, or twine; Poetry does it all. Let me show you!

To start, here is the complete pyproject.toml file for my SnakeMD repo:

# Poetry settings
[tool.poetry]
name = "SnakeMD"
description = "A markdown generation library for Python."
version = "2.2.0b1"
license = "MIT"

authors = [
    "Jeremy Grifski <jeremy.grifski@therenegadecoder.com>"
]

readme = "README.md"
homepage = "https://www.snakemd.io"
repository = "https://github.com/TheRenegadeCoder/SnakeMD"
documentation = "https://www.snakemd.io/en/latest/docs/"

classifiers=[
    "Development Status :: 5 - Production/Stable",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3.8",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Operating System :: OS Independent",
    "Topic :: Documentation :: Sphinx",
]

[tool.poetry.urls]
Changelog = "https://www.snakemd.io/en/latest/version-history/"

[tool.poetry.dependencies]
python = "^3.8"

[tool.poetry.group.test.dependencies]
coverage = "^7.2"
markdown = "^3.4"
pytest = "^7.3"

[tool.poetry.group.docs.dependencies]
sphinx = "^6.2"
sphinx-issues = "^3.0"
sphinx_rtd_theme = "^1.2"

[tool.poetry.group.analysis.dependencies]
black = "^23.3"
isort = "^5.12"
pydocstringformatter = "^v0.7"
pylint = "^2.17"

# Pytest settings
[tool.pytest.ini_options]
minversion = "7.3"
testpaths = [
    "tests"
]
log_file = "tests/pytest.log"
log_file_level = "DEBUG"
log_file_format = "%(asctime)s [%(levelname)s] (%(filename)s:%(lineno)s) %(message)s"
log_file_date_format = "%Y-%m-%d %H:%M:%S"

# Coverage settings
[tool.coverage.run]
branch = true
source = [
    "snakemd"
]

[tool.coverage.report]
fail_under = 95

# Black formatting settings
[tool.black]
line-length = 88
target-version = ['py38', 'py39', 'py310', 'py311']

# Pylint settings
[tool.pylint.format]
max-line-length = 88
fail-under = 9.5
max-args = 8

[tool.pylint.messages_control]
disable = [
    "C0302"
]

# isort setttings
[tool.isort]
profile = "black"
py_version = 311

# pydocstringformatter settings
[tool.pydocstringformatter]
write = true
strip-whitespaces = true
split-summary-body = false
max-line-length = 88

# Build system settings
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Most likely, it looks like a wall of text at about 100 lines. But, it’s really cool! Not only does it contain all the information we need to be able to deploy a package, but it also contains all of the dependencies as well as settings for various tools like testing and linting.

For our purposes, I’d like to go through how we can replicate the old workflow with a handful of commands. To start, there is no need to create a virtual environment, Poetry handles this automatically through the install command:

poetry install

While we don’t need to activate the virtual environment like before—though we can if we like—we do need to preface anything we do now with poetry run.

In either case, Poetry will resolve all the dependencies for us, and place them in a lock file. I am obsessed with the concept of a lock file. It pins every single dependency to a specific version, so anyone running our code is running it with the exact same dependencies. Whenever you want to update any dependencies, you can quickly generate a new lock file as follows:

poetry lock

There was frankly no easy way to do this before, and I didn’t even bother covering it in the old workflow. You could potentially pin all your versions in your requirements.txt file, but you’d have to resolve the dependencies more-or-less yourself. You’d also just have to be intimately familiar with all your dependencies, which isn’t reasonable.

Anyway, as soon as you’re ready to package your code for deployment, you run the following command:

poetry build

And to publish? Well, you guessed it:

poetry publish

In fact, you can roll them both into one command:

poetry publish --build

Call to Action

Overall, it’s hard to overstate how much simpler my life has gotten since switching over. I don’t have to bother with dependency management. I don’t have to think about deploying packages. I don’t have to bother with maintaining multiple key files for a package. Hell, I don’t even have to worry about virtual environments.

All I have to worry about is what goes in the pyproject.toml file. Poetry handles the rest. For anyone who had the displeasure of working with the setuptools ecosystem in the past, you know how welcome this change is.

And for folks who have been using Poetry for a while, you’re probably laughing at all of us who took so long to get here. I really mean it when I say that I’ve rushed to switch all my projects over. It’s saved me so much time and energy because it just works.

Every day, it seems like tech folks are obsessing over the latest fad related to productivity, right? Like how many times have you seen someone tweet, “ChatGPT saves me 1,000s of hours a week.” Yet, here I am completely bewildered by the quality of life I’ve gained from switching over to Poetry.

With all that said, I appreciate you taking the time to read this article and following me on my developer journey. As usual, if you liked this article, you might like some of these:

• How to Version Your Python Projects for Pip
• I Finally Figured Out Python’s Module and Package System
• 5 Ways to Write Hello World in Python

Likewise, if you want to support the site a bit more, you can always check out my list of ways to grow the site. Alternatively, you can browse some of these Python resources (#ad):

• Effective Python: 90 Specific Ways to Write Better Python
• Python Tricks: A Buffet of Awesome Python Features
• Python Programming: An Introduction to Computer Science

Otherwise, take care and see you next time!