Migrating to Poetry 2.x With Some Best Practices

Poetry 2.x was released in early 2025, and we just got around to migrating several of our open-source projects to the new major version. As a result, I wanted to share some of the lessons learned. Introducing Poetry 2.0 Earlier this year, poetry released its second version, resulting in the usual backwards compatibility issues. Now, in alignment with PEP 621, most of the poetry metadata can be listed in the [project] table of the pyproject.toml file. This is great news for a variety of reasons. For example, because the python standard now has a way of specifying most of a project's metadata, you should be able to more easily swap between build tools (e.g., poetry, setuptools, flit, etc.). With that said, any changes to the standard is going to require a bit of rework for any existing projects. In our case, we had several projects relying on 1.x versions of poetry, so it was a little bit of a challenge to migrate to poetry 2.x. In addition, the process (and some of my helpful colleagues *cough* rzuckerm *cough*) taught me several poetry best practices. As a result, I wanted to share some of them with you! Anatomy of a TOML File Before we can even talk about the changes between Poetry 1.x and Poetry 2.x, I figured I'd learn a little bit about the TOML file format and share that with you. Then, I'll share the before and after of the pyproject.toml file in the Sample Programs repo that I help maintain. To start, it's helpful to know that TOML (Tom's Obvious Minimal Language) is really not that different from most data config file formats like JSON and YAML. The only differences being that TOML basically eliminates any need for nesting, so the files are often a little easier to browse as they're almost completely flat. Of course, without nesting, it may be less obvious to see how values are related in a TOML file, but that's not the point of this article. Anyway, TOML works by using key/value pairs just like JSON and YAML (e.g., color = "purple" vs. {"color": "purple"} vs. color: purple, respectively). The difference being that if you want to show hierarchical relationships, you use what TOML calls a table: [item] color = "purple" size = "large" type = "sword" Of course, a table is really just another word for a dictionary. In this case, the [item] table has the key/value pairs for "color", "size", and "type." Naturally, you might be wondering how you would create nested tables (i.e., sub-tables), and that's entirely possible with only a slight difference in syntax: [item] color = "purple" size = "large" type = "sword" [item.sprite] path = "path/to/sprite.png" mode = "read" author = "unknown" Of course, you have to be careful not to define a table that overwrites an existing key/value pair. Beyond that, there aren't really many other features of TOML files beyond arrays, but you'll see some of those shortly. With that said, let's first take a look at an existing pyproject.toml file for Poetry 1.x. Then, we'll look at the updated version for poetry 2.x. After that, we'll look at some tips for upgrading your files. Examining a 1.x pyproject.toml File For our Sample Programs repo, we make use of a couple of dependencies to ensure we can generate READMEs and test all of our code. To track those dependencies, we've made the conscious choice to use poetry, which results in a pyproject.toml file that looks like this: [tool.poetry] name = "sample-programs" version = "2023.05.04" description = "Sample Programs in Every Programming Language" readme = "README.md" license = "MIT" package-mode = false # Initiator of the collection authors = [ "Jeremy Grifski " ] # Major contributors along the way maintainers = [ "auroq", "rzuckerm", "stuin", "noah11012", "heksterb", "msj2", "chrboe", "2Clutch", "alope107" ] [tool.poetry.dependencies] python = "^3.8" ronbun = "^0.8.1" glotter2 = "^0.9.0" [tool.pytest.ini_options] console_output_style = "count" As you can see, there is no [project] table here. Instead, the main table is titled [tool.poetry] meaning that a separate table would need to be made for each build tool. Otherwise, there shouldn't really be any surprises here. We have a table for all our project metadata, and then we have tables for dependencies and tool options. Examining a 2.x pyproject.toml File Naturally, our pyproject.toml file had to change a bit to accommodate the new standard. To give you a feel for what new config files might look like, here's the updated pyproject.toml file: [project] name = "sample-programs" version = "2025.03.10" description = "Sample Programs in Every Programming Language" readme = ".github/README.md" requires-python = ">=3.9, =0.8.1, =0.10.1,<0.11.0)" ] # Initiator of the collection authors = [ {name = "Jeremy Grifski", email = "jeremy.grifski@therenegadecoder.com"} ] # Major contributors along the way maintainers = [ {name = "auroq"}, {name = "rzuckerm"}, {name = "stuin"}, {name = "noah11012"}, {name = "heksterb"}, {name = "msj2"}, {name = "chrboe"}, {name = "2Clutch"}, {name = "alope107"} ] [tool.poetry] package-mode = false [tool.pytest.ini_options] console_output_style = "count" As you can see, there are a few key changes. First, the [tools.poetry] table has been relabeled as the [project] table. In addition, the Python version was moved out of the dependencies table and mapped directly to the new "requires-python" key as a string. Likewise, the dependencies were moved out of the [tool.poetry.dependencies] table and mapped directly to the new "dependencies" key as an array. In addition, there are a handful of minor changes that may cause you some headache. For example, when listing authors and maintainers, the expectation is to use inline tables rather than strings directly. Likewise, there are still holdover features from the [tool.poetry] table that cannot be placed in the [project] table for the time being. Migrating to Poetry 2.x the Hard Way Now that we are familiar with the TOML file format and some of the differences between Poetry 1.x and Poetry 2.x, let's talk about how to migrate to 2.x the hard way. I kept this section on the "hard way" just to detail my struggles with migrating the file by hand. If you would prefer not to takeaway the wrong lesson, feel free to jump ahead. First and foremost, Poetry 2.x has a wonderful command you can use to validate your existing pyproject.toml file; it's called poetry check, and you can run it at any time to check the format of your config file. For example, if I take the 1.x config file from above and drop it into some random folder on my computer, I will get the following warnings from poetry 2.1.1 when I run poetry check: poetry check Error: Declared README file does not exist: README.md Warning: [tool.poetry.name] is deprecated. Use [project.name] instead. Warning: [tool.poetry.version] is set but 'version' is not in [project.dynamic]. If it is static use [project.version]. If it is dynamic, add 'version' to [project.dynamic]. If you want to set the version dynamically via `poetry build --local-version` or you are using a plugin, which sets the version dynamically, you should define the version in [tool.poetry] and add 'version' to [project.dynamic]. Warning: [tool.poetry.description] is deprecated. Use [project.description] instead. Warning: [tool.poetry.readme] is set but 'readme' is not in [project.dynamic]. If it is static use [project.readme]. If it is dynamic, add 'readme' to [project.dynamic]. If you want to define multiple readmes, you should define them in [tool.poetry] and add 'readme' to [project.dynamic]. Warning: [tool.poetry.license] is deprecated. Use [project.license] instead. Warning: [tool.poetry.authors] is deprecated. Use [project.authors] instead. Warning: [tool.poetry.maintainers] is deprecated. Use [project.maintainers] instead. If you're like me, then you're probably just going to try to resolve these warnings one-by-one. Ignoring the README issue, we can start with the warning that reads: "[tool.poetry.name] is deprecated. Use [project.name] instead." That one's easy enough! We just have to move the "name" key into the [project] table. The same goes for "version", "description", "readme", "license", "authors", and "maintainers". In fact, you may find that just renaming the [tool.poetry] table to [project] is all you need to do. However, once you do this, you will be greeted with new warnings when you run poetry check: poetry check The Poetry configuration is invalid: - project.authors[0] must be object In this case, the error is as I stated before: authors can no longer be strings. So, you might have to go through and manually change all of your strings to inline tables (i.e., objects). It's a pain that even caused me to make a typo or two, but it can be done. Once you're done updating your authors, you'll probably have to do the same thing for maintainers (if that's something you track). Then, when you run a poetry check again, you'll be greeted with some new warnings: Warning: [tool.poetry.dependencies] is set but [project.dependencies] is not and 'dependencies' is not in [project.dynamic]. You should either migrate [tool.poetry.depencencies] to [project.dependencies] (if you do not need Poetry-specific features) or add [project.dependencies] in addition to [tool.poetry.dependencies] or add 'dependencies' to [project.dynamic]. Warning: [tool.poetry.dependencies.python] is set but [project.requires-python] is not set and 'requires-python' is not in [project.dynamic]. At this point, I'm going to direct your attention to the last warning first. Specifically, it states that the Python version must be declared under a new project key called "requires-python". While this should be a relatively easy change, there's one catch: the carat syntax that you may have previously used is not supported (e.g., "^3.8"). Instead, you'll have to specify the Python version explicitly using relational operators (e.g., ">=3.8,<4.0"). Of course, if you're going to set a minimum Python version, it should probably be at least 3.9 as 3.8 is no longer supported in Poetry 2.x. But don't blame Poetry! Python has a strict release cycle that results in a minor version reaching end of life annually. For instance, Python 3.9 will reach end-of-life as soon as October 2025. Finally, all that's left is to migrate the dependencies over as stated in the first warning above. Specifically, Poetry is giving you the option to set your dependencies statically or have them managed dynamically by a separate tool. Most likely, you're going to want to set them statically. Unfortunately, the format for setting dependencies is quite different from Poetry 1.x. Rather than having a sequence of key/value pairs, you'll need to set your dependencies as strings in an array (almost exactly the opposite of what we have to do for authors). Like the "requires-python" key, dependencies also do not respect the carat syntax. Ultimately, migrating dependencies is a painful process. This is by far the largest source of errors as you'll be manually converting the dependencies to strings with the new version syntax. Just make sure you sync your lock file to these manual changes by running poetry lock when you're done. You can follow that up with poetry install to get all the dependencies listed in the lock file. Fortunately, in the later sections, I'll show you a better way of handling dependencies. Of course, before we get there, I should mention one last thing: poetry check is not foolproof, at least at the time of writing. If you naively renamed your [tool.poetry] table to [project] like I did, some of the poetry specific keys may still be hanging around. It could be that Poetry respects these keys even if they're in the wrong section. However, other tools, like the "Even Better TOML" VS Code extension, will flag these keys as unexpected: Additional properties are not allowed ('package-mode' was unexpected) Even Better TOML Otherwise, I wouldn't have known to move the "package-mode" key to the [tool.poetry] table. Anyway, I just wanted to mention that before we move on to some better practices. Migrating to Poetry 2.x Using Best Practices The primary difference between migrating the hard way and following best practices is when you get to the part where you start migrating dependencies. As it turns out, it's not really recommended to mess with the dependencies in your config file directly. Instead, folks like rzuckerm recommend using the command line tools provided by poetry directly to manipulate any dependencies. Specifically, there is a command that you probably already use to add dependencies to Poetry: poetry add. While you can use this tool to add the latest version of any library to your project, it's more beneficial to us to maintain the exact versions we're already using. To do that, we just need to modify the command slightly: poetry add ronbun@^0.8.1 Unfortunately, Poetry doesn't automatically adopt the new standard when you try to do this. As a result, I recommend manually adding the dependencies key under the [project] table as follows: [project] dependencies = [] Now when you run the add command, the dependency will show up in the new array rather than as a key/value pair in the [tool.poetry.dependencies] table. Note that Poetry is not going to delete the listed dependencies in the [tool.poetry.dependencies] table, so you may end up with something like this: dependencies = ["ronbun (>=0.9.0,<0.10.0)"] [tool.poetry.dependencies] glotter2 = "^0.9.0" ronbun = "^0.8.1" Notice how the new version of ronbun is listed after running the add command but the old version is still listed as well. It will be up to you to delete the [tool.poetry.dependencies] table when you're done migrating each dependency by hand. A word of caution! If you choose to upgrade any of your dependencies as you're performing this migration, the dependencies between [project] and [tool.poetry.dependencies] will be out of sync. At that point, the poetry add command will no longer work. Instead, you will be greeted with the following error: Cannot enrich dependency with incompatible constraints: ronbun (>=0.9.0, =0.8.1,<0.9.0) As a result, you will probably want to delete the broken dependency from the [tool.poetry.dependencies] table manually. The upshot here being that you can continue to use the carat syntax, and it will be translated for you. When you're done, you might choose to run poetry update to update dependency versions and install them (as well as update the lock file), but know that the add command already keeps everything in sync. Migrating to Poetry 2.x Using Tooling As it turns out, some folks have already put together tools to hopefully provide pyproject.toml migration automatically. One tool that I found is called poetry-plugin-migrate, and it ran me through a series of prompts to migrate the pyproject.toml file for me. Because in my example the readme.md file did not exist, the plugin didn't actually work on the first try. That said, when I removed the readme key, I was given the following file after answering all the prompts with "static" as my preference: [tool.poetry] package-mode = false # Initiator of the collection # Major contributors along the way requires-poetry = '>=2.0,<3.0' [tool.pytest.ini_options] console_output_style = "count" [project] name = "sample-programs" description = "Sample Programs in Every Programming Language" license = "MIT" version = "2023.05.04" authors = [{name = "Jeremy Grifski", email = "jeremy.grifski@therenegadecoder.com"}] maintainers = [{name = "alope107"}, {name = "2Clutch"}, {name = "chrboe"}, {name = "msj2"}, {name = "heksterb"}, {name = "noah11012"}, {name = "stuin"}, {name = "rzuckerm"}, {name = "auroq"}] requires-python = '>=3.8, =0.8.1, =0.9.0,<0.10.0)'] Even here, I think the file needs some cleanup for readability, but I was quite pleased with how it turned out. And of course, the plugin itself recommends running poetry lock && poetry install immediately after the new pyproject.toml file is generated. In addition, the plugin is nice enough to provide a backup of your old file if you run into issues. Reflecting on the Process Because all software changes over time, it can be challenging to deal with all the hiccups along the way. For instance, I use a series of plugins on this site (e.g., the syntax highlighter you see in this article), and they often get fewer and fewer updates over time leading to deprecation by the plugin repository. Of course, I see this pretty regularly even in Python with packages that no longer work because they're not being updated (e.g., discord.py went on a huge hiatus a few years ago). That's just the reality of the business. On the opposite end, you have projects making major progress as they improve and mature. In this case, Python started moving away from the cryptic requirements.txt and setup.py files to a move data driven configuration file format. This has been a long time coming for Python, but it leads to changes that are needed in existing projects. For every project like ours that meets the demands of the new technology, probably a dozen become legacy code. As I grow older, my free time gets eaten away, so I can easily see how projects slowly make their way to the GitHub graveyard (how's that for a throwback?). These days, it's hard to even write for this site when I'm trying to raise a toddler, grade assignments for my courses, and plan trips to see friends and family. Hell, next week I'm helping my sister move! Then in just a few weeks I'll be running a study abroad program in Japan. I think all of this is to say thank you to the wonderful people who maintain open-source projects. You're doing great work! I have a special admiration for rzuckerm, who has put countless hours into projects like Sample Programs. There have been others along the way as well that helped it mature, and I assume life also caught up to them as well (e.g., auroq comes to mind). Not sure how to quite wrap this one up, so I'll just say that hopefully this article helps someone. If so, there's plenty more where that came from: Migrating From Eclipse to VS Code: The Many Hurdles How to Migrate to SnakeMD 2.0.0 How to Move Your Extensions Folder in VS Code Otherwise, thanks again for taking the time to read this! If you want to take your support to the next level, check out my list of ways to grow the site. If not, no worries! Hopefully, you will be back.