I like direnv too. But if you're not planning to use uv, you might want to give pipenv a try - the officially recommended tool for the purpose. Pipenv has just one command to create a virtual environment and install packages. And while pipenv can handle traditional requirements.text file, it's real strength is pipfile - a richer format with supporting lock files.
Pipenv doesn't automatically activate the venv on entry into a shell. But a shell plugin named pipenv-activate supports this. It does what you use direnv for in this case, without an envrc file in the source.
One major difference of pipenv from vanilla venv is that pipenv creates the venv in a common location outside the project (like poetry does). But this shouldn't be a big problem, since you wouldn't commit the venv into VCS anyway.
I'm loving how we're just 3 levels down into "python installers/package management" and it's already a heap of radioactive waste. Within 3 comments, 6 different python packaging and/or environment management tools were mentioned, and as a seasoned python user, I haven't even heard of direnv yet. Every week, a new pip/py/v/dir/fuck/shit/tit/arse -env tool emerges and adds to the pile of turd that is python packaging. It's truly getting to parody level of incompetence that we are displaying in the python community here.
You know: uv works, pipenv works (even in 2024), pyenv works, direnv (non-Python but fits my use cases), pipx works, pip-tools work, pip/python -m venv/virtualenv work, poetry works (in its own opinionated way), apt/dnf/apk, etc work
uv manages python binaries, user python tools, venvs, project/script dependencies, lock files. Other tools do less or different e.g., pipenv may use pyenv to install the desired python version.
I've used all of these tools successfully (at different times and/or for different use cases).
uv is the best attempt to circumvent "no size fits all" so far.
If you think a language that has just one tool is better, then it just means your use-cases are very narrow.
I don't know any other language that have such variety of users, applications (besides C). There may be more popular languages but nothing with such range.
I'm sorry but you are showcasing everything that's wrong with the python packaging /environment ecosystem. You list 10 tools which all work to varying degrees of overlap between each other. Which one should one use? And you don't even need to answer that because the answer changes every year. Last year it was poetry, this year it's uv. Next year it will be some other silly attempt.
This is a bad joke. It's a mature language where the answer to: "I want to manage my programming language version and installed libraries" is that you have to try a dozen different tools, each of which will cover some but not all of your requirements to do this one simple thing.
> I've used all of these tools successfully (at different times and/or for different use cases).
And you see nothing wrong with that? Pretty much every modern language has one way (or at most just a small handful of ways) to build a library and manage your dependencies/environment. The reason is that packaging and environment management is a side show. A necessary evil we have to do so that the actual crux of our work can be done. When I set out on a project, I don't say: "I want to spend a week figuring out which is the current environment management tool supported by the python mindshare". I don't want to deal with a dozen different ways of installing the dependencies of a package. This is insane. When I pick up a language, I want to know what is the way of managing dependencies and packaging up my stuff. And I don't want this to change on a yearly basis, doubly so for a mature language.
> If you think a language that has just one tool is better, then it just means your use-cases are very narrow.
Yes, I prefer that there's a choice of tools for say time series analysis or running a web service. Competition in those areas is good. That's how innovation is driven forward.
When it comes to package and environment management, I don't want innovation, I want stability. I want one agreed way to do it so that I don't need to fuck around with things which are completely orthogonal to the work I actually want to do and that will put bread on the table. I don't want to spend brain cycles keeping up to date with yet another hare brained way of defining what is a python package.
In my view, the reason why we are in this quagmire is because the roots of python are in a fairly simple scripting language, and the packaging has not well escaped these roots. You can have loose python files, you can have directories containing python files which are automagically modules, then we had to hack in a way to package collections of modules and manage their different versions. It's all hacks upon hacks upon hacks, and it has to be backwards compatible all the way to being able to run a loose python file.
I'm not saying this is easy to solve. It's the posterchild of the "Situation: there are 15 competing standards" XKCD. The time to solve it would've been 15 years ago while Guido was still the BDFL. There are too many stakeholders now to get any sort of consensus.
Well, that heavily depends on what you want to do. Python has a number of concerns when it comes to code and package management, some of which are not present in most other languages. Here's an incomplete list, off the top of my head:
1. installation and management of installed packages
2. management of Python versions present in an environment
3. management of virtual environments (Python version + packages installed and ready to use, in an isolated context)
4. building of distribution packages (nowadays pretty much only wheels, with or without native extensions, which depend on the target platform)
5. publishing of distribution packages (to PyPI or a compatible repo)
6. defining repeatable deployment environments (a subset of #1)
Most developers face some combination of above problems, and various tools offer solutions for certain combinations -- with a few covering all of them, to different levels of quality. It is crucial to understand your needs and select the tool(s) that offer the right solutions in the way that fits your usual workflow the best.
This article [0] is a good starting point to understanding the current Python packaging landscape, with a clear overview of which problem is covered by which tool.
And in this very thread, people seem to accept that this is fine. There's nothing wrong with the fact that there are 6 separate tools just for building a package, some support publishing, some don't, some also manage environments or python versions? Which of these are currently supported, which are deprecated, which are going to become deprecated in 2025?
But also there's a tool which only does publishing (twine)? The diagram is not even correct, because conda itself requires package building, except it's about building conda packages, which are cross-platform/language and separate from building a python package.
Why is there a separate set of tools for package management and package publishing?
The blog post is indeed helpful to allow someone new to python to at least see what options there are and roll the dice, but it will also raise extremely serious alarm bells that there's something fundamentally rotten at the core of the python ecosystem.
The fact that I need to read some unofficial post from 2023 to gain an overview of the python packaging and environment management ecosystem is itself completely nuts. And I can guarantee you that by now this blog post is getting outdated, because some mad genius is cooking the new best tool and ready to unleash it on the unsuspecting world.
The only case where this question is rhetorical is when you do not really use Python enough to require deciding which management tool to use. Which tells me everything I need to know about you.
Yes you are right. I do not want to care about which management tool to use. Programming is a difficult to discipline as it is, the tools should make it easier, not more complicated. Some people might do programming purely as an exercise of self-fulfilment and they don't mind. For me it's a tool, a means to achieve some ends. If you think the python landscape and tools are in an optimal state, more power to you. Meanwhile, I have a dozen quants complaining at me that they are wasting time on irrelevant crap learning yet another set of pacakging tools and that the technologists need to figure out one consistent tool (or at least a stable set of tools) for managing packaging/environment concerns instead of inventing a new one each year.
But yes, let's go throwing around thinly veiled insults instead. This tells me everything I need to know about you :)
Yup - hard agree on all the Python parts, but I'll happily recommend direnv wherever possible. It doesn't do much, but it brings a ton of sanity and simplicity to shell env without including any installers - it's just an activator, and a very simple one, so it'll survive next year's python and npm and etc tool flavors with no issues.
pip solved a lot of baseline problems. It doesn't solve all of them. There were a bunch of failed attempts to be "better pip". They have not worked, really, but people stick around to some tools despite this.
I do think uv is different, on account of working and being very reactive to ecosystem pains.
(direnv is not Python-specific! It's just a tool to set env vars in a directory. But because Python venv's can work through setting two env vars....)
haha, thank you for summarising my thoughts on python package management... and very few people have mentioned poetry which is what we and most teams I know use.
I use poetry as well, but lately I've been looking at uv and had to actively stop myself because it'd be the 3rd tool I migrated to in the last 3 years.
Rust programmers seem to be lousy with ECS frameworks that never get used in any games but seem hell bent on proving that rust is the best language for game programming, and python programmers seem to break out in a case of "packaging tool building". I don't know what causes this. Perhaps some sort of pathological thinking that "I can do better"?
I really like python (and I like rust too), but if I were to take an honest look at the python packaging and environment ecosystem, I'd think that I'm being trolled. I lived through the age of setuptools.py, and while it was not good, at least there was only one approach really. Now we have a bazillion approaches that are all good, and zero consensus on what to use. Each individual tool is much better than what we had before, but the landscape has become so fractured that as a whole it's a complete shitshow.
Pipenv doesn't automatically activate the venv on entry into a shell. But a shell plugin named pipenv-activate supports this. It does what you use direnv for in this case, without an envrc file in the source.
One major difference of pipenv from vanilla venv is that pipenv creates the venv in a common location outside the project (like poetry does). But this shouldn't be a big problem, since you wouldn't commit the venv into VCS anyway.