Packaging Trends in Python: Highlights from the 2023 Developer Survey

Results from the seventh annual official Python Developers Survey were published today, offering a wealth of insight from the broader Python ecosystem. The survey was conducted by the Python Software Foundation and JetBrains from November 2023 – February 2024, capturing responses from more than 25,000 Python developers and enthusiasts across ~200 countries and regions.

This comprehensive survey covers everything from general Python usage patterns and trends to the specific tools and frameworks developers are using to build modern applications. It delves into the evolving landscape of Python packaging and virtual environments, while also exploring developer preferences for learning resources and best practices. The report is a must-read for anyone interested in the current state and future direction of the Python ecosystem.

Here are a few of the most important highlights:

Popularity and Use Cases

• Python is the main language used by 85% of respondents to the survey, and 15% as a secondary language.
• Data analysis continues to be the top use case for Python, followed by web development and machine learning.
• 40% of those working in data science also use SQL, Bash/Shell (30%), and JavaScript (30%).
• In the web development use case, 44% also use SQL, Bash/Shell (36%), JavaScript (62%), and HTML/CSS (53%).
• 48% of surveyed Python developers are involved in data exploration and processing, with 32% reporting that they train ML models or generate predictions from them. scikit-learn and PyTorch are the top two solutions used for these tasks.

Approximately 1 in 4 Python developers are brand new to the language: 25% of respondents have been using the language for less than a year, 16% for 1-2 years, 25% for 3-5 years, and 19% for 6-10 years. Only 13% of those surveyed have been programming in Python for 11+ years.

A staggering 37% of Python developers reported contributing to open-source projects last year, with the vast majority being code contributions.

Frameworks and Libraries

Python developers love their web frameworks, and they are used widely. Among data scientists, 77% report using a framework. That percentage is even higher at 97% for web developers. Usage is divided fairly evenly among a diverse set of the most popular frameworks.

• Web Development: Django and Flask remain popular, with FastAPI showing significant growth, joining them in the ranks.
• Data Science: Data scientists favor Flask and FastAPI fairly equally, with Django a close runner up. When it comes to data exploration, pandas (77%) and NumPy (72%) are the most used frameworks.
• General-purpose: Requests continues to be one of the most favored general-purpose library used with Python.

Development Tools

Python developers are equally split across Linux (55%) and Windows (55%) operating systems, followed by macOS (29%).

Among database users, 43% responded that they are using PostgreSQL, 34% SQLite, and 30% MySQL, followed by MongoDB (17%), and Redis (17%). PostegreSQL has been the most popular database among Python developers for three years in a row.

Reporting on their main IDE/editor, Visual Studio Code (32%) and PyCharm (31%) are the top choices for Python development environments.

Python Packaging Trends#

Python packaging trends are of particular interest to the team at Socket, and we’re following the evolution of tools closely. The vast majority of developers are using pip to manage dependencies, followed by Conda, and Poetry. It will be interesting to see how these tools are represented in next year’s survey, following the debut of the new uv package manager this year.

The survey also asked respondents about the formats used by developers to store application dependency information. The most popular format is requirements.txt (63%), followed by pyproject.toml (32%), and setup.py (17%). Other formats like Pipfile, environment.yml, and setup.cfg each have around 8% usage. A smaller percentage use hatch.toml (2%) or other formats (1%), while 11% do not store dependency information at all.

Socket supports the top four formats in this category for the Python ecosystem.

It’s no surprise that 80% of respondents install packages from PyPI. Other popular sources inculde GitHub (28%), Anaconda (16%), and local sources (14%).

These stats are roughly the same across data science and web development use cases, with PyPI being slightly more popular for web development and Anaconda used more among data scientists.

When you take into consideration that 37% of Python developers reported contributing to open-source projects last year, it’s not surprising that approximately 25% of respondents say they have packaged and published Python applications they developed to a package repository. Twine is by far the most popular packaging tool at 53%, followed by Poetry (33%).

The dominance of pip as the primary tool for managing dependencies, alongside emerging interest in new package managers like uv, reflects a Python ecosystem poised for innovation. This trend aligns with the demographic shift towards a younger, less experienced Python user base. With nearly half of the respondents having three years or less of Python experience, there's a growing demand for tools that are both user-friendly and capable of handling more complex, modern development challenges.