Reference

https://ryanking13.github.io/2021/07/11/python-packaging.html (Korean)

https://packaging.python.org/en/latest/guides/writing-pyproject-toml/

Packaging

Say you want to use a function from your project in a Jupyter notebook or another script outside your project directory. You try to import it and get ModuleNotFoundError. This happens because Python doesn’t know where to find your code.

The solution? Package your code properly so you can install and import it from anywhere.

In this post, I will summarize two ways to set up Python projects.

setup.py

Let’s say that your file structure is like below:

1
2
3
4
5
6
7
8

project_package_name/
├── project_package_name/      your package file 
│   ├── tests/                 test package  
│   │   ├──__init__.py      
│   │   ├──README.md
│   ├── __init__.py      
│   └── code.py                code of the main package  
└── setup.py         

Then in setup.py, you can write the code like below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# setuptools setup.py example
from setuptools import setup, find_packages

setup(
    name='myproject',
    version='1.0',
    packages=find_packages(include=['focal_project_file'],exclude = ("tests",)),
    install_requires=[
        'requests',
        'numpy>=1.14.5',
    ],
    author='Your Name',
    author_email='youremail@example.com',
    description='A short description of your project',
    url='https://github.com/yourusername/myproject',
)

• name : The name of the package, which pip will refer. The name can be different from the file name.
• version: The version number of your project. ( look at https://www.python.org/dev/peps/pep-0440/)
• packages: A list of packages to include/exclude in your distribution. You can just use find_packages() without any parameters. This function will automatically find the package files including __init__.py.
• install_requires: A list of dependencies required by your project.

Problems of setuptools

The fundamental issue of setup.py and setuptools lies in the fact that setup.py files are inherently dependent on setuptools. While setuptools has been widely used for building Python packages, it’s important to note that it is not part of Python’s standard library. This means that projects can potentially use alternative build systems. However, the current Python packaging system, relying on setup.py, creates a dependency on setuptools. Without setuptools, building packages becomes impossible.

Consequently, if developers wish to use a different build system other than setuptools, they still need to declare the use of that system within setup.py, which again requires setuptools.

The most interesting issue is that when developers require specific features supported by a particular version of setuptools, they need to specify this dependency within setup.py. However, if the version of setuptools installed on the system is different, it can lead to compatibility issue. This is the so-called chicken-and-egg problem.

pyproject.toml

Example code: sampleproject

To address the limitations mentioned above, pyproject.toml, which is a declarative configuration file that is not tied to a specific build system was proposed.

Example code

In the toml file, you can declare which build backend you will use. For instance, you can still use setuptools like below.

1
2
3

[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

For the require key, you can list of dependencies including the version constraint likerequire = ["setuptools >= 61.0"].

Detail explanation of the pyproject.toml with setuptools can be found here.

Once you have your toml file ready, you can install your package by running pip install -e .

Alternative Build Tools

poetry

https://python-poetry.org/

flit

https://flit.pypa.io/en/stable/