blog

commit

ff1e3d8a815a451a1ee38e709032730805d45760

parent

970a58570e15a9077043bfcafe026ba01f0f8aba

Author

Tobias Bengfort <tobias.bengfort@posteo.de>

Date

2023-11-04 22:12

refine python packaging post

Diffstat

1 files changed, 35 insertions, 14 deletions

diff --git a/_content/posts/2023-11-04-python-packaging/index.md b/_content/posts/2023-11-04-python-packaging/index.md

@@ -30,6 +30,10 @@ frontend may also have to install packages into the build environment. And an
   30    30 installer might have to act as a build frontend if the package is not available
   31    31 as a wheel.
   32    32 
   -1    33 For an excellent overview of the different tools and options for package
   -1    34 management, see [this post by Anna-Lena
   -1    35 Popkes](https://alpopkes.com/posts/python/packaging_tools/).
   -1    36 
   33    37 ## Don't create a package if you want an environment
   34    38 
   35    39 Many modern package managers like npm, cargo, or Poetry automatically create
@@ -73,30 +77,34 @@ later, but you really don't need it. Just run `python -m venv` instead.
   73    77 ## Use pyproject.toml to specify package meta data
   74    78 
   75    79 I stuck with `setup.py` and `setup.cfg` pretty long. The most important reason
   76    -1 was that editable installs were not supported when using `pyproject.toml`. But
   77    -1 now, setuptools (>= 64) and pip (>= 21.3) both have all the features I need.
   78    -1 However, [Ubuntu 22.04 is still on setuptools
   -1    80 was that editable installs were not supported when using pyproject.toml.
   -1    81 Fortunately, this is not a problem because setuptools can still read meta data
   -1    82 from those files.
   -1    83 
   -1    84 But the future is pyproject.toml and both setuptools (>= 64) and pip (>= 21.3)
   -1    85 now support editable installs. Note that [Ubuntu 22.04 is still on setuptools
   79    86 59](https://packages.ubuntu.com/jammy/python3-setuptools). I have started
   80    87 porting some projects to the new system. But I will probably wait with some
   81    88 more critical projects until the new features are widely available.
   82    89 
   83    90 The [setuptools
   84    91 documentation](https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html)
   85    -1 on pyproject.toml is solid and even though to my knowledge there is no tool for
   86    -1 automatic migration, porting an existing `setup.py` or `setup.cfg` to the new
   87    -1 syntax should be simple enough.
   -1    92 on pyproject.toml is solid and porting an existing `setup.py` or `setup.cfg` to
   -1    93 the new syntax should be simple enough. You can also use
   -1    94 [`ini2toml`](https://github.com/abravalheri/ini2toml) to automatically do the
   -1    95 conversion.
   88    96 
   89    -1 You can then build your package either by using `python -m build` (which I see
   90    -1 recommended in most places) or `python -m pip wheel .` (which doesn't require
   91    -1 an additional tool). To upload your package to PyPI you can use
   92    -1 [twine](https://twine.readthedocs.io/en/stable/).
   -1    97 Once you have created that file you can build your package either by using
   -1    98 `python -m build` (which I see recommended in most places) or `python -m pip
   -1    99 wheel .` (which doesn't require an additional tool). To upload your package to
   -1   100 PyPI you can use [twine](https://twine.readthedocs.io/en/stable/).
   93   101 
   94   102 ## Include data files
   95   103 
   96   104 setuptools will automatically include python files in the package. If you need
   97   105 to include other files, e.g. templates or translations, you traditionally had
   98   106 to use a separate `MANIFEST.in` file. That still works, but it can also be
   99    -1 included in `pyproject.toml` directly:
   -1   107 included in pyproject.toml directly:
  100   108 
  101   109 ```
  102   110 [tool.setuptools.package-data]
@@ -106,9 +114,23 @@ mypackage = [
  106   114 ]
  107   115 ```
  108   116 
   -1   117 ## Use backend-specific configuration for more complex packages
   -1   118 
   -1   119 So far we discussed pure python packages. Packages that contain C code or
   -1   120 similar are much more complicated for several reasons. First because they need
   -1   121 an additional compile step, and second because we need to build different
   -1   122 binary packages for different architectures.
   -1   123 
   -1   124 For setuptools you [still configure that in
   -1   125 `setup.py`](https://setuptools.pypa.io/en/latest/userguide/ext_modules.html).
   -1   126 (`setup.py` is not deprecated. It is just no longer necessary for simple
   -1   127 packages.) However, there are other, more specialized build backends like
   -1   128 [scikit-build-core](https://github.com/scikit-build/scikit-build-core) or
   -1   129 [meson-python](https://meson-python.readthedocs.io/en/latest/tutorials/introduction.html).
   -1   130 
  109   131 ## Configure other tools
  110   132 
  111    -1 Most tools can be configured using `pyproject.toml`, e.g.
   -1   133 Most tools can be configured using pyproject.toml, e.g.
  112   134 [pytest](https://docs.pytest.org/en/7.1.x/reference/customize.html#pyproject-toml),
  113   135 [coverage](https://coverage.readthedocs.io/en/latest/config.html), or
  114   136 [isort](https://pycqa.github.io/isort/docs/configuration/config_files.html#pyprojecttoml-preferred-format).
@@ -127,8 +149,7 @@ few years.
  127   149 In this article I stuck with setuptools, because that is the build backend I
  128   150 know best. However, setuptools has accumulated a lot of legacy code over the
  129   151 years. [flit](https://flit.pypa.io/en/latest/rationale.html) is another backend
  130    -1 that has "not being setuptools" as its main feature. I am not sure if this is
  131    -1 enough of a reason for a switch. But I might try it anyways.
   -1   152 that has "not being setuptools" as its main feature.
  132   153 
  133   154 I wouldn't say that python packaging is good now. But at least it has
  134   155 stabilized to a degree that I feel like we could actually, finally reap the