How to publish your own pip package

The Python Package Index, abbreviated as PyPI is the official third-party software repository for Python.

This is the primary way to install libraries in python. For example, if you want to use tensorflow package, just run the command pip install tensorflow and you will have it installed on your system.

Pip packages, in the most basic sense, are libraries or projects which can be downloaded to a location where all your python packages are stored locally. These can then be imported in your codes as and when required. So, when you run pip install tensorflow command, a copy of the tensorflow library is saved to your local system and you can access all of the codes and import any functions or classes via python.

Now, contrary to what many people might think, it is surprisingly easy to publish your own pip package. This guide will tell you exactly how.

Content for this article has been taken from the official pypi guide for creating pip packages which is available here:

Packaging Python Projects - Python Packaging User Guide

Creating package files

First, make sure your project is sufficiently organised and uploaded to GitHub, complete with all the necessary functions to be used for the pip package.

Now, create a directory structure as follows:

my_project
|---> LICENSE
|---> README.md
|---> my_package
    |---> __init__.py
    |---> <other package files>
|---> setup.py

Let’s look at each of these files one by one:

1. my_project: The name of your git repository.
2. LICENSE: It’s important for every package uploaded to the Python Package Index to include a license. This tells users who install your package the terms under which they can use your package. For help picking a license, see https://choosealicense.com/. Once you have chosen a license, open LICENSE and enter the license text.
3. README.md: Your readme file describing your project.
4. my_package: The name of the package you want to publish. This directory should have all the codes files required for running your library. my_package/__init__.py is required to import the directory as a package, and can simply be an empty file.
5. setup.py: This is the build script for setuptools. It tells setuptools about your package (such as the name and version) as well as which code files to include.

Creating setup.py

Open setup.py and enter the following content:

import setuptools

with open("README.md", "r", encoding="utf-8") as fh:
    long_description = fh.read()

setuptools.setup(
    name="my_package", # Replace with your own username
    version="0.0.1",
    author="Example Author",
    author_email="author@example.com",
    description="A small example package",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url="https://github.com/pypa/sampleproject",
    packages=setuptools.find_packages(),
    install_requires = ['tensorflow',
                       'keras>=2.4.3'],
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    python_requires='>=3.6',
)

setup() takes several arguments. The ones used here are described below:

• name is the distribution name of your package.
• version is the package version see PEP 440 for more details on versions.
• author and author_email are used to identify the author of the package.
• description is a short, one-sentence summary of the package.
• long_description is a detailed description of the package. This is shown on the package detail page on the Python Package Index. In this case, the long description is loaded from README.md which is a common pattern.
• long_description_content_type tells the index what type of markup is used for the long description. In this case, it’s Markdown.
• url is the URL for the homepage of the project. For many projects, this will just be a link to GitHub, GitLab, Bitbucket, or similar code hosting service.
• install_requires includes a list of all the dependencies which needs to be installed for the library to work.
• packages is a list of all Python import packages that should be included in the Distribution Package. Instead of listing each package manually, we can use find_packages() to automatically discover all packages and subpackages. In this case, the list of packages will be my_package as that’s the only package present.
• classifiers gives the index and pip some additional metadata about your package. In this case, the package is only compatible with Python 3, is licensed under the MIT license, and is OS-independent.
• python_requires, as the name suggests, is the python version required for the library to work.

If you have completed the above steps, your basic code structure for publishing the package is complete.

Generating distribution archives

The next step is to generate distribution packages for the package. These are archives that are uploaded to the Package Index and can be installed by pip.

Make sure you have the latest versions of setuptools and wheel installed:

pip install setuptools wheel

Now run this command from the same directory where setup.py is located:

python setup.py sdist bdist_wheel

This command should output a lot of text and once completed should generate two files in the dist directory:

dist/
  my_package-0.0.1-py3-none-any.whl
  my_package-0.0.1.tar.gz

Uploading your package to the Python Package Index!

The first thing you’ll need to do is register an account on PyPI. To register an account, go to this and complete the steps on the page.

Now you’ll create a PyPI API token so you will be able to securely upload your project. Click here and create a new API token.

Now that you are registered, you can use twine to upload the distribution packages. You’ll need to install Twine:

pip install twine

Once installed, run Twine to upload all of the archives under dist:

python -m twine upload dist/*

You will be prompted for a username and password. For the username, use __token__. For the password, use the token value, including the pypi- prefix.

After the command completes, you should see output similar to this:

Uploading distributions to https://pypi.org/legacy/
Enter your username: [your username]
Enter your password:
Uploading my_package-0.0.1-py3-none-any.whl
100%|█████████████████████| 4.65k/4.65k [00:01<00:00, 2.88kB/s]
Uploading my_package-0.0.1.tar.gz
100%|█████████████████████| 4.25k/4.25k [00:01<00:00, 3.05kB/s]

Once uploaded your package should be viewable on PyPI, for example, https://pypi.org/project/my_package.

If you have successfully followed the above process, congratulations, you have published your own pip package.

Finally, install your package from the real PyPI using pip install [my_package].

I have created my own package using the same method:

celeb-detector

The pip package homepage looks like this:

Project details for the same can be found at: https://github.com/shobhit9618/celeb_recognition

My article for the above project is available on medium at: https://medium.com/analytics-vidhya/celebrity-recognition-using-vggface-and-annoy-363c5df31f1e

Thank you for reading the article, suggestions for improvement are welcome in comments.