Protecting Python software supply chain
Software supply chain is an important aspect of secure software development. Recently, we’ve seen an increase in the number of software supply chain attacks. We can protect against some of these attacks by following best practices during the development, and by ensuring the right packages are used.
Many of these attacks and vulnerabilities are based on using the wrong dependency (e.g. via typosquatting or confusion of dependency source) or using an old or vulnerable version of a library. Using lockfiles allows us to pin the version of our dependencies and also perform integrity checks using hashes of the packages.
In this article we look at best practices in securing Python software supply chain by pinning library versions and using integrity checks. We also look at solutions for automatic notification about our vulnerable dependencies and related updates and patches. For more information about software supply chain security you can look at the Cybersecurity and Infrastructure Security Agency’s (CISA) paper and this PyCon talk.
In Python we usually specify the dependencies via a requirements file and listing them in setup.py file. We can specify the package versions, however, it doesn’t provide a view of dependency graph (dependency of the dependencies, etc.) or the hash of the packages. Fortunately, we can use pip-tools to address both scenarios.
Pinning versions, hashes, and dependency graph
After installing pip-tools (pip install pip-tools), we’ll use pip-compile to pin package versions, create hashes for our dependencies and also create the dependency graph. pip-compile would first look for a setup.py file and create a requirements.txt file from it. If you don’t have a setup.py file, it’s possible to create a file with the list of all dependencies (requirements.in) and feed that into pip-compile.
For example, in our demo project we use:
requests==2.1.0
flask
urllib3==1.25.2
xml2dict==0.2.2We need to run pip-compile to create the dependency graph from our setup.py file. If a specific version is specified, pip-compile will use it, otherwise, pip-compile would use the latest version of that library. Below is a truncated output of pip-compile and the generated requirements.txt file from our demo project. I’ve added some comments about the source and nature of the dependencies (direct vs. indirect).
$ pip-compile --generate-hashes
#
# This file is autogenerated by pip-compile with python 3.9
# To update, run:
#
# pip-compile --generate-hashes
#
flask==2.0.1 \
--hash=sha256:1c4c257b1892aec1398784c6379... \
--hash=sha256:a6209ca15eb63fc9385f38e4527...
# via pysupplychaindemo (setup.py) (direct dependency)
# demo.py -> flaskjinja2==3.0.1 \
--hash=sha256:1f06f2da51e7b56b8f238affdd... \
--hash=sha256:703f484b47a6af502e743c9122...
# via flask (dependency of a dependency)
# demo.py -> flask -> jinja2markupsafe==2.0.1 \
--hash=sha256:01a9b8ea66f1658938f65b93a8... \
...
--hash=sha256:fa130dd50c57d53368c9d59395...
# via jinja2 (dependency of a dependency of a dependency)
# demo.py -> flask -> jinja2 -> markupsafexml2dict==0.2.2 \
--hash=sha256:20e4b48926ba3537b57587496c46d2...
# via pysupplychaindemo (setup.py) (direct dependency)
# demo.py -> xml2dictKeeping the dependencies up-to-date and getting the security updates
Another important step is to make sure that the dependencies are up-to-date and get notifications when a vulnerability is found in one of our dependencies. Fortunately, there are already many tools such as Github Dependabot, Snyk, and PyUp that provide this functionality.
In this post we look at Github Dependabot and PyUp. You can look at the instructions to enable and configure PyUp and Dependabot. Once you setup these service, you’ll notice that bots would automatically create pull requests to upgrade the library versions for you.
