An ideal development pipeline (non-specific to python) looks like this:

  1. a developer commits some code (eg. on "master" branch) and pushes into a remote CI/CD service
  2. the code get automatically statically checked (lint, mypy, pep8 etc.)
  3. (optional) compile the source code into a binary executable
  4. runtime check the unit/integration tests (to get coverage/performance info along the way)
  5. on success a package is created and deployed to an "index" server

The user can fetch/install the package from the "index" server.

This works in an ideal world where the checks in steps #2 and #4 guarantee the code perfectly match the requirements and it has no bugs.

Thanks to wide availability of hosted CI/CD systems as github, gitlab, bitbucket and opensuse1 and "index" servers like Pypi, this pipeline is trivial to implement.

Leveraging branches to implement the Branched Based Deployment (BBD)

In the above ideal development pipeline a meaningful commit will result in a working package: but in real life an extra step is required to guarantee a fully functional and working package.

The extra step is the Quality assurance (QA) where testers use the package and find problems ranging from usability to requirement issues all way down to discover bug not found during testing.

So in the previous pipeline there should be an extra step to facilitate creation of beta packages for the QA process: this is where leveraging branches comes to play.

This is the lifecycle:

  1. commits are merged on a beta/M.N.O branch and pushed into the CD/CI
  2. the build (including the static and dynamic checks) will result in a package (eg. foobar-0.0.0.bN where N is a non decreasing number)
  3. QA teams can use the foobar-0.0.0.bN for testing
  4. the previous cycle is repeated many times until a stable commit is ready for release
  5. a specific commit is tagged, that will trigger a foobar-0.0.0 production package for users

  1. This was and in many ways still is one of the very first large scale CD system widely available, predating other systems by more than a decade