Task #7868

Story #8093: [EPIC] Release automation

[RELEASING] Build all artifacts before publishing any

Added by mdellweg 7 months ago. Updated 4 days ago.

Start date:
Due date:
% Done:


Estimated time:
Platform Release:
Sprint Candidate:
Sprint 98


In the past it became a problem, that some artifacts (bindings) failed to build after the main python package landed on pypi.

In order to avoid such situations, we should generate and collect all packages and bindings for a Release before stating to publish any.

Associated revisions

Revision eb08e95a View on GitHub
Added by 4 days ago

Adds a new release GitHub Actions workflow

The new release workflow has 3 stages:

Build artifacts - build the plugin Python packages or downloads them from PyPI.

Test - build a container from the package in the previous stage. Build the Python and Ruby clients. Build docs. Run unit and function tests using filesystem and S3 storage.

Publish - publish all packages to PyPI if they are not present there. Publish Ruby client to if it's not there. Publish docs. Push tag to GitHub.

fixes: #7868


#1 Updated by daviddavis 5 months ago

  • Parent task set to #8093

#2 Updated by daviddavis about 2 months ago

I believe this is the task we need to work on next in terms of improving our release automation.


Currently, at a high level, our workflow is this:

  1. Release person tags the release and pushes to the repo
  2. Release person triggers a release
  3. The release workflow runs the tests
  4. If the tests all pass, the release workflow builds the artifacts and then publishes them.

The artifacts include the pypi package, the bindings (ruby and python), and the docs.

New Workflow

I'd propose we create the following workflow:

  1. Release person triggers a release
  2. The release automation creates the tag and artifacts (note: this step does not push/publish the tag or artifacts)
  3. The release automation runs the tests against these artifacts
  4. If the tests pass, the release automation pushes the tag and the artifacts

I believe step 2 should also check for a pre-existing tag and use it if it already exists. This would allow the workflow to be idempotent. The scripts that publish the artifacts in step 4 are already idempotent.

I think this change to our workflow will help us to move toward the unified release pipeline where we can also test against the installer, etc as well.

#3 Updated by about 1 month ago

  • Groomed changed from No to Yes
  • Sprint set to Sprint 96

#4 Updated by rchan about 1 month ago

  • Sprint changed from Sprint 96 to Sprint 97

#5 Updated by about 1 month ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to

#6 Updated by pulpbot 17 days ago

  • Status changed from ASSIGNED to POST

#7 Updated by rchan 16 days ago

  • Sprint changed from Sprint 97 to Sprint 98

#8 Updated by 4 days ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

Please register to edit this issue

Also available in: Atom PDF