Project

Profile

Help

Task #7868

closed

Story #8093: [EPIC] Release automation

[RELEASING] Build all artifacts before publishing any

Added by mdellweg over 1 year ago. Updated about 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
No
Tags:
CI/CD
Sprint:
Sprint 98
Quarter:

Description

In the past it became a problem, that some artifacts (bindings) failed to build after the main python package landed on pypi.

In order to avoid such situations, we should generate and collect all packages and bindings for a Release before stating to publish any.

Actions #1

Updated by daviddavis over 1 year ago

  • Parent task set to #8093
Actions #2

Updated by daviddavis about 1 year ago

I believe this is the task we need to work on next in terms of improving our release automation.

Background

Currently, at a high level, our workflow is this:

  1. Release person tags the release and pushes to the repo
  2. Release person triggers a release
  3. The release workflow runs the tests
  4. If the tests all pass, the release workflow builds the artifacts and then publishes them.

The artifacts include the pypi package, the bindings (ruby and python), and the docs.

New Workflow

I'd propose we create the following workflow:

  1. Release person triggers a release
  2. The release automation creates the tag and artifacts (note: this step does not push/publish the tag or artifacts)
  3. The release automation runs the tests against these artifacts
  4. If the tests pass, the release automation pushes the tag and the artifacts

I believe step 2 should also check for a pre-existing tag and use it if it already exists. This would allow the workflow to be idempotent. The scripts that publish the artifacts in step 4 are already idempotent.

I think this change to our workflow will help us to move toward the unified release pipeline where we can also test against the installer, etc as well.

Actions #3

Updated by mdepaulo@redhat.com about 1 year ago

  • Groomed changed from No to Yes
  • Sprint set to Sprint 96
Actions #4

Updated by rchan about 1 year ago

  • Sprint changed from Sprint 96 to Sprint 97
Actions #5

Updated by dkliban@redhat.com about 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dkliban@redhat.com
Actions #6

Updated by pulpbot about 1 year ago

  • Status changed from ASSIGNED to POST
Actions #7

Updated by rchan about 1 year ago

  • Sprint changed from Sprint 97 to Sprint 98

Added by dkliban@redhat.com about 1 year ago

Revision eb08e95a

Adds a new release GitHub Actions workflow

The new release workflow has 3 stages:

Build artifacts - build the plugin Python packages or downloads them from PyPI.

Test - build a container from the package in the previous stage. Build the Python and Ruby clients. Build docs. Run unit and function tests using filesystem and S3 storage.

Publish - publish all packages to PyPI if they are not present there. Publish Ruby client to RubyGems.org if it's not there. Publish docs. Push tag to GitHub.

fixes: #7868 https://pulp.plan.io/issues/7868

Actions #8

Updated by dkliban@redhat.com about 1 year ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #9

Updated by daviddavis about 1 year ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF