Project

Profile

Help

Actions
Send by e-mail Copy link

Task #7884

closed

Move the pulp_installer Vagrant tests off Travis

Added by mdepaulo@redhat.com over 3 years ago. Updated about 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
mdepaulo@redhat.com
Category:
Installer - Moved to GitHub issues
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
(Total: 0:00 h)
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
CI/CD
Sprint:
Sprint 90
Quarter:

Description

The current plan: Use Qemu emulation on GHA. This is due to us discovering that the performance is tolerable. The advantages of simple security on ephemeral public CI and maintenance outweigh the performance advantage. As a related implication, users' forks/PRs can run on GHA.

Sub-issues 1 (0 open — 1 closed)

Issue #8228: Pulp Connection Timed Out on slow emulated machinesCLOSED - CURRENTRELEASEmdepaulo@redhat.comActions

Related issues

Related to Pulp - Task #7960: FIPS and support for ALLOWED_CONTENT_CHECKSUMSCLOSED - COMPLETE

Actions
Related to Pulp - Issue #8095: geerlingguy.postgresql role (pulp_installer) now errors under FIPS modeMODIFIEDfao89Actions
Blocks Pulp - Task #3800: Add FIPS to the CI matrix in the plugin_templateCLOSED - CURRENTRELEASEfao89

Actions
Actions
Copy link
 #1

Updated by mdepaulo@redhat.com over 3 years ago

  • Sprint set to Sprint 86
Actions
Copy link
 #2

Updated by mdepaulo@redhat.com over 3 years ago

  • Status changed from NEW to ASSIGNED
  • % Done changed from 0 to 10
Actions
Copy link
 #3

Updated by rchan over 3 years ago

  • Sprint changed from Sprint 86 to Sprint 87
Actions
Copy link
 #4

Updated by rchan over 3 years ago

  • Sprint changed from Sprint 87 to Sprint 88
Actions
Copy link
 #5

Updated by daviddavis over 3 years ago

  • Parent issue deleted (#7859)
Actions
Copy link
 #6

Updated by mdepaulo@redhat.com over 3 years ago

  • Related to Task #7960: FIPS and support for ALLOWED_CONTENT_CHECKSUMS added
Actions
Copy link
 #7

Updated by mdepaulo@redhat.com over 3 years ago

  • Description updated (diff)

This was the previous plan: To use CentOS CI. Largely because Foreman / forklift (Vagrant) is already using it. Work will involve:

  1. Finishing the onboarding process with the CentOS/Fedora infra team
  2. hooking CentOS CI's Jenkins into GitHub
  3. migrating from the Travis yml file to a Jenkinsfile
  4. Migrating scripts from Ubuntu to CentOS 7 or 8

The current plan: Use Qemu emulation on GHA. This is due to us discovering that the performance is tolerable. The advantages of simple security on ephemeral public CI and maintenance outweigh the performance advantage. As a related implication, users' forks/PRs can run on GHA.

Work is ongoing on this branch: https://github.com/pulp/pulp_installer/pull/503/files

Actions
Copy link
 #8

Updated by mdepaulo@redhat.com over 3 years ago

  • Related to Issue #8095: geerlingguy.postgresql role (pulp_installer) now errors under FIPS mode added
Actions
Copy link
 #9

Updated by daviddavis about 3 years ago

  • Blocks Task #3800: Add FIPS to the CI matrix in the plugin_template added
Actions
Copy link
 #10

Updated by rchan about 3 years ago

  • Sprint changed from Sprint 88 to Sprint 89
Actions
Copy link
 #11

Updated by mdepaulo@redhat.com about 3 years ago

FYI: Reference on using Qemu emulation: https://github.com/fort-nix/nix-bitcoin/pull/272#issuecomment-736655102 "The runtime increases by factor ~2.5 on x86-64, which is acceptable."

It takes longer for us than that, but still.

Actions
Copy link
 #12

Updated by rchan about 3 years ago

  • Sprint changed from Sprint 89 to Sprint 90
Actions
Copy link
 #13

Updated by mdepaulo@redhat.com about 3 years ago

From the 1/26 open floor:

  • Which pulp_installer FIPS/Vagrant tests, which take ~90 min, to run for PRs vs cronjobs vs tagged jobs?
    • Mike's proposal:
      • PRs:
        • pulp3-source-centos7-fips
        • pulp3-source-centos8-fips
      • cron:
        • pulp3-source-centos7-fips
        • pulp3-source-centos8-fips
        • pulp3-sandbox-centos7-fips
        • pulp3-sandbox-centos8-fips
        • pulp3-source-fedora32
        • pulp3-source-debian10
      • tagged:
        • pulp3-sandbox-centos7-fips
        • pulp3-sandbox-centos8-fips
      • branches:
        • None
      • We'd sometimes merge PRs while the 2 jobs are still running
      • Will re-evaluate once SELinux CI is finished

Added by Mike DePaulo about 3 years ago

Revision 13db1da1 | View on GitHub

Use Qemu emulation on GHA for Vagrant tests

Adapted from: https://github.com/pulp/pulplift/pull/66 "RFC: Testing nested Virtualization"

Implementation Includes:

  1. Upgrade Qemu from 4.4 to 5.2 from our PPA to address a severe bug affecting CentOS 7 guests, they could not even validate SSL certs with curl / yum or create the Pulp postgres database.
  2. Upgrade the rest of the virtualization stack on Ubuntu
  3. Address the EL8 vagrant-sshfs workaround task failing due to a GPG signature mismatch.
  4. Workaround a bug with VM storage on the newer virtualization stack.
  5. Switch the boxes used on CentOS 7 for more recent updates.
  6. Reducing how long the pulp health check may take, particularly when there is a connection timed out.

workaround #8095: FIPS failure in geerlingguy.postgresql by using an old version. https://pulp.plan.io/issues/8095

workaround #7993: pulp_installer fails to create the database on EL7 when LANG=C.UTF-8 https://pulp.plan.io/issues/7993

fixes: #7884 Move the pulp_installer Vagrant tests off Travis https://pulp.plan.io/issues/7884

Added by Mike DePaulo about 3 years ago

Revision 13db1da1 | View on GitHub

Use Qemu emulation on GHA for Vagrant tests

Adapted from: https://github.com/pulp/pulplift/pull/66 "RFC: Testing nested Virtualization"

Implementation Includes:

  1. Upgrade Qemu from 4.4 to 5.2 from our PPA to address a severe bug affecting CentOS 7 guests, they could not even validate SSL certs with curl / yum or create the Pulp postgres database.
  2. Upgrade the rest of the virtualization stack on Ubuntu
  3. Address the EL8 vagrant-sshfs workaround task failing due to a GPG signature mismatch.
  4. Workaround a bug with VM storage on the newer virtualization stack.
  5. Switch the boxes used on CentOS 7 for more recent updates.
  6. Reducing how long the pulp health check may take, particularly when there is a connection timed out.

workaround #8095: FIPS failure in geerlingguy.postgresql by using an old version. https://pulp.plan.io/issues/8095

workaround #7993: pulp_installer fails to create the database on EL7 when LANG=C.UTF-8 https://pulp.plan.io/issues/7993

fixes: #7884 Move the pulp_installer Vagrant tests off Travis https://pulp.plan.io/issues/7884

Actions
Copy link
 #14

Updated by pulpbot about 3 years ago

  • Status changed from ASSIGNED to POST
Actions
Copy link
 #15

Updated by Anonymous about 3 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #16

Updated by ipanova@redhat.com about 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions
Send by e-mail Copy link

Also available in: Atom PDF