Test #4359
closed2.18.1 Testing
Description
Notes¶
Testing Coverage for 2.18.1 will consist of the following
- Migration/Upgrade Testing
- 2.17.0 --> 2.18.1
- FIPS
- Non-FIPS
- 2.17.0 --> 2.18.1
- 2.18.1 Functional Testing Suite
- FIPS
- Non-FIPS
Files
Related issues
Updated by bherring almost 6 years ago
Updated by bherring almost 6 years ago
- Related to Test #4357: Pulp 2.19 master pulled epel django instead of pulp 2 django added
Updated by bherring almost 6 years ago
- Related to Test #4351: modules.yaml reference in repomd.xml does not use selected checksum added
Updated by bherring almost 6 years ago
- Related to Issue #4296: Stages API could deadlock when "discovering" content due to minsize added
Updated by bherring almost 6 years ago
- Related to deleted (Issue #4296: Stages API could deadlock when "discovering" content due to minsize)
Updated by bherring almost 6 years ago
- Related to Test #4269: Recursive and conservative recursive copy added
Updated by bherring almost 6 years ago
Updated by bherring almost 6 years ago
Updated by bherring almost 6 years ago
- File 2.18.1_FunctionalTesting_FIPS_2.18.1.b1_20190201.log 2.18.1_FunctionalTesting_FIPS_2.18.1.b1_20190201.log added
- File 2.18.1_FunctionalTesting_nonFIPS_2.18.1.b1_20190204.log 2.18.1_FunctionalTesting_nonFIPS_2.18.1.b1_20190204.log added
Notes¶
- There is a change required in 2.18.1 to have recursive_copy working as we expect
- There are failures in the build ENV that are being worked by PulpQE
- Resolved by pinning pytest to version 4.0.1
- https://github.com/pulp/pulp-ci/pull/638
- There are ENV failures with CRUD that are not seen locally -- added as a task after 2.18.1b1
- Solved by: https://github.com/pulp/pulp-ci/pull/638
- Uploading logs ran locally. Combination of Jenkins + Local logs has passing FIPS + Non-FIPS, sans the issue in #4375
- non-FIPS (CRUD Failing + #4375): https://pulp-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/Pulp2.18/job/pulp-2.18-dev-rhel7/22/
- non-FIPS Local re-run with all passing on 2.18.1.b1 attached
- FIPS attached (single #4375 failure)
Updated by bherring almost 6 years ago
- Related to Issue #4375: Recursive copy doesn't solve rich dependencies correctly added
Updated by bherring almost 6 years ago
- File Screenshot from 2019-02-06 07-45-25.png Screenshot from 2019-02-06 07-45-25.png added
- File Screenshot from 2019-02-06 07-51-18.png Screenshot from 2019-02-06 07-51-18.png added
Final Update¶
2.18.1b1 passed with the updates dalley posted on 2019-02-06.
Note that the Jenkin's failure with OpenSuse is due to a flakey test be re-investigated once QE moves to the new cloud in #4385
- 2.18.1-FIPS: https://pulp-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/Pulp2.18/job/pulp-2.18-dev-rhel7-fips/21/
- 2.18.1-nonFIPS: https://pulp-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/Pulp2.18/job/pulp-2.18-dev-rhel7/24/
Since Jenkin's links are mutable, attached are screenshots of the results.
Updated by bherring almost 6 years ago
- Status changed from ASSIGNED to MODIFIED
Updated by bherring almost 6 years ago
- Status changed from MODIFIED to CLOSED - COMPLETE
Updated by bherring almost 6 years ago
- Related to Issue #4387: Epel is shipping a new version of celery which doesn't work with Pulp added
Updated by bherring almost 6 years ago
Notes¶
After the posting on 2019-02-06 of an issue related to amqp not restarting properly in #4387, a change to our ansible installer was made to prefer pulp-created RPMs over EPEL (due to amqp failures). This resulted in overnight runs for FIPS and non-FIPS passed without issue.
There will be a discussion about the validity of this approach for test builds.
The known issue, python-amqp, was proven and identified. This should allow completion of the beta phase.
A proper resolution and/or advisement for installation of Pulp 2.18.1b1 for python-amqp will be forth-coming.
Screenshot attached of all green runs with the modified approach to non-FIPS and non-modified FIPS runs below.
Updated by bherring over 5 years ago
- Subject changed from 2.18.1 RC Testing to 2.18.1 BETA Testing
Updated by bherring over 5 years ago
- File PreGA-2171-2181_nonFips_upgrade.log PreGA-2171-2181_nonFips_upgrade.log added
- File PreGA-2180-2181_nonFips_upgrade.log PreGA-2180-2181_nonFips_upgrade.log added
- Subject changed from 2.18.1 BETA Testing to 2.18.1 Testing
Final Beta Bits Testing before GA¶
Testing with the final beta bits worked with known work-arounds.
If EPEL repos are present (or any repo providing a python2- packages), the following packages will need to be excluded from upgrade with Pulp 2.18.1:
exclude=python2-kombu python2-celery python2-amqp python2-vine
This will resolve the issue seen in #4387.
All other functional and upgrade tests worked along with a subset upgrade.
Updated by bherring over 5 years ago
- File 2.18.1GA-HotFixUpgradeTest 2.18.1GA-HotFixUpgradeTest added
- Status changed from CLOSED - COMPLETE to MODIFIED
Hot-Fix Test¶
A solution initially implemented was to tell the end user on an upgrade if the celery stack they has was too new and to downgrade.
The output of a failed yum update may have looked like the following:
"--> Processing Conflict: pulp-server-2.18.1-1.el7.noarch conflicts python2-celery >= 4.1",
"--> Finished Dependency Resolution",
"Error: pulp-server conflicts with python2-celery-4.2.1-3.el7.noarch",
"Error: Package: pulp-server-2.18.1-1.el7.noarch (pulp)",
" Requires: python2-celery < 4.1",
" Installed: python2-celery-4.2.1-3.el7.noarch (@epel)",
" python2-celery = 4.2.1-3.el7",
" Available: python2-celery-4.0.2-6.el7.noarch (pulp)",
" python2-celery = 4.0.2-6.el7",
"**********************************************************************",
"yum can be configured to try to resolve such errors by temporarily enabling",
"disabled repos and searching for missing dependencies.",
"To enable this functionality please set 'notify_only=0' in /etc/yum/pluginconf.d/search-disabled-repos.conf",
"**********************************************************************",
"",
" You could try using --skip-broken to work around the problem",
" You could try running: rpm -Va --nofiles --nodigest"
]
}
to retry, use: --limit @/root/code/pulp-ci/ci/ansible/pulp_server_upgrade.retry
It was re-decided the ultimate solution was an upgrade that just worked and didn't require the ,customer to jump through any special hoops to upgrade.
The new solution was to make all celery stack RPMs EPOCH'd in NEVRA. This allows them to supercede any other sources and be preferred. This also means the Pulp Dev team will be required to maintain security and update of these packages.
A listing of some of these packages can be seen in a passing upgrade with the new solution:
"Updated:",
" libmodulemd.x86_64 0:1.7.0-1.el7 ",
" pulp-admin-client.noarch 0:2.18.1-2.git.111.74e1961.el7 ",
" pulp-docker-admin-extensions.noarch 0:3.2.2-1.el7 ",
" pulp-docker-plugins.noarch 0:3.2.2-1.el7 ",
" pulp-ostree-admin-extensions.noarch 0:1.4.0-1.el7 ",
" pulp-ostree-plugins.noarch 0:1.4.0-1.el7 ",
" pulp-puppet-admin-extensions.noarch 0:2.18.1-1.el7 ",
" pulp-puppet-plugins.noarch 0:2.18.1-1.el7 ",
" pulp-puppet-tools.noarch 0:2.18.1-1.el7 ",
" pulp-rpm-admin-extensions.noarch 0:2.18.1-1.el7 ",
" pulp-rpm-plugins.noarch 0:2.18.1-1.el7 ",
" pulp-selinux.noarch 0:2.18.1-2.git.111.74e1961.el7 ",
" pulp-server.noarch 0:2.18.1-2.git.111.74e1961.el7 ",
" python-crane.noarch 0:3.3.0-1.el7 ",
" python-gofer.noarch 0:2.12.5-1.el7 ",
" python-gofer-qpid.noarch 0:2.12.5-1.el7 ",
" python-pulp-bindings.noarch 0:2.18.1-2.git.111.74e1961.el7 ",
" python-pulp-client-lib.noarch 0:2.18.1-2.git.111.74e1961.el7 ",
" python-pulp-common.noarch 0:2.18.1-2.git.111.74e1961.el7 ",
" python-pulp-docker-common.noarch 0:3.2.2-1.el7 ",
" python-pulp-oid_validation.noarch 0:2.18.1-2.git.111.74e1961.el7 ",
" python-pulp-ostree-common.noarch 0:1.4.0-1.el7 ",
" python-pulp-puppet-common.noarch 0:2.18.1-1.el7 ",
" python-pulp-repoauth.noarch 0:2.18.1-2.git.111.74e1961.el7 ",
" python-pulp-rpm-common.noarch 0:2.18.1-1.el7 ",
" python-pulp-streamer.noarch 0:2.18.1-2.git.111.74e1961.el7 ",
" python2-amqp.noarch 10:2.2.2-5.git.111.74e1961.el7 ",
" python2-billiard.x86_64 10:3.5.0.3-4.git.110.4fbf91e.el7 ",
" python2-celery.noarch 10:4.0.2-7.git.111.74e1961.el7 ",
" python2-kombu.noarch 10:4.0.2-11.git.111.74e1961.el7 ",
" python2-vine.noarch 10:1.1.3-5.git.110.4fbf91e.el7 ",
"",
"Complete!"
These bits were tested in beta and are now being pushed to the GA location.
Updated by bherring over 5 years ago
Updated by bherring over 5 years ago
- Status changed from MODIFIED to CLOSED - COMPLETE
Added by bherring over 5 years ago
Added by bherring over 5 years ago
Revision ff598dda | View on GitHub
Update pulp role to define defaults for repo_priorty and evalute run-time
As an addendum to PR #644 , the repo_priorty variable in the pulp role must be defined within the scope of the role (not the pulp_server playbook) as other playbooks use that role.
Making changes to:
- Declare a default of '99', which is the same as no priority
- Declare a fips priority of '55' for fips pulp installs
- Evaluate the proper priority within the pulp role for the pulp.repos from fips
- Remove the pre_task declaration from the pulp_server playbook
Retested on fips and non-fips with the pulp_server playbook as well as other playbooks dependent on the pulp role.
refs #4359
Updating Jenkins Pulp Upgrade Permutations after 2.18.1 GA
Following the GA of 2.18.1, adding the 2.18 upgrade to 2.19.
For the current time, 2.17 is not being removed.
Updated the
README
to remove a specific phrase about Node Pool.refs #4359