Pulp 2.11 Test Result Summary¶

Summary

Status: [ ] Not Started [ ] In Progress [ ] Paused [ ] Blocked [X ] Finished
Release Lead:
Started On:
Finished On:
Beta/RC URL: https://repos.fedorapeople.org/pulp/pulp/beta/2.11/
Redmine URL: http://https://pulp.plan.io/issues?query_id=61

The number of new features/stories were limited. Upgrade testing needed to be done manually and automation needed to be updated due to some issues found. moother due to the addition of automated test jobs in Jenkins. Automated tests have increased from 432 in 2.10 to 472 in 2.11 . The jobs were run nightly on EL6, EL7 , Fedora 23 and Fedora 24. Jenkins Jobs were set to run on master before beta builds. That helped in identifying issues early. As a result, the beta produced were stable. Manual testing was limited to the scenarios that do not yet have pulp smash tests. All the new features have been added to pulp-smash backlog to automate or already automated.

Pulp QE team approves the release of the 2.10 build for general consumption of our customers.

Beta History
Beta 1: August 4, 2016
Beta 2: August 19, 2016
Beta 3: August 29, 2016
RC 1: September 6, 2016
RC 2: September 13, 2016

Automation
New tests were added for the following new features

pulp_smash.tests.rpm.api_v2.test_rsync_distributor 9
pulp_smash.tests.rpm.api_v2.test_signatures_checked_for_copies 12
pulp_smash.tests.rpm.api_v2.test_signatures_checked_for_uploads 10
pulp_smash.tests.rpm.api_v2.test_signatures_saved_for_packages 10
pulp_smash.tests.rpm.api_v2.test_force_full 3

Status: [X] PASSED [ ] FAILED
System: x86_64
RHEL 6.8 RHEL 7.3
Installation PASS PASS
Smoke PASS PASS
API PASS PASS
CLI PASS PASS
Upgrade PASS PASS

NOTES:

2.11 Features and Bugs
https://pulp.plan.io/issues?query_id=61

Bugs Verified

Action Items
The following process improvements will be made as a result of this release
Automation: Continue to Improve automation coverage. And add more matrix to automated upgrade testing.
Manual Testing: Organize manual testing efforts better. Identify areas for upgrade testing and feature testing for manual testing that are not covered in the automation.