Project

Profile

Help

Pulp 2 11 Test Result Summary » History » Revision 2

Revision 1 (pthomas@redhat.com, 12/13/2016 02:42 AM) → Revision 2/11 (pthomas@redhat.com, 12/13/2016 06:34 PM)

# Pulp 2.11 Test Result Summary 

 **Summary** Summary 

 Status: \[ \] Not Started \[ \] In Progress \[ \] Paused \[ \] Blocked \[X \] Finished   
 Release Lead:   
 Started On: October 26   
 Finished On: December 13   
 Beta/RC URL: https://repos.fedorapeople.org/pulp/pulp/beta/2.11/   
 Redmine URL: http://https://pulp.plan.io/issues?query_id=61 

 The number of new features/stories were limited. Upgrade testing needed to be done manually and automation needed to be updated due to some issues found. moother due to the addition of automated test jobs in Jenkins. Automated tests have increased from 432 in 2.10 to 472 in 2.11 . The jobs were run nightly on EL6, EL7 , Fedora 23 and Fedora 24. Jenkins Jobs were set to run on master before beta builds. That helped in identifying issues early. As a result, the beta produced were stable. Manual testing was limited to the scenarios that do not yet have pulp smash tests. All the new features have been added to pulp-smash backlog to automate or already automated. 

 Pulp QE team approves the release of the 2.11 2.10 build to GA. for general consumption of our customers. 

 Beta History   
 Beta 1: October 26, August 4, 2016   
 Beta 2: November 3, August 19, 2016   
 Beta 3: November 23, August 29, 2016   
 RC 1: November 30, September 6, 2016   
 RC 2: December 9, September 13, 2016 

 Automation 

   
 New tests were added for the following new features 

 ~~~ 
 https://github.com/PulpQE/pulp-smash/issues/444 
 https://github.com/PulpQE/pulp-smash/issues/427 
 https://github.com/PulpQE/pulp-smash/issues/425 
 ~~~ pulp_smash.tests.rpm.api_v2.test_rsync_distributor 9   
 pulp_smash.tests.rpm.api_v2.test_signatures_checked_for_copies 12   
 pulp_smash.tests.rpm.api_v2.test_signatures_checked_for_uploads 10   
 pulp_smash.tests.rpm.api_v2.test_signatures_saved_for_packages 10   
 pulp_smash.tests.rpm.api_v2.test_force_full 3 

 Status: \[X\] PASSED \[ \] FAILED   
 System: x86_64   
 RHEL 6.8 RHEL 7.3   
 Installation PASS PASS   
 Smoke PASS PASS   
 API PASS PASS   
 CLI PASS PASS   
 Upgrade PASS PASS 

 NOTES: 

 2.11 Features and Bugs   
 https://pulp.plan.io/issues?query_id=61 

 \*New Features   
 \*============ Bugs Verified 

 Here are the specific stories done for 2.11: 

 ~~~ 
 Docker Support 
       2114      As a user, I can perform incremental publish with Docker rsync distributor 
 ~~~ 

 ~~~ 
 OSTree Support 
       2205      As a user, I want to pull older versions of trees available to atomic hosts. 
 ~~~ 

 ~~~ 
 Pulp 
       1983      As a user, an importer config change or content removal will cause the next sync to 
 be full 
       2172      Memory Improvements with Process Recycling 
       2186      As a user, pulp-manage-db refuses to run if other pulp processes are running locally 
 ~~~ 

 ~~~ 
 RPM Support 
       1806      As a user, I can use pulp-admin to upload drpms 
       2113      As a user, I can perform an incremental publish using ISO rsync distributor 
       2132      As a consumer of Pulp content I want ISO & ISO_rsync distributor to support 
 relative_url for consistency with yum 
       2211      As a user, I can sync a distribution unit with the existing yum_importer without 
 requiring yum metadata 
       2212      As a user I can see in the task report of copy operation list of units which did not 
 pass signature filter  
 ~~~ 

 View this list in redmine:   
 http://bit.ly/2eQz5Ui 

 \*Issues Addressed   
 \*================ 

 Here are the bug fixes specific to 2.11: 

 ~~~ 
 Pulp 
       2377      0025_importer_schema_change hits issue with last_sync being None 
       1977      basic auth in URL fails when using authenticated proxy 
       1069      extra Task ID in docs example to bind a repo 
       2133      Permissions docs should include some info on /v2/ 
       752       Suggestion to add -v remains even if you use -v 
 ~~~ 

 ~~~ 
 RPM Support 
       2224      Cannot sync from mirrorlists 
       2321      Pulp can't handle mirrorlists with invalid entries 
       2363      Pulp acts like all is well when a sync completely fails 
       2071      Creating yum distributor without https, http keys in config crashes 
 ~~~ 

 Action Items 

   
 The following process improvements will be made as a result of this release   
 Automation: Continue to Improve automation coverage. And add more matrix to automated upgrade testing.   
 Manual Testing: Organize manual testing efforts better. Identify areas for upgrade testing and feature testing for manual testing that are not covered in the automation. 

 New Pulp-smash issues added 

 https://github.com/PulpQE/pulp-smash/issues/454   
 https://github.com/PulpQE/pulp-smash/issues/453   
 https://github.com/PulpQE/pulp-smash/issues/431   
 https://github.com/PulpQE/pulp-smash/issues/421   
 https://github.com/PulpQE/pulp-smash/issues/420