Project

Profile

Help

Edit History

QE Test Summary ยป

Pulp 2 11 Test Result Summary » History » Sprint/Milestone 1

pthomas@redhat.com, 12/13/2016 02:42 AM

1 1 pthomas@redhat.com 
# Pulp 2.11 Test Result Summary
2 





    3
    
    
    
Summary





    4
    
    
    





    5
    
    
    
Status: \[ \] Not Started \[ \] In Progress \[ \] Paused \[ \] Blocked \[X \] Finished  





    6
    
    
    
Release Lead:  





    7
    
    
    
Started On:  





    8
    
    
    
Finished On:  





    9
    
    
    
Beta/RC URL: https://repos.fedorapeople.org/pulp/pulp/beta/2.11/  





    10
    
    
    
Redmine URL: http://https://pulp.plan.io/issues?query_id=61





    11
    
    
    





    12
    
    
    
The number of new features/stories were limited. Upgrade testing needed to be done manually and automation needed to be updated due to some issues found. moother due to the addition of automated test jobs in Jenkins. Automated tests have increased from 432 in 2.10 to 472 in 2.11 . The jobs were run nightly on EL6, EL7 , Fedora 23 and Fedora 24. Jenkins Jobs were set to run on master before beta builds. That helped in identifying issues early. As a result, the beta produced were stable. Manual testing was limited to the scenarios that do not yet have pulp smash tests. All the new features have been added to pulp-smash backlog to automate or already automated.





    13
    
    
    





    14
    
    
    
Pulp QE team approves the release of the 2.10 build for general consumption of our customers.





    15
    
    
    





    16
    
    
    
Beta History  





    17
    
    
    
Beta 1: August 4, 2016  





    18
    
    
    
Beta 2: August 19, 2016  





    19
    
    
    
Beta 3: August 29, 2016  





    20
    
    
    
RC 1: September 6, 2016  





    21
    
    
    
RC 2: September 13, 2016





    22
    
    
    





    23
    
    
    
Automation  





    24
    
    
    
New tests were added for the following new features





    25
    
    
    





    26
    
    
    
pulp_smash.tests.rpm.api_v2.test_rsync_distributor 9  





    27
    
    
    
pulp_smash.tests.rpm.api_v2.test_signatures_checked_for_copies 12  





    28
    
    
    
pulp_smash.tests.rpm.api_v2.test_signatures_checked_for_uploads 10  





    29
    
    
    
pulp_smash.tests.rpm.api_v2.test_signatures_saved_for_packages 10  





    30
    
    
    
pulp_smash.tests.rpm.api_v2.test_force_full 3





    31
    
    
    





    32
    
    
    
Status: \[X\] PASSED \[ \] FAILED  





    33
    
    
    
System: x86_64  





    34
    
    
    
RHEL 6.8 RHEL 7.3  





    35
    
    
    
Installation PASS PASS  





    36
    
    
    
Smoke PASS PASS  





    37
    
    
    
API PASS PASS  





    38
    
    
    
CLI PASS PASS  





    39
    
    
    
Upgrade PASS PASS





    40
    
    
    





    41
    
    
    
NOTES:





    42
    
    
    





    43
    
    
    
2.11 Features and Bugs  





    44
    
    
    
https://pulp.plan.io/issues?query_id=61





    45
    
    
    





    46
    
    
    
Bugs Verified





    47
    
    
    





    48
    
    
    
Action Items  





    49
    
    
    
The following process improvements will be made as a result of this release  





    50
    
    
    
Automation: Continue to Improve automation coverage. And add more matrix to automated upgrade testing.  





    51
    
    
    
Manual Testing: Organize manual testing efforts better. Identify areas for upgrade testing and feature testing for manual testing that are not covered in the automation.