Project

Profile

Help

Pulp 2 11 Test Result Summary » History » Sprint/Milestone 11

pthomas@redhat.com, 12/14/2016 06:53 PM

1 1 pthomas@redhat.com
# Pulp 2.11 Test Result Summary
2
3 2 pthomas@redhat.com
**Summary**
4 1 pthomas@redhat.com
5 7 pthomas@redhat.com
Status: \[ \] Not Started \[ \] In Progress \[ \] Paused \[ \] Blocked **\[X \] Finished**  
6 2 pthomas@redhat.com
Started On: October 26  
7 11 pthomas@redhat.com
Finished On: December 14  
8 1 pthomas@redhat.com
Beta/RC URL: https://repos.fedorapeople.org/pulp/pulp/beta/2.11/  
9
Redmine URL: http://https://pulp.plan.io/issues?query_id=61
10
11 7 pthomas@redhat.com
The number of new features/stories were limited. Upgrade testing needed to be done manually and automation needed to be updated due to some issues found. Automated tests have increased from 432 in 2.10 to 472 in 2.11 . The jobs were run nightly on EL6, EL7 , Fedora 23 and Fedora 24. Jenkins Jobs were set to run on master before beta builds. As a result, the beta produced were stable. Manual testing was limited to the scenarios that do not yet have pulp smash tests. All the new features have been added to pulp-smash backlog to automate or already automated.
12 1 pthomas@redhat.com
13 6 pthomas@redhat.com
**Pulp QE team approves the release of the 2.11 RC to GA.**
14 1 pthomas@redhat.com
15
Beta History  
16 2 pthomas@redhat.com
Beta 1: October 26, 2016  
17
Beta 2: November 3, 2016  
18
Beta 3: November 23, 2016  
19
RC 1: November 30, 2016  
20
RC 2: December 9, 2016
21 1 pthomas@redhat.com
22 2 pthomas@redhat.com
Automation
23 1 pthomas@redhat.com
24 2 pthomas@redhat.com
New tests were added for the following
25 1 pthomas@redhat.com
26 2 pthomas@redhat.com
~~~
27
https://github.com/PulpQE/pulp-smash/issues/444
28
https://github.com/PulpQE/pulp-smash/issues/427
29
https://github.com/PulpQE/pulp-smash/issues/425
30
~~~
31
32 8 pthomas@redhat.com
\[\[Status: \[X\] PASSED \[ \] FAILED  
33 1 pthomas@redhat.com
System: x86_64  
34
RHEL 6.8 RHEL 7.3  
35
Installation PASS PASS  
36
Smoke PASS PASS  
37
API PASS PASS  
38
CLI PASS PASS  
39 8 pthomas@redhat.com
Upgrade PASS PASS\]\]
40 1 pthomas@redhat.com
41
NOTES:
42
43
2.11 Features and Bugs  
44
https://pulp.plan.io/issues?query_id=61
45
46 10 pthomas@redhat.com
**New Features**
47 1 pthomas@redhat.com
48 10 pthomas@redhat.com
~~~
49
============
50
~~~
51
52 2 pthomas@redhat.com
Here are the specific stories done for 2.11:
53
54
~~~
55
Docker Support
56
      2114    As a user, I can perform incremental publish with Docker rsync distributor
57
~~~
58
59
~~~
60
OSTree Support
61
      2205    As a user, I want to pull older versions of trees available to atomic hosts.
62
~~~
63
64
~~~
65
Pulp
66
      1983    As a user, an importer config change or content removal will cause the next sync to
67
be full
68
      2172    Memory Improvements with Process Recycling
69
      2186    As a user, pulp-manage-db refuses to run if other pulp processes are running locally
70
~~~
71
72
~~~
73
RPM Support
74
      1806    As a user, I can use pulp-admin to upload drpms
75
      2113    As a user, I can perform an incremental publish using ISO rsync distributor
76
      2132    As a consumer of Pulp content I want ISO & ISO_rsync distributor to support
77
relative_url for consistency with yum
78
      2211    As a user, I can sync a distribution unit with the existing yum_importer without
79
requiring yum metadata
80
      2212    As a user I can see in the task report of copy operation list of units which did not
81
pass signature filter 
82
~~~
83
84 1 pthomas@redhat.com
View this list in redmine:  
85
http://bit.ly/2eQz5Ui
86 2 pthomas@redhat.com
87 10 pthomas@redhat.com
**Issues Addressed**
88
89 9 pthomas@redhat.com
\================
90 2 pthomas@redhat.com
91
Here are the bug fixes specific to 2.11:
92
93
~~~
94
Pulp
95
      2377    0025_importer_schema_change hits issue with last_sync being None
96
      1977    basic auth in URL fails when using authenticated proxy
97
      1069    extra Task ID in docs example to bind a repo
98
      2133    Permissions docs should include some info on /v2/
99
      752     Suggestion to add -v remains even if you use -v
100
~~~
101
102
~~~
103
RPM Support
104
      2224    Cannot sync from mirrorlists
105
      2321    Pulp can't handle mirrorlists with invalid entries
106
      2363    Pulp acts like all is well when a sync completely fails
107
      2071    Creating yum distributor without https, http keys in config crashes
108
~~~
109
110
Action Items
111
112 1 pthomas@redhat.com
The following process improvements will be made as a result of this release  
113
Automation: Continue to Improve automation coverage. And add more matrix to automated upgrade testing.  
114
Manual Testing: Organize manual testing efforts better. Identify areas for upgrade testing and feature testing for manual testing that are not covered in the automation.
115 2 pthomas@redhat.com
116
New Pulp-smash issues added
117
118
https://github.com/PulpQE/pulp-smash/issues/454  
119
https://github.com/PulpQE/pulp-smash/issues/453  
120
https://github.com/PulpQE/pulp-smash/issues/431  
121
https://github.com/PulpQE/pulp-smash/issues/421  
122
https://github.com/PulpQE/pulp-smash/issues/420