Project

Profile

Help

Pulp 3 Minimum Viable Product » History » Sprint/Milestone 105

bmbouter, 11/07/2017 09:08 PM
Updating based on tickets

1 1 bmbouter
# Pulp 3.0.0 Minimum Viable Product (MVP)
2
3 21 bmbouter
<span style="color:red;">Lines highlighted in red need more attention.</span>
4 20 bmbouter
5 37 bmbouter
## Overall Guarantees
6
7 57 mhrivnak
  - This is not a direct replacement for Pulp 2.
8
  - This is the minimum required for a 3.0.0 beta and GA.
9
  - All REST API calls will update the DB using transactions as necessary to ensure data integrity.
10 37 bmbouter
11 59 bmbouter
## Legend
12
13
\[done\] means merged and documented  
14
\[in-progress\] means started but not fully done  
15
If there is no label the effort has not yet been started
16
17 1 bmbouter
## Authentication
18
19 59 bmbouter
As an authenticated user I can manage user(s). \[done\]
20 17 semyers
21
  - Add a user
22
  - View user(s)
23 58 mhrivnak
  - Update any user detail
24 1 bmbouter
  - Delete a user
25
26 59 bmbouter
As an API user, I can have documentation to generate a JSON Web Token (JWT) without the server being online. \[done\]
27 1 bmbouter
28 59 bmbouter
As an administrator, I can disable JWT token expiration. This configuration is in the settings file and is system-wide. \[done\]
29 1 bmbouter
30 59 bmbouter
As an administrator, I can configure the JWT tokens to expire after a configurable amount of time. This configuration is in the settings file and is system-wide. \[done\]
31 1 bmbouter
32 59 bmbouter
The JWT shall have a username identifier \[done\]
33 1 bmbouter
34 59 bmbouter
<span style="color:red;">As an API user, I can authenticate any API call (except to request a JWT) with a JWT. (not certain if this should be the behavior) \[in progress\]</span>
35 17 semyers
36 59 bmbouter
As an API user, I can invalidate all existing JWT tokens for a given user. \[done\]
37 17 semyers
38 59 bmbouter
As an authenticated user, when deleting a user 'foo', all of user 'foo's existing JWTs are invalidated. \[done\]
39 17 semyers
40 59 bmbouter
As an un-authenticated user, I can obtain a JWT token by using a username and password. \[done\]
41 1 bmbouter
42
## Repositories
43
44 18 dkliban@redhat.com
As an authenticated user, I can list all repos.
45
46 60 bmbouter
  - All fields are included \[done\]
47
  - Pagination is supported \[done\]
48 61 mhrivnak
  - <span style="color:orange;">Filtering support</span>
49 18 dkliban@redhat.com
50
As an authenticated user, I can CRUD a repository
51
52 60 bmbouter
  - Create a repo \[done\]
53
  - Read a repo \[done\]
54
  - Update all mutable repo fields \[done\]
55
  - Delete a repo (asynchronous) \[done\]
56 18 dkliban@redhat.com
57
As an authenticated user, I can list a repository's associated importers and publishers
58
59 60 bmbouter
  - All fields are included \[done\]
60
  - Pagination is supported \[done\]
61 1 bmbouter
62 104 bmbouter
As an authenticated user, I can see the number of content unit types with counts for each \[done\]\[[3059](https://pulp.plan.io/issues/3059)\]
63 18 dkliban@redhat.com
64 51 bmbouter
## Importers
65
66
note: Importer attributes will commonly be available on importers, but aren't guaranteed to be used by all importers.
67
68 18 dkliban@redhat.com
As an authenticated user, I can CRUD an importer
69
70
  - Create an importer
71
  - Read an importer
72
  - Update all mutable importer fields
73 21 bmbouter
  - Delete an importer (asynchronous)
74 18 dkliban@redhat.com
75 64 mhrivnak
As an authenticated user I can configure the following attributes on an Importer: \[done\]
76 47 bmbouter
77 48 amacdona@redhat.com
  - validate (bool) \[optional: defaults to True\]: If true, the plugin will validate imported content.
78
  - ssl_ca_certificate (str) \[optional\] String containing a PEM encoded CA certificate used to validate the server certificate presented by the external source.
79
  - ssl_client_certificate (str) \[optional\] Contains a PEM encoded client certificate used for authentication.
80
  - ssl_client_key (str) \[optional\] Contains a PEM encoded private key used for authentication.
81
  - ssl_validation (bool) \[optional: defaults to True\]: If true, SSL peer validation must be performed.
82 65 mhrivnak
  - proxy_url (str) \[optional\] Contains the proxy URL. Format: scheme://user:password@host:port.
83 48 amacdona@redhat.com
  - username (str) \[optional\] Contains the username to be used for authentication when syncing.
84 1 bmbouter
  - password (str) \[optional\] Contains the password to be used for authentication when syncing.
85 63 mhrivnak
  - <span style="color:orange;">download_policy (str) \[optional\] Contains the downloading policy name. This is a choice of three options:  
86
 \- immediate (default) - Downloading occurs during sync. The sync task does not complete until downloading is done.  
87
 \- background - Downloading is started by the sync but occurs in the background. The sync task completes before downloading is complete.  
88
 \- on-demand - The sync task records everything that would be downloaded but does not download content. Downloading occurs on demand as driven by client requests for content.</span>
89 1 bmbouter
  - feed_url (str) \[optional\] Contains the URL of an external content source. This is optional.
90 63 mhrivnak
  - <span style="color:orange;">sync mode: (str) \[optional\]. It has two choices 'additive' and 'mirror':  
91
 \- additive (default) - all remote content is added to the local repository on sync. During sync no content is removed from the local repository.  
92
 \- mirror - the local content will mirror the remote content exactly, removing local content if not also present in the remote content.</span>
93 50 bmbouter
  - name (str) \[required\] Contains the name.
94
  - last_updated (datetime) \[read-only\] Contains the datetime of the last importer update.
95 53 bmbouter
  - last_synced (datetime) \[read-only\] Contains the datetime of the last importer sync.
96 51 bmbouter
97 52 bmbouter
## Publishers
98 51 bmbouter
99
note: Publisher attributes will commonly be available on publishers, but aren't guaranteed to be used by all publishers.
100 47 bmbouter
101 18 dkliban@redhat.com
As an authenticated user, I can CRUD a publisher
102
103
  - Create a publisher
104
  - Read a publisher
105 1 bmbouter
  - Update all mutable publisher fields
106
  - Delete a publisher (asynchronous)
107 50 bmbouter
108
As an authenticated user I can configure the following attributes on a Publisher:
109
110
  - relative_path (str) \[optional\] The (relative) path component of the published url.
111
  - name - (str) \[required\] contains the name.
112
  - last_published (datetime) \[read-only\] When the last successful publish occurred.
113
  - last_updated (datetime) \[read-only\] The datetime of the last publisher update.
114 56 ttereshc
  - <span style="color:red;">auto_publish(bool) - ??? consider adding auto-publish feature to MVP</span>
115 1 bmbouter
116 72 mhrivnak
## Sync and Publish
117 1 bmbouter
118 66 mhrivnak
As an authenticated user, I can trigger an importer to sync. \[done\]
119 19 bmbouter
120
  - I can follow the progress of all syncs. (Syncs are asynchronous.)
121
  - I cannot pass "sync" options.
122
  - Auto-publish is not included as an importer property.
123
124 66 mhrivnak
As an authenticated user, I can trigger a publisher to publish. \[done\]
125 19 bmbouter
126
  - I can follow the progress of all publishes. (Publishes are asynchronous.)
127
  - I cannot pass "publish" options.
128
129 71 mhrivnak
## Content Manipulation
130 25 bmbouter
131 73 mhrivnak
#### Uploading Artifacts
132 32 bmbouter
133 67 mhrivnak
As an authenticated user, I can create an Artifact by uploading a file. \[done\]
134 30 bmbouter
135 68 mhrivnak
As an authenticated user, I can specify a size and/or digest to validate the uploaded file. \[done\]
136
137 71 mhrivnak
#### Creating Content Units
138 35 bmbouter
139 70 mhrivnak
As an authenticated user, I can create a content unit by providing the content type (in the URL), references to Artifacts, and the metadata supplied in the POST body. \[done\]
140 1 bmbouter
141 102 bmbouter
#### Add / Remove Content from a Repository
142 29 bmbouter
143 99 bmbouter
As a user I can add content (specific existing content) to a repository
144 1 bmbouter
145 99 bmbouter
  - I can specify one or more content units by id/href
146
  - I can follow the progress. (adding are asynchronous).
147
148
As a user I can remove specified content from a repository
149
150
  - I can specify one or more content units by id/href
151
  - I can follow the progress. (removing are asynchronous).
152
153 101 bmbouter
#### Simple Copy
154 1 bmbouter
155 101 bmbouter
As a user, I can search (synchronous call) a repository's content using filtering.
156
157
  - note: In a separate call (async), I can add this content to another repository. This is effectively a copy operation in two calls.
158
159
As an authenticated user, I can import all content from one repository into another repository in a single async call. (Clone use case)
160
161
  - I can follow the progress. (adding are asynchronous).
162
163
#### Complex Copy
164
165
As a plugin writer I can provide a rich search features with abitrary viewsets. e.g. depsolving, verisoning, etc
166
167
Examples of specific plugin use cases motivating the above general viewset
168
169
  - As an authenticated user, I can add an Errata from one repository to another repository along with packages mentioned in the Errata and all their dependencies that are present in the source repository.
170
  - As an authenticated user, I can add bunch of dependencies and store n versions back for each RPM.
171
  - As an authenticated user, I can use depsolving and versioning copy features together
172
  - As an authenticated user, I can copy a puppet module and all of it's dependencies from one repository to another. 
173
  - As an authenticated user, I can depsolve units to be added to a destination repo based on an errata
174 71 mhrivnak
175 95 bmbouter
#### Delete
176 1 bmbouter
177 95 bmbouter
<span style="color:red;">As an authenticated user, I can delete a specific content unit</span>  
178
<span style="color:red;">\* If the content unit is still in at least one repository the delete fails with a listing of all repositories the unit is part of.</span>  
179
<span style="color:red;">\* Artifacts and associated files from the deleted unit are cleaned up</span>
180
181
<span style="color:red;">As an authenticated user, I can delete multiple content units with filtering</span>  
182
<span style="color:red;">\* If a content unit is still in at least one repository the delete fails with a listing of all repositories the unit is part of.</span>  
183
<span style="color:red;">\* Artifacts and associated files from deleted units are cleaned up</span>
184
185 74 mhrivnak
## <span style="color:red;">Versioned Repositories</span>
186 1 bmbouter
187 40 bmbouter
As an authenticated user, I can list the content in a particular repository version
188
189
  - All fields are included
190
  - Pagination is supported
191
  - <span style="color:red;">Filtering support</span>
192
193
As an authenticated user, I can discover a URL to the latest version of a repository  
194
<span class="resource repository the on attributes or endpoint, API dedicated a through \^ Is" style="color:red;"></span>
195 1 bmbouter
196 43 bmbouter
As an authenticated user, I can run a publisher without a repository version and have it default to the latest version.
197
198 1 bmbouter
<span style="color:red;">As an authenticated user, I can delete a repository version by specifying the version</span>
199
200 43 bmbouter
<span style="color:red;">As an authenticated user, I can upload multiple content(s?) and add create a single new version that adds all of them.</span>
201
202 95 bmbouter
## Orphan Content Units and Artifacts
203 43 bmbouter
204 95 bmbouter
As an authenticated user, I can cause an action that cleans up **both** orphaned content units and orphaned artifacts.
205 43 bmbouter
206 95 bmbouter
  - I cannot specify the units specifically (all types).
207
  - I can follow the progress of all cleanups. (Cleanups are asynchronous.)
208 1 bmbouter
209 22 bmbouter
## Task Management
210
211 76 mhrivnak
As an authenticated user, I can list all tasks
212
213 22 bmbouter
  - <span style="color:orange;">Filtering support on \['state', 'id', 'group'\]</span>
214 76 mhrivnak
  - <span style="color:orange;">This does not include associated progress reports</span>
215 22 bmbouter
216
As an authenticated user, I can see a detail view for a specific task \[done\]
217
218
  - all attributes of a task
219 76 mhrivnak
  - all associated progress reports
220 1 bmbouter
221
As an authenticated user, I can cancel a task \[done\]
222
223 77 mhrivnak
  - don't dare to use the DELETE verb!
224
225 1 bmbouter
As an authenticated user, I can delete tasks.
226
227 78 mhrivnak
## Task Group
228 1 bmbouter
229
<span style="color:red;">I can view a summary of the status of all tasks in a group</span>
230
231 82 mhrivnak
## Status
232 1 bmbouter
233 82 mhrivnak
As an unauthenticated user I can view the status of Pulp workers, resource managers, and celerybeats. \[done\]
234 42 dkliban@redhat.com
235 80 mhrivnak
As an unauthenticated user I can view the status of the web server's connection to the database and message broker. \[done\]
236 46 bmbouter
237 28 bmbouter
As an unauthenticated user I can view the versions of core and each installed plugin.
238 1 bmbouter
239 83 mhrivnak
## Plugin API
240 1 bmbouter
241 83 mhrivnak
As a plugin writer, I have a plugin API that is semantically versioned at 0.x separate from the REST API \[done\]
242 28 bmbouter
243 83 mhrivnak
As a plugin writer, I can report progress with a message and state \[done\]
244 28 bmbouter
245 83 mhrivnak
As a plugin writer, I can report progress with an optional suffix \[done\]
246 28 bmbouter
247 83 mhrivnak
As a plugin writer, I can report progress with a total count of things to do an the current count of things done \[done\]
248 28 bmbouter
249 83 mhrivnak
As a plugin writer, non-fatal exceptions on the Task and are included in the Task detail. non_fatal exceptions do not cause the Task to be marked as failed, but may be interpreted by the user as not fully successful. \[done\]
250 28 bmbouter
251 83 mhrivnak
As a plugin writer, the working directory is set before Task work is done and cleaned up afterwards. I should not need to interact with the file system outside of the working dir. \[done\]
252 28 bmbouter
253 84 bmbouter
<span style="color:red;">As a plugin writer, I can provide a subclassed Importer. The importer's responsibility is to synchronize the content of a Pulp repository with the content of a remote repository. (a circular import problem needs to be discussed and may cause this to change) \[done\]</span>
254 28 bmbouter
255 83 mhrivnak
<span style="color:red;">As a plugin writer, I can provide a subclassed Publisher. The publisher's responsibility is to publish content. (a circular import problem needs to be discussed and may cause this to change) \[done\]</span>
256 28 bmbouter
257 83 mhrivnak
As a plugin writer, I can define unit types by subclassing Content models to provide concrete content unit types to be manged by the platform. \[done\]
258 28 bmbouter
259 83 mhrivnak
As a plugin writer, I can interact with and create Artifacts \[done\]
260 28 bmbouter
261 83 mhrivnak
As a plugin writer, my app will be discovered by Pulp's app via an entry point provided by the plugin writer \[done\]
262 6 Ichimonji10
263 83 mhrivnak
As a plugin writer, I can use the plugin API to query content units/artifacts associated with a repository. \[done\]
264 1 bmbouter
265 8 Ichimonji10
As a plugin writer, I can add and remove content units to and from a repository. \[done\]
266
267 103 bmbouter
## Plugin Writer "live APIs"
268
269
<span style="color:red;">TBD</span>
270
271 94 bizhang
## CLI
272 93 bizhang
273 1 bmbouter
We will use coreapi-cli to generate a one to one mapping of cli commands to rest api schema #3068  
274 26 bmbouter
<span style="color:orange;">We will have a wrapper for coreapi-cli. This wrapper will handle parallel progress reporting</span>
275
276
## Download API
277
278
As a plugin writer, I can download files via
279
280
  - http://
281
  - https://
282
  - file://
283 27 bmbouter
284
As a plugin writer, I can configure a downloader with:
285 26 bmbouter
286
  - Basic Auth
287
  - SSL Cert Client Auth
288
  - Custom CAs will be configured via a "trust store" either on the system or similar. Pulp will not do anything to read/load/manage CAs directly.
289
290
As a plugin writer, I can provide arbitrary behaviors for customized downloaders
291
292
  - For example token authentication in the docker plugin
293
294
As a plugin writer, I can have connection pooling/reuse
295
296
As a plugin writer, I have proxy settings
297
298
  - proxy url (containing basic auth info)
299
300
As a plugin writer, I can have great logs
301
302
As a user, I have documentation about how to use something for bandwidth limiting
303
304
As a plugin writer, I can configure the validation mechanisms used at download time
305
306
  - checksum validation - minimum (md5, sha1, sha256, sha512)
307
  - size validation
308
309
<span style="color:red;">As a plugin writer, I expect units that are missing from the remote repository to not be created in Pulp when using the immediate download policy.</span>
310
311
<span style="color:red;">As a plugin writer, I expect units that are missing from the remote repository to be created in Pulp when using background or on_demand download policies.</span>
312
313
As a plugin writer I can configure mirror lists and rotate between the mirrors
314
315
  - round robin
316
  - nearest mirror support
317
318 1 bmbouter
As a plugin writer, the plugin API provides tooling whereby I can provide the content to be added and removed from the repository. This tooling supports both immediate and deferred downloading.
319 26 bmbouter
320 27 bmbouter
As a plugin writer I can manage the catalog by using ChangeSets
321 26 bmbouter
322
As a plugin writer, the plugin can participate in adding content for cases where the decision to add additional content is based content that has been downloaded.
323 1 bmbouter
324
As a plugin writer, I can fetch content myself (but I am not encouraged to do so) with code I write
325
326 56 ttereshc
As a plugin writer, I can CRUD content units
327 1 bmbouter
328 49 amacdona@redhat.com
## {color:red} Consumer Applicability
329
330 11 Ichimonji10
<span style="color:red;">Using consumer profiles and repo bindings I can compute applicability with 2.y parity  
331 49 amacdona@redhat.com
Performance needs to be awesome</span>
332 1 bmbouter
333 86 mhrivnak
<span style="color:red;">\_(Is the Pulp Consumer going away in Pulp 3? If so, is this section still appropriate?)\_</span>
334 36 bmbouter
335 86 mhrivnak
## Migrations only involving Pulp 3
336 1 bmbouter
337 87 mhrivnak
Users can run "pulp-manager migrate" to migrate the database and adjust state in other locations (filesystem, message broker, ...). \[done\]
338 1 bmbouter
339
## Web Server Integration
340
341 105 bmbouter
<span style="color:red;">As a user, I can have content efficiently served to me by Apache by Pulp using the X-SEND response headers \[[2914](https://pulp.plan.io/issues/2914)\]</span>  
342
<span style="color:red;">As a user, I can have content efficiently served to me by Nginx by Pulp using the X-Accel-Redirect response headers \[[2914](https://pulp.plan.io/issues/2914)\]</span>
343
344
<span style="color:red;">As a user, I can have an Ansible role to install Apache which enables Apache integration for Pulp and configures Apache to serve Pulp. \[[2921](https://pulp.plan.io/issues/2921)\]</span>  
345
<span style="color:red;">As a user, I can have an Ansible role to install Nginx which enables Nginx integration for Pulp and configures Nginx to serve Pulp. \[[2922](https://pulp.plan.io/issues/2922)\]</span>
346 87 mhrivnak
347 89 mhrivnak
## Glossary
348 88 mhrivnak
349 1 bmbouter
Repository - A named collection of content.
350
351 39 jortel@redhat.com
Artifact - A file associated with one content (unit). Artifacts are not shared between content (units). Create a content unit using an uploaded file ID as the source for its metadata. Create Artifacts associated with the content unit using an uploaded file ID for each; commit as a single transaction.
352
353
Content (unit) - A single piece of content manged by Pulp. Each file associated with a content (unit) is called an Artifact. Each content (unit) may have zero or many Artifacts.
354 95 bmbouter
355
Orphan Content (unit): A content unit that is a member of 0 repositories
356
357
Orphan Artifact: An Artifact that is associated with 0 Content Units and 0 Publications
358 98 bmbouter
359
Add (Content Unit): An operation causing a repository to contain a content unit(s)
360
361
Remove (content unit): An operation causing a repository to not contain a content unit(s)