Project

Profile

Help

Pulp 3 Minimum Viable Product » History » Sprint/Milestone 79

mhrivnak, 10/09/2017 10:17 PM

1 1 bmbouter
# Pulp 3.0.0 Minimum Viable Product (MVP)
2
3 21 bmbouter
<span style="color:red;">Lines highlighted in red need more attention.</span>
4 20 bmbouter
5 37 bmbouter
## Overall Guarantees
6
7 57 mhrivnak
  - This is not a direct replacement for Pulp 2.
8
  - This is the minimum required for a 3.0.0 beta and GA.
9
  - All REST API calls will update the DB using transactions as necessary to ensure data integrity.
10 37 bmbouter
11 59 bmbouter
## Legend
12
13
\[done\] means merged and documented  
14
\[in-progress\] means started but not fully done  
15
If there is no label the effort has not yet been started
16
17 1 bmbouter
## Authentication
18
19 59 bmbouter
As an authenticated user I can manage user(s). \[done\]
20 17 semyers
21
  - Add a user
22
  - View user(s)
23 58 mhrivnak
  - Update any user detail
24 1 bmbouter
  - Delete a user
25
26 59 bmbouter
As an API user, I can have documentation to generate a JSON Web Token (JWT) without the server being online. \[done\]
27 1 bmbouter
28 59 bmbouter
As an administrator, I can disable JWT token expiration. This configuration is in the settings file and is system-wide. \[done\]
29 1 bmbouter
30 59 bmbouter
As an administrator, I can configure the JWT tokens to expire after a configurable amount of time. This configuration is in the settings file and is system-wide. \[done\]
31 1 bmbouter
32 59 bmbouter
The JWT shall have a username identifier \[done\]
33 1 bmbouter
34 59 bmbouter
<span style="color:red;">As an API user, I can authenticate any API call (except to request a JWT) with a JWT. (not certain if this should be the behavior) \[in progress\]</span>
35 17 semyers
36 59 bmbouter
As an API user, I can invalidate all existing JWT tokens for a given user. \[done\]
37 17 semyers
38 59 bmbouter
As an authenticated user, when deleting a user 'foo', all of user 'foo's existing JWTs are invalidated. \[done\]
39 17 semyers
40 59 bmbouter
As an autheticated user, I can invalidate a user's JWTs in the same operation as updating the password. \[done\]
41 45 bmbouter
42 59 bmbouter
As an un-authenticated user, I can obtain a JWT token by using a username and password. \[done\]
43 1 bmbouter
44
## Repositories
45
46 18 dkliban@redhat.com
As an authenticated user, I can list all repos.
47
48 60 bmbouter
  - All fields are included \[done\]
49
  - Pagination is supported \[done\]
50 61 mhrivnak
  - <span style="color:orange;">Filtering support</span>
51 18 dkliban@redhat.com
52
As an authenticated user, I can CRUD a repository
53
54 60 bmbouter
  - Create a repo \[done\]
55
  - Read a repo \[done\]
56
  - Update all mutable repo fields \[done\]
57
  - Delete a repo (asynchronous) \[done\]
58 18 dkliban@redhat.com
59
As an authenticated user, I can list a repository's associated importers and publishers
60
61 60 bmbouter
  - All fields are included \[done\]
62
  - Pagination is supported \[done\]
63 1 bmbouter
64 62 mhrivnak
<span style="color:orange;">As an authenticated user, I can see the number of content unit types with counts for each</span>
65 18 dkliban@redhat.com
66 51 bmbouter
## Importers
67
68
note: Importer attributes will commonly be available on importers, but aren't guaranteed to be used by all importers.
69
70 18 dkliban@redhat.com
As an authenticated user, I can CRUD an importer
71
72
  - Create an importer
73
  - Read an importer
74
  - Update all mutable importer fields
75 21 bmbouter
  - Delete an importer (asynchronous)
76 18 dkliban@redhat.com
77 64 mhrivnak
As an authenticated user I can configure the following attributes on an Importer: \[done\]
78 47 bmbouter
79 48 amacdona@redhat.com
  - validate (bool) \[optional: defaults to True\]: If true, the plugin will validate imported content.
80
  - ssl_ca_certificate (str) \[optional\] String containing a PEM encoded CA certificate used to validate the server certificate presented by the external source.
81
  - ssl_client_certificate (str) \[optional\] Contains a PEM encoded client certificate used for authentication.
82
  - ssl_client_key (str) \[optional\] Contains a PEM encoded private key used for authentication.
83
  - ssl_validation (bool) \[optional: defaults to True\]: If true, SSL peer validation must be performed.
84 65 mhrivnak
  - proxy_url (str) \[optional\] Contains the proxy URL. Format: scheme://user:password@host:port.
85 48 amacdona@redhat.com
  - username (str) \[optional\] Contains the username to be used for authentication when syncing.
86 1 bmbouter
  - password (str) \[optional\] Contains the password to be used for authentication when syncing.
87 63 mhrivnak
  - <span style="color:orange;">download_policy (str) \[optional\] Contains the downloading policy name. This is a choice of three options:  
88
 \- immediate (default) - Downloading occurs during sync. The sync task does not complete until downloading is done.  
89
 \- background - Downloading is started by the sync but occurs in the background. The sync task completes before downloading is complete.  
90
 \- on-demand - The sync task records everything that would be downloaded but does not download content. Downloading occurs on demand as driven by client requests for content.</span>
91 1 bmbouter
  - feed_url (str) \[optional\] Contains the URL of an external content source. This is optional.
92 63 mhrivnak
  - <span style="color:orange;">sync mode: (str) \[optional\]. It has two choices 'additive' and 'mirror':  
93
 \- additive (default) - all remote content is added to the local repository on sync. During sync no content is removed from the local repository.  
94
 \- mirror - the local content will mirror the remote content exactly, removing local content if not also present in the remote content.</span>
95 50 bmbouter
  - name (str) \[required\] Contains the name.
96
  - last_updated (datetime) \[read-only\] Contains the datetime of the last importer update.
97 53 bmbouter
  - last_synced (datetime) \[read-only\] Contains the datetime of the last importer sync.
98 51 bmbouter
99 52 bmbouter
## Publishers
100 51 bmbouter
101
note: Publisher attributes will commonly be available on publishers, but aren't guaranteed to be used by all publishers.
102 47 bmbouter
103 18 dkliban@redhat.com
As an authenticated user, I can CRUD a publisher
104
105
  - Create a publisher
106
  - Read a publisher
107 1 bmbouter
  - Update all mutable publisher fields
108
  - Delete a publisher (asynchronous)
109 50 bmbouter
110
As an authenticated user I can configure the following attributes on a Publisher:
111
112
  - relative_path (str) \[optional\] The (relative) path component of the published url.
113
  - name - (str) \[required\] contains the name.
114
  - last_published (datetime) \[read-only\] When the last successful publish occurred.
115
  - last_updated (datetime) \[read-only\] The datetime of the last publisher update.
116 56 ttereshc
  - <span style="color:red;">auto_publish(bool) - ??? consider adding auto-publish feature to MVP</span>
117 1 bmbouter
118 72 mhrivnak
## Sync and Publish
119 1 bmbouter
120 66 mhrivnak
As an authenticated user, I can trigger an importer to sync. \[done\]
121 19 bmbouter
122
  - I can follow the progress of all syncs. (Syncs are asynchronous.)
123
  - I cannot pass "sync" options.
124
  - Auto-publish is not included as an importer property.
125
126 66 mhrivnak
As an authenticated user, I can trigger a publisher to publish. \[done\]
127 19 bmbouter
128
  - I can follow the progress of all publishes. (Publishes are asynchronous.)
129
  - I cannot pass "publish" options.
130
131 71 mhrivnak
## Content Manipulation
132 25 bmbouter
133 73 mhrivnak
#### Uploading Artifacts
134 32 bmbouter
135 67 mhrivnak
As an authenticated user, I can create an Artifact by uploading a file. \[done\]
136 30 bmbouter
137 68 mhrivnak
As an authenticated user, I can specify a size and/or digest to validate the uploaded file. \[done\]
138
139 71 mhrivnak
#### Creating Content Units
140 35 bmbouter
141 70 mhrivnak
As an authenticated user, I can create a content unit by providing the content type (in the URL), references to Artifacts, and the metadata supplied in the POST body. \[done\]
142 1 bmbouter
143 69 mhrivnak
#### Content Management / Copy
144 29 bmbouter
145 73 mhrivnak
<span style="color:red;">As an authenticated user, I can add and remove one or more units to and from a destination repo.</span>
146 1 bmbouter
147
  - <span style="color:red;">Filtering support for specifying the unit(s)</span>
148 73 mhrivnak
  - <span style="color:red;">I can follow the progress. (adding and removing are asynchronous).</span>
149 71 mhrivnak
150
#### <span style="color:red;">Content Removal</span>
151 1 bmbouter
152 74 mhrivnak
## <span style="color:red;">Versioned Repositories</span>
153 1 bmbouter
154 40 bmbouter
As an authenticated user, I can list the content in a particular repository version
155
156
  - All fields are included
157
  - Pagination is supported
158
  - <span style="color:red;">Filtering support</span>
159
160
As an authenticated user, I can discover a URL to the latest version of a repository  
161
<span class="resource repository the on attributes or endpoint, API dedicated a through \^ Is" style="color:red;"></span>
162
163
As an authenticated user, I can run a publisher without a repository version and have it default to the latest version.
164
165
<span style="color:red;">As an authenticated user, I can delete a repository version by specifying the version</span>
166
167
<span style="color:red;">As an authenticated user, I can upload multiple content(s?) and add create a single new version that adds all of them.</span>
168 1 bmbouter
169 20 bmbouter
## Orphans
170
171 43 bmbouter
<span style="color:red;">As an authenticated user, I can clean up orphaned content units</span>  
172 1 bmbouter
<span style="color:red;">\* I can follow the progress of all cleanups. (Cleanups are asynchronous.)</span>
173 43 bmbouter
174
<span style="color:red;">As an authenticated user, I can delete a specific content unit</span>  
175
<span style="color:red;">\* If the content unit is still in at least one repository the delete fails with a listing of all repositories the unit is part of.</span>  
176
<span style="color:red;">\* Artifacts and associated files from the deleted unit are cleaned up</span>
177
178
<span style="color:red;">As an authenticated user, I can delete multiple content units with filtering</span>  
179
<span style="color:red;">\* If a content unit is still in at least one repository the delete fails with a listing of all repositories the unit is part of.</span>  
180
<span style="color:red;">\* Artifacts and associated files from deleted units are cleaned up</span>
181
182
<span style="color:red;">As an authenticated user, I see all (orphans) units that are not in any repositories</span>
183 1 bmbouter
184 22 bmbouter
## Task Management
185
186
As an authenticated user, I can list all tasks
187
188 76 mhrivnak
  - <span style="color:orange;">Filtering support on \['state', 'id', 'group'\]</span>
189
  - <span style="color:orange;">This does not include associated progress reports</span>
190 22 bmbouter
191 76 mhrivnak
As an authenticated user, I can see a detail view for a specific task \[done\]
192 22 bmbouter
193
  - all attributes of a task
194
  - all associated progress reports
195
196 76 mhrivnak
As an authenticated user, I can cancel a task \[done\]
197 1 bmbouter
198
  - don't dare to use the DELETE verb!
199
200 77 mhrivnak
As an authenticated user, I can delete tasks.
201
202 1 bmbouter
## Task Group
203
204 78 mhrivnak
<span style="color:red;">I can view a summary of the status of all tasks in a group</span>
205 1 bmbouter
206
## Status
207
208 42 dkliban@redhat.com
As an unauthenticated user I can view the status of Pulp workers, resource managers, and celerybeats.
209 1 bmbouter
210 42 dkliban@redhat.com
As an unauthenticated user I can view the status of httpd's connection to the database and message broker.
211
212 46 bmbouter
<span style="color:red;">As an administrator, the WSGI app will not start if all migrations have not been applied</span>
213
214 28 bmbouter
## Plugin API
215 1 bmbouter
216 28 bmbouter
As a plugin writer, I have a plugin API that is semantically versioned at 0.x separate from the REST API
217 1 bmbouter
218 28 bmbouter
As a plugin writer, I can report progress with a message and state
219
220
As a plugin writer, I can report progress with an optional suffix
221
222
As a plugin writer, I can report progress with a total count of things to do an the current count of things done
223
224
As a plugin writer, non-fatal exceptions a on the Task and are included in the Task detail. non_fatal exceptions do not cause the Task to be marked as failed, but may be interpreted by the user as not fully successful.
225
226
As a plugin writer, the working directory is set before Task work is done and cleaned up afterwards. I should not need to interact with the file system outside of the working dir.
227
228
As a plugin writer, I can provide a subclassed Importer. The importer's responsibility is to synchronize the content of a Pulp repository with the content of a remote repository.
229
230
As a plugin writer, I can provide a subclassed Publisher. The publisher's responsibility is to publish content.
231
232
As a plugin writer, I can define unit types by subclassing Content models to provide concrete content unit types to be manged by the platform.
233
234
As a plugin writer, I can interact with and create Artifacts
235
236
As a plugin writer, my app will be discovered by Pulp's app via an entry point provided by the plugin writer
237
238
As a plugin writer, I can use the plugin API to query content units/artifacts associated with a repository.
239 6 Ichimonji10
240 41 jortel@redhat.com
As a plugin writer, I can add and remove content units to and from a repository.
241 1 bmbouter
242 8 Ichimonji10
## CLI
243
244 54 bmbouter
<span style="color:red;">We will port what is there with as little effort as possible *(Does this mean that porting will be easy for developers, or that switching from the Pulp 2-3 CLI will be easy for users? If the former, isn't this an implementation detail that doesn't belong in an MVP document? If the latter, does this mean that we're going to carry forward the issues with pulp-admin, like a lack of status codes?)*</span>
245 1 bmbouter
246 54 bmbouter
<span style="color:red;">repo CRUD</span>  
247
<span style="color:red;">CRUD for importers</span>  
248
<span style="color:red;">CRUD for publishers</span>  
249
<span style="color:red;">trigger syncs</span>  
250
<span style="color:red;">trigger publish</span>  
251
<span style="color:red;">list content in a repo</span>  
252
<span style="color:red;">upload</span>  
253
<span style="color:red;">server status</span>  
254
<span style="color:red;">list and cancel tasks</span>  
255
<span style="color:red;">authn via basic auth</span>  
256
<span style="color:red;">\_(Should the supported set of operations be stated in terms of "The capabilities listed in the 'Authenctication,' 'Repositories,' and 'Filter' sections will be supported by the CLI."?)\_</span>
257 1 bmbouter
258 26 bmbouter
## Download API
259
260
As a plugin writer, I can download files via
261
262
  - http://
263
  - https://
264
  - file://
265
266
As a plugin writer, I can configure a downloader with:
267 27 bmbouter
268
  - Basic Auth
269 26 bmbouter
  - SSL Cert Client Auth
270
  - Custom CAs will be configured via a "trust store" either on the system or similar. Pulp will not do anything to read/load/manage CAs directly.
271
272
As a plugin writer, I can provide arbitrary behaviors for customized downloaders
273
274
  - For example token authentication in the docker plugin
275
276
As a plugin writer, I can have connection pooling/reuse
277
278
As a plugin writer, I have proxy settings
279
280
  - proxy url (containing basic auth info)
281
282
As a plugin writer, I can have great logs
283
284
As a user, I have documentation about how to use something for bandwidth limiting
285
286
As a plugin writer, I can configure the validation mechanisms used at download time
287
288
  - checksum validation - minimum (md5, sha1, sha256, sha512)
289
  - size validation
290
291
<span style="color:red;">As a plugin writer, I expect units that are missing from the remote repository to not be created in Pulp when using the immediate download policy.</span>
292
293
<span style="color:red;">As a plugin writer, I expect units that are missing from the remote repository to be created in Pulp when using background or on_demand download policies.</span>
294
295
As a plugin writer I can configure mirror lists and rotate between the mirrors
296
297
  - round robin
298
  - nearest mirror support
299
300
As a plugin writer, the plugin API provides tooling whereby I can provide the content to be added and removed from the repository. This tooling supports both immediate and deferred downloading.
301
302 1 bmbouter
As a plugin writer I can manage the catalog by using ChangeSets
303 26 bmbouter
304 27 bmbouter
As a plugin writer, the plugin can participate in adding content for cases where the decision to add additional content is based content that has been downloaded.
305 26 bmbouter
306
As a plugin writer, I can fetch content myself (but I am not encouraged to do so) with code I write
307 1 bmbouter
308
As a plugin writer, I can CRUD content units
309
310 56 ttereshc
## {color:red} Consumer Applicability
311 1 bmbouter
312 49 amacdona@redhat.com
<span style="color:red;">Using consumer profiles and repo bindings I can compute applicability with 2.y parity  
313
Performance needs to be awesome</span>
314 11 Ichimonji10
315 49 amacdona@redhat.com
<span style="color:red;">\_(Is the Pulp Consumer going away in Pulp 3? If so, is this section still appropriate?)\_</span>
316 1 bmbouter
317
## Plugin compatibility
318
319
rpm will work with platform  
320
puppet will work with platform  
321
ostree will work with platform  
322
python will work with platform  
323
file_plugin will work with platform  
324
docker will work with platform
325
326
## Migrations
327 20 bmbouter
328
users can run an executable similar to pulp-manage-db that is not named pulp-manage-db *(Why the change in name?)*
329 36 bmbouter
330
<span style="color:red;">What about migrating fields that we don't use in 3.0 but will use in 3.1+. For example the auto-publish feature?</span>
331 1 bmbouter
332
## Glossary
333
334 39 jortel@redhat.com
Repository - A named collection of content.
335
336
Artifact - A file associated with one content (unit). Artifacts are not shared between content (units). Create a content unit using an uploaded file ID as the source for its metadata. Create Artifacts associated with the content unit using an uploaded file ID for each; commit as a single transaction.
337
338
Content (unit) - A single piece of content manged by Pulp. Each file associated with a content (unit) is called an Artifact. Each content (unit) may have zero or many Artifacts.