Project

Profile

Help

Pulp 3 Minimum Viable Product » History » Sprint/Milestone 58

mhrivnak, 10/09/2017 08:31 PM

1 1 bmbouter
# Pulp 3.0.0 Minimum Viable Product (MVP)
2
3 21 bmbouter
<span style="color:red;">Lines highlighted in red need more attention.</span>
4 20 bmbouter
5 37 bmbouter
## Overall Guarantees
6
7 57 mhrivnak
  - This is not a direct replacement for Pulp 2.
8
  - This is the minimum required for a 3.0.0 beta and GA.
9
  - All REST API calls will update the DB using transactions as necessary to ensure data integrity.
10 37 bmbouter
11 1 bmbouter
## Authentication
12
13 17 semyers
As an authenticated user I can manage user(s).
14
15
  - Add a user
16
  - View user(s)
17
  - Update any user detail
18
  - Delete a user
19
20
As an API user, I can have documentation to generate a JSON Web Token (JWT) without the server being online.
21
22 58 mhrivnak
As an administrator, I can disable JWT token expiration. This configuration is in the settings file and is system-wide.
23 1 bmbouter
24 58 mhrivnak
As an administrator, I can configure the JWT tokens to expire after a configurable amount of time. This configuration is in the settings file and is system-wide.
25 1 bmbouter
26 58 mhrivnak
The JWT shall have a username identifier
27 1 bmbouter
28 58 mhrivnak
<span style="color:red;">As an API user, I can authenticate any API call (except to request a JWT) with a JWT. (not certain if this should be the behavior)</span>
29 17 semyers
30 58 mhrivnak
As an API user, I can invalidate all existing JWT tokens for a given user.
31 17 semyers
32 58 mhrivnak
As an authenticated user, when deleting a user 'foo', all of user 'foo's existing JWTs are invalidated.
33 17 semyers
34 58 mhrivnak
As an autheticated user, I can invalidate a user's JWTs in the same operation as updating the password.
35 45 bmbouter
36 58 mhrivnak
As an un-authenticated user, I can obtain a JWT token by using a username and password.
37 1 bmbouter
38
## Repositories
39
40 18 dkliban@redhat.com
As an authenticated user, I can list all repos.
41
42
  - All fields are included
43
  - Pagination is supported
44 22 bmbouter
  - <span style="color:red;">Filtering support</span>
45 18 dkliban@redhat.com
46
As an authenticated user, I can CRUD a repository
47
48
  - Create a repo
49
  - Read a repo
50
  - Update all mutable repo fields
51 21 bmbouter
  - Delete a repo (asynchronous)
52 18 dkliban@redhat.com
53
As an authenticated user, I can list a repository's associated importers and publishers
54
55
  - All fields are included
56
  - Pagination is supported
57 22 bmbouter
  - <span style="color:red;">Filtering support</span>
58 18 dkliban@redhat.com
59 20 bmbouter
<span style="color:red;">As an authenticated user, I can summarize content in a repo (including counts)</span>
60 18 dkliban@redhat.com
61 51 bmbouter
## Importers
62
63
note: Importer attributes will commonly be available on importers, but aren't guaranteed to be used by all importers.
64
65 18 dkliban@redhat.com
As an authenticated user, I can CRUD an importer
66
67
  - Create an importer
68
  - Read an importer
69
  - Update all mutable importer fields
70 21 bmbouter
  - Delete an importer (asynchronous)
71 18 dkliban@redhat.com
72 50 bmbouter
As an authenticated user I can configure the following attributes on an Importer:
73 47 bmbouter
74 48 amacdona@redhat.com
  - validate (bool) \[optional: defaults to True\]: If true, the plugin will validate imported content.
75
  - ssl_ca_certificate (str) \[optional\] String containing a PEM encoded CA certificate used to validate the server certificate presented by the external source.
76
  - ssl_client_certificate (str) \[optional\] Contains a PEM encoded client certificate used for authentication.
77
  - ssl_client_key (str) \[optional\] Contains a PEM encoded private key used for authentication.
78
  - ssl_validation (bool) \[optional: defaults to True\]: If true, SSL peer validation must be performed.
79
  - proxy_url (str) \[optional\] Contains the proxy URL. Format: scheme://user:password@host:port.
80
  - username (str) \[optional\] Contains the username to be used for authentication when syncing.
81 1 bmbouter
  - password (str) \[optional\] Contains the password to be used for authentication when syncing.
82 50 bmbouter
  - download_policy (str) \[optional\] Contains the downloading policy name. This is a choice of three options:  
83
    \- immediate (default) - Downloading occurs during sync. The sync task does not complete until downloading is done.  
84
    \- background - Downloading is started by the sync but occurs in the background. The sync task completes before downloading is complete.  
85
    \- on-demand - The sync task records everything that would be downloaded but does not download content. Downloading occurs on demand as driven by client requests for content.
86 1 bmbouter
  - feed_url (str) \[optional\] Contains the URL of an external content source. This is optional.
87 50 bmbouter
  - sync mode: (str) \[optional\]. It has two choices 'additive' and 'mirror':  
88
    \- additive (default) - all remote content is added to the local repository on sync. During sync no content is removed from the local repository.  
89
    \- mirror - the local content will mirror the remote content exactly, removing local content if not also present in the remote content.
90
  - name (str) \[required\] Contains the name.
91
  - last_updated (datetime) \[read-only\] Contains the datetime of the last importer update.
92 55 bmbouter
  - last_synced (datetime) \[read-only\] Contains the datetime of the last importer sync.
93 53 bmbouter
  - <span style="color:red;">max_count (int) - ???</span>
94 51 bmbouter
95 52 bmbouter
## Publishers
96 51 bmbouter
97
note: Publisher attributes will commonly be available on publishers, but aren't guaranteed to be used by all publishers.
98 47 bmbouter
99 18 dkliban@redhat.com
As an authenticated user, I can CRUD a publisher
100
101
  - Create a publisher
102
  - Read a publisher
103 1 bmbouter
  - Update all mutable publisher fields
104
  - Delete a publisher (asynchronous)
105 50 bmbouter
106
As an authenticated user I can configure the following attributes on a Publisher:
107
108
  - relative_path (str) \[optional\] The (relative) path component of the published url.
109
  - name - (str) \[required\] contains the name.
110
  - last_published (datetime) \[read-only\] When the last successful publish occurred.
111
  - last_updated (datetime) \[read-only\] The datetime of the last publisher update.
112 56 ttereshc
  - <span style="color:red;">auto_publish(bool) - ??? consider adding auto-publish feature to MVP</span>
113 1 bmbouter
114
## Content Manipulation
115
116 19 bmbouter
As an authenticated user, I can trigger an importer to sync.
117
118
  - I can follow the progress of all syncs. (Syncs are asynchronous.)
119
  - I cannot pass "sync" options.
120
  - Auto-publish is not included as an importer property.
121
122
As an authenticated user, I can trigger a publisher to publish.
123
124
  - I can follow the progress of all publishes. (Publishes are asynchronous.)
125
  - I cannot pass "publish" options.
126
127 25 bmbouter
## Upload & Copy
128
129 32 bmbouter
#### Getting bits from the client to Pulp
130
131 35 bmbouter
As an authenticated user, I can request a file ID from the server to upload a file with
132 19 bmbouter
133 35 bmbouter
As an authenticated user, I can upload a file with the server provided file ID, an optional chunk size, and an optional offset.
134 19 bmbouter
135 35 bmbouter
As an authenticated user, I can rely on Pulp to auto-delete uploaded files after a configurable time. (Eg: 6 hours).
136 30 bmbouter
137 35 bmbouter
As a user, I can delete an uploaded file by file ID
138 30 bmbouter
139 1 bmbouter
#### Creating Artifacts and Content Units
140 35 bmbouter
141 29 bmbouter
As a user, I can reference a file ID at Artifact creation time.
142 1 bmbouter
143 37 bmbouter
As an authenticated user, I can create a content unit by providing the content type, its Artifacts using file upload IDs for each Artifact, and the metadata supplied in the POST body. This call is atomic, either all Artifacts and the content unit are created in the database and on the filesystem or none are.
144 31 bmbouter
145 1 bmbouter
As an authenticated user, I can reuse a file ID to create multiple Artifacts without uploading the file twice.
146
147 37 bmbouter
#### Unit Management / Copy
148 29 bmbouter
149 41 jortel@redhat.com
As an authenticated user, I can add and remove one or more units to and from a destination repo.
150 1 bmbouter
151
  - <span style="color:red;">Filtering support for specifying the unit(s)</span>
152 41 jortel@redhat.com
  - I can follow the progress. (adding and removing are asynchronous).
153 1 bmbouter
154
## Versioned Repositories
155
156 40 bmbouter
As an authenticated user, I can list the content in a particular repository version
157
158
  - All fields are included
159
  - Pagination is supported
160
  - <span style="color:red;">Filtering support</span>
161
162
As an authenticated user, I can discover a URL to the latest version of a repository  
163
<span class="resource repository the on attributes or endpoint, API dedicated a through \^ Is" style="color:red;"></span>
164
165
As an authenticated user, I can run a publisher without a repository version and have it default to the latest version.
166
167
<span style="color:red;">As an authenticated user, I can delete a repository version by specifying the version</span>
168
169
<span style="color:red;">As an authenticated user, I can upload multiple content(s?) and add create a single new version that adds all of them.</span>
170 1 bmbouter
171 20 bmbouter
## Orphans
172
173 43 bmbouter
<span style="color:red;">As an authenticated user, I can clean up orphaned content units</span>  
174 1 bmbouter
<span style="color:red;">\* I can follow the progress of all cleanups. (Cleanups are asynchronous.)</span>
175 43 bmbouter
176
<span style="color:red;">As an authenticated user, I can delete a specific content unit</span>  
177
<span style="color:red;">\* If the content unit is still in at least one repository the delete fails with a listing of all repositories the unit is part of.</span>  
178
<span style="color:red;">\* Artifacts and associated files from the deleted unit are cleaned up</span>
179
180
<span style="color:red;">As an authenticated user, I can delete multiple content units with filtering</span>  
181
<span style="color:red;">\* If a content unit is still in at least one repository the delete fails with a listing of all repositories the unit is part of.</span>  
182
<span style="color:red;">\* Artifacts and associated files from deleted units are cleaned up</span>
183
184
<span style="color:red;">As an authenticated user, I see all (orphans) units that are not in any repositories</span>
185 1 bmbouter
186
## Filter
187
188 44 bmbouter
<span style="color:red;">I can filter all nouns *(What is the meaning of "filter?" What is a noun?)*</span>
189 1 bmbouter
190 22 bmbouter
## Task Management
191
192
As an authenticated user, I can list all tasks
193
194
  - Filtering support on \['state', 'id', 'group'\]
195
  - This does not include associated progress reports
196
197
As an authenticated user, I can see a detail view for a specific task
198
199
  - all attributes of a task
200
  - all associated progress reports
201
202
As an authenticated user, I can cancel a task
203 1 bmbouter
204
  - don't dare to use the DELETE verb!
205
206
## Task Group
207
208
I can view a summary of the status of all tasks in a group
209
210
## Event Listener Notifier
211
212 12 Ichimonji10
I can receive serialized task info via AMQP on each task save
213
214 1 bmbouter
*Can this be restated in more pedantic terms? Does this mean that an arbitrary host can attach itself to Pulp's AMQP message bus and get updates on the progress of tasks?*
215
216
## Status
217
218 42 dkliban@redhat.com
As an unauthenticated user I can view the status of Pulp workers, resource managers, and celerybeats.
219 1 bmbouter
220 42 dkliban@redhat.com
As an unauthenticated user I can view the status of httpd's connection to the database and message broker.
221
222 46 bmbouter
<span style="color:red;">As an administrator, the WSGI app will not start if all migrations have not been applied</span>
223
224 28 bmbouter
## Plugin API
225 1 bmbouter
226 28 bmbouter
As a plugin writer, I have a plugin API that is semantically versioned at 0.x separate from the REST API
227 1 bmbouter
228 28 bmbouter
As a plugin writer, I can report progress with a message and state
229
230
As a plugin writer, I can report progress with an optional suffix
231
232
As a plugin writer, I can report progress with a total count of things to do an the current count of things done
233
234
As a plugin writer, non-fatal exceptions a on the Task and are included in the Task detail. non_fatal exceptions do not cause the Task to be marked as failed, but may be interpreted by the user as not fully successful.
235
236
As a plugin writer, the working directory is set before Task work is done and cleaned up afterwards. I should not need to interact with the file system outside of the working dir.
237
238
As a plugin writer, I can provide a subclassed Importer. The importer's responsibility is to synchronize the content of a Pulp repository with the content of a remote repository.
239
240
As a plugin writer, I can provide a subclassed Publisher. The publisher's responsibility is to publish content.
241
242
As a plugin writer, I can define unit types by subclassing Content models to provide concrete content unit types to be manged by the platform.
243
244
As a plugin writer, I can interact with and create Artifacts
245
246
As a plugin writer, my app will be discovered by Pulp's app via an entry point provided by the plugin writer
247
248
As a plugin writer, I can use the plugin API to query content units/artifacts associated with a repository.
249 6 Ichimonji10
250 41 jortel@redhat.com
As a plugin writer, I can add and remove content units to and from a repository.
251 1 bmbouter
252 8 Ichimonji10
## CLI
253
254 54 bmbouter
<span style="color:red;">We will port what is there with as little effort as possible *(Does this mean that porting will be easy for developers, or that switching from the Pulp 2-3 CLI will be easy for users? If the former, isn't this an implementation detail that doesn't belong in an MVP document? If the latter, does this mean that we're going to carry forward the issues with pulp-admin, like a lack of status codes?)*</span>
255 1 bmbouter
256 54 bmbouter
<span style="color:red;">repo CRUD</span>  
257
<span style="color:red;">CRUD for importers</span>  
258
<span style="color:red;">CRUD for publishers</span>  
259
<span style="color:red;">trigger syncs</span>  
260
<span style="color:red;">trigger publish</span>  
261
<span style="color:red;">list content in a repo</span>  
262
<span style="color:red;">upload</span>  
263
<span style="color:red;">server status</span>  
264
<span style="color:red;">list and cancel tasks</span>  
265
<span style="color:red;">authn via basic auth</span>  
266
<span style="color:red;">\_(Should the supported set of operations be stated in terms of "The capabilities listed in the 'Authenctication,' 'Repositories,' and 'Filter' sections will be supported by the CLI."?)\_</span>
267 1 bmbouter
268 26 bmbouter
## Download API
269
270
As a plugin writer, I can download files via
271
272
  - http://
273
  - https://
274
  - file://
275
276
As a plugin writer, I can configure a downloader with:
277 27 bmbouter
278
  - Basic Auth
279 26 bmbouter
  - SSL Cert Client Auth
280
  - Custom CAs will be configured via a "trust store" either on the system or similar. Pulp will not do anything to read/load/manage CAs directly.
281
282
As a plugin writer, I can provide arbitrary behaviors for customized downloaders
283
284
  - For example token authentication in the docker plugin
285
286
As a plugin writer, I can have connection pooling/reuse
287
288
As a plugin writer, I have proxy settings
289
290
  - proxy url (containing basic auth info)
291
292
As a plugin writer, I can have great logs
293
294
As a user, I have documentation about how to use something for bandwidth limiting
295
296
As a plugin writer, I can configure the validation mechanisms used at download time
297
298
  - checksum validation - minimum (md5, sha1, sha256, sha512)
299
  - size validation
300
301
<span style="color:red;">As a plugin writer, I expect units that are missing from the remote repository to not be created in Pulp when using the immediate download policy.</span>
302
303
<span style="color:red;">As a plugin writer, I expect units that are missing from the remote repository to be created in Pulp when using background or on_demand download policies.</span>
304
305
As a plugin writer I can configure mirror lists and rotate between the mirrors
306
307
  - round robin
308
  - nearest mirror support
309
310
As a plugin writer, the plugin API provides tooling whereby I can provide the content to be added and removed from the repository. This tooling supports both immediate and deferred downloading.
311
312 1 bmbouter
As a plugin writer I can manage the catalog by using ChangeSets
313 26 bmbouter
314 27 bmbouter
As a plugin writer, the plugin can participate in adding content for cases where the decision to add additional content is based content that has been downloaded.
315 26 bmbouter
316
As a plugin writer, I can fetch content myself (but I am not encouraged to do so) with code I write
317 1 bmbouter
318
As a plugin writer, I can CRUD content units
319
320 56 ttereshc
## {color:red} Consumer Applicability
321 1 bmbouter
322 49 amacdona@redhat.com
<span style="color:red;">Using consumer profiles and repo bindings I can compute applicability with 2.y parity  
323
Performance needs to be awesome</span>
324 11 Ichimonji10
325 49 amacdona@redhat.com
<span style="color:red;">\_(Is the Pulp Consumer going away in Pulp 3? If so, is this section still appropriate?)\_</span>
326 1 bmbouter
327
## Plugin compatibility
328
329
rpm will work with platform  
330
puppet will work with platform  
331
ostree will work with platform  
332
python will work with platform  
333
file_plugin will work with platform  
334
docker will work with platform
335
336
## Migrations
337 20 bmbouter
338
users can run an executable similar to pulp-manage-db that is not named pulp-manage-db *(Why the change in name?)*
339 36 bmbouter
340
<span style="color:red;">What about migrating fields that we don't use in 3.0 but will use in 3.1+. For example the auto-publish feature?</span>
341 1 bmbouter
342
## Glossary
343
344 39 jortel@redhat.com
Repository - A named collection of content.
345
346
Artifact - A file associated with one content (unit). Artifacts are not shared between content (units). Create a content unit using an uploaded file ID as the source for its metadata. Create Artifacts associated with the content unit using an uploaded file ID for each; commit as a single transaction.
347
348
Content (unit) - A single piece of content manged by Pulp. Each file associated with a content (unit) is called an Artifact. Each content (unit) may have zero or many Artifacts.