Project

Profile

Help

Pulp 3 Minimum Viable Product » History » Sprint/Milestone 48

amacdona@redhat.com, 06/13/2017 08:32 PM

1 1 bmbouter
# Pulp 3.0.0 Minimum Viable Product (MVP)
2
3 21 bmbouter
<span style="color:red;">Lines highlighted in red need more attention.</span>
4 20 bmbouter
5 37 bmbouter
## Overall Guarantees
6
7 39 jortel@redhat.com
All REST API calls will update the DB using transactions as necessary to ensure data integrity.
8 37 bmbouter
9 1 bmbouter
## Authentication
10
11 17 semyers
As an authenticated user I can manage user(s).
12
13
  - Add a user
14
  - View user(s)
15
  - Update any user detail
16
  - Delete a user
17
18
As an API user, I can have documentation to generate a JSON Web Token (JWT) without the server being online.
19
20 45 bmbouter
<span style="color:red;">As an administrator, I can disable JWT token expiration. This configuration is in the settings file and is system-wide.</span>
21 1 bmbouter
22 45 bmbouter
<span style="color:red;">As an administrator, I can configure the JWT tokens to expire after a configurable amount of time. This configuration is in the settings file and is system-wide.</span>
23 1 bmbouter
24 45 bmbouter
<span style="color:red;">The JWT shall have a username identifier</span>
25 1 bmbouter
26 45 bmbouter
<span style="color:red;">As an API user, I can authenticate any API call (except to request a JWT) with a JWT.</span>
27 17 semyers
28 45 bmbouter
<span style="color:red;">As an API user, I can invalidate all existing JWT tokens for a given user.</span>
29 17 semyers
30 45 bmbouter
<span style="color:red;">As an authenticated user, when deleting a user 'foo', all of user 'foo's existing JWTs are invalidated.</span>
31 17 semyers
32 45 bmbouter
<span style="color:red;">As an autheticated user, I can invalidate a user's JWTs in the same operation as updating the password.</span>
33
34
<span style="color:red;">As an un-authenticated user, I can obtain a JWT token by using a username and password.</span>
35 1 bmbouter
36
## Repositories
37
38 18 dkliban@redhat.com
As an authenticated user, I can list all repos.
39
40
  - All fields are included
41
  - Pagination is supported
42 22 bmbouter
  - <span style="color:red;">Filtering support</span>
43 18 dkliban@redhat.com
44
As an authenticated user, I can CRUD a repository
45
46
  - Create a repo
47
  - Read a repo
48
  - Update all mutable repo fields
49 21 bmbouter
  - Delete a repo (asynchronous)
50 18 dkliban@redhat.com
51
As an authenticated user, I can list a repository's associated importers and publishers
52
53
  - All fields are included
54
  - Pagination is supported
55 22 bmbouter
  - <span style="color:red;">Filtering support</span>
56 18 dkliban@redhat.com
57 20 bmbouter
<span style="color:red;">As an authenticated user, I can summarize content in a repo (including counts)</span>
58 18 dkliban@redhat.com
59
As an authenticated user, I can CRUD an importer
60
61
  - Create an importer
62
  - Read an importer
63
  - Update all mutable importer fields
64 21 bmbouter
  - Delete an importer (asynchronous)
65 18 dkliban@redhat.com
66 47 bmbouter
As an authenticated user I need to be able to configure the following attributes on an Importer:
67
68 48 amacdona@redhat.com
  - validate (bool) \[optional: defaults to True\]: If true, the plugin will validate imported content.
69
  - ssl_ca_certificate (str) \[optional\] String containing a PEM encoded CA certificate used to validate the server certificate presented by the external source.
70
  - ssl_client_certificate (str) \[optional\] Contains a PEM encoded client certificate used for authentication.
71
  - ssl_client_key (str) \[optional\] Contains a PEM encoded private key used for authentication.
72
  - ssl_validation (bool) \[optional: defaults to True\]: If true, SSL peer validation must be performed.
73
  - proxy_url (str) \[optional\] Contains the proxy URL. Format: scheme://user:password@host:port.
74
  - username (str) \[optional\] Contains the username to be used for authentication when syncing.
75
  - password (str) \[optional\] Contains the password to be used for authentication when syncing.
76
  - download_policy (str) \[optional: defaults to \`immediate\`\] Contains the downloading policy name. (immediate|background|on-demand). 
77
  - feed_url (str) \[optional\] Contains the URL of an external content source. This is optional.
78 47 bmbouter
79 18 dkliban@redhat.com
As an authenticated user, I can CRUD a publisher
80
81
  - Create a publisher
82
  - Read a publisher
83
  - Update all mutable publisher fields
84 21 bmbouter
  - Delete a publisher (asynchronous)
85 1 bmbouter
86
## Content Manipulation
87
88 19 bmbouter
As an authenticated user, I can trigger an importer to sync.
89
90
  - I can follow the progress of all syncs. (Syncs are asynchronous.)
91
  - I cannot pass "sync" options.
92
  - Auto-publish is not included as an importer property.
93
94
As an authenticated user, I can trigger a publisher to publish.
95
96
  - I can follow the progress of all publishes. (Publishes are asynchronous.)
97
  - I cannot pass "publish" options.
98
99 25 bmbouter
## Upload & Copy
100
101 32 bmbouter
#### Getting bits from the client to Pulp
102
103 35 bmbouter
As an authenticated user, I can request a file ID from the server to upload a file with
104 19 bmbouter
105 35 bmbouter
As an authenticated user, I can upload a file with the server provided file ID, an optional chunk size, and an optional offset.
106 19 bmbouter
107 35 bmbouter
As an authenticated user, I can rely on Pulp to auto-delete uploaded files after a configurable time. (Eg: 6 hours).
108 30 bmbouter
109 35 bmbouter
As a user, I can delete an uploaded file by file ID
110 30 bmbouter
111 1 bmbouter
#### Creating Artifacts and Content Units
112 35 bmbouter
113 29 bmbouter
As a user, I can reference a file ID at Artifact creation time.
114 1 bmbouter
115 37 bmbouter
As an authenticated user, I can create a content unit by providing the content type, its Artifacts using file upload IDs for each Artifact, and the metadata supplied in the POST body. This call is atomic, either all Artifacts and the content unit are created in the database and on the filesystem or none are.
116 31 bmbouter
117 1 bmbouter
As an authenticated user, I can reuse a file ID to create multiple Artifacts without uploading the file twice.
118
119 37 bmbouter
#### Unit Management / Copy
120 29 bmbouter
121 41 jortel@redhat.com
As an authenticated user, I can add and remove one or more units to and from a destination repo.
122 1 bmbouter
123
  - <span style="color:red;">Filtering support for specifying the unit(s)</span>
124 41 jortel@redhat.com
  - I can follow the progress. (adding and removing are asynchronous).
125 1 bmbouter
126
## Versioned Repositories
127
128 40 bmbouter
As an authenticated user, I can list the content in a particular repository version
129
130
  - All fields are included
131
  - Pagination is supported
132
  - <span style="color:red;">Filtering support</span>
133
134
As an authenticated user, I can discover a URL to the latest version of a repository  
135
<span class="resource repository the on attributes or endpoint, API dedicated a through \^ Is" style="color:red;"></span>
136
137
As an authenticated user, I can run a publisher without a repository version and have it default to the latest version.
138
139
<span style="color:red;">As an authenticated user, I can delete a repository version by specifying the version</span>
140
141
<span style="color:red;">As an authenticated user, I can upload multiple content(s?) and add create a single new version that adds all of them.</span>
142 1 bmbouter
143 20 bmbouter
## Orphans
144
145 43 bmbouter
<span style="color:red;">As an authenticated user, I can clean up orphaned content units</span>  
146 1 bmbouter
<span style="color:red;">\* I can follow the progress of all cleanups. (Cleanups are asynchronous.)</span>
147 43 bmbouter
148
<span style="color:red;">As an authenticated user, I can delete a specific content unit</span>  
149
<span style="color:red;">\* If the content unit is still in at least one repository the delete fails with a listing of all repositories the unit is part of.</span>  
150
<span style="color:red;">\* Artifacts and associated files from the deleted unit are cleaned up</span>
151
152
<span style="color:red;">As an authenticated user, I can delete multiple content units with filtering</span>  
153
<span style="color:red;">\* If a content unit is still in at least one repository the delete fails with a listing of all repositories the unit is part of.</span>  
154
<span style="color:red;">\* Artifacts and associated files from deleted units are cleaned up</span>
155
156
<span style="color:red;">As an authenticated user, I see all (orphans) units that are not in any repositories</span>
157 1 bmbouter
158
## Filter
159
160 44 bmbouter
<span style="color:red;">I can filter all nouns *(What is the meaning of "filter?" What is a noun?)*</span>
161 1 bmbouter
162 22 bmbouter
## Task Management
163
164
As an authenticated user, I can list all tasks
165
166
  - Filtering support on \['state', 'id', 'group'\]
167
  - This does not include associated progress reports
168
169
As an authenticated user, I can see a detail view for a specific task
170
171
  - all attributes of a task
172
  - all associated progress reports
173
174
As an authenticated user, I can cancel a task
175 1 bmbouter
176
  - don't dare to use the DELETE verb!
177
178
## Task Group
179
180
I can view a summary of the status of all tasks in a group
181
182
## Event Listener Notifier
183
184 12 Ichimonji10
I can receive serialized task info via AMQP on each task save
185
186 1 bmbouter
*Can this be restated in more pedantic terms? Does this mean that an arbitrary host can attach itself to Pulp's AMQP message bus and get updates on the progress of tasks?*
187
188
## Status
189
190 42 dkliban@redhat.com
As an unauthenticated user I can view the status of Pulp workers, resource managers, and celerybeats.
191 1 bmbouter
192 42 dkliban@redhat.com
As an unauthenticated user I can view the status of httpd's connection to the database and message broker.
193
194 46 bmbouter
<span style="color:red;">As an administrator, the WSGI app will not start if all migrations have not been applied</span>
195
196 28 bmbouter
## Plugin API
197 1 bmbouter
198 28 bmbouter
As a plugin writer, I have a plugin API that is semantically versioned at 0.x separate from the REST API
199 1 bmbouter
200 28 bmbouter
As a plugin writer, I can report progress with a message and state
201
202
As a plugin writer, I can report progress with an optional suffix
203
204
As a plugin writer, I can report progress with a total count of things to do an the current count of things done
205
206
As a plugin writer, non-fatal exceptions a on the Task and are included in the Task detail. non_fatal exceptions do not cause the Task to be marked as failed, but may be interpreted by the user as not fully successful.
207
208
As a plugin writer, the working directory is set before Task work is done and cleaned up afterwards. I should not need to interact with the file system outside of the working dir.
209
210
As a plugin writer, I can provide a subclassed Importer. The importer's responsibility is to synchronize the content of a Pulp repository with the content of a remote repository.
211
212
As a plugin writer, I can provide a subclassed Publisher. The publisher's responsibility is to publish content.
213
214
As a plugin writer, I can define unit types by subclassing Content models to provide concrete content unit types to be manged by the platform.
215
216
As a plugin writer, I can interact with and create Artifacts
217
218
As a plugin writer, my app will be discovered by Pulp's app via an entry point provided by the plugin writer
219
220
As a plugin writer, I can use the plugin API to query content units/artifacts associated with a repository.
221 6 Ichimonji10
222 41 jortel@redhat.com
As a plugin writer, I can add and remove content units to and from a repository.
223 1 bmbouter
224 8 Ichimonji10
## CLI
225
226 1 bmbouter
We will port what is there with as little effort as possible *(Does this mean that porting will be easy for developers, or that switching from the Pulp 2-3 CLI will be easy for users? If the former, isn't this an implementation detail that doesn't belong in an MVP document? If the latter, does this mean that we're going to carry forward the issues with pulp-admin, like a lack of status codes?)*
227
228
repo CRUD  
229
CRUD for importers  
230
CRUD for publishers  
231
trigger syncs  
232
trigger publish  
233
list content in a repo  
234
upload  
235 8 Ichimonji10
server status  
236
list and cancel tasks  
237 1 bmbouter
authn via basic auth  
238 24 bmbouter
*(Should the supported set of operations be stated in terms of "The capabilities listed in the 'Authenctication,' 'Repositories,' and 'Filter' sections will be supported by the CLI."?)*
239 1 bmbouter
240 26 bmbouter
## Download API
241
242
As a plugin writer, I can download files via
243
244
  - http://
245
  - https://
246
  - file://
247
248
As a plugin writer, I can configure a downloader with:
249 27 bmbouter
250
  - Basic Auth
251 26 bmbouter
  - SSL Cert Client Auth
252
  - Custom CAs will be configured via a "trust store" either on the system or similar. Pulp will not do anything to read/load/manage CAs directly.
253
254
As a plugin writer, I can provide arbitrary behaviors for customized downloaders
255
256
  - For example token authentication in the docker plugin
257
258
As a plugin writer, I can have connection pooling/reuse
259
260
As a plugin writer, I have proxy settings
261
262
  - proxy url (containing basic auth info)
263
264
As a plugin writer, I can have great logs
265
266
As a user, I have documentation about how to use something for bandwidth limiting
267
268
As a plugin writer, I can configure the validation mechanisms used at download time
269
270
  - checksum validation - minimum (md5, sha1, sha256, sha512)
271
  - size validation
272
273
<span style="color:red;">As a plugin writer, I expect units that are missing from the remote repository to not be created in Pulp when using the immediate download policy.</span>
274
275
<span style="color:red;">As a plugin writer, I expect units that are missing from the remote repository to be created in Pulp when using background or on_demand download policies.</span>
276
277
As a plugin writer I can configure mirror lists and rotate between the mirrors
278
279
  - round robin
280
  - nearest mirror support
281
282
As a plugin writer, the plugin API provides tooling whereby I can provide the content to be added and removed from the repository. This tooling supports both immediate and deferred downloading.
283
284 1 bmbouter
As a plugin writer I can manage the catalog by using ChangeSets
285 26 bmbouter
286 27 bmbouter
As a plugin writer, the plugin can participate in adding content for cases where the decision to add additional content is based content that has been downloaded.
287 26 bmbouter
288
As a plugin writer, I can fetch content myself (but I am not encouraged to do so) with code I write
289 1 bmbouter
290
As a plugin writer, I can CRUD content units
291
292
## Consumer Applicability
293
294
Using consumer profiles and repo bindings I can compute applicability with 2.y parity  
295 11 Ichimonji10
Performance needs to be awesome
296
297 1 bmbouter
*(Is the Pulp Consumer going away in Pulp 3? If so, is this section still appropriate?)*
298
299
## Plugin compatibility
300
301
rpm will work with platform  
302
puppet will work with platform  
303
ostree will work with platform  
304
python will work with platform  
305
file_plugin will work with platform  
306
docker will work with platform
307
308
## Migrations
309 20 bmbouter
310
users can run an executable similar to pulp-manage-db that is not named pulp-manage-db *(Why the change in name?)*
311 36 bmbouter
312
<span style="color:red;">What about migrating fields that we don't use in 3.0 but will use in 3.1+. For example the auto-publish feature?</span>
313 1 bmbouter
314
## Glossary
315
316 39 jortel@redhat.com
Repository - A named collection of content.
317
318
Artifact - A file associated with one content (unit). Artifacts are not shared between content (units). Create a content unit using an uploaded file ID as the source for its metadata. Create Artifacts associated with the content unit using an uploaded file ID for each; commit as a single transaction.
319
320
Content (unit) - A single piece of content manged by Pulp. Each file associated with a content (unit) is called an Artifact. Each content (unit) may have zero or many Artifacts.