Project

Profile

Help

Pulp 3 Minimum Viable Product » History » Sprint/Milestone 40

bmbouter, 05/23/2017 06:07 PM
Posting results of the "versioned repos Part I" MVP call

1 1 bmbouter
# Pulp 3.0.0 Minimum Viable Product (MVP)
2
3 21 bmbouter
<span style="color:red;">Lines highlighted in red need more attention.</span>
4 20 bmbouter
5 37 bmbouter
## Overall Guarantees
6
7 39 jortel@redhat.com
All REST API calls will update the DB using transactions as necessary to ensure data integrity.
8 37 bmbouter
9 1 bmbouter
## Authentication
10
11 17 semyers
As an authenticated user I can manage user(s).
12
13
  - Add a user
14
  - View user(s)
15
  - Update any user detail
16
  - Delete a user
17
18
As an API user, I can have documentation to generate a JSON Web Token (JWT) without the server being online.
19
20
A user authenticated with HTTP/HTTPS "Basic" auth can acquire a non-expiring JWT to access the API.
21
22
  - The JWT shall have a created timestamp which can be used to invalidate
23
  - The JWT shall have a user identifier (its primary key)
24
25
I can authenticate to any API call using HTTP/HTTPS "Basic" auth with a username and password that Pulp stores.
26
27
  - The password is stored as a hash (for security).
28
29
As an API user, I can authenticate any API call with a JWT.
30
31
As an API user, I can invalidate all JWT tokens for a given user issued earlier than now.
32
33
As an authenticated user, when deleting a user 'foo', all of user 'foo's JWTs are invalidated.
34 1 bmbouter
35
## Repositories
36
37 18 dkliban@redhat.com
As an authenticated user, I can list all repos.
38
39
  - All fields are included
40
  - Pagination is supported
41 22 bmbouter
  - <span style="color:red;">Filtering support</span>
42 18 dkliban@redhat.com
43
As an authenticated user, I can CRUD a repository
44
45
  - Create a repo
46
  - Read a repo
47
  - Update all mutable repo fields
48 21 bmbouter
  - Delete a repo (asynchronous)
49 18 dkliban@redhat.com
50
As an authenticated user, I can list a repository's associated importers and publishers
51
52
  - All fields are included
53
  - Pagination is supported
54 22 bmbouter
  - <span style="color:red;">Filtering support</span>
55 18 dkliban@redhat.com
56 20 bmbouter
<span style="color:red;">As an authenticated user, I can summarize content in a repo (including counts)</span>
57 18 dkliban@redhat.com
58
As an authenticated user, I can CRUD an importer
59
60
  - Create an importer
61
  - Read an importer
62
  - Update all mutable importer fields
63 21 bmbouter
  - Delete an importer (asynchronous)
64 18 dkliban@redhat.com
65
As an authenticated user, I can CRUD a publisher
66
67
  - Create a publisher
68
  - Read a publisher
69
  - Update all mutable publisher fields
70 21 bmbouter
  - Delete a publisher (asynchronous)
71 1 bmbouter
72
## Content Manipulation
73
74 19 bmbouter
As an authenticated user, I can trigger an importer to sync.
75
76
  - I can follow the progress of all syncs. (Syncs are asynchronous.)
77
  - I cannot pass "sync" options.
78
  - Auto-publish is not included as an importer property.
79
80
As an authenticated user, I can trigger a publisher to publish.
81
82
  - I can follow the progress of all publishes. (Publishes are asynchronous.)
83
  - I cannot pass "publish" options.
84
85 25 bmbouter
## Upload & Copy
86
87 32 bmbouter
#### Getting bits from the client to Pulp
88
89 35 bmbouter
As an authenticated user, I can request a file ID from the server to upload a file with
90 19 bmbouter
91 35 bmbouter
As an authenticated user, I can upload a file with the server provided file ID, an optional chunk size, and an optional offset.
92 19 bmbouter
93 35 bmbouter
As an authenticated user, I can rely on Pulp to auto-delete uploaded files after a configurable time. (Eg: 6 hours).
94 30 bmbouter
95 35 bmbouter
As a user, I can delete an uploaded file by file ID
96 30 bmbouter
97 1 bmbouter
#### Creating Artifacts and Content Units
98 35 bmbouter
99 29 bmbouter
As a user, I can reference a file ID at Artifact creation time.
100 1 bmbouter
101 37 bmbouter
As an authenticated user, I can create a content unit by providing the content type, its Artifacts using file upload IDs for each Artifact, and the metadata supplied in the POST body. This call is atomic, either all Artifacts and the content unit are created in the database and on the filesystem or none are.
102 31 bmbouter
103 1 bmbouter
As an authenticated user, I can reuse a file ID to create multiple Artifacts without uploading the file twice.
104
105 37 bmbouter
#### Unit Management / Copy
106 29 bmbouter
107 37 bmbouter
As an authenticated user, I can associate and disassociate one or more units with a destination repo.
108 1 bmbouter
109
  - <span style="color:red;">Filtering support for specifying the unit(s)</span>
110
  - I can follow the progress. (association and disassociation are asynchronous).
111
112
## Versioned Repositories
113
114 40 bmbouter
As an authenticated user, I can list the content in a particular repository version
115
116
  - All fields are included
117
  - Pagination is supported
118
  - <span style="color:red;">Filtering support</span>
119
120
As an authenticated user, I can discover a URL to the latest version of a repository  
121
<span class="resource repository the on attributes or endpoint, API dedicated a through \^ Is" style="color:red;"></span>
122
123
As an authenticated user, I can run a publisher without a repository version and have it default to the latest version.
124
125
<span style="color:red;">As an authenticated user, I can delete a repository version by specifying the version</span>
126
127
<span style="color:red;">As an authenticated user, I can upload multiple content(s?) and add create a single new version that adds all of them.</span>
128 1 bmbouter
129 20 bmbouter
## Orphans
130
131 1 bmbouter
<span style="color:red;">As an authenticated user, I can clean up orphans</span>  
132
<span style="color:red;">\* I can follow the progress of all cleanups. (Cleanups are asynchronous.)</span>
133
134
## Filter
135
136
I can filter all nouns *(What is the meaning of "filter?" What is a noun?)*
137
138 22 bmbouter
## Task Management
139
140
As an authenticated user, I can list all tasks
141
142
  - Filtering support on \['state', 'id', 'group'\]
143
  - This does not include associated progress reports
144
145
As an authenticated user, I can see a detail view for a specific task
146
147
  - all attributes of a task
148
  - all associated progress reports
149
150
As an authenticated user, I can cancel a task
151 1 bmbouter
152
  - don't dare to use the DELETE verb!
153
154
## Task Group
155
156
I can view a summary of the status of all tasks in a group
157
158
## Event Listener Notifier
159
160 12 Ichimonji10
I can receive serialized task info via AMQP on each task save
161
162 1 bmbouter
*Can this be restated in more pedantic terms? Does this mean that an arbitrary host can attach itself to Pulp's AMQP message bus and get updates on the progress of tasks?*
163
164
## Status
165
166
I can view the status of all pulp components  
167
I can view an overall health attribute  
168
I can view information about unapplied migrations
169
170 28 bmbouter
## Plugin API
171 1 bmbouter
172 28 bmbouter
As a plugin writer, I have a plugin API that is semantically versioned at 0.x separate from the REST API
173 1 bmbouter
174 28 bmbouter
As a plugin writer, I can report progress with a message and state
175
176
As a plugin writer, I can report progress with an optional suffix
177
178
As a plugin writer, I can report progress with a total count of things to do an the current count of things done
179
180
As a plugin writer, non-fatal exceptions a on the Task and are included in the Task detail. non_fatal exceptions do not cause the Task to be marked as failed, but may be interpreted by the user as not fully successful.
181
182
As a plugin writer, the working directory is set before Task work is done and cleaned up afterwards. I should not need to interact with the file system outside of the working dir.
183
184
As a plugin writer, I can provide a subclassed Importer. The importer's responsibility is to synchronize the content of a Pulp repository with the content of a remote repository.
185
186
As a plugin writer, I can provide a subclassed Publisher. The publisher's responsibility is to publish content.
187
188
As a plugin writer, I can define unit types by subclassing Content models to provide concrete content unit types to be manged by the platform.
189
190
As a plugin writer, I can interact with and create Artifacts
191
192
As a plugin writer, my app will be discovered by Pulp's app via an entry point provided by the plugin writer
193
194
As a plugin writer, I can use the plugin API to query content units/artifacts associated with a repository.
195 6 Ichimonji10
196 1 bmbouter
As a plugin writer, I can assocate/disassociate content units with a repository.
197
198 8 Ichimonji10
## CLI
199
200 1 bmbouter
We will port what is there with as little effort as possible *(Does this mean that porting will be easy for developers, or that switching from the Pulp 2-3 CLI will be easy for users? If the former, isn't this an implementation detail that doesn't belong in an MVP document? If the latter, does this mean that we're going to carry forward the issues with pulp-admin, like a lack of status codes?)*
201
202
repo CRUD  
203
CRUD for importers  
204
CRUD for publishers  
205
trigger syncs  
206
trigger publish  
207
list content in a repo  
208
upload  
209 8 Ichimonji10
server status  
210
list and cancel tasks  
211 1 bmbouter
authn via basic auth  
212 24 bmbouter
*(Should the supported set of operations be stated in terms of "The capabilities listed in the 'Authenctication,' 'Repositories,' and 'Filter' sections will be supported by the CLI."?)*
213 1 bmbouter
214 26 bmbouter
## Download API
215
216
As a plugin writer, I can download files via
217
218
  - http://
219
  - https://
220
  - file://
221
222
As a plugin writer, I can configure a downloader with:
223 27 bmbouter
224
  - Basic Auth
225 26 bmbouter
  - SSL Cert Client Auth
226
  - Custom CAs will be configured via a "trust store" either on the system or similar. Pulp will not do anything to read/load/manage CAs directly.
227
228
As a plugin writer, I can provide arbitrary behaviors for customized downloaders
229
230
  - For example token authentication in the docker plugin
231
232
As a plugin writer, I can have connection pooling/reuse
233
234
As a plugin writer, I have proxy settings
235
236
  - proxy url (containing basic auth info)
237
238
As a plugin writer, I can have great logs
239
240
As a user, I have documentation about how to use something for bandwidth limiting
241
242
As a plugin writer, I can configure the validation mechanisms used at download time
243
244
  - checksum validation - minimum (md5, sha1, sha256, sha512)
245
  - size validation
246
247
<span style="color:red;">As a plugin writer, I expect units that are missing from the remote repository to not be created in Pulp when using the immediate download policy.</span>
248
249
<span style="color:red;">As a plugin writer, I expect units that are missing from the remote repository to be created in Pulp when using background or on_demand download policies.</span>
250
251
As a plugin writer I can configure mirror lists and rotate between the mirrors
252
253
  - round robin
254
  - nearest mirror support
255
256
As a plugin writer, the plugin API provides tooling whereby I can provide the content to be added and removed from the repository. This tooling supports both immediate and deferred downloading.
257
258 1 bmbouter
As a plugin writer I can manage the catalog by using ChangeSets
259 26 bmbouter
260 27 bmbouter
As a plugin writer, the plugin can participate in adding content for cases where the decision to add additional content is based content that has been downloaded.
261 26 bmbouter
262
As a plugin writer, I can fetch content myself (but I am not encouraged to do so) with code I write
263
264
As a plugin writer, I can CRUD content units
265
266
## Alternate Content Source Support
267
268
<span class="Etc? Python? RPM? this? support will importers of types which things, importer-related an is this Assuming else? Something locations? of list prioritized a from content download to attempt may importer single a that mean it Does file://? and HTTPS HTTP, than other protocol a via downloaded be can content that mean this Does means. this what clarify Please" style="color:red;"></span>
269 1 bmbouter
270
  - Streamer parity with 2.y
271
272
## Consumer Applicability
273
274
Using consumer profiles and repo bindings I can compute applicability with 2.y parity  
275 11 Ichimonji10
Performance needs to be awesome
276
277 1 bmbouter
*(Is the Pulp Consumer going away in Pulp 3? If so, is this section still appropriate?)*
278
279
## Plugin compatibility
280
281
rpm will work with platform  
282
puppet will work with platform  
283
ostree will work with platform  
284
python will work with platform  
285
file_plugin will work with platform  
286
docker will work with platform
287
288
## Migrations
289 20 bmbouter
290
users can run an executable similar to pulp-manage-db that is not named pulp-manage-db *(Why the change in name?)*
291 36 bmbouter
292
<span style="color:red;">What about migrating fields that we don't use in 3.0 but will use in 3.1+. For example the auto-publish feature?</span>
293 1 bmbouter
294
## Glossary
295
296 39 jortel@redhat.com
Repository - A named collection of content.
297
298 1 bmbouter
Artifact - A file associated with one content (unit). Artifacts are not shared between content (units). Create a content unit using an uploaded file ID as the source for its metadata. Create Artifacts associated with the content unit using an uploaded file ID for each; commit as a single transaction.
299 36 bmbouter
300 1 bmbouter
Content (unit) - A single piece of content manged by Pulp. Each file associated with a content (unit) is called an Artifact. Each content (unit) may have zero or many Artifacts.
301
302 39 jortel@redhat.com
Associate - Create a relationship between individual content (unit) and a repository. When content is *associated* to a repository, it has been added to a repository in the sense that a repository is a collection of content.
303 1 bmbouter
304 39 jortel@redhat.com
<span style="color:red;">Wonder if we should just refer to this \^ as *adding content to a repository*?</span>
305
306
Disassociate - Delete the relationship between individual content (unit) and a repository. When content is *disassociated* from a repository, it has been removed from the repository in the sense that a repository is a collection of content.
307
308
<span style="color:red;">Wonder if we should just refer to this \^ as *removing content from a repository*?</span>