Project

Profile

Help

Actions

Pulp 3 Minimum Viable Product » History » Revision 92

« Previous | Revision 92/167 (diff) | Next »
bizhang, 10/10/2017 11:06 PM


Pulp 3.0.0 Minimum Viable Product (MVP)

Lines highlighted in red need more attention.

Overall Guarantees

  • This is not a direct replacement for Pulp 2.
  • This is the minimum required for a 3.0.0 beta and GA.
  • All REST API calls will update the DB using transactions as necessary to ensure data integrity.

Legend

[done] means merged and documented
[in-progress] means started but not fully done
If there is no label the effort has not yet been started

Authentication

As an authenticated user I can manage user(s). [done]

  • Add a user
  • View user(s)
  • Update any user detail
  • Delete a user

As an API user, I can have documentation to generate a JSON Web Token (JWT) without the server being online. [done]

As an administrator, I can disable JWT token expiration. This configuration is in the settings file and is system-wide. [done]

As an administrator, I can configure the JWT tokens to expire after a configurable amount of time. This configuration is in the settings file and is system-wide. [done]

The JWT shall have a username identifier [done]

As an API user, I can authenticate any API call (except to request a JWT) with a JWT. (not certain if this should be the behavior) [in progress]

As an API user, I can invalidate all existing JWT tokens for a given user. [done]

As an authenticated user, when deleting a user 'foo', all of user 'foo's existing JWTs are invalidated. [done]

As an autheticated user, I can invalidate a user's JWTs in the same operation as updating the password. [done]

As an un-authenticated user, I can obtain a JWT token by using a username and password. [done]

Repositories

As an authenticated user, I can list all repos.

  • All fields are included [done]
  • Pagination is supported [done]
  • Filtering support

As an authenticated user, I can CRUD a repository

  • Create a repo [done]
  • Read a repo [done]
  • Update all mutable repo fields [done]
  • Delete a repo (asynchronous) [done]

As an authenticated user, I can list a repository's associated importers and publishers

  • All fields are included [done]
  • Pagination is supported [done]

As an authenticated user, I can see the number of content unit types with counts for each

Importers

note: Importer attributes will commonly be available on importers, but aren't guaranteed to be used by all importers.

As an authenticated user, I can CRUD an importer

  • Create an importer
  • Read an importer
  • Update all mutable importer fields
  • Delete an importer (asynchronous)

As an authenticated user I can configure the following attributes on an Importer: [done]

  • validate (bool) [optional: defaults to True]: If true, the plugin will validate imported content.
  • ssl_ca_certificate (str) [optional] String containing a PEM encoded CA certificate used to validate the server certificate presented by the external source.
  • ssl_client_certificate (str) [optional] Contains a PEM encoded client certificate used for authentication.
  • ssl_client_key (str) [optional] Contains a PEM encoded private key used for authentication.
  • ssl_validation (bool) [optional: defaults to True]: If true, SSL peer validation must be performed.
  • proxy_url (str) [optional] Contains the proxy URL. Format: scheme://user:password@host:port.
  • username (str) [optional] Contains the username to be used for authentication when syncing.
  • password (str) [optional] Contains the password to be used for authentication when syncing.
  • download_policy (str) [optional] Contains the downloading policy name. This is a choice of three options:
    - immediate (default) - Downloading occurs during sync. The sync task does not complete until downloading is done.
    - background - Downloading is started by the sync but occurs in the background. The sync task completes before downloading is complete.
    - on-demand - The sync task records everything that would be downloaded but does not download content. Downloading occurs on demand as driven by client requests for content.
  • feed_url (str) [optional] Contains the URL of an external content source. This is optional.
  • sync mode: (str) [optional]. It has two choices 'additive' and 'mirror':
    - additive (default) - all remote content is added to the local repository on sync. During sync no content is removed from the local repository.
    - mirror - the local content will mirror the remote content exactly, removing local content if not also present in the remote content.
  • name (str) [required] Contains the name.
  • last_updated (datetime) [read-only] Contains the datetime of the last importer update.
  • last_synced (datetime) [read-only] Contains the datetime of the last importer sync.

Publishers

note: Publisher attributes will commonly be available on publishers, but aren't guaranteed to be used by all publishers.

As an authenticated user, I can CRUD a publisher

  • Create a publisher
  • Read a publisher
  • Update all mutable publisher fields
  • Delete a publisher (asynchronous)

As an authenticated user I can configure the following attributes on a Publisher:

  • relative_path (str) [optional] The (relative) path component of the published url.
  • name - (str) [required] contains the name.
  • last_published (datetime) [read-only] When the last successful publish occurred.
  • last_updated (datetime) [read-only] The datetime of the last publisher update.
  • auto_publish(bool) - ??? consider adding auto-publish feature to MVP

Sync and Publish

As an authenticated user, I can trigger an importer to sync. [done]

  • I can follow the progress of all syncs. (Syncs are asynchronous.)
  • I cannot pass "sync" options.
  • Auto-publish is not included as an importer property.

As an authenticated user, I can trigger a publisher to publish. [done]

  • I can follow the progress of all publishes. (Publishes are asynchronous.)
  • I cannot pass "publish" options.

Content Manipulation

Uploading Artifacts

As an authenticated user, I can create an Artifact by uploading a file. [done]

As an authenticated user, I can specify a size and/or digest to validate the uploaded file. [done]

Creating Content Units

As an authenticated user, I can create a content unit by providing the content type (in the URL), references to Artifacts, and the metadata supplied in the POST body. [done]

Content Management / Copy

As an authenticated user, I can add and remove one or more units to and from a destination repo.

  • Filtering support for specifying the unit(s)
  • I can follow the progress. (adding and removing are asynchronous).

Content Removal

Versioned Repositories

As an authenticated user, I can list the content in a particular repository version

  • All fields are included
  • Pagination is supported
  • Filtering support

As an authenticated user, I can discover a URL to the latest version of a repository

As an authenticated user, I can run a publisher without a repository version and have it default to the latest version.

As an authenticated user, I can delete a repository version by specifying the version

As an authenticated user, I can upload multiple content(s?) and add create a single new version that adds all of them.

Orphans

As an authenticated user, I can clean up orphaned content units
* I can follow the progress of all cleanups. (Cleanups are asynchronous.)

As an authenticated user, I can delete a specific content unit
* If the content unit is still in at least one repository the delete fails with a listing of all repositories the unit is part of.
* Artifacts and associated files from the deleted unit are cleaned up

As an authenticated user, I can delete multiple content units with filtering
* If a content unit is still in at least one repository the delete fails with a listing of all repositories the unit is part of.
* Artifacts and associated files from deleted units are cleaned up

As an authenticated user, I see all (orphans) units that are not in any repositories

Task Management

As an authenticated user, I can list all tasks

  • Filtering support on ['state', 'id', 'group']
  • This does not include associated progress reports

As an authenticated user, I can see a detail view for a specific task [done]

  • all attributes of a task
  • all associated progress reports

As an authenticated user, I can cancel a task [done]

  • don't dare to use the DELETE verb!

As an authenticated user, I can delete tasks.

Task Group

I can view a summary of the status of all tasks in a group

Status

As an unauthenticated user I can view the status of Pulp workers, resource managers, and celerybeats. [done]

As an unauthenticated user I can view the status of the web server's connection to the database and message broker. [done]

As an unauthenticated user I can view the versions of core and each installed plugin.

Plugin API

As a plugin writer, I have a plugin API that is semantically versioned at 0.x separate from the REST API [done]

As a plugin writer, I can report progress with a message and state [done]

As a plugin writer, I can report progress with an optional suffix [done]

As a plugin writer, I can report progress with a total count of things to do an the current count of things done [done]

As a plugin writer, non-fatal exceptions on the Task and are included in the Task detail. non_fatal exceptions do not cause the Task to be marked as failed, but may be interpreted by the user as not fully successful. [done]

As a plugin writer, the working directory is set before Task work is done and cleaned up afterwards. I should not need to interact with the file system outside of the working dir. [done]

As a plugin writer, I can provide a subclassed Importer. The importer's responsibility is to synchronize the content of a Pulp repository with the content of a remote repository. (a circular import problem needs to be discussed and may cause this to change) [done]

As a plugin writer, I can provide a subclassed Publisher. The publisher's responsibility is to publish content. (a circular import problem needs to be discussed and may cause this to change) [done]

As a plugin writer, I can define unit types by subclassing Content models to provide concrete content unit types to be manged by the platform. [done]

As a plugin writer, I can interact with and create Artifacts [done]

As a plugin writer, my app will be discovered by Pulp's app via an entry point provided by the plugin writer [done]

As a plugin writer, I can use the plugin API to query content units/artifacts associated with a repository. [done]

As a plugin writer, I can add and remove content units to and from a repository. [done]

CLI

{color:orange} We will use coreapi-cli to generate a one to one mapping of cli commands to rest api schema %
{color:orange} We will have a wrapper for coreapi-cli. This wrapper will handle parallel progress reporting %

Download API

As a plugin writer, I can download files via

  • http://
  • https://
  • file://

As a plugin writer, I can configure a downloader with:

  • Basic Auth
  • SSL Cert Client Auth
  • Custom CAs will be configured via a "trust store" either on the system or similar. Pulp will not do anything to read/load/manage CAs directly.

As a plugin writer, I can provide arbitrary behaviors for customized downloaders

  • For example token authentication in the docker plugin

As a plugin writer, I can have connection pooling/reuse

As a plugin writer, I have proxy settings

  • proxy url (containing basic auth info)

As a plugin writer, I can have great logs

As a user, I have documentation about how to use something for bandwidth limiting

As a plugin writer, I can configure the validation mechanisms used at download time

  • checksum validation - minimum (md5, sha1, sha256, sha512)
  • size validation

As a plugin writer, I expect units that are missing from the remote repository to not be created in Pulp when using the immediate download policy.

As a plugin writer, I expect units that are missing from the remote repository to be created in Pulp when using background or on_demand download policies.

As a plugin writer I can configure mirror lists and rotate between the mirrors

  • round robin
  • nearest mirror support

As a plugin writer, the plugin API provides tooling whereby I can provide the content to be added and removed from the repository. This tooling supports both immediate and deferred downloading.

As a plugin writer I can manage the catalog by using ChangeSets

As a plugin writer, the plugin can participate in adding content for cases where the decision to add additional content is based content that has been downloaded.

As a plugin writer, I can fetch content myself (but I am not encouraged to do so) with code I write

As a plugin writer, I can CRUD content units

{color:red} Consumer Applicability

Using consumer profiles and repo bindings I can compute applicability with 2.y parity
Performance needs to be awesome

_(Is the Pulp Consumer going away in Pulp 3? If so, is this section still appropriate?)_

Migrations only involving Pulp 3

Users can run "pulp-manager migrate" to migrate the database and adjust state in other locations (filesystem, message broker, ...). [done]

Web Server Integration

The content app when deployed with Apache web server or nginx uses their respective features to serve files efficiently.

Glossary

Repository - A named collection of content.

Artifact - A file associated with one content (unit). Artifacts are not shared between content (units). Create a content unit using an uploaded file ID as the source for its metadata. Create Artifacts associated with the content unit using an uploaded file ID for each; commit as a single transaction.

Content (unit) - A single piece of content manged by Pulp. Each file associated with a content (unit) is called an Artifact. Each content (unit) may have zero or many Artifacts.

Updated by bizhang over 6 years ago · 92 revisions