Pulp

Catalog model object:

Unique Index (path, importer_id)

Catalog:

• path A unit storage path.
• unit_id The content unit ID.
• revision The revision is used to group entries into snapshots.
• importer_id An importer ID.
• url The download URL
• checksum The checksum used to verify downloaded file. Is: <algorithm>:<digest>
• data Misc data (dict) used by the importer.

The streamer will use only the latest revision of matching entries.
Importers steps:

1. Create new entries using an incremented revision.
2. Remove all entries with older revisions.

There may be another way of doing this but the goal is to ensure the streamer can use the catalog concurrently with it being updated without locking. And, that the catalog can be self-cleaning.

@startuml
database Catalog
actor Apache
control Twisted

== Population ==

Importer -> Catalog : Find Latest Revision
Importer -> Importer: Increment Revision
Importer -> Catalog : Add New Entries
note right
**path**: /var/lib/pulp/content/file-1.rpm
**importer_id**: 1234
**url**: http://fedora/f21/file-1.rpm
end note

Importer -> Catalog : Delete Older Revisions

== Downloading ==

Apache -> Streamer : Download File
note right: http://var/lib/pulp/content/file-1.rpm

Streamer -> Catalog : Find Matching Entry
note right: Find by path: /var/lib/pulp/content/file-1.rpm

Streamer -> Plugins : Get Importer
Plugins --> Streamer : Importer
Streamer -> Importer : Get Downloader
Importer --> Streamer : Downloader
note right
The downloader is
completely configured.
end note
Streamer -> Twisted : Run Downloader
@enduml