Project

Profile

Help

Task #3708

Add model for Modulemd-defaults

Added by ipanova@redhat.com 11 months ago. Updated 5 days ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
% Done:

100%

Platform Release:
2.17.0
Blocks Release:
Backwards Incompatible:
No
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 39

Description

Add a Modulemd-defaults model

Module_name - string
Module_stream - string
Module_profiles - dict
repo_id - string

Example:
Modulemd-defaults
unit_key - repo_id, module_name
Module_name : bar
Def_stream: x
Def_profile : { x : [], y: [], z: [] }
repo_id: foo

this spec should be taken as the reference for modelling https://github.com/fedora-modularity/libmodulemd/blob/master/mod-defaults/spec.v1.yaml


Related issues

Blocked by RPM Support - Task #3661: Add model for modular content CLOSED - CURRENTRELEASE Actions
Blocks RPM Support - Story #3206: Support for module metadata in pulp_rpm CLOSED - CURRENTRELEASE Actions
Blocks RPM Support - Story #3659: Add a migration for transition from basic to advanced modular support CLOSED - CURRENTRELEASE Actions
Blocks RPM Support - Story #3766: As a user I can manage modulemd-defaults content CLOSED - CURRENTRELEASE Actions

Associated revisions

History

#1 Updated by ipanova@redhat.com 11 months ago

this is still in discussion on what and how manage the modulemd-defaults.

#2 Updated by ipanova@redhat.com 11 months ago

  • Blocked by Task #3661: Add model for modular content added

#3 Updated by ipanova@redhat.com 11 months ago

  • Blocks Story #3206: Support for module metadata in pulp_rpm added

#4 Updated by ipanova@redhat.com 11 months ago

  • Description updated (diff)

#5 Updated by ipanova@redhat.com 11 months ago

  • Sprint Candidate changed from No to Yes

#6 Updated by dkliban@redhat.com 11 months ago

  • Description updated (diff)
  • Groomed changed from No to Yes

#7 Updated by rchan 11 months ago

  • Sprint set to Sprint 38

#8 Updated by ipanova@redhat.com 11 months ago

  • Groomed changed from Yes to No
  • Sprint deleted (Sprint 38)

ungrooming and removing from the sprint due to the lack of the specific usecases.
we are waiting on more input to properly model and think trough the implementation logic

#9 Updated by rmcgover 10 months ago

Hi, here's some details and use-cases about module defaults for my team. Not all of these are about the model as such, but this seems to be the most relevant issue to add the info.

Use case: As a consumer of Pulp-published yum repos, I want to read modulemd-defaults documents from the YAML stream in the repodata file of type "modules", so that dnf can resolve default streams/profiles when installing a module.

Must-have. The modules file published by Pulp in yum repodata, which is a stream of YAML documents, should contain modulemd-defaults documents of the form seen in https://github.com/fedora-modularity/libmodulemd/blob/master/mod-defaults/spec.v1.yaml ( as well as the modulemd documents already covered by other issues).

In the modules stream, for each module name, there should be either 0 or 1 matching modulemd-defaults documents.

Use case: As a publisher of content via Pulp, I want to use Pulp's APIs to upload modulemd-defaults so that they can be included in yum publishes for consumption by dnf.

Must have.

The fields we'd want to specify during upload are: module, stream, profiles. (Currently no requirement for intents).

Use case: As a publisher of content via Pulp, I want to be able to change the stream/profiles of an existing module default as a single operation, so that I can change data consistently.

Must have. I called out this use-case because a basic implementation may end up as "If you want to change a module default, then remove the existing one and add a new one". I don't think that would be acceptable because it means a module default temporarily disappears and may remain missing for a while if the process was interrupted part-way through, and I think clients would be significantly disrupted if a yum publish happened during that time.

There'd be a few ways to accomplish this, one way which seems workable is simply to use the usual import/upload API and have the upload overwrite any unit with the same unit key. That seems to be the way other models work (e.g. erratum), so maybe that's the way it'd work by default anyway.

Use case: As a developer of a UI displaying metadata stored in Pulp, I want to be able to query modulemd-default information so that I can show users summary info, such as which stream they'll get by default for a module.

Must have. Essentially, imagine a web page (backed by API queries to Pulp) which wants to show information to users along the lines of "For nodejs module, streams 6 and 9 are available, and if you only specify 'nodejs' you'll get stream 9". Pulp's API should expose enough data so that this could be implemented. (It looks like the proposed model would already achieve this)

Notes on constraints:

The one constraint I'm pretty confident should be in place is: for any given module name in a repo, there should be only 0 or 1 matching modulemd-defaults units. (I think it's implied already by the proposed unit key)

There are some other constraints which might come to mind such as: module/stream/profiles in a default must match with a module in the repo. However it seems to me we probably should NOT have those constraints. The reason is that it seems to complicate upload.

It's not acceptable if we'd be forced to upload a modulemd unit as one operation and then upload a modulemd-default as another, as that could be interrupted part way through and leave a module without a default, which is bad. A default without a module on the other hand is fine (confirmed in dnf), so I think we'd usually want to put modulemd-default units in place before the modules themselves - meaning that we can't enforce that a default refers to an existing module.

General usage pattern

It's not concrete yet what our workflow will be for introducing new modules & defaults, but it seems likely to be one of these:

1. Before we introduce new modules into a repo using our usual publishing tools, we run a one-time task separately to put modulemd-defaults into place for the modules we're about to introduce. That would probably happen in bursts around new minor version releases of our products (e.g. a dozen times per year). With this approach we may put modulemd-defaults in place some hours or days before we put the modules in place.

2. Or: our tools which upload modules into Pulp know what the module defaults are supposed to be. Before uploading a module, the tool first queries the Pulp repo to determine if the default is already in place (and correct). If it's not, it puts the default in place before uploading the module. With this approach the modulemd-defaults imports would be integrated into our usual content publish workflow and would happen immediately before introducing a new module.

Other notes

For me the module currently written in description looks sufficient except for:

Module_stream - integer

That might be a mistake, it seems to me that "stream" can be an arbitrary string.

#10 Updated by ipanova@redhat.com 10 months ago

  • Description updated (diff)

#11 Updated by ipanova@redhat.com 10 months ago

  • Subject changed from Define model for Modulemd-defaults to Add model for Modulemd-defaults

#12 Updated by ipanova@redhat.com 10 months ago

  • Description updated (diff)

#13 Updated by ipanova@redhat.com 10 months ago

  • Description updated (diff)

#14 Updated by ttereshc 10 months ago

  • Groomed changed from No to Yes

#15 Updated by ipanova@redhat.com 10 months ago

  • Sprint set to Sprint 38

#16 Updated by ttereshc 10 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc

#17 Updated by ttereshc 10 months ago

  • Blocks Story #3659: Add a migration for transition from basic to advanced modular support added

#18 Updated by rchan 10 months ago

  • Sprint changed from Sprint 38 to Sprint 39

#19 Updated by ttereshc 10 months ago

  • Sprint changed from Sprint 39 to Sprint 38

#20 Updated by ttereshc 10 months ago

  • Sprint changed from Sprint 38 to Sprint 39

#21 Updated by ttereshc 10 months ago

  • Status changed from ASSIGNED to POST

#22 Updated by rchan 10 months ago

  • Sprint/Milestone set to 2.17.0

Adding to 2.17.0 milestone. This is one of the required deliverables.

#23 Updated by dkliban@redhat.com 9 months ago

  • Blocks Story #3766: As a user I can manage modulemd-defaults content added

#24 Updated by ttereshc 9 months ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#25 Updated by ipanova@redhat.com 9 months ago

  • Platform Release set to 2.17.0

#26 Updated by ipanova@redhat.com 9 months ago

  • Platform Release changed from 2.17.0 to master

#27 Updated by ipanova@redhat.com 9 months ago

  • Platform Release deleted (master)

#28 Updated by ipanova@redhat.com 9 months ago

  • Platform Release set to 2.17.0

#29 Updated by ipanova@redhat.com 8 months ago

  • Status changed from MODIFIED to ON_QA

#30 Updated by ipanova@redhat.com 8 months ago

  • Status changed from ON_QA to CLOSED - CURRENTRELEASE

#31 Updated by bmbouter 5 days ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF