Project

Profile

Help

Issue #6214

closed

Module dependency metadata is stored in an overly-simplified format

Added by dalley almost 3 years ago. Updated almost 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 67
Quarter:

Description

The dependency segment of the module metadata is stored in the database as follows:

            "dependencies": {                                                                                                                                                                                                                 
                "ant": [                                                                                                                                                                                                                      
                    "1.10"                                                                                                                                                                                                                    
                ],                                                                                                                                                                                                                            
                "maven": [                                                                                                                                                                                                                    
                    "3.5"                                                                                                                                                                                                                     
                ],                                                                                                                                                                                                                            
                "platform": [                                                                                                                                                                                                                 
                    "f31"                                                                                                                                                                                                                     
                ]                                                                                                                                                                                                                             
            },     

(example from the eclipse module from fedora 31 modular release repo)

I think this is over-simplified slightly. I think module stream is allowed to have different dependencies on different platforms. The "platform" key of the dictionary is used as a tag for the dictionary of dependencies, meaning "the dependencies listed on this dict apply to each of these platforms".

This means that the outer level of the dependencies should be a list, not a dict. This is the way it is stored in Pulp 2.

For this same module, Pulp 2 stores it as:

"dependencies": [
    {
        "maven": [
            "3.5"
        ],
        "platform": [
            "f31"
        ],
        "ant": [
            "1.10"
        ]
    },
]

I think that flattening out the dependencies as we have done is incorrect. It's difficult to say for sure because I can't find any modules that have different dependencies on different platforms, or even different values for platform to begin with.

In any case, part of the Pulp 2 depsolving algorithm (which I am copying to Pulp 3) is built around this format, and it was originally copied from Module Build Service code, which also works this way. But since a data migration to un-flatten this might be painful (depending on whether multi-platform module streams actually exist), and changing the algorithm is probably unwise, we should pause and try to figure out how exactly the metadata is supposed to work (if anybody knows).

Actions #1

Updated by ttereshc almost 3 years ago

I agree with you that the structure is oversimplified and it doesn't cover the case when there are different dependencies for different platforms.

Here are some examples for different platforms:

module duck 0 (0.8)

dependencies:
  - buildrequires:
      walrus: [0.71, 5.21]
      platform: [f28]
    requires:
      walrus: [0.71, 5.21]
      platform: [f28]
  - buildrequires:
      platform: [el8]
    requires:
      platform: [el8]

Notice that module duck 0 (0.7) has also multiple platforms but they have the same set of dependencies: 

dependencies:
  - buildrequires:
      walrus: []
      platform: [-f29, -f30]
    requires:
      walrus: []
      platform: [-f29, -f30]

I hope migration is not terribly painful. We can't unflatten the structure based on the data from DB only if there are multiple platforms specified. But in such cases we can look at its artifact where original yaml snippet is stored and just recreate this field.

Actions #2

Updated by dalley almost 3 years ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 67
Actions #3

Updated by ppicka almost 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ppicka
Actions #4

Updated by ppicka almost 3 years ago

  • Status changed from ASSIGNED to POST

Added by ppicka almost 3 years ago

Revision 50b410ae

Modulemd dependecies fix

Modulemd depenedencies is now stored corectly in database. Django migration for dependecies added.

closes: #6214 https://pulp.plan.io/issues/6214

Actions #5

Updated by ppicka almost 3 years ago

  • Status changed from POST to MODIFIED
Actions #6

Updated by ttereshc almost 3 years ago

  • Project changed from Pulp to RPM Support
  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
  • Sprint/Milestone set to Pulp RPM 3.2.0

Also available in: Atom PDF