Project

Profile

Help

Issue #6191

closed

Module metadata is stored improperly in the database

Added by dalley almost 5 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 66
Quarter:

Description

JSONField accepts Python-native types, and encodes them itself. But we're pre-encoding the JSON, and passing in a string. This results in module metadata being stored improperly in the database.

Correct (RPM package Requires, Provides, Suggests, etc.)

pulp=> SELECT requires from rpm_package;
                                                             requires                                                              
-----------------------------------------------------------------------------------------------------------------------------------
 [["squirrel", null, null, null, null, false], ["walrus", null, null, null, null, false]]
 []
 [["bear", null, null, null, null, false], ["lion", null, null, null, null, false], ["tiger", null, null, null, null, false]]
 []
 []
 []
 []
 [["wolf", null, null, null, null, false]]
 [["cockateel", null, null, null, null, false], ["lion", null, null, null, null, false]]
 []
 [["trout", null, null, null, null, false]]
 []
 []
 []
 [["kangaroo", null, null, null, null, false]]
 []
 []
 []
 []
 [["mouse", null, null, null, null, false], ["penguin", null, null, null, null, false]]
 []
 [["wolf", null, null, null, null, false]]
 [["dolphin", null, null, null, null, false]]
 [["dolphin", null, null, null, null, false], ["zebra", null, null, null, null, false]]
 []
 [["camel", null, null, null, null, false], ["fox", null, null, null, null, false]]
 []
 [["crow", null, null, null, null, false], ["elephant", null, null, null, null, false]]
 []
 [["cat", null, null, null, null, false], ["horse", null, null, null, null, false]]
 []
 [["shark", null, null, null, null, false], ["stork", null, null, null, null, false]]
 []
 [["dolphin", null, null, null, null, false], ["gorilla", null, null, null, null, false], ["pike", null, null, null, null, false]]
 [["whale", null, null, null, null, false]]
(35 rows)

Incorrect (Modulemd artifacts)

pulp=> SELECT artifacts from rpm_modulemd;
                      artifacts                       
------------------------------------------------------
 "[\"duck-0:0.8-1.noarch\"]"
 "[\"duck-0:0.7-1.noarch\"]"
 "[\"duck-0:0.6-1.noarch\"]"
 "[\"duck-0:0.8-1.noarch\"]"
 "[\"duck-0:0.8-1.noarch\"]"
 "[\"duck-0:0.8-1.noarch\", \"frog-0:0.1-1.noarch\"]"
 "[\"kangaroo-0:0.3-1.noarch\"]"
 "[\"kangaroo-0:0.2-1.noarch\"]"
 "[\"walrus-0:5.21-1.noarch\"]"
 "[\"walrus-0:0.71-1.noarch\"]"
(10 rows)

This also shows up through the API:

        {
            "arch": "x86_64",
            "artifact": "/pulp/api/v3/artifacts/e5098a5e-4254-4d3f-abcb-96624ed738b4/",
            "artifacts": "[\"walrus-0:0.71-1.noarch\"]",
            "context": "c0ffee42",
            "dependencies": "{}",
            "name": "walrus",
            "packages": [
                "54c9f42a-0026-4bbe-a0fc-72d226d72040"
            ],
            "pulp_created": "2020-02-18T21:16:43.144976Z",
            "pulp_href": "/pulp/api/v3/content/rpm/modulemds/a4d07838-fe43-4ee9-a847-eb9a7072bf50/",
            "stream": "0.71",
            "version": "20180707144203"
        }

The same is true of "profiles" on modulemd-defaults

        {
            "artifact": "/pulp/api/v3/artifacts/d131bbd8-47ee-4424-81a8-aa7e6c189b5e/",
            "module": "duck",
            "profiles": "[\"default\"]",
            "pulp_created": "2020-02-18T21:16:43.129878Z",
            "pulp_href": "/pulp/api/v3/content/rpm/modulemd_defaults/0a86deca-e551-4268-9c34-99c6ec937dfd/",
            "stream": "0"
        }

Fixing this will require a data migration

Actions #1

Updated by dalley almost 5 years ago

  • Priority changed from Normal to High
  • Triaged changed from No to Yes
  • Sprint set to Sprint 66

I'm marking this as triaged because it's obviously a big issue and it's blocking my RPM plugin copy work.

Actions #2

Updated by dalley almost 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley
Actions #3

Updated by dalley almost 5 years ago

  • Status changed from ASSIGNED to POST

Added by dalley almost 5 years ago

Revision 245fbffc | View on GitHub

Fix an issue causing module metadata to be stored incorrectly

And add a data migration to fix already-created modules and module-defaults.

closes: #6191 https://pulp.plan.io/issues/6191

Added by dalley almost 5 years ago

Revision 4dd45497 | View on GitHub

Fix an issue causing module metadata to be stored incorrectly

And add a data migration to fix already-created modules and module-defaults.

closes: #6191 https://pulp.plan.io/issues/6191

Actions #4

Updated by dalley almost 5 years ago

  • Status changed from POST to MODIFIED
Actions #5

Updated by dalley almost 5 years ago

Actions #6

Updated by ttereshc almost 5 years ago

  • Sprint/Milestone set to Pulp RPM 3.2.0
Actions #7

Updated by ttereshc almost 5 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF