Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #9570

closed

RHEL 9 beta contains one module stream, but it isn't imported into pulp

Added by jsherril@redhat.com about 3 years ago. Updated almost 3 years ago.

Status:
CLOSED - NOTABUG
Priority:
Normal
Assignee:
ggainey
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

For some reason when syncing the rhel 9 beta appstream repo:

https://cdn.redhat.com/content/beta/rhel9/9/x86_64/appstream/os/

The repository is fully synced (and metadata is mirrored properly), but the database/api has no modules listed:

      "rpm.modulemd_defaults": {
        "count": 1,
        "href": "/pulp/api/v3/content/rpm/modulemd_defaults/?repository_version=/pulp/api/v3/repositories/rpm/rpm/1f4c31f6-cd64-41b2-af3e-381bcde36248/versions/1/"
      },
      "rpm.package": {
        "count": 5189,
        "href": "/pulp/api/v3/content/rpm/packages/?repository_version=/pulp/api/v3/repositories/rpm/rpm/1f4c31f6-cd64-41b2-af3e-381bcde36248/versions/1/"
      },
      "rpm.packagecategory": {
        "count": 5,
        "href": "/pulp/api/v3/content/rpm/packagecategories/?repository_version=/pulp/api/v3/repositories/rpm/rpm/1f4c31f6-cd64-41b2-af3e-381bcde36248/versions/1/"
      },
      "rpm.packageenvironment": {
        "count": 4,
        "href": "/pulp/api/v3/content/rpm/packageenvironments/?repository_version=/pulp/api/v3/repositories/rpm/rpm/1f4c31f6-cd64-41b2-af3e-381bcde36248/versions/1/"
      },
      "rpm.packagegroup": {
        "count": 62,
        "href": "/pulp/api/v3/content/rpm/packagegroups/?repository_version=/pulp/api/v3/repositories/rpm/rpm/1f4c31f6-cd64-41b2-af3e-381bcde36248/versions/1/"
      },
      "rpm.repo_metadata_file": {
        "count": 1,
        "href": "/pulp/api/v3/content/rpm/repo_metadata_files/?repository_version=/pulp/api/v3/repositories/rpm/rpm/1f4c31f6-cd64-41b2-af3e-381bcde36248/versions/1/"
      }

notice that there is a modulemd_defaults, but no modulemd.

Centos 9-stream looks fine:

      "rpm.package": {
        "count": 13301,
        "href": "/pulp/api/v3/content/rpm/packages/?repository_version=/pulp/api/v3/repositories/rpm/rpm/79787071-9217-421d-864b-e1d03ee480c6/versions/1/"
      },
      "rpm.packagecategory": {
        "count": 5,
        "href": "/pulp/api/v3/content/rpm/packagecategories/?repository_version=/pulp/api/v3/repositories/rpm/rpm/79787071-9217-421d-864b-e1d03ee480c6/versions/1/"
      },
      "rpm.distribution_tree": {
        "count": 1,
        "href": "/pulp/api/v3/content/rpm/distribution_trees/?repository_version=/pulp/api/v3/repositories/rpm/rpm/79787071-9217-421d-864b-e1d03ee480c6/versions/1/"
      },
      "rpm.modulemd_defaults": {
        "count": 45,
        "href": "/pulp/api/v3/content/rpm/modulemd_defaults/?repository_version=/pulp/api/v3/repositories/rpm/rpm/79787071-9217-421d-864b-e1d03ee480c6/versions/1/"
      },
      "rpm.packagegroup": {
        "count": 59,
        "href": "/pulp/api/v3/content/rpm/packagegroups/?repository_version=/pulp/api/v3/repositories/rpm/rpm/79787071-9217-421d-864b-e1d03ee480c6/versions/1/"
      },
      "rpm.modulemd": {
        "count": 245,
        "href": "/pulp/api/v3/content/rpm/modulemds/?repository_version=/pulp/api/v3/repositories/rpm/rpm/79787071-9217-421d-864b-e1d03ee480c6/versions/1/"
      },
      "rpm.packageenvironment": {
        "count": 4,
        "href": "/pulp/api/v3/content/rpm/packageenvironments/?repository_version=/pulp/api/v3/repositories/rpm/rpm/79787071-9217-421d-864b-e1d03ee480c6/versions/1/"
      }
Actions
Copy link
 #2

Updated by ggainey about 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ggainey
Actions
Copy link
 #3

Updated by ggainey about 3 years ago

I just tested against core/master and it seemed to work:

rhel9-app-beta-cdn...
"/pulp/api/v3/remotes/rpm/rpm/23a06b17-d685-4ab8-8ad9-98d31382e839/"
"/pulp/api/v3/repositories/rpm/rpm/995ffa57-0971-47b8-a865-a5b03870647b/"
Started background task /pulp/api/v3/tasks/f7a55929-8876-47de-8abe-60ee00f2dee4/
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Done.
(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 sync_test]$ http :/pulp/api/v3/tasks/f7a55929-8876-47de-8abe-60ee00f2dee4/
{
    "child_tasks": [],
    "created_resources": [
        "/pulp/api/v3/repositories/rpm/rpm/995ffa57-0971-47b8-a865-a5b03870647b/versions/1/",
        "/pulp/api/v3/publications/rpm/rpm/35e571df-2e49-4006-b630-6684bfe9107a/"
    ],
    "error": null,
    "finished_at": "2021-11-17T16:59:20.263548Z",
    "logging_cid": "4a36571848ef476ebaac2979b2f473ea",
    "name": "pulp_rpm.app.tasks.synchronizing.synchronize",
    "parent_task": null,
    "progress_reports": [
        {
            "code": "sync.downloading.metadata",
            "done": 7,
            "message": "Downloading Metadata Files",
            "state": "completed",
            "suffix": null,
            "total": null
        },
        {
            "code": "sync.parsing.modulemds",
            "done": 1,
            "message": "Parsed Modulemd",
            "state": "completed",
            "suffix": null,
            "total": 1
        },
        {
            "code": "sync.parsing.modulemd_defaults",
            "done": 1,
            "message": "Parsed Modulemd-defaults",
            "state": "completed",
            "suffix": null,
            "total": 1
        },
        {
            "code": "sync.downloading.artifacts",
            "done": 5190,
            "message": "Downloading Artifacts",
            "state": "completed",
            "suffix": null,
            "total": null
        },
        {
            "code": "associating.content",
            "done": 5263,
            "message": "Associating Content",
            "state": "completed",
            "suffix": null,
            "total": null
        },
        {
            "code": "sync.parsing.packages",
            "done": 5189,
            "message": "Parsed Packages",
            "state": "completed",
            "suffix": null,
            "total": null
        },
        {
            "code": "sync.parsing.comps",
            "done": 71,
            "message": "Parsed Comps",
            "state": "completed",
            "suffix": null,
            "total": 71
        },
        {
            "code": "sync.parsing.advisories",
            "done": 0,
            "message": "Parsed Advisories",
            "state": "completed",
            "suffix": null,
            "total": 0
        },
        {
            "code": "publish.generating_metadata",
            "done": 1,
            "message": "Generating repository metadata",
            "state": "completed",
            "suffix": null,
            "total": 1
        }
    ],
    "pulp_created": "2021-11-17T16:27:35.931196Z",
    "pulp_href": "/pulp/api/v3/tasks/f7a55929-8876-47de-8abe-60ee00f2dee4/",
    "reserved_resources_record": [
        "/pulp/api/v3/repositories/rpm/rpm/995ffa57-0971-47b8-a865-a5b03870647b/",
        "shared:/pulp/api/v3/remotes/rpm/rpm/23a06b17-d685-4ab8-8ad9-98d31382e839/"
    ],
    "started_at": "2021-11-17T16:27:35.989214Z",
    "state": "completed",
    "task_group": null,
    "worker": "/pulp/api/v3/workers/c946b4c5-e176-40dd-859d-a0d51007d9f7/"
}

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 sync_test]$ psql -U pulp -d pulp --host 127.0.0.1
psql (10.17)
Type "help" for help.

pulp=> select name, stream, version, context, static_context from rpm_modulemd;
      name       | stream |     version     | context | static_context 
-----------------+--------+-----------------+---------+----------------
 container-tools | latest | 920210922061925 | el9     | t
(1 row)

pulp=>

I'll reset my env to core/3.14 and friends and retest, will report back here on results.

Actions
Copy link
 #4

Updated by ggainey about 3 years ago

Can't recreate under core/3.14 and friends either. Tried --mirror, still unable to reproduce. I'm using these versions of pulp-et-al:

  "versions": [
    {
      "component": "core",
      "version": "3.14.10.dev"
    },
    {
      "component": "file",
      "version": "1.8.3.dev"
    },
    {
      "component": "rpm",
      "version": "3.14.9.dev"
    },
    {
      "component": "container",
      "version": "2.8.2.dev"
    },
    {
      "component": "deb",
      "version": "2.13.2.dev"
    },
    {
      "component": "certguard",
      "version": "1.4.1.dev"
    },
    {
      "component": "pulp_2to3_migration",
      "version": "0.12.2.dev"
    }
  ],

The repo doesn't look like anything's changed since 30-OCT.

- Are there any pulp-errors in journalctl from around sync-time?

Actions
Copy link
 #5

Updated by ggainey about 3 years ago

Another thought from dralley - what version of libmodulemd is running on the box?

Actions
Copy link
 #6

Updated by jsherril@redhat.com almost 3 years ago

libmodulemd-2.9.4-2.el8.x86_64 is what i was using

Actions
Copy link
 #7

Updated by dalley almost 3 years ago

@Justin that version of libmodulemd does not support the EL9 metadata format, it should be updated to 2.12+. Either via shipping the package ourselves or by limiting support to EL 8.5+ / whichever has the appropriate libmodulemd versions for EL9 suport.

https://bugzilla.redhat.com/show_bug.cgi?id=1894573

Could you update that package and retry the reproducer?

Actions
Copy link
 #8

Updated by jsherril@redhat.com almost 3 years ago

  • Status changed from ASSIGNED to CLOSED - NOTABUG

I can confirm that libmodulemd-2.13.0-1.el8.x86_64 fixed this for me!

Thanks

Actions
Send by e-mail Copy link

Also available in: Atom PDF