Project

Profile

Help

Issue #9111

can't sync rhel 8.4 baseos kickstart repo unless md5 (and sha1) is enabled

Added by jsherril@redhat.com 2 months ago. Updated about 2 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 101
Quarter:

Description

Syncing rhel 8.4 base os kickstart results in an error:

    traceback: |2
        File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py", line 266, in _perform_task
          result = func(*args, **kwargs)
        File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 422, in synchronize
          version = dv.create()
        File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 151, in create
          loop.run_until_complete(pipeline)
        File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
          return future.result()
        File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
          await asyncio.gather(*futures)
        File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
          await self.run()
        File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 693, in run
          result = await downloader.run()
        File "/usr/lib/python3.6/site-packages/pulpcore/download/http.py", line 258, in run
          return await download_wrapper()
        File "/usr/lib/python3.6/site-packages/backoff/_async.py", line 133, in retry
          ret = await target(*args, **kwargs)
        File "/usr/lib/python3.6/site-packages/pulpcore/download/http.py", line 256, in download_wrapper
          return await self._run(extra_data=extra_data)
        File "/usr/lib/python3.6/site-packages/pulp_rpm/app/downloaders.py", line 93, in _run
          to_return = await self._handle_response(response)
        File "/usr/lib/python3.6/site-packages/pulpcore/download/http.py", line 212, in _handle_response
          await self.finalize()
        File "/usr/lib/python3.6/site-packages/pulpcore/download/base.py", line 162, in finalize
          self.validate_digests()
        File "/usr/lib/python3.6/site-packages/pulpcore/download/base.py", line 211, in validate_digests
          if expected_digest != self._digests[algorithm].hexdigest():
    description: "'md5'"

digging around the metadata, i didn't see any md5 checksums anywhere except for the 'extra_files.json':

{
    "data": [
        {
            "checksums": {
                "md5": "feb4d252ee63634debea654b446e830b",
                "sha1": "a73fad5aeb5642d1b2108885010c4e7a547a1204",
                "sha256": "c4117d0e325cde392981626edbd1484c751f0216689a171e4b7547e8800acc21"
            },
            "file": "RPM-GPG-KEY-redhat-release",
            "size": 5134
        },
        {
            "checksums": {
                "md5": "3c24137e12ece142a27bbf825c256936",
                "sha1": "a72daf8585b41529269cdffcca3a0b3d4e2f21cd",
                "sha256": "3f8644b35db4197e7689d0a034bdef2039d92e330e6b22217abfa6b86a1fc0fa"
            },
            "file": "RPM-GPG-KEY-redhat-beta",
            "size": 1669
        },
        {
            "checksums": {
                "md5": "b234ee4d69f5fce4486a80fdaf4a4263",
                "sha1": "4cc77b90af91e615a64ae04893fdffa7939db84c",
                "sha256": "8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643"
            },
            "file": "GPL",
            "size": 18092
        },
        {
            "checksums": {
                "md5": "597eec87ff5541f220b19036afdac59d",
                "sha1": "bd6006ea5e4e03be575a2d0252b8654130dc43ed",
                "sha256": "1d652b91107d3f2e1c57804eb0077dd45a24ab407dd6ebc0be0412380d8191ca"
            },
            "file": "EULA",
            "size": 8266
        }
    ],
    "header": {
        "version": "1.0"
    }
}

In my case i had sha1 and sha256 enabled, but not md5. I don't think pulp should validate ALL the checksums if those particular checksums aren't enabled within pulp.

And its possible that i'm barking up the wrong tree with this extra_files.json, its just the only file i could find in all the metadata (including treeinfo) that included an md5 checksum

Associated revisions

Revision 1e0e4539 View on GitHub
Added by dalley 2 months ago

Filter out disallowed checksums for files in extra_files.json

[nocoverage]

closes: #9111 https://pulp.plan.io/issues/9111

History

#1 Updated by jsherril@redhat.com 2 months ago

This was found on pulp-rpm 3.13.3

#2 Updated by jsherril@redhat.com 2 months ago

  • Description updated (diff)

#3 Updated by jsherril@redhat.com 2 months ago

I can also reproduce the problem with this repo: https://fixtures.pulpproject.org/rpm-distribution-tree/

probably easier than rhel8.4

#4 Updated by dalley 2 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley
  • Sprint/Milestone set to 3.14.0
  • Sprint set to Sprint 101

#5 Updated by dalley 2 months ago

  • Status changed from ASSIGNED to POST

#7 Updated by dalley 2 months ago

  • Status changed from POST to MODIFIED

#8 Updated by pulpbot about 2 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF