Project

Profile

Help

Issue #8411

Inconsistent data in upstream remote repo causes inconsistent of pulp content

Added by binlinf0 7 months ago. Updated 4 days ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 106
Quarter:

Description

The issues started inconstancy of repo metadata checksum and actual checksum in upstream remote repo. With the patch https://gist.github.com/daviddavis/2e6ab1872d97230d144a6cd1f9d05e31, the sync to upstream failed with following errors: Received checksum b8b257c32135daf51e703d439594f1a676871d7d for http://something/something/flume-1.9.0-1.noarch.rpm but expected c281a94a354178c42800d47b63479c2621772351

Once the upstream repo was fixed, the subsequence sync keep failing with: "error": { "description": "Package matching query does not exist.", "traceback": " File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/worker.py", line 886, in perform_job\n rv = job.perform()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/job.py", line 664, in perform\n self._result = self._execute()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/job.py", line 670, in _execute\n return self.func(*self.args, **self.kwargs)\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 266, in synchronize\n dv.create()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create\n loop.run_until_complete(pipeline)\n File "/opt/python/3.7.3/lib64/python3.7/asyncio/base_events.py", line 584, in run_until_complete\n return future.result()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline\n await asyncio.gather(*futures)\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py", line 43, in call\n await self.run()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/content_stages.py", line 105, in run\n d_content.content.q()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method\n return getattr(self.get_queryset(), name)(*args, **kwargs)\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/query.py", line 408, in get\n self.model._meta.object_name\n"

We deleted the repo which failed sync and run "delete localhost/pulp/api/v3/orphans/" but keep getting the same errors.

Query the database doesn't provide any result of the package which cause the issue originally

=> select name from rpm_package where name like 'flume%' limit 100; name

(0 rows)

=> select checksum from rpm_checksum where checksum like '%594f1a676871d7d' OR checksum like '%63479c2621772351'; checksum

(0 rows)

=> select name from rpm_package where name like 'flume%' limit 100; name

(0 rows)

=> select checksum from rpm_checksum where checksum like '%594f1a676871d7d' OR checksum like '%63479c2621772351'; checksum

(0 rows)

History

#1 Updated by dalley 6 months ago

I think step 1 is to figure out why a DigestValidationError that was (or should have been) raised didn't prevent the invalid artifact file from working its way through the pipeline and being saved.

Adding David's patch to make sure it gets logged is probably also a good idea.

#2 Updated by fao89 6 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 93

#3 Updated by rchan 6 months ago

  • Sprint changed from Sprint 93 to Sprint 94

#4 Updated by rchan 5 months ago

  • Sprint changed from Sprint 94 to Sprint 95

#5 Updated by rchan 5 months ago

  • Sprint changed from Sprint 95 to Sprint 96

#6 Updated by rchan 5 months ago

  • Sprint changed from Sprint 96 to Sprint 97

#7 Updated by rchan 4 months ago

  • Sprint changed from Sprint 97 to Sprint 98

#8 Updated by rchan 3 months ago

  • Sprint changed from Sprint 98 to Sprint 99

#9 Updated by rchan 3 months ago

  • Sprint changed from Sprint 99 to Sprint 100

#10 Updated by rchan 2 months ago

  • Sprint changed from Sprint 100 to Sprint 101

#11 Updated by ipanova@redhat.com about 2 months ago

  • Sprint changed from Sprint 101 to Sprint 102

#12 Updated by rchan about 2 months ago

  • Sprint changed from Sprint 102 to Sprint 103

#13 Updated by rchan about 1 month ago

  • Sprint changed from Sprint 103 to Sprint 104

#14 Updated by rchan 18 days ago

  • Sprint changed from Sprint 104 to Sprint 105

#15 Updated by rchan 4 days ago

  • Sprint changed from Sprint 105 to Sprint 106

Please register to edit this issue

Also available in: Atom PDF