Project

Profile

Help

Issue #8411

Inconsistent data in upstream remote repo causes inconsistent of pulp content

Added by binlinf0 about 2 months ago. Updated 13 days ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 96
Quarter:

Description

The issues started inconstancy of repo metadata checksum and actual checksum in upstream remote repo. With the patch https://gist.github.com/daviddavis/2e6ab1872d97230d144a6cd1f9d05e31, the sync to upstream failed with following errors: Received checksum b8b257c32135daf51e703d439594f1a676871d7d for http://something/something/flume-1.9.0-1.noarch.rpm but expected c281a94a354178c42800d47b63479c2621772351

Once the upstream repo was fixed, the subsequence sync keep failing with: "error": { "description": "Package matching query does not exist.", "traceback": " File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/worker.py", line 886, in perform_job\n rv = job.perform()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/job.py", line 664, in perform\n self._result = self._execute()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/rq/job.py", line 670, in _execute\n return self.func(*self.args, **self.kwargs)\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 266, in synchronize\n dv.create()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create\n loop.run_until_complete(pipeline)\n File "/opt/python/3.7.3/lib64/python3.7/asyncio/base_events.py", line 584, in run_until_complete\n return future.result()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline\n await asyncio.gather(*futures)\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py", line 43, in call\n await self.run()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/pulpcore/plugin/stages/content_stages.py", line 105, in run\n d_content.content.q()\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method\n return getattr(self.get_queryset(), name)(*args, **kwargs)\n File "/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/models/query.py", line 408, in get\n self.model._meta.object_name\n"

We deleted the repo which failed sync and run "delete localhost/pulp/api/v3/orphans/" but keep getting the same errors.

Query the database doesn't provide any result of the package which cause the issue originally

=> select name from rpm_package where name like 'flume%' limit 100; name

(0 rows)

=> select checksum from rpm_checksum where checksum like '%594f1a676871d7d' OR checksum like '%63479c2621772351'; checksum

(0 rows)

=> select name from rpm_package where name like 'flume%' limit 100; name

(0 rows)

=> select checksum from rpm_checksum where checksum like '%594f1a676871d7d' OR checksum like '%63479c2621772351'; checksum

(0 rows)

History

#1 Updated by dalley about 2 months ago

I think step 1 is to figure out why a DigestValidationError that was (or should have been) raised didn't prevent the invalid artifact file from working its way through the pipeline and being saved.

Adding David's patch to make sure it gets logged is probably also a good idea.

#2 Updated by fao89 about 2 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 93

#3 Updated by rchan about 1 month ago

  • Sprint changed from Sprint 93 to Sprint 94

#4 Updated by rchan 27 days ago

  • Sprint changed from Sprint 94 to Sprint 95

#5 Updated by rchan 13 days ago

  • Sprint changed from Sprint 95 to Sprint 96

Please register to edit this issue

Also available in: Atom PDF