Project

Profile

Help

Issue #5847

Pulp error when syncing rhel 8 baseos

Added by iballou almost 2 years ago. Updated about 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 77
Quarter:

Description

Ping me for the full repo. Found in pulp-rpm 3.0.0rc1 (2019-11-19)

Error: No declared artifact with relative path \"repodata/ff44f60d-80ee-47e8-9932-6522073764f5\" for content \"<RepoMetadataFile: pk=e2740ada-e07f-4a80-a076-160f3f541850>\""}

Traceback:

{"pulp_tasks"=>
  [{"pulp_href"=>"/pulp/api/v3/tasks/c58b7d28-453b-4e8a-8060-ea6bcf8395bb/",
    "pulp_created"=>"2019-12-06T16:58:26.987+00:00",
    "state"=>"failed",
    "name"=>"pulp_rpm.app.tasks.synchronizing.synchronize",
    "started_at"=>"2019-12-06T16:58:27.234+00:00",
    "finished_at"=>"2019-12-06T16:58:29.609+00:00",
    "error"=>
     {"traceback"=>
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 822, in perform_job\n" +
       "    rv = job.perform()\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 605, in perform\n" +
       "    self._result = self._execute()\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 611, in _execute\n" +
       "    return self.func(*self.args, **self.kwargs)\n" +
       "  File \"/usr/local/lib/pulp/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py\", line 147, in synchronize\n" +
       "    dv.create()\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/declarative_version.py\", line 149, in create\n" +
       "    loop.run_until_complete(pipeline)\n" +
       "  File \"/usr/lib64/python3.6/asyncio/base_events.py\", line 484, in run_until_complete\n" +
       "    return future.result()\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/api.py\", line 209, in create_pipeline\n" +
       "    await asyncio.gather(*futures)\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/api.py\", line 43, in __call__\n" +
       "    await self.run()\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/artifact_stages.py\", line 219, in run\n" +
       "    RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/artifact_stages.py\", line 263, in _needed_remote_artifacts\n" +
       "    c=d_content.content))\n",
      "description"=>
       "No declared artifact with relative path \"repodata/ff44f60d-80ee-47e8-9932-6522073764f5\" for content \"<RepoMetadataFile: pk=e2740ada-e07f-4a80-a076-160f3f541850>\""},
    "worker"=>"/pulp/api/v3/workers/8e8407f4-4840-4f21-b1ef-8084c8a08a5e/",
    "spawned_tasks"=>[],
    "progress_reports"=>
     [{"message"=>"Associating Content",
       "code"=>"associating.content",
       "state"=>"canceled",
       "done"=>0},
      {"message"=>"Downloading Artifacts",
       "code"=>"downloading.artifacts",
       "state"=>"canceled",
       "done"=>0},
      {"message"=>"Downloading Metadata Files",
       "code"=>"downloading.metadata",
       "state"=>"canceled",
       "done"=>2},
      {"message"=>"Parsed Comps",
       "code"=>"parsing.comps",
       "state"=>"running",
       "total"=>42,
       "done"=>7},
      {"message"=>"Parsed Advisories",
       "code"=>"parsing.advisories",
       "state"=>"running",
       "total"=>87,
       "done"=>87},
      {"message"=>"Parsed Packages",
       "code"=>"parsing.packages",
       "state"=>"waiting",
       "done"=>0}],
    "created_resources"=>[],
    "reserved_resources_record"=>
     ["/pulp/api/v3/repositories/rpm/rpm/674ba43b-2e86-44f7-81aa-78326bccae72/",
      "/pulp/api/v3/remotes/rpm/rpm/e6b4fd16-b0f0-4f5d-8db8-7b112bbb8a77/"]}],
 "create_version"=>true,
 "poll_attempts"=>{"total"=>3, "failed"=>1}}

To reproduce:

  1. sync rhel 8 baseos repo
  2. sync rhel 8 appstream repo

Do these in this order, waiting for 1) to complete and then doing 2). You may not have to wait, but i did to reproduce. You can also do 2) before 1. The important part is having one synced before the other.

Associated revisions

Revision dcf82ef7 View on GitHub
Added by ppicka over 1 year ago

Repo metadata relative path

Extend RepoMetaData with relative_path to avoid sync issue repositories contains productid.

[nocoverage]

closes: #5847 https://pulp.plan.io/issues/5847

History

#1 Updated by iballou almost 2 years ago

The sync was on demand.

#2 Updated by ggainey almost 2 years ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 63

#3 Updated by fao89 almost 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fao89

#4 Updated by fao89 almost 2 years ago

I tested with staging repos, and it seems like this problem was solved on the last PRs before GA:

(pulp) [vagrant@pulp3-source-fedora30 pulp_rpm]$ pytest -vv -r sx --color=yes --pyargs --capture=no --durations=0 pulp_rpm.tests.performance.test_publish                       
===================================== test session starts =====================================                                                                                               
platform linux -- Python 3.7.3, pytest-5.3.1, py-1.8.0, pluggy-0.13.1 -- /usr/local/lib/pulp/bin/python3                                                                                      
cachedir: .pytest_cache                                                                                                                                                                       
rootdir: /home/vagrant/devel/pulp_rpm                                                                                                                                                         
collected 4 items                                                                                                                                                                             

pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/appstream/os/

->     Sync => Waiting time (s): 0.08308 | Service time (s): 50.538572

->     Publish => Waiting time (s): 0.094765 | Service time (s): 39.823955
PASSED                                         
pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/baseos/os/

->     Sync => Waiting time (s): 0.065093 | Service time (s): 40.073583

->     Publish => Waiting time (s): 0.074289 | Service time (s): 23.161167
PASSED                                         
pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/appstream/kickstart/

->     Sync => Waiting time (s): 0.072525 | Service time (s): 51.66728

->     Publish => Waiting time (s): 0.079725 | Service time (s): 18.832368
PASSED                                         
pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/baseos/kickstart/

->     Sync => Waiting time (s): 0.063364 | Service time (s): 12.500118

->     Publish => Waiting time (s): 0.074592 | Service time (s): 7.158012
PASSED                                         

=================================== slowest test durations ====================================
119.24s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream
95.18s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream
86.64s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos
34.62s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos
0.39s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream
0.00s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos
0.00s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream
0.00s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream
================================ 4 passed in 336.30s (0:05:36) ================================

#6 Updated by ggainey over 1 year ago

  • Tags Katello added
  • Tags deleted (Katello-P2)

#7 Updated by jsherril@redhat.com over 1 year ago

  • Status changed from CLOSED - WORKSFORME to NEW

reopening and will add updated repro steps to description

#8 Updated by jsherril@redhat.com over 1 year ago

  • Description updated (diff)
  • Triaged changed from Yes to No

#9 Updated by jsherril@redhat.com over 1 year ago

  • Severity changed from 2. Medium to 3. High

#10 Updated by ttereshc over 1 year ago

  • Assignee deleted (fao89)
  • Triaged changed from No to Yes

#11 Updated by ttereshc over 1 year ago

  • Sprint/Milestone set to Pulp 3.x RPM (Katello 4.1)
  • Sprint changed from Sprint 63 to Sprint 75

#12 Updated by ttereshc over 1 year ago

  • Priority changed from Normal to High

#13 Updated by rchan over 1 year ago

  • Sprint changed from Sprint 75 to Sprint 76

#14 Updated by ppicka over 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ppicka

#15 Updated by ggainey over 1 year ago

Specific rel-paths causing us grief:

repodata/adff8e65-8c3c-4ffb-bb72-5d4116fab7c9
repodata/a22b89e3-765e-45bf-97f0-60e53435f3e5

#16 Updated by dkliban@redhat.com over 1 year ago

The problem is that the RepoMetadataFile does not have enough of a uniqueness constraint. It needs to have the relative_path added to it's unique_together. This change will require writing a migration that will take the relative_path value from the ContentArtifact and copy it over into the RepoMetadataFile.relative_path field.

We can achieve this by addubg the relative_path field to the RepoMetadataFile model and then generating a migration for this field. Then we should manually create a data migration and specify the auto generated migration as it's dependency[0].

[0] https://docs.djangoproject.com/en/2.2/topics/migrations/#data-migrations

#17 Updated by pulpbot over 1 year ago

  • Status changed from ASSIGNED to POST

#18 Updated by rchan over 1 year ago

  • Sprint changed from Sprint 76 to Sprint 77

#19 Updated by ppicka over 1 year ago

  • Status changed from POST to MODIFIED

#20 Updated by ttereshc about 1 year ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

#21 Updated by ttereshc about 1 year ago

  • Sprint/Milestone changed from Pulp 3.x RPM (Katello 4.1) to Pulp RPM 3.5.0

Please register to edit this issue

Also available in: Atom PDF