Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #5847

closed

Pulp error when syncing rhel 8 baseos

Added by iballou almost 5 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
ppicka
Sprint/Milestone:
Pulp RPM 3.5.0
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 77
Quarter:

Description

Ping me for the full repo. Found in pulp-rpm 3.0.0rc1 (2019-11-19)

Error: No declared artifact with relative path \"repodata/ff44f60d-80ee-47e8-9932-6522073764f5\" for content \"<RepoMetadataFile: pk=e2740ada-e07f-4a80-a076-160f3f541850>\""}

Traceback:

{"pulp_tasks"=>
  [{"pulp_href"=>"/pulp/api/v3/tasks/c58b7d28-453b-4e8a-8060-ea6bcf8395bb/",
    "pulp_created"=>"2019-12-06T16:58:26.987+00:00",
    "state"=>"failed",
    "name"=>"pulp_rpm.app.tasks.synchronizing.synchronize",
    "started_at"=>"2019-12-06T16:58:27.234+00:00",
    "finished_at"=>"2019-12-06T16:58:29.609+00:00",
    "error"=>
     {"traceback"=>
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 822, in perform_job\n" +
       "    rv = job.perform()\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 605, in perform\n" +
       "    self._result = self._execute()\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 611, in _execute\n" +
       "    return self.func(*self.args, **self.kwargs)\n" +
       "  File \"/usr/local/lib/pulp/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py\", line 147, in synchronize\n" +
       "    dv.create()\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/declarative_version.py\", line 149, in create\n" +
       "    loop.run_until_complete(pipeline)\n" +
       "  File \"/usr/lib64/python3.6/asyncio/base_events.py\", line 484, in run_until_complete\n" +
       "    return future.result()\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/api.py\", line 209, in create_pipeline\n" +
       "    await asyncio.gather(*futures)\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/api.py\", line 43, in __call__\n" +
       "    await self.run()\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/artifact_stages.py\", line 219, in run\n" +
       "    RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/artifact_stages.py\", line 263, in _needed_remote_artifacts\n" +
       "    c=d_content.content))\n",
      "description"=>
       "No declared artifact with relative path \"repodata/ff44f60d-80ee-47e8-9932-6522073764f5\" for content \"<RepoMetadataFile: pk=e2740ada-e07f-4a80-a076-160f3f541850>\""},
    "worker"=>"/pulp/api/v3/workers/8e8407f4-4840-4f21-b1ef-8084c8a08a5e/",
    "spawned_tasks"=>[],
    "progress_reports"=>
     [{"message"=>"Associating Content",
       "code"=>"associating.content",
       "state"=>"canceled",
       "done"=>0},
      {"message"=>"Downloading Artifacts",
       "code"=>"downloading.artifacts",
       "state"=>"canceled",
       "done"=>0},
      {"message"=>"Downloading Metadata Files",
       "code"=>"downloading.metadata",
       "state"=>"canceled",
       "done"=>2},
      {"message"=>"Parsed Comps",
       "code"=>"parsing.comps",
       "state"=>"running",
       "total"=>42,
       "done"=>7},
      {"message"=>"Parsed Advisories",
       "code"=>"parsing.advisories",
       "state"=>"running",
       "total"=>87,
       "done"=>87},
      {"message"=>"Parsed Packages",
       "code"=>"parsing.packages",
       "state"=>"waiting",
       "done"=>0}],
    "created_resources"=>[],
    "reserved_resources_record"=>
     ["/pulp/api/v3/repositories/rpm/rpm/674ba43b-2e86-44f7-81aa-78326bccae72/",
      "/pulp/api/v3/remotes/rpm/rpm/e6b4fd16-b0f0-4f5d-8db8-7b112bbb8a77/"]}],
 "create_version"=>true,
 "poll_attempts"=>{"total"=>3, "failed"=>1}}

To reproduce:

  1. sync rhel 8 baseos repo
  2. sync rhel 8 appstream repo

Do these in this order, waiting for 1) to complete and then doing 2). You may not have to wait, but i did to reproduce. You can also do 2) before 1. The important part is having one synced before the other.

Actions
Copy link
 #1

Updated by iballou almost 5 years ago

The sync was on demand.

Actions
Copy link
 #2

Updated by ggainey almost 5 years ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 63
Actions
Copy link
 #3

Updated by fao89 almost 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fao89
Actions
Copy link
 #4

Updated by fao89 almost 5 years ago

I tested with staging repos, and it seems like this problem was solved on the last PRs before GA:

(pulp) [vagrant@pulp3-source-fedora30 pulp_rpm]$ pytest -vv -r sx --color=yes --pyargs --capture=no --durations=0 pulp_rpm.tests.performance.test_publish                       
===================================== test session starts =====================================                                                                                               
platform linux -- Python 3.7.3, pytest-5.3.1, py-1.8.0, pluggy-0.13.1 -- /usr/local/lib/pulp/bin/python3                                                                                      
cachedir: .pytest_cache                                                                                                                                                                       
rootdir: /home/vagrant/devel/pulp_rpm                                                                                                                                                         
collected 4 items                                                                                                                                                                             

pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/appstream/os/

->     Sync => Waiting time (s): 0.08308 | Service time (s): 50.538572

->     Publish => Waiting time (s): 0.094765 | Service time (s): 39.823955
PASSED                                         
pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/baseos/os/

->     Sync => Waiting time (s): 0.065093 | Service time (s): 40.073583

->     Publish => Waiting time (s): 0.074289 | Service time (s): 23.161167
PASSED                                         
pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/appstream/kickstart/

->     Sync => Waiting time (s): 0.072525 | Service time (s): 51.66728

->     Publish => Waiting time (s): 0.079725 | Service time (s): 18.832368
PASSED                                         
pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/baseos/kickstart/

->     Sync => Waiting time (s): 0.063364 | Service time (s): 12.500118

->     Publish => Waiting time (s): 0.074592 | Service time (s): 7.158012
PASSED                                         

=================================== slowest test durations ====================================
119.24s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream
95.18s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream
86.64s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos
34.62s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos
0.39s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream
0.00s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos
0.00s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream
0.00s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream
================================ 4 passed in 336.30s (0:05:36) ================================
Actions
Copy link
 #6

Updated by ggainey over 4 years ago

  • Tags Katello added
  • Tags deleted (Katello-P2)
Actions
Copy link
 #7

Updated by jsherril@redhat.com over 4 years ago

  • Status changed from CLOSED - WORKSFORME to NEW

reopening and will add updated repro steps to description

Actions
Copy link
 #8

Updated by jsherril@redhat.com over 4 years ago

  • Description updated (diff)
  • Triaged changed from Yes to No
Actions
Copy link
 #9

Updated by jsherril@redhat.com over 4 years ago

  • Severity changed from 2. Medium to 3. High
Actions
Copy link
 #10

Updated by ttereshc over 4 years ago

  • Assignee deleted (fao89)
  • Triaged changed from No to Yes
Actions
Copy link
 #11

Updated by ttereshc over 4 years ago

  • Sprint/Milestone set to Pulp 3.x RPM (Katello 4.1)
  • Sprint changed from Sprint 63 to Sprint 75
Actions
Copy link
 #12

Updated by ttereshc over 4 years ago

  • Priority changed from Normal to High
Actions
Copy link
 #13

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 75 to Sprint 76
Actions
Copy link
 #14

Updated by ppicka over 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ppicka
Actions
Copy link
 #15

Updated by ggainey over 4 years ago

Specific rel-paths causing us grief:

repodata/adff8e65-8c3c-4ffb-bb72-5d4116fab7c9
repodata/a22b89e3-765e-45bf-97f0-60e53435f3e5
Actions
Copy link
 #16

Updated by dkliban@redhat.com over 4 years ago

The problem is that the RepoMetadataFile does not have enough of a uniqueness constraint. It needs to have the relative_path added to it's unique_together. This change will require writing a migration that will take the relative_path value from the ContentArtifact and copy it over into the RepoMetadataFile.relative_path field.

We can achieve this by addubg the relative_path field to the RepoMetadataFile model and then generating a migration for this field. Then we should manually create a data migration and specify the auto generated migration as it's dependency[0].

[0] https://docs.djangoproject.com/en/2.2/topics/migrations/#data-migrations

Actions
Copy link
 #17

Updated by pulpbot over 4 years ago

  • Status changed from ASSIGNED to POST
Actions
Copy link
 #18

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 76 to Sprint 77

Added by ppicka over 4 years ago

Revision dcf82ef7 | View on GitHub

Repo metadata relative path

Extend RepoMetaData with relative_path to avoid sync issue repositories contains productid.

[nocoverage]

closes: #5847 https://pulp.plan.io/issues/5847

Actions
Copy link
 #19

Updated by ppicka over 4 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #20

Updated by ttereshc over 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions
Copy link
 #21

Updated by ttereshc over 4 years ago

  • Sprint/Milestone changed from Pulp 3.x RPM (Katello 4.1) to Pulp RPM 3.5.0
Actions
Send by e-mail Copy link

Also available in: Atom PDF