Project

Profile

Help

Issue #5848

closed

Pulp error when syncing rhel 8 appstream

Added by iballou over 4 years ago. Updated almost 4 years ago.

Status:
CLOSED - WORKSFORME
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 63
Quarter:

Description

Ping me for the exact repo.

Error: DETAIL: Failing row contains (91efe203-4e7c-48be-b53e-fffa893a0315, 2019-12-06 17:02:12.040401+00, 2019-12-06 17:02:12.040436+00, null, 5ee2f1c3-1ff5-4a45-bfb4-0b17cfcd1e41, 2d3544b0-6d3d-43f7-84d1-e934cf9823a7, null).

Traceback:

{"pulp_tasks"=>
  [{"pulp_href"=>"/pulp/api/v3/tasks/fb324522-835f-425e-b0ed-75f8f6a7107e/",
    "pulp_created"=>"2019-12-06T16:56:30.486+00:00",
    "state"=>"failed",
    "name"=>"pulp_rpm.app.tasks.synchronizing.synchronize",
    "started_at"=>"2019-12-06T16:56:30.592+00:00",
    "finished_at"=>"2019-12-06T17:02:12.072+00:00",
    "error"=>
     {"traceback"=>
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 822, in perform_job\n" +
       "    rv = job.perform()\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 605, in perform\n" +
       "    self._result = self._execute()\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 611, in _execute\n" +
       "    return self.func(*self.args, **self.kwargs)\n" +
       "  File \"/usr/local/lib/pulp/src/pulp-rpm/pulp_rpm/app/tasks/synchronizing.py\", line 147, in synchronize\n" +
       "    dv.create()\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/plugin/stages/declarative_version.py\", line 149, in create\n" +
       "    loop.run_until_complete(pipeline)\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/app/models/repository.py\", line 657, in __exit__\n" +
       "    self.repository.finalize_new_version(self)\n" +
       "  File \"/usr/local/lib/pulp/src/pulp-rpm/pulp_rpm/app/models.py\", line 1372, in finalize_new_version\n" +
       "    resolve_module_packages(new_version, previous_version)\n" +
       "  File \"/usr/local/lib/pulp/src/pulp-rpm/pulp_rpm/app/modulemd.py\", line 39, in resolve_module_packages\n" +
       "    version.add_content(packages_to_add)\n" +
       "  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/app/models/repository.py\", line 512, in add_content\n" +
       "    RepositoryContent.objects.bulk_create(repo_content)\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py\", line 82, in manager_method\n" +
       "    return getattr(self.get_queryset(), name)(*args, **kwargs)\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py\", line 468, in bulk_create\n" +
       "    self._batched_insert(objs_with_pk, fields, batch_size, ignore_conflicts=ignore_conflicts)\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py\", line 1204, in _batched_insert\n" +
       "    ignore_conflicts=ignore_conflicts,\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py\", line 1186, in _insert\n" +
       "    return query.get_compiler(using=using).execute_sql(return_id)\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py\", line 1335, in execute_sql\n" +
       "    cursor.execute(sql, params)\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 67, in execute\n" +
       "    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 76, in _execute_with_wrappers\n" +
       "    return executor(sql, params, many, context)\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 84, in _execute\n" +
       "    return self.cursor.execute(sql, params)\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py\", line 89, in __exit__\n" +
       "    raise dj_exc_value.with_traceback(traceback) from exc_value\n" +
       "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py\", line 84, in _execute\n" +
       "    return self.cursor.execute(sql, params)\n",
      "description"=>
       "null value in column \"content_id\" violates not-null constraint\n" +
       "DETAIL:  Failing row contains (91efe203-4e7c-48be-b53e-fffa893a0315, 2019-12-06 17:02:12.040401+00, 2019-12-06 17:02:12.040436+00, null, 5ee2f1c3-1ff5-4a45-bfb4-0b17cfcd1e41, 2d3544b0-6d3d-43f7-84d1-e934cf9823a7, null).\n"},
    "worker"=>"/pulp/api/v3/workers/45bd950e-4b1a-4dfa-b9bf-95967a3b4470/",
    "spawned_tasks"=>[],
    "progress_reports"=>
     [{"message"=>"Parsed Packages",
       "code"=>"parsing.packages",
       "state"=>"completed",
       "total"=>9130,
       "done"=>9130},
      {"message"=>"Parsed Advisories",
       "code"=>"parsing.advisories",
       "state"=>"completed",
       "total"=>171,
       "done"=>171},
      {"message"=>"Parsed Comps",
       "code"=>"parsing.comps",
       "state"=>"completed",
       "total"=>67,
       "done"=>67},
      {"message"=>"Downloading Metadata Files",
       "code"=>"downloading.metadata",
       "state"=>"completed",
       "done"=>5},
      {"message"=>"Downloading Artifacts",
       "code"=>"downloading.artifacts",
       "state"=>"completed",
       "done"=>1},
      {"message"=>"Associating Content",
       "code"=>"associating.content",
       "state"=>"completed",
       "done"=>9550},
      {"message"=>"Parse Modulemd",
       "code"=>"parsing.modulemds",
       "state"=>"completed",
       "total"=>141,
       "done"=>141},
      {"message"=>"Parse Modulemd-defaults",
       "code"=>"parsing.modulemddefaults",
       "state"=>"completed",
       "total"=>40,
       "done"=>40}],
    "created_resources"=>[""],
    "reserved_resources_record"=>
     ["/pulp/api/v3/repositories/rpm/rpm/5ee2f1c3-1ff5-4a45-bfb4-0b17cfcd1e41/",
      "/pulp/api/v3/remotes/rpm/rpm/48fc85b6-c42b-4679-acda-dbb4e24e2e18/"]}],
 "create_version"=>true,
 "poll_attempts"=>{"total"=>42, "failed"=>1}}
Actions #1

Updated by iballou over 4 years ago

The sync was on demand.

Actions #2

Updated by iballou over 4 years ago

I also get the same error for CentOS 8 AppStream too.

Actions #3

Updated by ggainey over 4 years ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 63
Actions #4

Updated by fao89 over 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fao89
Actions #5

Updated by fao89 over 4 years ago

I tested with staging repos, and it seems like this problem was solved on the last PRs before GA:

(pulp) [vagrant@pulp3-source-fedora30 pulp_rpm]$ pytest -vv -r sx --color=yes --pyargs --capture=no --durations=0 pulp_rpm.tests.performance.test_publish                       
===================================== test session starts =====================================                                                                                               
platform linux -- Python 3.7.3, pytest-5.3.1, py-1.8.0, pluggy-0.13.1 -- /usr/local/lib/pulp/bin/python3                                                                                      
cachedir: .pytest_cache                                                                                                                                                                       
rootdir: /home/vagrant/devel/pulp_rpm                                                                                                                                                         
collected 4 items                                                                                                                                                                             

pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/appstream/os/

->     Sync => Waiting time (s): 0.08308 | Service time (s): 50.538572

->     Publish => Waiting time (s): 0.094765 | Service time (s): 39.823955
PASSED                                         
pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/baseos/os/

->     Sync => Waiting time (s): 0.065093 | Service time (s): 40.073583

->     Publish => Waiting time (s): 0.074289 | Service time (s): 23.161167
PASSED                                         
pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/appstream/kickstart/

->     Sync => Waiting time (s): 0.072525 | Service time (s): 51.66728

->     Publish => Waiting time (s): 0.079725 | Service time (s): 18.832368
PASSED                                         
pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos 
http://cdn.stage.redhat.com/content/dist/rhel8/8.0/x86_64/baseos/kickstart/

->     Sync => Waiting time (s): 0.063364 | Service time (s): 12.500118

->     Publish => Waiting time (s): 0.074592 | Service time (s): 7.158012
PASSED                                         

=================================== slowest test durations ====================================
119.24s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream
95.18s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream
86.64s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos
34.62s call     pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos
0.39s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream
0.00s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos
0.00s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream
0.00s setup    pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_baseos
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_appstream
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_baseos
0.00s teardown pulp_rpm/tests/performance/test_publish.py::PublishTestCase::test_rhel8_kickstart_appstream
================================ 4 passed in 336.30s (0:05:36) ================================
Actions #7

Updated by ggainey almost 4 years ago

  • Tags Katello added
  • Tags deleted (Katello-P2)

Also available in: Atom PDF