Project

Profile

Help

Issue #8489

artifact_stage fix breaks pulp_container

Added by ggainey 7 months ago. Updated 6 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

The fix for #8377 causes a lot of failures in pulp_container CI. An example:

Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: pulp [bdb67b82aca74c9eaec69b99e5319b3c]: rq.worker:ERROR: Traceback (most recent call last):
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "core_remoteartifact_content_artifact_id_remote_id_be64c19e_uniq
"
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: DETAIL:  Key (content_artifact_id, remote_id)=(a935046c-f958-408a-9c30-091cc311f003, 9c7d0f6e-76c7-42b0-b38d-20d84400b200) already exists.
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: The above exception was the direct cause of the following exception:
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: Traceback (most recent call last):
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/app/models/content.py", line 82, in bulk_get_or_create
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return super().bulk_create(objs, batch_size=batch_size)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 468, in bulk_create
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: self._batched_insert(objs_with_pk, fields, batch_size, ignore_conflicts=ignore_conflicts)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1204, in _batched_insert
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: ignore_conflicts=ignore_conflicts,
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return query.get_compiler(using=using).execute_sql(return_id)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return executor(sql, params, many, context)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: raise dj_exc_value.with_traceback(traceback) from exc_value
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_remoteartifact_content_artifact_id_remote_id_be64c19e_uniq"
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: DETAIL:  Key (content_artifact_id, remote_id)=(a935046c-f958-408a-9c30-091cc311f003, 9c7d0f6e-76c7-42b0-b38d-20d84400b200) already exists.
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: During handling of the above exception, another exception occurred:
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: Traceback (most recent call last):
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "core_remoteartifact_content_artifact_id_remote_id_be64c19e_uniq
"
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: DETAIL:  Key (content_artifact_id, remote_id)=(a935046c-f958-408a-9c30-091cc311f003, 9c7d0f6e-76c7-42b0-b38d-20d84400b200) already exists.
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: The above exception was the direct cause of the following exception:
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: Traceback (most recent call last):
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/app/models/content.py", line 87, in bulk_get_or_create
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: objs[i].save()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django_lifecycle/mixins.py", line 134, in save
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: save(*args, **kwargs)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 744, in save
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: force_update=force_update, update_fields=update_fields)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 782, in save_base
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: force_update, using, update_fields,
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 873, in _save_table
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 911, in _do_insert
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: using=using, raw=raw)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return query.get_compiler(using=using).execute_sql(return_id)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return executor(sql, params, many, context)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: raise dj_exc_value.with_traceback(traceback) from exc_value
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_remoteartifact_content_artifact_id_remote_id_be64c19e_uniq"
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: DETAIL:  Key (content_artifact_id, remote_id)=(a935046c-f958-408a-9c30-091cc311f003, 9c7d0f6e-76c7-42b0-b38d-20d84400b200) already exists.
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: During handling of the above exception, another exception occurred:
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: Traceback (most recent call last):
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 975, in perform_job
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: rv = job.perform()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 696, in perform
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: self._result = self._execute()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 719, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.func(*self.args, **self.kwargs)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulp_container/pulp_container/app/tasks/synchronize.py", line 44, in synchronize
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return dv.create()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/declarative_version.py", line 147, in create
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: loop.run_until_complete(pipeline)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return future.result()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: await asyncio.gather(*futures)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/api.py", line 43, in __call__
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: await self.run()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/artifact_stages.py", line 264, in run
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/app/models/content.py", line 89, in bulk_get_or_create
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: objs[i] = objs[i].__class__.objects.get(objs[i].q())
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 408, in get
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: self.model._meta.object_name
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: pulpcore.app.models.content.RemoteArtifact.DoesNotExist: RemoteArtifact matching query does not exist.
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: Traceback (most recent call last):
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "core_remoteartifact_content_artifact_id_remote_id_be64c19e_uniq"
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: DETAIL:  Key (content_artifact_id, remote_id)=(a935046c-f958-408a-9c30-091cc311f003, 9c7d0f6e-76c7-42b0-b38d-20d84400b200) already exists.
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: The above exception was the direct cause of the following exception:
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: Traceback (most recent call last):
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/app/models/content.py", line 82, in bulk_get_or_create
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return super().bulk_create(objs, batch_size=batch_size)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 468, in bulk_create
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: self._batched_insert(objs_with_pk, fields, batch_size, ignore_conflicts=ignore_conflicts)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1204, in _batched_insert
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: ignore_conflicts=ignore_conflicts,
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return query.get_compiler(using=using).execute_sql(return_id)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return executor(sql, params, many, context)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: raise dj_exc_value.with_traceback(traceback) from exc_value
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_remoteartifact_content_artifact_id_remote_id_be64c19e_uniq"
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: DETAIL:  Key (content_artifact_id, remote_id)=(a935046c-f958-408a-9c30-091cc311f003, 9c7d0f6e-76c7-42b0-b38d-20d84400b200) already exists.
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: During handling of the above exception, another exception occurred:
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: Traceback (most recent call last):
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "core_remoteartifact_content_artifact_id_remote_id_be64c19e_uniq
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: DETAIL:  Key (content_artifact_id, remote_id)=(a935046c-f958-408a-9c30-091cc311f003, 9c7d0f6e-76c7-42b0-b38d-20d84400b200) already exists.
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: The above exception was the direct cause of the following exception:
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: Traceback (most recent call last):
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/app/models/content.py", line 87, in bulk_get_or_create
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: objs[i].save()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django_lifecycle/mixins.py", line 134, in save
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: save(*args, **kwargs)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 744, in save
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: force_update=force_update, update_fields=update_fields)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 782, in save_base
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: force_update, using, update_fields,
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 873, in _save_table
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 911, in _do_insert
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: using=using, raw=raw)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return query.get_compiler(using=using).execute_sql(return_id)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return executor(sql, params, many, context)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: raise dj_exc_value.with_traceback(traceback) from exc_value
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.cursor.execute(sql, params)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_remoteartifact_content_artifact_id_remote_id_be64c19e_uniq"
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: DETAIL:  Key (content_artifact_id, remote_id)=(a935046c-f958-408a-9c30-091cc311f003, 9c7d0f6e-76c7-42b0-b38d-20d84400b200) already exists.
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: During handling of the above exception, another exception occurred:
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: Traceback (most recent call last):
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 975, in perform_job
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: rv = job.perform()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 696, in perform
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: self._result = self._execute()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 719, in _execute
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return self.func(*self.args, **self.kwargs)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulp_container/pulp_container/app/tasks/synchronize.py", line 44, in synchronize
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return dv.create()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/declarative_version.py", line 147, in create
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: loop.run_until_complete(pipeline)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return future.result()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: await asyncio.gather(*futures)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/api.py", line 43, in __call__
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: await self.run()
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/artifact_stages.py", line 264, in run
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/home/vagrant/devel/pulpcore/pulpcore/app/models/content.py", line 89, in bulk_get_or_create
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: objs[i] = objs[i].__class__.objects.get(objs[i].q())
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 408, in get
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: self.model._meta.object_name
Mar 31 16:30:26 pulp2-nightly-pulp3-source-centos7.padre-fedora.example.com rq[7364]: pulpcore.app.models.content.RemoteArtifact.DoesNotExist: RemoteArtifact matching query does not exist.

Associated revisions

Revision 47e5cc2a View on GitHub
Added by ggainey 7 months ago

Teach artifact_stages:_needed_remote_artifacts() to not return dupes.

fixes #8489 [nocoverage]

History

#1 Updated by ggainey 7 months ago

The fix in pulpcore/plugin/stages/artifact_stages:_needed_remote_artifacts() resulted in the returned llist holding duplicates - the pk of the RemoteArtifact was different, for the same sha256. Ultimately, these wind their way through to the content_stages portion of the pipeline and blow up with the error above.

#2 Updated by pulpbot 7 months ago

  • Status changed from ASSIGNED to POST

#3 Updated by ggainey 7 months ago

  • Status changed from POST to MODIFIED

#4 Updated by mdellweg 6 months ago

  • Sprint/Milestone set to 3.12.0

#5 Updated by pulpbot 6 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF