Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #8166

closed

Migration fails with ForeignKeyViolation: insert or update on table "pulp_2to3_migration_pulp2erratum" violates foreign key constraint "pulp_2to3_migration

Added by jsherril@redhat.com about 3 years ago. Updated about 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Urgent
Assignee:
ggainey
Sprint/Milestone:
0.8.0
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 90
Quarter:

Description

On a user database, the following was observed:

pulp: rq.worker:ERROR: Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 240, in _commit
return self.connection.commit()
psycopg2.errors.ForeignKeyViolation: insert or update on table "pulp_2to3_migration_pulp2erratum" violates foreign key constraint "pulp_2to3_migration__pulp2content_id_157e4b7e_fk_pulp_2to3"
DETAIL:  Key (pulp2content_id)=(92d6bb6c-531c-43c7-82d8-dc51d752d00e) is not present in table "pulp_2to3_migration_pulp2content".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
rv = job.perform()
File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
self._result = self._execute()
File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
return self.func(*self.args, **self.kwargs)
File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 137, in migrate_from_pulp2
pre_migrate_all_content(plan)
File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 68, in pre_migrate_all_content
pre_migrate_content_type(content_model, mutable_type, lazy_type, premigrate_hook)
File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 272, in pre_migrate_content_type
content_model.pulp_2to3_detail.pre_migrate_content_detail(pulp2content_batch)
File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/pulp_2to3_models.py", line 370, in pre_migrate_content_detail
batch_size=DEFAULT_BATCH_SIZE)
File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 480, in bulk_create
obj_without_pk._state.db = self.db
File "/usr/lib/python3.6/site-packages/django/db/transaction.py", line 240, in __exit__
connection.commit()
File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 262, in commit
self._commit()
File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 240, in _commit
return self.connection.commit()
File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 240, in _commit
return self.connection.commit()
django.db.utils.IntegrityError: insert or update on table "pulp_2to3_migration_pulp2erratum" violates foreign key constraint "pulp_2to3_migration__pulp2content_id_157e4b7e_fk_pulp_2to3"
DETAIL:  Key (pulp2content_id)=(92d6bb6c-531c-43c7-82d8-dc51d752d00e) is not present in table "pulp_2to3_migration_pulp2content".
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 240, in _commit
return self.connection.commit()
psycopg2.errors.ForeignKeyViolation: insert or update on table "pulp_2to3_migration_pulp2erratum" violates foreign key constraint "pulp_2to3_migration__pulp2content_id_157e4b7e_fk_pulp_2to3"
DETAIL:  Key (pulp2content_id)=(92d6bb6c-531c-43c7-82d8-dc51d752d00e) is not present in table "pulp_2to3_migration_pulp2content".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
rv = job.perform()
File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
self._result = self._execute()
File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
return self.func(*self.args, **self.kwargs)
File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 137, in migrate_from_pulp2
pre_migrate_all_content(plan)
File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 68, in pre_migrate_all_content
pre_migrate_content_type(content_model, mutable_type, lazy_type, premigrate_hook)
File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 272, in pre_migrate_content_type
content_model.pulp_2to3_detail.pre_migrate_content_detail(pulp2content_batch)
File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/pulp_2to3_models.py", line 370, in pre_migrate_content_detail
batch_size=DEFAULT_BATCH_SIZE)
File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 480, in bulk_create
obj_without_pk._state.db = self.db
File "/usr/lib/python3.6/site-packages/django/db/transaction.py", line 240, in __exit__
connection.commit()
File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 262, in commit
self._commit()
File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 240, in _commit
return self.connection.commit()
File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 240, in _commit
return self.connection.commit()
django.db.utils.IntegrityError: insert or update on table "pulp_2to3_migration_pulp2erratum" violates foreign key constraint "pulp_2to3_migration__pulp2content_id_157e4b7e_fk_pulp_2to3"
DETAIL:  Key (pulp2content_id)=(92d6bb6c-531c-43c7-82d8-dc51d752d00e) is not present in table "pulp_2to3_migration_pulp2content".
Actions
Copy link
 #1

Updated by ttereshc about 3 years ago

  • Priority changed from High to Urgent
  • Sprint/Milestone set to 0.9.0
  • Sprint set to Sprint 89
Actions
Copy link
 #2

Updated by ggainey about 3 years ago

Hard to figure out what's going on without more info.

Could I ask anyone who is seeing this, to grab some more data for us?

  • log info
  • erratum
  • specific repo

To find the erratum, grab the "(pulp2content_id)=" from th etraceback and look for it in mongo:

$ mongo
MongoDB shell version v3.4.24
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.24
...
> use pulp_database
switched to db pulp_database
> db.units_erratum.find({ "_id": "ID-HERE" });
{ "_id" : "9f6c6313-3fcb-4409-b3fa-0bd69b3fad77", "pulp_user_metadata" : {  }, "_last_updated" : 1611955176, "errata_id" : "RHBA-2020:4548", "status" : "final", "updated" : "2020-11-03 19:39:00 UTC", "description" : "For detailed information on changes in this release, see the Red Hat\nEnterprise Linux 8.3 Release Notes linked from the References section.", "issued" : "2020-11-03 12:14:39 UTC", "pushcount" : "2", "references" : [ { "href" : "https://access.redhat.com/errata/RHBA-2020:4548", "type" : "self", "id" : "RHBA-2020:4548", "title" : "RHBA-2020:4548" }, { "href" : "https://bugzilla.redhat.com/show_bug.cgi?id=1829822", "type" : "bugzilla", "id" : "1829822", "title" : "Rebase gpgme to version 1.13.1 to accommodate gnupg-2.2.20" }, { "href" : "https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/8.3_release_notes/index", "type" : "other", "id" : "ref_0", "title" : "other_reference_0" } ], "reboot_suggested" : false, "relogin_suggested" : false, "restart_suggested" : false, "from" : "release-engineering@redhat.com", "severity" : "None", "rights" : "Copyright 2020 Red Hat Inc", "version" : "2", "release" : "0", "type" : "bugfix", "pkglist" : [ ], "title" : "gpgme bug fix and enhancement update", "solution" : "Before applying this update, make sure all previously released errata\nrelevant to your system have been applied.\n\nFor details on how to apply this update, refer to:\n\nhttps://access.redhat.com/articles/11258", "summary" : "An update for gpgme is now available for Red Hat Enterprise Linux 8.", "_ns" : "units_erratum", "_content_type_id" : "erratum" }
>

To get the repo and remote(s):

$ mongo
> use pulp_database
> db.repo_content_units.find({"unit_id": "ID-HERE"});
{ "_id" : ObjectId("60147be84f26977bc0cc7b49"), "repo_id" : "rhel8-base", "unit_id" : "9f6c6313-3fcb-4409-b3fa-0bd69b3fad77", "unit_type_id" : "erratum", "updated" : "2021-01-29T21:19:36Z", "created" : "2021-01-29T21:19:36Z" }
> db.repo_importers.find({ "repo_id": "REPO-ID-ABOVE"} );
{...}

Hopefully we can recreate this more directly than via "Something went wrong in one of the 34K repos in this dataset. Go fish." :)

Actions
Copy link
 #3

Updated by ggainey about 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ggainey
Actions
Copy link
 #4

Updated by rchan about 3 years ago

  • Sprint changed from Sprint 89 to Sprint 90
Actions
Copy link
 #5

Updated by ggainey about 3 years ago

  • Status changed from ASSIGNED to POST

Added by ggainey about 3 years ago

Revision c7d0e2db | View on GitHub

Removed no-longer-needed pre_migration codepath.

Removes part of the changes for #7280.

fixes #8166

Added by ggainey about 3 years ago

Revision c7d0e2db | View on GitHub

Removed no-longer-needed pre_migration codepath.

Removes part of the changes for #7280.

fixes #8166

Added by ggainey about 3 years ago

Revision c7d0e2db | View on GitHub

Removed no-longer-needed pre_migration codepath.

Removes part of the changes for #7280.

fixes #8166

Actions
Copy link
 #6

Updated by ggainey about 3 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #8

Updated by ttereshc about 3 years ago

  • Sprint/Milestone changed from 0.9.0 to 0.8.0
Actions
Copy link
 #9

Updated by pulpbot about 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions
Send by e-mail Copy link

Also available in: Atom PDF