Project

Profile

Help

Issue #9029

Problems when syncing identical repositories simultaneously

Added by dalley 3 months ago. Updated 29 days ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 103
Quarter:

Description

Unguarded uses of bulk_create() are not resilient to races when repositories are synced simultaneously, causing one or both of the syncs to fail.

            if addons:
                Addon.objects.bulk_create(addons)
            if checksums:
                Checksum.objects.bulk_create(checksums)
            if images:
                Image.objects.bulk_create(images)
            if variants:
                Variant.objects.bulk_create(variants)

Related issues

Related to Pulp - Issue #8967: "duplicate key value violates unique constraint" when syncing two repositories with identical content in parallelNEW<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to RPM Support - Backport #9267: Backport #9029 "Problems when syncing identical repositories simultaneously" to 3.14.zCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 74db3255 View on GitHub
Added by dalley about 1 month ago

Fix some errors when identical content is synced simultaneously

closes: #9029 https://pulp.plan.io/issues/9029

History

#1 Updated by dalley 3 months ago

  • Description updated (diff)

#2 Updated by dalley 2 months ago

  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes
  • Sprint set to Sprint 101

#3 Updated by ggainey 2 months ago

pulpcore uses of bulk_create():

(master) ~/github/Pulp3/pulpcore/pulpcore/app $ find . -name \*.py | xargs grep "bulk_create("
./models/repository.py:        RepositoryContent.objects.bulk_create(repo_content)
./models/repository.py:            RepositoryVersionContentDetails.objects.bulk_create(counts_list)
./models/content.py:                return super().bulk_create(objs, batch_size=batch_size)

#4 Updated by dalley about 2 months ago

  • Related to Issue #8967: "duplicate key value violates unique constraint" when syncing two repositories with identical content in parallel added

#5 Updated by ipanova@redhat.com about 2 months ago

  • Sprint changed from Sprint 101 to Sprint 102

#7 Updated by dalley about 2 months ago

  • Priority changed from Normal to High
  • Tags Katello added

#8 Updated by dalley about 2 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley

#9 Updated by pulpbot about 2 months ago

  • Status changed from ASSIGNED to POST

#10 Updated by rchan about 1 month ago

  • Sprint changed from Sprint 102 to Sprint 103

#11 Updated by dalley about 1 month ago

  • Status changed from POST to MODIFIED

#12 Updated by dalley about 1 month ago

  • Sprint/Milestone set to 3.15.0

#13 Updated by dalley about 1 month ago

  • Copied to Backport #9267: Backport #9029 "Problems when syncing identical repositories simultaneously" to 3.14.z added

#15 Updated by pulpbot 29 days ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF