Project

Profile

Help

Issue #902

closed

race condition in condit's init_unit()

Added by mhrivnak almost 9 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.6.0
Platform Release:
2.6.2
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

When calling init_unit, there is a race condition here:

"/usr/lib/python2.7/site-packages/pulp/server/managers/content/query.py", line 290

When doing a sync of repos with similar content, it is possible that the conflict exhibited below will occur. The fix is likely to change to a try...except workflow that no-ops if the directory exists. Histro yfrom the downstream bug below.

I kicked off a sync of the following repos, all at once

RHEL7 Kickstart (7.0 & 7.1)
RHEL7 Common (7Server, 7.0, 7.1)
RHEL7 Server (7Server, 7.0, 7.1)

In syslog I get a few errors about File exists during the sync. But the final error, I'll upload as a follow-up post.

Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.plugins.conduits.mixins:ERROR: (4689-91168) Exception from server requesting unit filename for relative path [OpenEXR-libs/1.7.1/7.el7/x86_64/3b8c0f8dbfc0b6ef0befc1d5f3ee82942fe825164b86c62e6b29632a9974fcff/OpenEXR-libs-1.7.1-7.el7.x86_64.rpm]
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.plugins.conduits.mixins:ERROR: (4689-91168) Traceback (most recent call last):
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.plugins.conduits.mixins:ERROR: (4689-91168)   File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/mixins.py", line 477, in init_unit
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.plugins.conduits.mixins:ERROR: (4689-91168)     path = content_query_manager.request_content_unit_file_path(type_id, relative_path)
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.plugins.conduits.mixins:ERROR: (4689-91168)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/query.py", line 290, in request_content_unit_file_path
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.plugins.conduits.mixins:ERROR: (4689-91168)     os.makedirs(unit_dir)
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.plugins.conduits.mixins:ERROR: (4689-91168)   File "/usr/lib64/python2.7/os.py", line 157, in makedirs
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.plugins.conduits.mixins:ERROR: (4689-91168)     mkdir(name, mode)
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.plugins.conduits.mixins:ERROR: (4689-91168) OSError: [Errno 17] File exists: '/var/lib/pulp/content/rpm/OpenEXR-libs/1.7.1/7.el7/x86_64/3b8c0f8dbfc0b6ef0befc1d5f3ee82942fe825164b86c62e6b29632a9974fcff'
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168) <bound method ContainerListener.download_succeeded of <pulp_rpm.plugins.importers.yum.repomd.alternate.ContainerListener object at 0x417eb90>>
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168) Traceback (most recent call last):
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)   File "/usr/lib/python2.7/site-packages/pulp/server/content/sources/container.py", line 148, in _forward
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)     method(request)
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/alternate.py", line 126, in download_succeeded
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)     self.content_listener.download_succeeded(report)
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/listener.py", line 81, in download_succeeded
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)     unit = self.sync_conduit.init_unit(model.TYPE, model.unit_key, model.metadata, model.relative_path)
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)   File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/mixins.py", line 477, in init_unit
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)     path = content_query_manager.request_content_unit_file_path(type_id, relative_path)
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/query.py", line 290, in request_content_unit_file_path
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)     os.makedirs(unit_dir)
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)   File "/usr/lib64/python2.7/os.py", line 157, in makedirs
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168)     mkdir(name, mode)
Apr 17 13:33:52 satellite6.rdu.salab.redhat.com pulp[4689]: pulp.server.content.sources.container:ERROR: (4689-91168) ImporterConduitException: [Errno 17] File exists: '/var/lib/pulp/content/rpm/OpenEXR-libs/1.7.1/7.el7/x86_64/3b8c0f8dbfc0b6ef0befc1d5f3ee82942fe825164b86c62e6b29632a9974fcff'

Also available in: Atom PDF