Project

Profile

Help

Issue #2047

closed

directory creation race condition during publish

Added by mhrivnak almost 8 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.8.5
Platform Release:
2.9.3
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Easy Fix, Pulp 2
Sprint:
Quarter:

Description

If two repos are published concurrently and share a parent directory, there is a chance they will try to create the parent directory at the same time, and the loser will fail with a traceback like this:

traceback: ! "Traceback (most recent call last):\n  File \"/usr/lib/python2.6/site-packages/celery/app/trace.py\",
    line 240, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py\",
    line 328, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n
    \ File \"/usr/lib/python2.6/site-packages/celery/app/trace.py\", line 437, in
    __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py\",
    line 99, in publish\n    transfer_repo, conduit, call_config)\n  File \"/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py\",
    line 127, in _do_publish\n    publish_report = publish_repo(transfer_repo, conduit,
    call_config)\n  File \"/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py\",
    line 495, in wrap_f\n    return f(*args, **kwargs)\n  File \"/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py\",
    line 143, in publish_repo\n    return self._publisher.publish()\n  File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
    line 558, in publish\n    return self.process_lifecycle()\n  File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
    line 503, in process_lifecycle\n    super(PluginStep, self).process_lifecycle()\n
    \ File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
    line 127, in process_lifecycle\n    step.process()\n  File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
    line 204, in process\n    self._process_block()\n  File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
    line 246, in _process_block\n    self.process_main()\n  File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
    line 766, in process_main\n    os.makedirs(publish_dir_parent, 0750)\n  File \"/usr/lib64/python2.6/os.py\",
    line 157, in makedirs\n    mkdir(name, mode)\nOSError: [Errno 17] File exists:
    '/var/lib/pulp/published/yum/https/repos/Default_Organization/Library/content_view_name/custom/somename'\n"

Instead of checking for existence, then creating, the code should try to create and handle the exception if it exists.

Here is one such occurrence: https://github.com/pulp/pulp/blob/pulp-2.8.5-1/server/pulp/plugins/util/publish_step.py#L904

And there are two others in the same file.

Also available in: Atom PDF