Actions
Issue #2047
closeddirectory creation race condition during publish
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.8.5
Platform Release:
2.9.3
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Easy Fix, Pulp 2
Sprint:
Quarter:
Description
If two repos are published concurrently and share a parent directory, there is a chance they will try to create the parent directory at the same time, and the loser will fail with a traceback like this:
traceback: ! "Traceback (most recent call last):\n File \"/usr/lib/python2.6/site-packages/celery/app/trace.py\",
line 240, in trace_task\n R = retval = fun(*args, **kwargs)\n File \"/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py\",
line 328, in __call__\n return super(Task, self).__call__(*args, **kwargs)\n
\ File \"/usr/lib/python2.6/site-packages/celery/app/trace.py\", line 437, in
__protected_call__\n return self.run(*args, **kwargs)\n File \"/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py\",
line 99, in publish\n transfer_repo, conduit, call_config)\n File \"/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py\",
line 127, in _do_publish\n publish_report = publish_repo(transfer_repo, conduit,
call_config)\n File \"/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py\",
line 495, in wrap_f\n return f(*args, **kwargs)\n File \"/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py\",
line 143, in publish_repo\n return self._publisher.publish()\n File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
line 558, in publish\n return self.process_lifecycle()\n File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
line 503, in process_lifecycle\n super(PluginStep, self).process_lifecycle()\n
\ File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
line 127, in process_lifecycle\n step.process()\n File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
line 204, in process\n self._process_block()\n File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
line 246, in _process_block\n self.process_main()\n File \"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py\",
line 766, in process_main\n os.makedirs(publish_dir_parent, 0750)\n File \"/usr/lib64/python2.6/os.py\",
line 157, in makedirs\n mkdir(name, mode)\nOSError: [Errno 17] File exists:
'/var/lib/pulp/published/yum/https/repos/Default_Organization/Library/content_view_name/custom/somename'\n"
Instead of checking for existence, then creating, the code should try to create and handle the exception if it exists.
Here is one such occurrence: https://github.com/pulp/pulp/blob/pulp-2.8.5-1/server/pulp/plugins/util/publish_step.py#L904
And there are two others in the same file.
Actions
handles mkdirs race condition
closes #2047 https://pulp.plan.io/issues/2047