Actions
Issue #9146
closedsync of file:// repos tries to move files from /var/lib/pulp/sync_imports/
Status:
CLOSED - CURRENTRELEASE
Priority:
Urgent
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 102
Quarter:
Description
When syncing file:// repos it appears that pulp is trying to move files from that directory. In our case we copy files that are world readable, but not writable by the pulp user and see a permission error:
Jul 26 18:27:11 devel2 pulpcore-worker-1: pulp [994770242e2842ecade44f158e7de8d0]: pulpcore.tasking.pulpcore_worker:INFO: Task 4c40418c-8a76-4a83-a6ee-4d2b5326c9d4 failed ([Errno 13] Pe
rmission denied: '/var/lib/pulp/sync_imports/test_repos/zoo/kangaroo-0.2-1.noarch.rpm')
Jul 26 18:27:11 devel2 pulpcore-worker-1: pulp [994770242e2842ecade44f158e7de8d0]: pulpcore.tasking.pulpcore_worker:INFO: File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpc
ore_worker.py", line 272, in _perform_task
Jul 26 18:27:11 devel2 pulpcore-worker-1: result = func(*args, **kwargs)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 427, in synchronize
Jul 26 18:27:11 devel2 pulpcore-worker-1: version = dv.create()
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 151, in create
Jul 26 18:27:11 devel2 pulpcore-worker-1: loop.run_until_complete(pipeline)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Jul 26 18:27:11 devel2 pulpcore-worker-1: return future.result()
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Jul 26 18:27:11 devel2 pulpcore-worker-1: await asyncio.gather(*futures)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Jul 26 18:27:11 devel2 pulpcore-worker-1: await self.run()
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 248, in run
Jul 26 18:27:11 devel2 pulpcore-worker-1: d_artifact.artifact for d_artifact in da_to_save
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/app/models/content.py", line 82, in bulk_get_or_create
Jul 26 18:27:11 devel2 pulpcore-worker-1: return super().bulk_create(objs, batch_size=batch_size)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Jul 26 18:27:11 devel2 pulpcore-worker-1: return getattr(self.get_queryset(), name)(*args, **kwargs)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 468, in bulk_create
Jul 26 18:27:11 devel2 pulpcore-worker-1: self._batched_insert(objs_with_pk, fields, batch_size, ignore_conflicts=ignore_conflicts)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1204, in _batched_insert
Jul 26 18:27:11 devel2 pulpcore-worker-1: ignore_conflicts=ignore_conflicts,
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Jul 26 18:27:11 devel2 pulpcore-worker-1: return query.get_compiler(using=using).execute_sql(return_id)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1376, in execute_sql
Jul 26 18:27:11 devel2 pulpcore-worker-1: for sql, params in self.as_sql():
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django_readonly_field/compiler.py", line 31, in as_sql
Jul 26 18:27:11 devel2 pulpcore-worker-1: return super(ReadonlySQLCompilerMixin, self).as_sql()
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1320, in as_sql
Jul 26 18:27:11 devel2 pulpcore-worker-1: for obj in self.query.objs
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1320, in <listcomp>
Jul 26 18:27:11 devel2 pulpcore-worker-1: for obj in self.query.objs
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1319, in <listcomp>
Jul 26 18:27:11 devel2 pulpcore-worker-1: [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1270, in pre_save_val
Jul 26 18:27:11 devel2 pulpcore-worker-1: return field.pre_save(obj, add=True)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/app/models/fields.py", line 68, in pre_save
Jul 26 18:27:11 devel2 pulpcore-worker-1: return super().pre_save(model_instance, add)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/fields/files.py", line 289, in pre_save
Jul 26 18:27:11 devel2 pulpcore-worker-1: file.save(file.name, file.file, save=False)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/fields/files.py", line 88, in save
Jul 26 18:27:11 devel2 pulpcore-worker-1: self.name = self.storage.save(name, content, max_length=self.field.max_length)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 54, in save
Jul 26 18:27:11 devel2 pulpcore-worker-1: return self._save(name, content)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/app/models/storage.py", line 68, in _save
Jul 26 18:27:11 devel2 pulpcore-worker-1: file_move_safe(content.temporary_file_path(), full_path)
Jul 26 18:27:11 devel2 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/core/files/move.py", line 80, in file_move_safe
Jul 26 18:27:11 devel2 pulpcore-worker-1: os.remove(old_file_name)
These repos are in /var/lib/pulp/sync_imports/ (and settings.py is configured properly).
Related issues
Actions
Move files to artifact storage from WORKING_DIRECTORY.
Copy files from all other sources. Since users are able to change the WORKING_DIRECTORY settings, this patch also ensures that we check that the WORKING_DIRECTORY and MEDIA_ROOT are on the same device. Django's FILE_UPLOAD_TEMP_DIR is now set to the value of WORKING_DIRECTORY.
fixes: #9146 https://pulp.plan.io/issues/9146/