Project

Profile

Help

Issue #7676

closed

django-cleanup can silently delete files being used by existing artifacts in some circumstances

Added by dalley about 4 years ago. Updated almost 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Urgent
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
4. Urgent
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 84
Quarter:

Description

Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=centos8-baseos, version=1
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp: rq.worker:ERROR: Traceback (most recent call last):
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: rv = job.perform()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: self._result = self._execute()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return self.func(*self.args, **self.kwargs)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/tasks/migrate.py", line 142, in migrate_from_pulp2
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: create_repoversions_publications_distributions(plan)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 323, in create_repoversions_publications_distributions
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: task_func(*task_args)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 270, in complex_repo_migration
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: migrated_repo.pulp3_repository_version
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 391, in migrate_repo_distributor
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp2dist, repo_version)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/rpm/repository.py", line 74, in migrate_to_pulp3
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: publish(repo_version.pk, checksum_types=checksum_types)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 283, in publish
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: publication_data.populate()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 224, in populate
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: self.handle_sub_repos(distribution_tree)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 165, in handle_sub_repos
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: artifact_file = storage.open(original_treeinfo_content_artifact.artifact.file.name)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/files/storage.py", line 36, in open
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return self._open(name, mode)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/files/storage.py", line 224, in _open
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return File(open(self.path(name), mode))
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/artifact/48/66eceaf78ffabb1d25774932085c6653f9f654411d7669db7b91e2ef0fd2a3'
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: Traceback (most recent call last):
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: rv = job.perform()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: self._result = self._execute()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return self.func(*self.args, **self.kwargs)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/tasks/migrate.py", line 142, in migrate_from_pulp2
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: create_repoversions_publications_distributions(plan)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 323, in create_repoversions_publications_distributions
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: task_func(*task_args)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 270, in complex_repo_migration
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: migrated_repo.pulp3_repository_version
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 391, in migrate_repo_distributor
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp2dist, repo_version)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/rpm/repository.py", line 74, in migrate_to_pulp3
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: publish(repo_version.pk, checksum_types=checksum_types)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 283, in publish
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: publication_data.populate()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 224, in populate
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: self.handle_sub_repos(distribution_tree)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 165, in handle_sub_repos
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: artifact_file = storage.open(original_treeinfo_content_artifact.artifact.file.name)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/files/storage.py", line 36, in open
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return self._open(name, mode)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/files/storage.py", line 224, in _open
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return File(open(self.path(name), mode))
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/artifact/48/66eceaf78ffabb1d25774932085c6653f9f654411d7669db7b91e2ef0fd2a3'
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp: rq.worker:INFO: 26855@pulp2-nightly-pulp3-source-centos7.localhost.example.com: c009a655-bb3e-4665-9b2e-b3100b82b060
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp: rq.worker:INFO: 26855@pulp2-nightly-pulp3-source-centos7.localhost.example.com: Job OK (c009a655-bb3e-4665-9b2e-b3100b82b060)

Related issues

Related to Pulp - Issue #7316: Files are not being deleted from storage when calling the method delete()CLOSED - CURRENTRELEASEbmbouterActions
Related to Pulp - Issue #7755: Repair any corruption of the Pulp data store to clean up after #7676CLOSED - CURRENTRELEASEdalleyActions
Related to Pulp - Backport #7757: Backport to 3.7 #7676 django-cleanup can silently delete files being used by existing artifacts in some circumstancesCLOSED - CURRENTRELEASEttereshc

Actions
Related to Pulp - Backport #7758: Backport to 3.8 #7676 django-cleanup can silently delete files being used by existing artifacts in some circumstancesCLOSED - CURRENTRELEASEttereshc

Actions
Has duplicate Migration Plugin - Issue #7744: pulp2 data is deleted during migrationCLOSED - DUPLICATEActions

Also available in: Atom PDF