Project

Profile

Help

Issue #7676

django-cleanup can silently delete files being used by existing artifacts in some circumstances

Added by dalley about 2 months ago. Updated about 1 month ago.

Status:
MODIFIED
Priority:
Urgent
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
4. Urgent
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 84
Quarter:

Description

Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=centos8-baseos, version=1
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp: rq.worker:ERROR: Traceback (most recent call last):
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: rv = job.perform()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: self._result = self._execute()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return self.func(*self.args, **self.kwargs)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/tasks/migrate.py", line 142, in migrate_from_pulp2
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: create_repoversions_publications_distributions(plan)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 323, in create_repoversions_publications_distributions
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: task_func(*task_args)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 270, in complex_repo_migration
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: migrated_repo.pulp3_repository_version
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 391, in migrate_repo_distributor
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp2dist, repo_version)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/rpm/repository.py", line 74, in migrate_to_pulp3
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: publish(repo_version.pk, checksum_types=checksum_types)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 283, in publish
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: publication_data.populate()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 224, in populate
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: self.handle_sub_repos(distribution_tree)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 165, in handle_sub_repos
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: artifact_file = storage.open(original_treeinfo_content_artifact.artifact.file.name)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/files/storage.py", line 36, in open
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return self._open(name, mode)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/files/storage.py", line 224, in _open
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return File(open(self.path(name), mode))
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/artifact/48/66eceaf78ffabb1d25774932085c6653f9f654411d7669db7b91e2ef0fd2a3'
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: Traceback (most recent call last):
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: rv = job.perform()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: self._result = self._execute()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return self.func(*self.args, **self.kwargs)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/tasks/migrate.py", line 142, in migrate_from_pulp2
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: create_repoversions_publications_distributions(plan)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 323, in create_repoversions_publications_distributions
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: task_func(*task_args)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 270, in complex_repo_migration
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: migrated_repo.pulp3_repository_version
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/migration.py", line 391, in migrate_repo_distributor
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp2dist, repo_version)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/rpm/repository.py", line 74, in migrate_to_pulp3
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: publish(repo_version.pk, checksum_types=checksum_types)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 283, in publish
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: publication_data.populate()
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 224, in populate
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: self.handle_sub_repos(distribution_tree)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/publishing.py", line 165, in handle_sub_repos
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: artifact_file = storage.open(original_treeinfo_content_artifact.artifact.file.name)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/files/storage.py", line 36, in open
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return self._open(name, mode)
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/files/storage.py", line 224, in _open
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: return File(open(self.path(name), mode))
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/artifact/48/66eceaf78ffabb1d25774932085c6653f9f654411d7669db7b91e2ef0fd2a3'
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp: rq.worker:INFO: 26855@pulp2-nightly-pulp3-source-centos7.localhost.example.com: c009a655-bb3e-4665-9b2e-b3100b82b060
Oct 08 13:30:37 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[26855]: pulp: rq.worker:INFO: 26855@pulp2-nightly-pulp3-source-centos7.localhost.example.com: Job OK (c009a655-bb3e-4665-9b2e-b3100b82b060)

Related issues

Related to Pulp - Issue #7316: Files are not being deleted from storage when calling the method delete()NEW<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to Pulp - Issue #7755: Repair any corruption of the Pulp data store to clean up after #7676MODIFIED<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to Pulp - Backport #7757: Backport to 3.7 #7676 django-cleanup can silently delete files being used by existing artifacts in some circumstancesMODIFIED

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to Pulp - Backport #7758: Backport to 3.8 #7676 django-cleanup can silently delete files being used by existing artifacts in some circumstancesCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Has duplicate Migration Plugin - Issue #7744: pulp2 data is deleted during migrationCLOSED - DUPLICATE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision cf31c5a9 View on GitHub
Added by dalley about 1 month ago

Revert "Delete associated files from the storage"

This reverts commit f508d6b2ba4d624d64bcdbfa6c851afe159c6fac.

closes: #7676 https://pulp.plan.io/issues/7676

History

#1 Updated by dalley about 2 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley

#2 Updated by ttereshc about 2 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 83

#3 Updated by dalley about 2 months ago

Can only reproduce this when also migrating the centos8-baseos repo simultaneously with centos8-kickstart. If attempting to migrate either of them individually, it works, even when migrating the full set of content from both.

#4 Updated by dalley about 2 months ago

I notice that when I migrate centos8-baseos and then sync, all the package groups and the distribution tree are removed and the same number of new units added, which possibly signals something wrong with the migration?

#5 Updated by rchan about 2 months ago

  • Sprint changed from Sprint 83 to Sprint 84

#6 Updated by dalley about 1 month ago

So this is interesting.

The file is created, properly, but gets deleted at some point during the migration. If I pause it in the right place I can see the file there on the filesystem and read it and it's got all the right contents. And then when I continue the migration it fails because the file can't be found. Weird.

#7 Updated by dalley about 1 month ago

I figured out how to make the file undeletable [0] and got this error when continuing the migration

Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: pulp: django_cleanup.handlers:ERROR: There was an exception deleting the file `/var/lib/pulp/artifact/48/66eceaf78ffabb1d25774932085c6653f9f654411d7669db7b
91e2ef0fd2a3` on field `core.artifact.file`
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: Traceback (most recent call last):
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: return self.cursor.execute(sql, params)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "core_artifact_pkey"
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: DETAIL:  Key (pulp_id)=(5ee472a8-aae9-447c-b7a1-74aa0efee3a5) already exists.
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: The above exception was the direct cause of the following exception:
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: Traceback (most recent call last):
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/home/vagrant/devel/pulpcore/pulpcore/app/models/content.py", line 87, in bulk_get_or_create
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: return super().bulk_create(objs, batch_size=batch_size)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 468, in bulk_create
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: self._batched_insert(objs_with_pk, fields, batch_size, ignore_conflicts=ignore_conflicts)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1204, in _batched_insert
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: ignore_conflicts=ignore_conflicts,
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: return query.get_compiler(using=using).execute_sql(return_id)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: cursor.execute(sql, params)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: return executor(sql, params, many, context)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: return self.cursor.execute(sql, params)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: raise dj_exc_value.with_traceback(traceback) from exc_value
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: return self.cursor.execute(sql, params)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_artifact_pkey"
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: DETAIL:  Key (pulp_id)=(5ee472a8-aae9-447c-b7a1-74aa0efee3a5) already exists.
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: During handling of the above exception, another exception occurred:
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: Traceback (most recent call last):
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django_cleanup/handlers.py", line 96, in run_on_commit
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: file_.delete(save=False)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/fields/files.py", line 105, in delete
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: self.storage.delete(self.name)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/files/storage.py", line 303, in delete
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: os.remove(name)
Oct 27 04:46:58 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[7711]: PermissionError: [Errno 1] Operation not permitted: '/var/lib/pulp/artifact/48/66eceaf78ffabb1d25774932085c6653f9f654411d7669db7b91e2ef0fd2a3'

[0] https://www.tecmint.com/make-file-directory-undeletable-immutable-in-linux/

#8 Updated by dalley about 1 month ago

Alright, I think I know what's going on.

Here's the code of our mixin that provides bulk_get_or_create()

        objs = list(objs)
        try:
            with transaction.atomic():
                return super().bulk_create(objs, batch_size=batch_size)
        except IntegrityError:
            for i in range(len(objs)):
                try:
                    with transaction.atomic():
                        objs[i].save()
                except IntegrityError:
                    objs[i] = objs[i].__class__.objects.get(objs[i].q())
        return objs

When it falls back to saving the objects manually, and attempts to save the Artifact in question, it fails on a uniqueness constraint. Which is fine and expected and all of that, except that about 2 months ago we added django-cleanup.

https://github.com/pulp/pulpcore/commit/f508d6b2ba4d624d64bcdbfa6c851afe159c6fac#diff-4d7c51b1efe9043e44439a949dfd92e5827321b34082903477fd04876edb7552

And django-cleanup sees that our model failed to save and decides to helpfully clean up the file that was "left behind" except that in this case it's actually shared with the existing Artifact which is already saved.

I think this would be a problem anywhere that multiple content units share the same artifact, and the artifact has already been downloaded prior to the second content unit being processed?

We notice this only because the RPM plugin migration because it's a place where we have two of the same DeclarativeContent running through the pipeline, and we end up reading the distribution tree file at publish time - most plugins don't actually try to open the file, so you wouldn't notice until requesting the file via the content app. Or maybe not? Do we handle this case where the file disappeared in the content app and try to correct the corruption?

#9 Updated by dalley about 1 month ago

  • Project changed from Migration Plugin to Pulp
  • Subject changed from Publishing fails during the migration of the centos8-baseos repository to django-cleanup can silently delete files being used by existing artifacts in some circumstances
  • Priority changed from High to Urgent

#10 Updated by dalley about 1 month ago

Confirmed that removing the django-cleanup middleware resolves the original issue.

#11 Updated by dalley about 1 month ago

  • Severity changed from 2. Medium to 4. Urgent

#12 Updated by bmbouter about 1 month ago

dalley can you remove the middleware as a PR and also reopen whatever issue was the one that introduced it in the first place?

#13 Updated by dalley about 1 month ago

bmbouter, yes, but we should probably think about how we try to find and fix existing corruption.

  • In a migration?
  • In a global integrity check feature?
  • Lazily, in the content app?

Is it good enough to just delete the existing Artifact and then let the lazy sync machinery re-download?

#14 Updated by dalley about 1 month ago

  • Has duplicate Issue #7744: pulp2 data is deleted during migration added

#15 Updated by bmbouter about 1 month ago

dalley wrote:

bmbouter, yes, but we should probably think about how we try to find and fix existing corruption.

  • In a migration?
  • In a global integrity check feature? We had wondered if we were going to make this before. We currently have a "per-repository" repair, so this would be a "global" version of that. I'm in favor of this.
  • Lazily, in the content app? I think it already does this actually, but we need to check... IIRC, it came up with Katello. I have not checked. If not I'm in favor of this.

Is it good enough to just delete the existing Artifact and then let the lazy sync machinery re-download? I think it is.

#16 Updated by pulpbot about 1 month ago

  • Status changed from ASSIGNED to POST

#17 Updated by daviddavis about 1 month ago

  • Related to Issue #7316: Files are not being deleted from storage when calling the method delete() added

#18 Updated by dalley about 1 month ago

  • Status changed from POST to MODIFIED

#19 Updated by dalley about 1 month ago

  • Related to Issue #7755: Repair any corruption of the Pulp data store to clean up after #7676 added

#20 Updated by ttereshc about 1 month ago

  • Related to Backport #7757: Backport to 3.7 #7676 django-cleanup can silently delete files being used by existing artifacts in some circumstances added

#21 Updated by ttereshc about 1 month ago

  • Related to Backport #7758: Backport to 3.8 #7676 django-cleanup can silently delete files being used by existing artifacts in some circumstances added

Please register to edit this issue

Also available in: Atom PDF