Project

Profile

Help

Story #6749

closed

As a core or plugin developer, i want to be able to share temporary files between the viewset and triggered tasks

Added by mdellweg over 2 years ago. Updated about 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Current Problems

  1. Deleting the temporary Artifact can cause a race condition where the second fails or no deletes occur
  2. FK exception when the Artifact being deleted already exists and is associated with other content
  3. Backups would incorrectly include both long-term Artifacts and short-term files.
  4. Security concern that these temporary files are exposed to users as Artifacts and can be associated to Content, etc

Background

Sometimes, files must be brought forward from the viewset to the executing task, that are not meant to end up being artifacts in the end. On the other hand, different pulp services are not guaranteed to share a common filesystem (like /usr/share/pulp).

Proposal

The design is to have a service that allows to create files with the same storage technology that the artifacts use, where unique names are generated, and the task to use the file has the duty to remove it again from the storage. It is necessary, that those files do neither collide with any artifact that happens to be identical, nor another such file created for a different task.

The Plan

  1. Create a new model called PulpTemporaryFile that has a pulp_id and a file.
  2. Files should be saved based on pulp_id (eg MEDIA_ROOT/tmp/files/29/beb7f3-1a70-40da-86fc-5f9006670baa)
  3. Add PulpTemporaryFile to the plugin api and add documentation

Related issues

Related to Ansible Plugin - Task #6718: Stop using artifacts when importing collectionsCLOSED - CURRENTRELEASEfao89

Actions
Related to Ansible Plugin - Issue #5666: Failed imports leave orphaned artifacts blocking subsequent attemptsCLOSED - DUPLICATEActions
Related to Pulp - Refactor #7270: NoArtifactContentUploadViewSet should use PulpTemporaryFile instead of Artifact to store temporary dataCLOSED - CURRENTRELEASElmjachky

Actions
Actions #1

Updated by mdellweg over 2 years ago

  • Description updated (diff)
Actions #2

Updated by fao89 about 2 years ago

  • Related to Task #6718: Stop using artifacts when importing collections added
Actions #3

Updated by fao89 about 2 years ago

  • Related to Issue #5666: Failed imports leave orphaned artifacts blocking subsequent attempts added
Actions #4

Updated by bmbouter about 2 years ago

  • Description updated (diff)
Actions #5

Updated by daviddavis about 2 years ago

  • Description updated (diff)
Actions #6

Updated by daviddavis about 2 years ago

  • Description updated (diff)
Actions #7

Updated by bmbouter about 2 years ago

  • Description updated (diff)
Actions #8

Updated by bmbouter about 2 years ago

  • Description updated (diff)
Actions #9

Updated by daviddavis about 2 years ago

  • Description updated (diff)
Actions #10

Updated by daviddavis about 2 years ago

  • Description updated (diff)
Actions #11

Updated by daviddavis about 2 years ago

  • Description updated (diff)
Actions #12

Updated by fao89 about 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fao89
Actions #13

Updated by pulpbot about 2 years ago

  • Status changed from ASSIGNED to POST

Added by Fabricio Aguiar about 2 years ago

Revision 6b3be357

Introducing PulpTemporaryFile

https://pulp.plan.io/issues/6749 closes #6749

Actions #14

Updated by Anonymous about 2 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #15

Updated by lmjachky about 2 years ago

  • Related to Refactor #7270: NoArtifactContentUploadViewSet should use PulpTemporaryFile instead of Artifact to store temporary data added
Actions #16

Updated by dkliban@redhat.com about 2 years ago

  • Sprint/Milestone set to 3.6.0
Actions #17

Updated by pulpbot about 2 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF