Project

Profile

Help

Issue #8695

closed

FileSystemExporter cannot cross os boundary

Added by bmbouter over 3 years ago. Updated about 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 102
Quarter:

Description

When I run the test: pytest -v -r sx --color=yes --pyargs pulpcore.tests.functional.api.using_plugin.test_filesystemexport::FilesystemExportTestCase::test_delete -x on Fedora33 I receive the error:

E pulp_smash.pulp3.bindings.PulpTaskError: (PulpTaskError(...), "Pulp task failed ([Errno 18] Invalid cross-device link: '/var/lib/pulp/media/artifact/de/1e4813fedff7ee0223d63c4aaba1b9b3399cbff0edf059083be7ab1a157f9d' -> '/tmp/10be1fa3-87e1-49f9-8653-27d4df1dfd2c/2.iso')")'

The suggestion came up that it should us a symlink not a hardlink. Here is the background IRC convo:

2021-05-04 11:33:22     dalley  in general though, it's the kind of error you get when you try to move or link a file between two separate filesystems
2021-05-04 11:33:28     bmbouter        I installed last night so I thought I didn't get it yet
2021-05-04 11:33:29     dalley  and /tmp is a separate one I believe
2021-05-04 11:33:29     ggainey (this isn't PulpExport, it's another kind-of export, fwiw)
2021-05-04 11:33:48     bmbouter        that's actually what I'm going to try to fix (the django based issue)
2021-05-04 11:34:40     bmbouter        yeah I have Django==2.2.20
2021-05-04 11:35:55     bmbouter        dalley: lgtm, https://github.com/pulp/pulpcore/pull/1302#pullrequestreview-651382325
2021-05-04 11:37:08     dalley  bmbouter, do I need to say something about the signature of ContentAssociation stage changing or should I just add a default
2021-05-04 11:37:49     dalley  or, honestly, should we just remove both from the API
2021-05-04 11:37:56     ggainey ok, so, filesystemexporter 'assumes' you can link betwen where the artifacts are, and where you told it to export to : https://github.com/pulp/pulpcore/blob/master/pulpcore/app/models/exporter.py#L99-L104
2021-05-04 11:38:05     ggainey bmbouter: ^^
2021-05-04 11:38:27     ggainey I haven't seen the error before tho, fwiw
2021-05-04 11:38:38     bmbouter        I think it has to be with me doing this in a vagrant env
2021-05-04 11:39:17     ggainey yeah, could be
2021-05-04 11:39:34     ggainey hrm, which vagrant box are you on? lemme run the test on one of mine
2021-05-04 11:39:51     dalley  I've been dealing with similar issues in my side project. trying to recursively copy files a directory /tmp to somewhere in /home is sadly much more difficult than one would hope
2021-05-04 11:41:30     ggainey I have a cat in my ofc snoring so loudly I can hear them thru my headphones :)
2021-05-04 11:42:55     bmbouter        ha
2021-05-04 11:42:59     ggainey bmbouter: oh fun - test runs fine on my centos7 vagrant, fails on my f33 w/your error
2021-05-04 11:43:10     bmbouter        I'm on pulp3-source-fedora33
2021-05-04 11:43:39     ggainey yeah same
2021-05-04 11:43:47     bmbouter        well that's comforting (no joke)
2021-05-04 11:43:54     ggainey and the 2to3 centos box doesn't have /tmp on its own FS
2021-05-04 11:44:00     bmbouter        mmm
2021-05-04 11:44:12     bmbouter        so what do you think we should do for resolution for dev envs
2021-05-04 11:44:24     bmbouter        I mean the test isn't wrong par-se
2021-05-04 11:44:36     bmbouter        probably something about the f33 env is wrong...
2021-05-04 11:46:53     <--     quba42 (~qubap42@aftr-82-135-82-151.dynamic.mnet-online.de) has quit (Quit: Leaving)
2021-05-04 11:47:42     ggainey bmbouter: hrm - one could make the case that the code should be using os.symlink() instead of creating a hardlink
2021-05-04 11:48:30     ggainey because right now it makes Assumptions about the partitioning of your system, that it prob shouldn't
2021-05-04 11:49:17     ggainey having /tmp on its own FS isn't terribly uncommon - and having F33 set up this way, has shown us a potential problem, so yay? :)

Related issues

Blocked by Pulp - Task #8860: Move FilesystemExporter out of plugin API into pulpcoreCLOSED - CURRENTRELEASEdaviddavis

Actions
Actions #1

Updated by daviddavis over 3 years ago

In the past, we've discussed adding an option to FileSystemExporter to toggle between symlink/hardlink. Perhaps that's a solution to this problem?

Actions #2

Updated by fao89 over 3 years ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 96
Actions #3

Updated by rchan over 3 years ago

  • Sprint changed from Sprint 96 to Sprint 97
Actions #4

Updated by daviddavis over 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to daviddavis
Actions #5

Updated by rchan over 3 years ago

  • Sprint changed from Sprint 97 to Sprint 98
Actions #6

Updated by pulpbot over 3 years ago

  • Status changed from ASSIGNED to POST
Actions #7

Updated by daviddavis over 3 years ago

  • Blocked by Task #8860: Move FilesystemExporter out of plugin API into pulpcore added
Actions #8

Updated by rchan over 3 years ago

  • Sprint changed from Sprint 98 to Sprint 99
Actions #9

Updated by rchan over 3 years ago

  • Sprint changed from Sprint 99 to Sprint 100
Actions #10

Updated by rchan over 3 years ago

  • Sprint changed from Sprint 100 to Sprint 101
Actions #11

Updated by daviddavis over 3 years ago

  • Sprint/Milestone set to 3.15.0
Actions #12

Updated by ipanova@redhat.com over 3 years ago

  • Sprint changed from Sprint 101 to Sprint 102

Added by daviddavis over 3 years ago

Revision b50780d9 | View on GitHub

Add method field to FileSystemExporter

fixes #8695

Actions #13

Updated by daviddavis over 3 years ago

  • Status changed from POST to MODIFIED
Actions #14

Updated by pulpbot about 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF