Issue #8695
closedFileSystemExporter cannot cross os boundary
Description
When I run the test: pytest -v -r sx --color=yes --pyargs pulpcore.tests.functional.api.using_plugin.test_filesystemexport::FilesystemExportTestCase::test_delete -x
on Fedora33 I receive the error:
E pulp_smash.pulp3.bindings.PulpTaskError: (PulpTaskError(...), "Pulp task failed ([Errno 18] Invalid cross-device link: '/var/lib/pulp/media/artifact/de/1e4813fedff7ee0223d63c4aaba1b9b3399cbff0edf059083be7ab1a157f9d' -> '/tmp/10be1fa3-87e1-49f9-8653-27d4df1dfd2c/2.iso')")'
The suggestion came up that it should us a symlink not a hardlink. Here is the background IRC convo:
2021-05-04 11:33:22 dalley in general though, it's the kind of error you get when you try to move or link a file between two separate filesystems
2021-05-04 11:33:28 bmbouter I installed last night so I thought I didn't get it yet
2021-05-04 11:33:29 dalley and /tmp is a separate one I believe
2021-05-04 11:33:29 ggainey (this isn't PulpExport, it's another kind-of export, fwiw)
2021-05-04 11:33:48 bmbouter that's actually what I'm going to try to fix (the django based issue)
2021-05-04 11:34:40 bmbouter yeah I have Django==2.2.20
2021-05-04 11:35:55 bmbouter dalley: lgtm, https://github.com/pulp/pulpcore/pull/1302#pullrequestreview-651382325
2021-05-04 11:37:08 dalley bmbouter, do I need to say something about the signature of ContentAssociation stage changing or should I just add a default
2021-05-04 11:37:49 dalley or, honestly, should we just remove both from the API
2021-05-04 11:37:56 ggainey ok, so, filesystemexporter 'assumes' you can link betwen where the artifacts are, and where you told it to export to : https://github.com/pulp/pulpcore/blob/master/pulpcore/app/models/exporter.py#L99-L104
2021-05-04 11:38:05 ggainey bmbouter: ^^
2021-05-04 11:38:27 ggainey I haven't seen the error before tho, fwiw
2021-05-04 11:38:38 bmbouter I think it has to be with me doing this in a vagrant env
2021-05-04 11:39:17 ggainey yeah, could be
2021-05-04 11:39:34 ggainey hrm, which vagrant box are you on? lemme run the test on one of mine
2021-05-04 11:39:51 dalley I've been dealing with similar issues in my side project. trying to recursively copy files a directory /tmp to somewhere in /home is sadly much more difficult than one would hope
2021-05-04 11:41:30 ggainey I have a cat in my ofc snoring so loudly I can hear them thru my headphones :)
2021-05-04 11:42:55 bmbouter ha
2021-05-04 11:42:59 ggainey bmbouter: oh fun - test runs fine on my centos7 vagrant, fails on my f33 w/your error
2021-05-04 11:43:10 bmbouter I'm on pulp3-source-fedora33
2021-05-04 11:43:39 ggainey yeah same
2021-05-04 11:43:47 bmbouter well that's comforting (no joke)
2021-05-04 11:43:54 ggainey and the 2to3 centos box doesn't have /tmp on its own FS
2021-05-04 11:44:00 bmbouter mmm
2021-05-04 11:44:12 bmbouter so what do you think we should do for resolution for dev envs
2021-05-04 11:44:24 bmbouter I mean the test isn't wrong par-se
2021-05-04 11:44:36 bmbouter probably something about the f33 env is wrong...
2021-05-04 11:46:53 <-- quba42 (~qubap42@aftr-82-135-82-151.dynamic.mnet-online.de) has quit (Quit: Leaving)
2021-05-04 11:47:42 ggainey bmbouter: hrm - one could make the case that the code should be using os.symlink() instead of creating a hardlink
2021-05-04 11:48:30 ggainey because right now it makes Assumptions about the partitioning of your system, that it prob shouldn't
2021-05-04 11:49:17 ggainey having /tmp on its own FS isn't terribly uncommon - and having F33 set up this way, has shown us a potential problem, so yay? :)
Related issues
Updated by daviddavis over 3 years ago
In the past, we've discussed adding an option to FileSystemExporter to toggle between symlink/hardlink. Perhaps that's a solution to this problem?
Updated by fao89 over 3 years ago
- Triaged changed from No to Yes
- Sprint set to Sprint 96
Updated by daviddavis over 3 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to daviddavis
Updated by pulpbot over 3 years ago
- Status changed from ASSIGNED to POST
Updated by daviddavis over 3 years ago
- Blocked by Task #8860: Move FilesystemExporter out of plugin API into pulpcore added
Updated by ipanova@redhat.com over 3 years ago
- Sprint changed from Sprint 101 to Sprint 102
Added by daviddavis over 3 years ago
Updated by daviddavis over 3 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulpcore|b50780d93c2b6b070caf478b1a43c1dae8376abf.
Updated by pulpbot over 3 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Add method field to FileSystemExporter
fixes #8695