Project

Profile

Help

Issue #5658

closed

Copying modulemd_defaults over an uploaded unit fails: Not a directory

Added by rmcgover over 2 years ago. Updated about 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.20.0
Platform Release:
2.21.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 63
Quarter:

Description

Copying a modulemd_defaults unit from one repo to another, where a modulemd_defaults unit with the same "name" was formerly uploaded into the target repo, fails with "[Errno 20] Not a directory".

Steps to reproduce

In dev env:

0. Prepare two modulemd_defaults YAML with same name:

[vagrant@pulp2 ~]$ cat mod1.yaml 
---
document: modulemd-defaults
version: 1
data:
  module: varnish
  stream: 6
  profiles:
    6: [common]
...

[vagrant@pulp2 ~]$ cat mod2.yaml 
---
document: modulemd-defaults
version: 1
data:
  module: varnish
  stream: 7
  profiles:
    7: [common]
...

And ensure a clean RPM repo exists for target of copy (e.g. zoo2):

pulp-admin  rpm repo create --repo-id zoo2

1. Upload a modulemd_defaults to zoo

$ curl -k -u admin:admin -X POST https://localhost/pulp/api/v2/content/uploads/
{"upload_id": "7a55199f-59aa-49e0-b808-26bee49b0c01", "_href": "/pulp/api/v2/content/uploads/7a55199f-59aa-49e0-b808-26bee49b0c01/"}

$ curl -k -u admin:admin -X PUT --data-binary @mod1.yaml https://localhost/pulp/api/v2/content/uploads/7a55199f-59aa-49e0-b808-26bee49b0c01/0/

$ curl -v -k -u admin:admin -X POST -H 'Content-Type: application/json' -d '{"upload_id":"7a55199f-59aa-49e0-b808-26bee49b0c01", "unit_type_id": "modulemd_defaults", "unit_key": {}}' https://localhost/pulp/api/v2/repositories/zoo/actions/import_upload/

# ...and wait for task to succeed

2. Upload a modulemd_defaults to zoo2

Similar commands as step 1, but use the other YAML file and repo.

$ curl -k -u admin:admin -X POST https://localhost/pulp/api/v2/content/uploads/
$ curl -k -u admin:admin -X PUT --data-binary @mod2.yaml https://localhost/pulp/api/v2/content/uploads/4c445ccc-a96d-4212-97d9-d60bddd642ce/0/
$ curl -v -k -u admin:admin -X POST -H 'Content-Type: application/json' -d '{"upload_id":"4c445ccc-a96d-4212-97d9-d60bddd642ce", "unit_type_id":"modulemd_defaults", "unit_key": {}}' https://localhost/pulp/api/v2/repositories/zoo2/actions/import_upload/
# ...and wait for task to succeed

After this step, zoo2 contains this association:

    {
        "_id": {
            "$oid": "5dbf63be741ab32483de58b5"
        },
        "created": "2019-11-03T23:33:18Z",
        "metadata": {
            "_content_type_id": "modulemd_defaults",
            "_id": "ff24ba39-925b-42fd-af6e-19cfdb74eac4",
            "_last_updated": 1572823998,
            "_ns": "units_modulemd_defaults",
            "_storage_path": "/var/lib/pulp/content/units/modulemd_defaults/75/593dd2cf41e10676befac02877d4f4da4d854cbc648d4dfb1a61ca19986dff",
            "checksum": "152166f328559f2c1b4bcd403e47801a1092bc86d93560cff723653b9cb57c16",
            "downloaded": true,
            "name": "varnish",
            "profiles": {
                "7": [
                    "common"
                ]
            },
            "pulp_user_metadata": {},
            "repo_id": "zoo2",
            "stream": "7"
        },
        "repo_id": "zoo2",
        "unit_id": "ff24ba39-925b-42fd-af6e-19cfdb74eac4",
        "unit_type_id": "modulemd_defaults",
        "updated": "2019-11-03T23:33:18Z"
    }

3. Copy from zoo to zoo2

$ pulp-admin  rpm repo copy all --from-repo-id zoo --to-repo-id zoo2
This command may be exited via ctrl+c without affecting the request.

[\]
Running...

Task Failed

[Errno 20] Not a directory:
'/var/lib/pulp/content/units/modulemd_defaults/75/593dd2cf41e10676befac02877d4f4
da4d854cbc648d4dfb1a61ca19986dff/tmpxUWq1E'

Actual results

1. Copy task fails:

pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416) Exception from importer [yum_importer] while importing units into repository [zoo2]
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416) Traceback (most recent call last):
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp/server/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)     units=transfer_units)
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/importer.py", line 60, in import_units
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)     return associate.associate(source_repo, dest_repo, import_conduit, config, units)
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/associate.py", line 72, in associate
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)     for unit in units if not isinstance(unit, models.RPM)
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/associate.py", line 471, in _associate_unit
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)     return associate_copy_for_repo(unit, dest_repo, True)
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/associate.py", line 526, in associate_copy_for_repo
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)     new_unit.safe_import_content(unit._storage_path)
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp/server/pulp/server/db/model/__init__.py", line 941, in safe_import_content
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)     self.import_content(path, location)
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp/server/pulp/server/db/model/__init__.py", line 915, in import_content
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)     storage.put(self, path, location)
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp/server/pulp/server/content/storage.py", line 128, in put
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)     fd, temp_destination = tempfile.mkstemp(dir=os.path.dirname(destination))
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)   File "/usr/lib64/python2.7/tempfile.py", line 314, in mkstemp
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)     return _mkstemp_inner(dir, prefix, suffix, flags)
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)   File "/usr/lib64/python2.7/tempfile.py", line 244, in _mkstemp_inner
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416)     fd = _os.open(file, flags, 0600)
pulp.server.managers.repo.unit_association:ERROR: [b368ddc9] (20873-28416) OSError: [Errno 20] Not a directory: '/var/lib/pulp/content/units/modulemd_defaults/75/593dd2cf41e10676befac02877d4f4da4d854cbc648d4dfb1a61ca19986dff/tmpxUWq1E'
pulp.server.async.tasks:INFO: [b368ddc9] Task failed : [b368ddc9-4701-45e4-8eb2-32b5fa9c6fa2]
celery.app.trace:ERROR: [b368ddc9] (20873-28416) Task pulp.server.managers.repo.unit_association.associate_from_repo[b368ddc9-4701-45e4-8eb2-32b5fa9c6fa2] raised unexpected: OSError(20, 'Not a directory')
celery.app.trace:ERROR: [b368ddc9] (20873-28416) Traceback (most recent call last):
celery.app.trace:ERROR: [b368ddc9] (20873-28416)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
celery.app.trace:ERROR: [b368ddc9] (20873-28416)     R = retval = fun(*args, **kwargs)
celery.app.trace:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 686, in __call__
celery.app.trace:ERROR: [b368ddc9] (20873-28416)     return super(Task, self).__call__(*args, **kwargs)
celery.app.trace:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 108, in __call__
celery.app.trace:ERROR: [b368ddc9] (20873-28416)     return super(PulpTask, self).__call__(*args, **kwargs)
celery.app.trace:ERROR: [b368ddc9] (20873-28416)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
celery.app.trace:ERROR: [b368ddc9] (20873-28416)     return self.run(*args, **kwargs)
celery.app.trace:ERROR: [b368ddc9] (20873-28416)   File "/home/vagrant/devel/pulp/server/pulp/server/managers/repo/unit_association.py", line 291, in associate_from_repo
celery.app.trace:ERROR: [b368ddc9] (20873-28416)     raise (e, None, sys.exc_info()[2])
celery.app.trace:ERROR: [b368ddc9] (20873-28416) OSError: [Errno 20] Not a directory: '/var/lib/pulp/content/units/modulemd_defaults/75/593dd2cf41e10676befac02877d4f4da4d854cbc648d4dfb1a61ca19986dff/tmpxUWq1E'

2. Target repo is left with no modulemd_defaults unit for the relevant "name" (i.e. the unit which was formerly present, is removed).

3. Filesystem is left with a file under content/units/modulemd_defaults which is not referenced by any existing unit (and so can't be cleaned up using Pulp's API).

Expected results

Copy task succeeds.

Additional info

The issue seems to be caused by the fact that two different storage path structures are used for modulemd_defaults units, depending on whether a unit was uploaded or associated.

Where "digest" is derived from the unit key (repo_id, name), then:

- if unit was uploaded, it's stored at content/units/<unit_type>/<digest[:2]>/<digest[2:]>
- if unit was associated, it's stored at content/units/<unit_type>/<digest[:2]>/<digest[2:]>/<basename-from-src>

So, in the upload case, the leading portion of the path "content/units/<unit_type>/<digest[0:2]>/<digest[2:]>" is used as a file, while in the associate case it's used as a directory (with no logic for detecting if the same path was formerly used as a file).

Tested with pulp 706d6c14b8adb9db034866, pulp_rpm cc35af04da2b97.

Actions #2

Updated by ttereshc over 2 years ago

  • Tags Pulp 2 added
Actions #3

Updated by ipanova@redhat.com over 2 years ago

here is the code that appends basename-from-src during the copy https://github.com/pulp/pulp_rpm/blob/2-master/plugins/pulp_rpm/plugins/importers/yum/associate.py#L507
and here we create tempfle expecting the directory but instead we find a file https://github.com/pulp/pulp/blob/2-master/server/pulp/server/content/storage.py#L128

Actions #4

Updated by ggainey over 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ggainey
Actions #5

Updated by ttereshc over 2 years ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 61
Actions #6

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 61 to Sprint 62
Actions #7

Updated by ggainey over 2 years ago

  • Status changed from ASSIGNED to POST

Great prob-dtermination info - thanks folks.

Taught pulp to handle the exception case, in code that was already handling an exception-case. Not very elegant, but appears to get the job done.

Submitted PR:

https://github.com/pulp/pulp/pull/3966

Actions #8

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 62 to Sprint 63

Added by ggainey over 2 years ago

Revision bbc84606

Fix modulemd_defaults upload/copy collision traceback

closes #5658

Actions #9

Updated by ggainey over 2 years ago

  • Status changed from POST to MODIFIED
Actions #10

Updated by ipanova@redhat.com over 2 years ago

  • Platform Release set to 2.21.1

Added by ggainey about 2 years ago

Revision f8d80ebf

Fix modulemd_defaults upload/copy collision traceback

closes #5658

(cherry picked from commit bbc84606ce1ed033b5465a44ffeac38d74d712e6)

Actions #12

Updated by ipanova@redhat.com about 2 years ago

  • Status changed from MODIFIED to 5
Actions #13

Updated by ipanova@redhat.com about 2 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

Also available in: Atom PDF