Issue #9636
closedFileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format
Description
Ticket moved to GitHub: "pulp/pulp_rpm/2321":https://github.com/pulp/pulp_rpm/issues/2321
Pulp expects custom metadata files to have a specific format for filenames: -.
During pulp-2to3-migration, users get FileNotFoundError: [Errno 2] No such file or directory: ' '
because the filename is only a checksum :/, which we remove to determine the filename, so the path we detect becomes an empty string.
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 478, in migrate_repo_distributor
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: pulp2dist, repo_version)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py", line 91, in migrate_to_pulp3
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 344, in publish
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: publication_data.populate()
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 253, in populate
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: self.repomdrecords = self.prepare_metadata_files(main_content)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 99, in prepare_metadata_files
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: with open(path, "wb") as new_file:
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: FileNotFoundError: [Errno 2] No such file or directory: ''
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: pulp: rq.worker:INFO: 27738@1002a1103081001.xxx.com: c0d58c5a-9ff1-4d40-bbb3-9c24fdf0fdb4
Nov 22 21:31:34 1002a1103081001 pulpcore-resource-manager: pulp: rq.worker:INFO: resource-manager: 43f43c1a-d09e-46bd-99fe-73b6e2ee397c
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-3: pulp: rq.worker:INFO: 27740@1002a1103081001.xxx.com: Job OK (b25f3fa2-401c-425c-92eb-b49a61415617)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-2: pulp: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=XXX-Red_Hat_Ansible_Engine_2_7_RPMs_for_Red_Hat_Enterprise_Linux_7_Server_x86_64, version=1
Some repositories like https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhv-mgmt-agent/4/os/repodata/repomd.xml have filenames in the form of UUID.
Observed with pulp2 to pulp3 migration only so far, need to test sync.
Related issues
Updated by ttereshc about 3 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ttereshc
- Sprint set to Sprint 111
Updated by ttereshc about 3 years ago
Sync is affected as well. UUID names are supported already, so only the case, when filename is just a checksum, needs to be handled.
Updated by ttereshc about 3 years ago
- Copied to Backport #9644: Backport #9636 'FileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format' to 3.11.z added
Updated by pulpbot about 3 years ago
- Status changed from ASSIGNED to POST
Added by ttereshc about 3 years ago
Updated by ttereshc about 3 years ago
- Status changed from POST to MODIFIED
Applied in changeset b7ba9350b476f9b641ca84f33f3057f37c32afb0.
Updated by ttereshc about 3 years ago
- Copied to Backport #9649: Backport #9636 'FileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format' to 3.14.z added
Updated by ttereshc about 3 years ago
- Copied to Backport #9650: Backport #9636 'FileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format' to 3.16.z added
Updated by fao89 almost 3 years ago
- Description updated (diff)
- Status changed from MODIFIED to CLOSED - DUPLICATE
Fix FileNotFoundError during sync or pulp-2to3 migration
It happens when a checksum is used as a filename for a custom repo metadata file.
All kudos for the fix go to @hao-yu.
closes #9636 https://pulp.plan.io/issues/9636