Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #9636

closed

FileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format

Added by ttereshc almost 3 years ago. Updated almost 3 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
ttereshc
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 111
Quarter:

Description

Ticket moved to GitHub: "pulp/pulp_rpm/2321":https://github.com/pulp/pulp_rpm/issues/2321

Pulp expects custom metadata files to have a specific format for filenames: -.

During pulp-2to3-migration, users get FileNotFoundError: [Errno 2] No such file or directory: ' ' because the filename is only a checksum :/, which we remove to determine the filename, so the path we detect becomes an empty string.

Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 478, in migrate_repo_distributor
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: pulp2dist, repo_version)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py", line 91, in migrate_to_pulp3
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 344, in publish
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: publication_data.populate()
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 253, in populate
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: self.repomdrecords = self.prepare_metadata_files(main_content)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 99, in prepare_metadata_files
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: with open(path, "wb") as new_file:
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: FileNotFoundError: [Errno 2] No such file or directory: ''
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-1: pulp: rq.worker:INFO: 27738@1002a1103081001.xxx.com: c0d58c5a-9ff1-4d40-bbb3-9c24fdf0fdb4
Nov 22 21:31:34 1002a1103081001 pulpcore-resource-manager: pulp: rq.worker:INFO: resource-manager: 43f43c1a-d09e-46bd-99fe-73b6e2ee397c
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-3: pulp: rq.worker:INFO: 27740@1002a1103081001.xxx.com: Job OK (b25f3fa2-401c-425c-92eb-b49a61415617)
Nov 22 21:31:34 1002a1103081001 pulpcore-worker-2: pulp: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=XXX-Red_Hat_Ansible_Engine_2_7_RPMs_for_Red_Hat_Enterprise_Linux_7_Server_x86_64, version=1

Some repositories like https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhv-mgmt-agent/4/os/repodata/repomd.xml have filenames in the form of UUID.

Observed with pulp2 to pulp3 migration only so far, need to test sync.

Related issues

Copied to RPM Support - Backport #9644: Backport #9636 'FileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format' to 3.11.z CLOSED - DUPLICATE

Actions
Copied to RPM Support - Backport #9649: Backport #9636 'FileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format' to 3.14.zCLOSED - CURRENTRELEASEdalley

Actions
Copied to RPM Support - Backport #9650: Backport #9636 'FileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format' to 3.16.zCLOSED - CURRENTRELEASEdalley

Actions
Actions
Copy link
 #1

Updated by ttereshc almost 3 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by ttereshc almost 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc
  • Sprint set to Sprint 111
Actions
Copy link
 #3

Updated by ttereshc almost 3 years ago

Sync is affected as well. UUID names are supported already, so only the case, when filename is just a checksum, needs to be handled.

Actions
Copy link
 #4

Updated by ttereshc almost 3 years ago

  • Copied to Backport #9644: Backport #9636 'FileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format' to 3.11.z added
Actions
Copy link
 #5

Updated by pulpbot almost 3 years ago

  • Status changed from ASSIGNED to POST

Added by ttereshc almost 3 years ago

Revision b7ba9350 | View on GitHub

Fix FileNotFoundError during sync or pulp-2to3 migration

It happens when a checksum is used as a filename for a custom repo metadata file.

All kudos for the fix go to @hao-yu.

closes #9636 https://pulp.plan.io/issues/9636

Actions
Copy link
 #7

Updated by ttereshc almost 3 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #8

Updated by ttereshc almost 3 years ago

  • Copied to Backport #9649: Backport #9636 'FileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format' to 3.14.z added
Actions
Copy link
 #9

Updated by ttereshc almost 3 years ago

  • Copied to Backport #9650: Backport #9636 'FileNotFoundError: [Errno 2] No such file or directory: '' if custom repo metadata file does not have an expected filename format' to 3.16.z added
Actions
Copy link
 #10

Updated by fao89 almost 3 years ago

  • Description updated (diff)
  • Status changed from MODIFIED to CLOSED - DUPLICATE
Actions
Send by e-mail Copy link

Also available in: Atom PDF