Actions
Issue #4856
closed
Uploading ISOs of same name to a repo creates duplicates, makes repo unusable
Status:
CLOSED - NOTABUG
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:
Description
Uploading an ISO file into a repo which already contains an ISO of the same name, but with different content, will cause the repo to end up with multiple units of the same name. This breaks the repo, as PULP_MANIFEST will contain two entries for the same file, making the repo unable to be synced by another Pulp instance.
Steps to reproduce¶
In dev env:
# Make an ISO repo
$ pulp-admin -u admin -p admin iso repo create --repo-id zoo-iso
# Upload a file
$ echo hamburger > food
$ pulp-admin -u admin -p admin iso repo uploads upload --repo-id zoo-iso -f food
# Upload another file of same name
$ echo pizza > food
$ pulp-admin -u admin -p admin iso repo uploads upload --repo-id zoo-iso -f food
# Publish it
$ pulp-admin -u admin -p admin iso repo publish run --repo-id zoo-iso
# Check generated PULP_MANIFEST
$ curl -k https://localhost/pulp/isos/zoo-iso/PULP_MANIFEST
food,8cec6a9cb8cb1941243b138e50ab8e2dd1da2a380dd2c7c63928003f82173d9b,10
food,fff8acd78f7528c143cb5a6971f911d3869368cbc177f3f4404d945c6accc08d,6
Actual behavior¶
- Repo contains two ISO units named "food"
- PULP_MANIFEST contains two entries named "food"
- Pulp can't sync from the published repo.
Expected behavior¶
- Repo contains one ISO unit named "food" with checksum fff8acd7... (the second uploaded file)
- PULP_MANIFEST contains one entry named "food"
- Pulp can sync from the published repo.
Additional info¶
Might be a regression from Pulp ~2.8 but I'm not sure.
Seems almost a duplicate of https://pulp.plan.io/issues/3047, but I only see sync being tested there, even though issue description mentions uploads.
Tested with pulp_rpm 0f5b2e7c.
Updated by bherring over 5 years ago
- Related to Issue #4857: Pulp 2 Nightly Regression in test_iso_crud.ISOUpdateTestCase.test_all added
Updated by ipanova@redhat.com over 5 years ago
rmcgover do you happen to have the fix in place already? or could you possibly submit a PR?
Updated by ttereshc over 5 years ago
- Related to deleted (Issue #4857: Pulp 2 Nightly Regression in test_iso_crud.ISOUpdateTestCase.test_all)
Updated by ttereshc over 5 years ago
- Related to Issue #4857: Pulp 2 Nightly Regression in test_iso_crud.ISOUpdateTestCase.test_all added
Updated by ttereshc over 5 years ago
- Related to deleted (Issue #4857: Pulp 2 Nightly Regression in test_iso_crud.ISOUpdateTestCase.test_all)
Updated by ipanova@redhat.com over 5 years ago
rmcgover it seems like Pr for this issue https://pulp.plan.io/issues/4857 is fixing the problem. I was not able to reproduce this from 2- master. Can you confirm and close?
[ipanova@ina pulp_rpm]$ pulp-admin -u admin -p admin iso repo create --repo-id zoo-iso
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/
Successfully created repository [zoo-iso]
[ipanova@ina pulp_rpm]$ echo hamburger > food
[ipanova@ina pulp_rpm]$ pulp-admin -u admin -p admin iso repo uploads upload --repo-id zoo-iso -f food
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/
+----------------------------------------------------------------------+
Unit Upload
+----------------------------------------------------------------------+
Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: food
... completed
Creating upload requests on the server...
[==================================================] 100%
Initializing: food
... completed
Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.
Uploading: food
[==================================================] 100%
10/10 bytes
... completed
Importing into the repository...
This command may be exited via ctrl+c without affecting the request.
[\]
Running...
Task Succeeded
Deleting the upload request...
... completed
[ipanova@ina pulp_rpm]$ pulp-admin iso repo isos --repo-id zoo-iso
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/
Name: food
Size: 10
Checksum: 8cec6a9cb8cb1941243b138e50ab8e2dd1da2a380dd2c7c63928003f82173d9b
[ipanova@ina pulp_rpm]$ echo pizza > food
[ipanova@ina pulp_rpm]$ pulp-admin -u admin -p admin iso repo uploads upload --repo-id zoo-iso -f food
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/
+----------------------------------------------------------------------+
Unit Upload
+----------------------------------------------------------------------+
Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: food
... completed
Creating upload requests on the server...
[==================================================] 100%
Initializing: food
... completed
Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.
Uploading: food
[==================================================] 100%
6/6 bytes
... completed
Importing into the repository...
This command may be exited via ctrl+c without affecting the request.
[\]
Running...
Task Succeeded
Deleting the upload request...
... completed
[ipanova@ina pulp_rpm]$ pulp-admin iso repo isos --repo-id zoo-iso
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/
Name: food
Size: 6
Checksum: fff8acd78f7528c143cb5a6971f911d3869368cbc177f3f4404d945c6accc08d
[ipanova@ina pulp_rpm]$ pulp-admin -u admin -p admin iso repo publish run --repo-id zoo-iso
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/
+----------------------------------------------------------------------+
Publishing Repository [zoo-iso]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
The repository was successfully published.
Task Succeeded
[ipanova@ina pulp_rpm]$ curl -k https://localhost/pulp/isos/zoo-iso/PULP_MANIFEST
food,fff8acd78f7528c143cb5a6971f911d3869368cbc177f3f4404d945c6accc08d,6
[ipanova@ina pulp_rpm]$ sudo ls -l /var/lib/pulp/published/https/isos/zoo-iso/food
lrwxrwxrwx. 1 apache apache 97 May 28 10:53 /var/lib/pulp/published/https/isos/zoo-iso/food -> /var/lib/pulp/content/units/iso/21/403956117c5aebaece1fd64e2c3ebb0d86b8123e31f9aad365ce97c7bef498
[ipanova@ina pulp_rpm]$ cat /var/lib/pulp/content/units/iso/21/403956117c5aebaece1fd64e2c3ebb0d86b8123e31f9aad365ce97c7bef498
pizza
[ipanova@ina pulp_rpm]$ echo tacos > food
[ipanova@ina pulp_rpm]$ pulp-admin -u admin -p admin iso repo uploads upload --repo-id zoo-iso -f food
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/
+----------------------------------------------------------------------+
Unit Upload
+----------------------------------------------------------------------+
Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: food
... completed
Creating upload requests on the server...
[==================================================] 100%
Initializing: food
... completed
Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.
Uploading: food
[==================================================] 100%
6/6 bytes
... completed
Importing into the repository...
This command may be exited via ctrl+c without affecting the request.
[\]
Running...
Task Succeeded
Deleting the upload request...
... completed
[ipanova@ina pulp_rpm]$ pulp-admin iso repo isos --repo-id zoo-iso
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/
Name: food
Size: 6
Checksum: e3e27097e49379044410e50f80781f63b0606eb93577a2d8ee4380eebf9b38bd
[ipanova@ina pulp_rpm]$ pulp-admin -u admin -p admin iso repo publish run --repo-id zoo-iso
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/
+----------------------------------------------------------------------+
Publishing Repository [zoo-iso]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
The repository was successfully published.
Task Succeeded
[ipanova@ina pulp_rpm]$ curl -k https://localhost/pulp/isos/zoo-iso/PULP_MANIFEST
food,e3e27097e49379044410e50f80781f63b0606eb93577a2d8ee4380eebf9b38bd,6
[ipanova@ina pulp_rpm]$ sudo ls -l /var/lib/pulp/published/https/isos/zoo-iso/food
lrwxrwxrwx. 1 apache apache 97 May 28 10:55 /var/lib/pulp/published/https/isos/zoo-iso/food -> /var/lib/pulp/content/units/iso/e1/a29a7a8b1e8ca8f06c32d2dd7c24ab337d56821a48f657965c4265e370dfd9
[ipanova@ina pulp_rpm]$ cat /var/lib/pulp/content/units/iso/e1/a29a7a8b1e8ca8f06c32d2dd7c24ab337d56821a48f657965c4265e370dfd9
tacos
[ipanova@ina pulp_rpm]$
Updated by rmcgover over 5 years ago
Yes, I couldn't reproduce it now as well.
Actually, there still seems to be a closely related issue - corrupt repo can still be created if you put ISOs into the repo by copy, rather than direct upload. e.g. this process:
- create empty repos: upload-repo, target-repo
- upload food=hamburger to upload-repo
- iso repo copy --from-repo-id upload-repo --to-repo-id target-repo
- upload food=pizza to upload-repo
- iso repo copy --from-repo-id upload-repo --to-repo-id target-repo
End result: upload-repo contains only one "food" (good) but target-repo contains two, and has broken PULP_MANIFEST.
Would it be clearer to close this issue and file a new one for that?
Updated by ipanova@redhat.com over 5 years ago
- Status changed from NEW to CLOSED - NOTABUG
rmcgover i will close this issue. I was able to reproduce copy scenario so I opened a new issue https://pulp.plan.io/issues/4892
Actions
.