Project

Profile

Help

Issue #6693

closed

pulp_rpm copy with dest_base_version = 0 fails to copy content/create new repo version

Added by jdjeffers almost 4 years ago. Updated almost 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 72
Quarter:

Description

Here's an example of a session where the same source repo version (populated with content) is used to copy content into another repo using the dest base version of 0.

The first copy successfully copied all content units. The second copy specified a single unit (that exists in the source repo) but the copy task completes with no crreated resources (no new repo version).

Here's a trace output pulled from application logs:

        2020-05-08T14:49:25 [D|kat|] Calling API: RpmCopyApi.copy_content ...
        2020-05-08T14:49:25 [D|kat|] HTTP request body param ~BEGIN~
        | {"config":[{"source_repo_version":"/pulp/api/v3/repositories/rpm/rpm/0c72a809-e5b5-43c0-b687-8049f8005753/versions/1/","dest_repo":"/pulp/api/v3/repositories/rpm/rpm/d0326c7c-8767-4951-80be-595b54aca32f/","dest_base_version":0,"content":["/pulp/api/v3/content/rpm/packages/1141b0e0-c20a-4c8f-acb8-08eda489433e/","/pulp/api/v3/content/rpm/packages/202c8513-cddd-47e5-9004-a7e00cf9a03f/","/pulp/api/v3/content/rpm/packages/23e532c7-b2bc-4cb7-b91c-51a9846dc09d/","/pulp/api/v3/content/rpm/packages/23f99068-2fa4-4fd6-a277-303d3b53713a/","/pulp/api/v3/content/rpm/packages/2ae2a30a-bb8d-4a50-90d3-dc28805a8e23/","/pulp/api/v3/content/rpm/packages/2c5add2b-c599-4bdf-adbf-790164e88ca5/","/pulp/api/v3/content/rpm/packages/317005b2-34b5-4e19-bb48-2f34b4e7a1a8/","/pulp/api/v3/content/rpm/packages/4650ba2c-a141-4720-b509-b9477adc2348/","/pulp/api/v3/content/rpm/packages/47f4047e-ad17-47de-9683-74622bfa67f6/","/pulp/api/v3/content/rpm/packages/56c938cd-1c82-42d4-bea1-ce24e828217b/","/pulp/api/v3/content/rpm/packages/5dfb1bf0-0373-4421-92ad-7bef1fa90078/","/pulp/api/v3/content/rpm/packages/5ef359e1-7ec9-4a95-9fff-1ac05ba39440/","/pulp/api/v3/content/rpm/packages/6af8994c-f6d8-4088-b2e5-fd8e08c7e62a/","/pulp/api/v3/content/rpm/packages/6b2fbf72-dbd7-4393-980e-c0585ded7ffd/","/pulp/api/v3/content/rpm/packages/6d328de6-2542-410b-b1c2-fd1666c5903d/","/pulp/api/v3/content/rpm/packages/70a8b050-6ac4-435d-b30f-946a3f91b760/","/pulp/api/v3/content/rpm/packages/7219bd2d-e380-4a6f-a2ae-11553cbeec51/","/pulp/api/v3/content/rpm/packages/7ba12c51-951e-44c5-bb58-8feace81431d/","/pulp/api/v3/content/rpm/packages/89160b1c-9f0c-461b-ad92-4e9d1f49d02d/","/pulp/api/v3/content/rpm/packages/8b3ca359-2631-42dc-a1fd-4d31cd2e3ea7/","/pulp/api/v3/content/rpm/packages/98fb6079-11b3-4ec4-9fb6-13e466cc8e5e/","/pulp/api/v3/content/rpm/packages/a1951362-3f81-412d-a129-6da89711a836/","/pulp/api/v3/content/rpm/packages/a562ba07-8080-4ed7-9be9-21a4cf01a9ae/","/pulp/api/v3/content/rpm/packages/a708d1cf-9e20-46d4-abfc-07950a8e352c/","/pulp/api/v3/content/rpm/packages/ae344a41-baf1-41f2-b93a-83f67324ff13/","/pulp/api/v3/content/rpm/packages/aee67379-d66b-44e7-a6c4-a2b7ab254a63/","/pulp/api/v3/content/rpm/packages/bb3f3588-db36-4aeb-89d5-70329bc01b90/","/pulp/api/v3/content/rpm/packages/bda56f12-9114-42c6-aa61-21c1c75dd741/","/pulp/api/v3/content/rpm/packages/cf6314f9-c4ed-44fd-8a00-c74f28203c4a/","/pulp/api/v3/content/rpm/packages/d4ab21ee-6c1f-4304-abe5-40e292556706/","/pulp/api/v3/content/rpm/packages/d5943c51-be60-48e2-b090-5a81feafb548/","/pulp/api/v3/content/rpm/packages/f4408e80-0096-4e39-b7cd-5730e4f01157/","/pulp/api/v3/content/rpm/advisories/937fc1d0-a227-4691-a05b-e55a01b6ab3f/","/pulp/api/v3/content/rpm/advisories/52bd7427-7f95-4b7b-af17-46677ea08077/","/pulp/api/v3/content/rpm/advisories/73694a78-178b-4c1d-98ee-0755e62e4ce3/","/pulp/api/v3/content/rpm/advisories/149d10f7-818a-4c44-8de1-7a2c61011d86/"]}],"dependency_solving":false}
         | ~END~
         |
        2020-05-08T14:49:25 [D|kat|] HTTP response body ~BEGIN~
         | {"task":"/pulp/api/v3/tasks/703f6300-57d1-4de5-a721-822543c2e9e5/"}
         | ~END~
         |
         
        | Data: #
         
        2020-05-08T14:49:27 [D|kat|] Calling API: RpmCopyApi.copy_content ...
        2020-05-08T14:49:27 [D|kat|] HTTP request body param ~BEGIN~
         | {"config":[{"source_repo_version":"/pulp/api/v3/repositories/rpm/rpm/0c72a809-e5b5-43c0-b687-8049f8005753/versions/1/","dest_repo":"/pulp/api/v3/repositories/rpm/rpm/d0326c7c-8767-4951-80be-595b54aca32f/","dest_base_version":0,"content":["/pulp/api/v3/content/rpm/packages/56c938cd-1c82-42d4-bea1-ce24e828217b/"]}],"dependency_solving":false}
         | ~END~
         |
        2020-05-08T14:49:27 [D|kat|] HTTP response body ~BEGIN~
         | {"task":"/pulp/api/v3/tasks/b872eb19-6a93-4183-b196-c7de76586f05/"}
         | ~END~
         |
         | Data: #
         
         
         
         
         

Reply to "Untitled"
Author
Title
Language
Your paste - Paste your paste here
2020-05-08T14:49:25 [D|kat|] Calling API: RpmCopyApi.copy_content ...
2020-05-08T14:49:25 [D|kat|] HTTP request body param ~BEGIN~ 
| {"config":[{"source_repo_version":"/pulp/api/v3/repositories/rpm/rpm/0c72a809-e5b5-43c0-b687-8049f8005753/versions/1/","dest_repo":"/pulp/api/v3/repositories/rpm/rpm/d0326c7c-8767-4951-80be-595b54aca32f/","dest_base_version":0,"content":["/pulp/api/v3/content/rpm/packages/1141b0e0-c20a-4c8f-acb8-08eda489433e/","/pulp/api/v3/content/rpm/packages/202c8513-cddd-47e5-9004-a7e00cf9a03f/","/pulp/api/v3/content/rpm/packages/23e532c7-b2bc-4cb7-b91c-51a9846dc09d/","/pulp/api/v3/content/rpm/packages/23f99068-2fa4-4fd6-a277-303d3b53713a/","/pulp/api/v3/content/rpm/packages/2ae2a30a-bb8d-4a50-90d3-dc28805a8e23/","/pulp/api/v3/content/rpm/packages/2c5add2b-c599-4bdf-adbf-790164e88ca5/","/pulp/api/v3/content/rpm/packages/317005b2-34b5-4e19-bb48-2f34b4e7a1a8/","/pulp/api/v3/content/rpm/packages/4650ba2c-a141-4720-b509-b9477adc2348/","/pulp/api/v3/content/rpm/packages/47f4047e-ad17-47de-9683-74622bfa67f6/","/pulp/api/v3/content/rpm/packages/56c938cd-1c82-42d4-bea1-ce24e828217b/","/pulp/api/v3/content/rpm/packages/5dfb1bf0-0373-4421-92ad-7bef1fa90078/","/pulp/api/v3/content/rpm/packages/5ef359e1-7ec9-4a95-9fff-1ac05ba39440/","/pulp/api/v3/content/rpm/packages/6af8994c-f6d8-4088-b2e5-fd8e08c7e62a/","/pulp/api/v3/content/rpm/packages/6b2fbf72-dbd7-4393-980e-c0585ded7ffd/","/pulp/api/v3/content/rpm/packages/6d328de6-2542-410b-b1c2-fd1666c5903d/","/pulp/api/v3/content/rpm/packages/70a8b050-6ac4-435d-b30f-946a3f91b760/","/pulp/api/v3/content/rpm/packages/7219bd2d-e380-4a6f-a2ae-11553cbeec51/","/pulp/api/v3/content/rpm/packages/7ba12c51-951e-44c5-bb58-8feace81431d/","/pulp/api/v3/content/rpm/packages/89160b1c-9f0c-461b-ad92-4e9d1f49d02d/","/pulp/api/v3/content/rpm/packages/8b3ca359-2631-42dc-a1fd-4d31cd2e3ea7/","/pulp/api/v3/content/rpm/packages/98fb6079-11b3-4ec4-9fb6-13e466cc8e5e/","/pulp/api/v3/content/rpm/packages/a1951362-3f81-412d-a129-6da89711a836/","/pulp/api/v3/content/rpm/packages/a562ba07-8080-4ed7-9be9-21a4cf01a9ae/","/pulp/api/v3/content/rpm/packages/a708d1cf-9e20-46d4-abfc-07950a8e352c/","/pulp/api/v3/content/rpm/packages/ae344a41-baf1-41f2-b93a-83f67324ff13/","/pulp/api/v3/content/rpm/packages/aee67379-d66b-44e7-a6c4-a2b7ab254a63/","/pulp/api/v3/content/rpm/packages/bb3f3588-db36-4aeb-89d5-70329bc01b90/","/pulp/api/v3/content/rpm/packages/bda56f12-9114-42c6-aa61-21c1c75dd741/","/pulp/api/v3/content/rpm/packages/cf6314f9-c4ed-44fd-8a00-c74f28203c4a/","/pulp/api/v3/content/rpm/packages/d4ab21ee-6c1f-4304-abe5-40e292556706/","/pulp/api/v3/content/rpm/packages/d5943c51-be60-48e2-b090-5a81feafb548/","/pulp/api/v3/content/rpm/packages/f4408e80-0096-4e39-b7cd-5730e4f01157/","/pulp/api/v3/content/rpm/advisories/937fc1d0-a227-4691-a05b-e55a01b6ab3f/","/pulp/api/v3/content/rpm/advisories/52bd7427-7f95-4b7b-af17-46677ea08077/","/pulp/api/v3/content/rpm/advisories/73694a78-178b-4c1d-98ee-0755e62e4ce3/","/pulp/api/v3/content/rpm/advisories/149d10f7-818a-4c44-8de1-7a2c61011d86/"]}],"dependency_solving":false}
 | ~END~
 |
2020-05-08T14:49:25 [D|kat|] HTTP response body ~BEGIN~
 | {"task":"/pulp/api/v3/tasks/703f6300-57d1-4de5-a721-822543c2e9e5/"}
 | ~END~
 |

| Data: #

2020-05-08T14:49:27 [D|kat|] Calling API: RpmCopyApi.copy_content ...
2020-05-08T14:49:27 [D|kat|] HTTP request body param ~BEGIN~
 | {"config":[{"source_repo_version":"/pulp/api/v3/repositories/rpm/rpm/0c72a809-e5b5-43c0-b687-8049f8005753/versions/1/","dest_repo":"/pulp/api/v3/repositories/rpm/rpm/d0326c7c-8767-4951-80be-595b54aca32f/","dest_base_version":0,"content":["/pulp/api/v3/content/rpm/packages/56c938cd-1c82-42d4-bea1-ce24e828217b/"]}],"dependency_solving":false}
 | ~END~
 |
2020-05-08T14:49:27 [D|kat|] HTTP response body ~BEGIN~
 | {"task":"/pulp/api/v3/tasks/b872eb19-6a93-4183-b196-c7de76586f05/"}
 | ~END~
 |
 | Data: #

"versions": [
    {
      "component": "pulpcore",
      "version": "3.4.0.dev0"
    },
    {
      "component": "pulp_2to3_migration",
      "version": "0.2.0b2"
    },
    {
      "component": "pulp_rpm",
      "version": "3.4.0b1.dev0"
    },
    {
      "component": "pulp_file",
      "version": "0.4.0.dev0"
    },
    {
      "component": "pulp_container",
      "version": "1.4.0.dev0"
    }
  ]

pulp_2to3_migration_client (0.2.0b2)
pulp_ansible_client (0.2.0b13.dev01588633276)
pulp_container_client (1.2.0)
pulp_file_client (0.2.0)
pulp_rpm_client (3.3.0)
pulpcore_client (3.2.1)

Actions #1

Updated by dkliban@redhat.com almost 4 years ago

  • Description updated (diff)
Actions #2

Updated by dalley almost 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley
Actions #3

Updated by dalley almost 4 years ago

@jdjeffers, would you be able to apply this patch to your Pulp installation and test it?

--- a/pulp_rpm/app/tasks/copy.py
+++ b/pulp_rpm/app/tasks/copy.py
@@ -82,7 +82,8 @@ def copy_content(config, dependency_solving):
         source_repo_version = RepositoryVersion.objects.get(pk=entry["source_repo_version"])
         dest_repo = RpmRepository.objects.get(pk=entry["dest_repo"])
 
-        if entry.get("dest_base_version"):
+        dest_version_provided = bool(entry.get("dest_base_version"))
+        if dest_version_provided:
             dest_repo_version = RepositoryVersion.objects.get(pk=entry["dest_base_version"])
         else:
             dest_repo_version = dest_repo.latest_version()
@@ -96,7 +97,8 @@ def copy_content(config, dependency_solving):
             content_to_copy = source_repo_version.content.filter(content_filter)
             content_to_copy |= find_children_of_content(content_to_copy, source_repo_version)
 
-            with dest_repo.new_version() as new_version:
+            base_version = dest_repo_version if dest_version_provided else None
+            with dest_repo.new_version(base_version=base_version) as new_version:
                 new_version.add_content(content_to_copy)
 
             continue
@@ -129,5 +131,6 @@ def copy_content(config, dependency_solving):
         for from_repo, units in content_to_copy.items():
             src_repo_version = libsolv_repo_names[from_repo]
             dest_repo_version = repo_mapping[src_repo_version]
-            with dest_repo_version.repository.new_version() as new_version:
+            base_version = dest_repo_version if dest_version_provided else None
+            with dest_repo_version.repository.new_version(base_version=base_version) as new_version:
                 new_version.add_content(Content.objects.filter(pk__in=units))

If not, can you provide instructions on exactly how I should set up my repos when I attempt to reproduce it? As in, are there more repository versions present than just the ones mentioned, and which remote url was used, etc.

Actions #4

Updated by jdjeffers almost 4 years ago

I can test that out today, thank you.

Actions #5

Updated by jdjeffers almost 4 years ago

In the mean time, the repository URL used was https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/

The steps in the scenario,

Create 2 repos with that upstream URL. Sync the first repo ("source repo"), obsevere all units present.

Copy all units from source repo into the 2nd repo ("target repo"). Observer that the target repo has all the same units, and that the latest version for both repositories is now /versions/1/.

Copy a single unit from source repo into target repo, specifying the target repo base version is 0.

The copy completes, but there is no new version (target repo latest version is still /versions/1/, and the content units are all present.

Actions #6

Updated by jdjeffers almost 4 years ago

Initial testing of the suggested patch didn't seem to resolve the issue.

As a workaround (without the patch) deleting all content from the target repo (and getting a new version), seemed to work, if that helps with the scheduling this issue.

Actions #7

Updated by pulpbot almost 4 years ago

  • Status changed from ASSIGNED to POST
Actions #8

Updated by ttereshc almost 4 years ago

  • Project changed from Pulp to RPM Support

Added by dalley almost 4 years ago

Revision 204950a5 | View on GitHub

Fix an edge case with 'dest_base_version'

closes: #6693 https://pulp.plan.io/issues/6693

Actions #9

Updated by dalley almost 4 years ago

  • Status changed from POST to MODIFIED

Added by dalley almost 4 years ago

Revision 0e199dca | View on GitHub

Fix an edge case with 'dest_base_version'

closes: #6693 https://pulp.plan.io/issues/6693 (cherry picked from commit 204950a5fc4f15d05ee1970b2aa0e0f1ae6cf651)

Actions #10

Updated by dalley almost 4 years ago

Added by dalley almost 4 years ago

Revision 8a5f3504 | View on GitHub

Fix an edge case with 'dest_base_version'

closes: #6693 https://pulp.plan.io/issues/6693 (cherry picked from commit 204950a5fc4f15d05ee1970b2aa0e0f1ae6cf651)

Actions #11

Updated by dalley almost 4 years ago

Actions #12

Updated by ttereshc almost 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
  • Sprint/Milestone set to Pulp RPM 3.3.2
  • Triaged changed from No to Yes
  • Sprint set to Sprint 72

Also available in: Atom PDF