Issue #6693: pulp_rpm copy with dest_base_version = 0 fails to copy content/create new repo version - RPM Support - Pulp

Actions

Send by e-mail Copy link

Issue #6693

closed

pulp_rpm copy with dest_base_version = 0 fails to copy content/create new repo version

Added by jdjeffers over 4 years ago. Updated over 4 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

dalley

Sprint/Milestone:

Pulp RPM 3.3.2

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version:

Platform Release:

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Sprint:

Sprint 72

Quarter:

Description

Here's an example of a session where the same source repo version (populated with content) is used to copy content into another repo using the dest base version of 0.

The first copy successfully copied all content units. The second copy specified a single unit (that exists in the source repo) but the copy task completes with no crreated resources (no new repo version).

Here's a trace output pulled from application logs:

        2020-05-08T14:49:25 [D|kat|] Calling API: RpmCopyApi.copy_content ...
        2020-05-08T14:49:25 [D|kat|] HTTP request body param ~BEGIN~
        | {"config":[{"source_repo_version":"/pulp/api/v3/repositories/rpm/rpm/0c72a809-e5b5-43c0-b687-8049f8005753/versions/1/","dest_repo":"/pulp/api/v3/repositories/rpm/rpm/d0326c7c-8767-4951-80be-595b54aca32f/","dest_base_version":0,"content":["/pulp/api/v3/content/rpm/packages/1141b0e0-c20a-4c8f-acb8-08eda489433e/","/pulp/api/v3/content/rpm/packages/202c8513-cddd-47e5-9004-a7e00cf9a03f/","/pulp/api/v3/content/rpm/packages/23e532c7-b2bc-4cb7-b91c-51a9846dc09d/","/pulp/api/v3/content/rpm/packages/23f99068-2fa4-4fd6-a277-303d3b53713a/","/pulp/api/v3/content/rpm/packages/2ae2a30a-bb8d-4a50-90d3-dc28805a8e23/","/pulp/api/v3/content/rpm/packages/2c5add2b-c599-4bdf-adbf-790164e88ca5/","/pulp/api/v3/content/rpm/packages/317005b2-34b5-4e19-bb48-2f34b4e7a1a8/","/pulp/api/v3/content/rpm/packages/4650ba2c-a141-4720-b509-b9477adc2348/","/pulp/api/v3/content/rpm/packages/47f4047e-ad17-47de-9683-74622bfa67f6/","/pulp/api/v3/content/rpm/packages/56c938cd-1c82-42d4-bea1-ce24e828217b/","/pulp/api/v3/content/rpm/packages/5dfb1bf0-0373-4421-92ad-7bef1fa90078/","/pulp/api/v3/content/rpm/packages/5ef359e1-7ec9-4a95-9fff-1ac05ba39440/","/pulp/api/v3/content/rpm/packages/6af8994c-f6d8-4088-b2e5-fd8e08c7e62a/","/pulp/api/v3/content/rpm/packages/6b2fbf72-dbd7-4393-980e-c0585ded7ffd/","/pulp/api/v3/content/rpm/packages/6d328de6-2542-410b-b1c2-fd1666c5903d/","/pulp/api/v3/content/rpm/packages/70a8b050-6ac4-435d-b30f-946a3f91b760/","/pulp/api/v3/content/rpm/packages/7219bd2d-e380-4a6f-a2ae-11553cbeec51/","/pulp/api/v3/content/rpm/packages/7ba12c51-951e-44c5-bb58-8feace81431d/","/pulp/api/v3/content/rpm/packages/89160b1c-9f0c-461b-ad92-4e9d1f49d02d/","/pulp/api/v3/content/rpm/packages/8b3ca359-2631-42dc-a1fd-4d31cd2e3ea7/","/pulp/api/v3/content/rpm/packages/98fb6079-11b3-4ec4-9fb6-13e466cc8e5e/","/pulp/api/v3/content/rpm/packages/a1951362-3f81-412d-a129-6da89711a836/","/pulp/api/v3/content/rpm/packages/a562ba07-8080-4ed7-9be9-21a4cf01a9ae/","/pulp/api/v3/content/rpm/packages/a708d1cf-9e20-46d4-abfc-07950a8e352c/","/pulp/api/v3/content/rpm/packages/ae344a41-baf1-41f2-b93a-83f67324ff13/","/pulp/api/v3/content/rpm/packages/aee67379-d66b-44e7-a6c4-a2b7ab254a63/","/pulp/api/v3/content/rpm/packages/bb3f3588-db36-4aeb-89d5-70329bc01b90/","/pulp/api/v3/content/rpm/packages/bda56f12-9114-42c6-aa61-21c1c75dd741/","/pulp/api/v3/content/rpm/packages/cf6314f9-c4ed-44fd-8a00-c74f28203c4a/","/pulp/api/v3/content/rpm/packages/d4ab21ee-6c1f-4304-abe5-40e292556706/","/pulp/api/v3/content/rpm/packages/d5943c51-be60-48e2-b090-5a81feafb548/","/pulp/api/v3/content/rpm/packages/f4408e80-0096-4e39-b7cd-5730e4f01157/","/pulp/api/v3/content/rpm/advisories/937fc1d0-a227-4691-a05b-e55a01b6ab3f/","/pulp/api/v3/content/rpm/advisories/52bd7427-7f95-4b7b-af17-46677ea08077/","/pulp/api/v3/content/rpm/advisories/73694a78-178b-4c1d-98ee-0755e62e4ce3/","/pulp/api/v3/content/rpm/advisories/149d10f7-818a-4c44-8de1-7a2c61011d86/"]}],"dependency_solving":false}
         | ~END~
         |
        2020-05-08T14:49:25 [D|kat|] HTTP response body ~BEGIN~
         | {"task":"/pulp/api/v3/tasks/703f6300-57d1-4de5-a721-822543c2e9e5/"}
         | ~END~
         |
         
        | Data: #
         
        2020-05-08T14:49:27 [D|kat|] Calling API: RpmCopyApi.copy_content ...
        2020-05-08T14:49:27 [D|kat|] HTTP request body param ~BEGIN~
         | {"config":[{"source_repo_version":"/pulp/api/v3/repositories/rpm/rpm/0c72a809-e5b5-43c0-b687-8049f8005753/versions/1/","dest_repo":"/pulp/api/v3/repositories/rpm/rpm/d0326c7c-8767-4951-80be-595b54aca32f/","dest_base_version":0,"content":["/pulp/api/v3/content/rpm/packages/56c938cd-1c82-42d4-bea1-ce24e828217b/"]}],"dependency_solving":false}
         | ~END~
         |
        2020-05-08T14:49:27 [D|kat|] HTTP response body ~BEGIN~
         | {"task":"/pulp/api/v3/tasks/b872eb19-6a93-4183-b196-c7de76586f05/"}
         | ~END~
         |
         | Data: #
         
         
         
         
         

Reply to "Untitled"
Author
Title
Language
Your paste - Paste your paste here
2020-05-08T14:49:25 [D|kat|] Calling API: RpmCopyApi.copy_content ...
2020-05-08T14:49:25 [D|kat|] HTTP request body param ~BEGIN~ 
| {"config":[{"source_repo_version":"/pulp/api/v3/repositories/rpm/rpm/0c72a809-e5b5-43c0-b687-8049f8005753/versions/1/","dest_repo":"/pulp/api/v3/repositories/rpm/rpm/d0326c7c-8767-4951-80be-595b54aca32f/","dest_base_version":0,"content":["/pulp/api/v3/content/rpm/packages/1141b0e0-c20a-4c8f-acb8-08eda489433e/","/pulp/api/v3/content/rpm/packages/202c8513-cddd-47e5-9004-a7e00cf9a03f/","/pulp/api/v3/content/rpm/packages/23e532c7-b2bc-4cb7-b91c-51a9846dc09d/","/pulp/api/v3/content/rpm/packages/23f99068-2fa4-4fd6-a277-303d3b53713a/","/pulp/api/v3/content/rpm/packages/2ae2a30a-bb8d-4a50-90d3-dc28805a8e23/","/pulp/api/v3/content/rpm/packages/2c5add2b-c599-4bdf-adbf-790164e88ca5/","/pulp/api/v3/content/rpm/packages/317005b2-34b5-4e19-bb48-2f34b4e7a1a8/","/pulp/api/v3/content/rpm/packages/4650ba2c-a141-4720-b509-b9477adc2348/","/pulp/api/v3/content/rpm/packages/47f4047e-ad17-47de-9683-74622bfa67f6/","/pulp/api/v3/content/rpm/packages/56c938cd-1c82-42d4-bea1-ce24e828217b/","/pulp/api/v3/content/rpm/packages/5dfb1bf0-0373-4421-92ad-7bef1fa90078/","/pulp/api/v3/content/rpm/packages/5ef359e1-7ec9-4a95-9fff-1ac05ba39440/","/pulp/api/v3/content/rpm/packages/6af8994c-f6d8-4088-b2e5-fd8e08c7e62a/","/pulp/api/v3/content/rpm/packages/6b2fbf72-dbd7-4393-980e-c0585ded7ffd/","/pulp/api/v3/content/rpm/packages/6d328de6-2542-410b-b1c2-fd1666c5903d/","/pulp/api/v3/content/rpm/packages/70a8b050-6ac4-435d-b30f-946a3f91b760/","/pulp/api/v3/content/rpm/packages/7219bd2d-e380-4a6f-a2ae-11553cbeec51/","/pulp/api/v3/content/rpm/packages/7ba12c51-951e-44c5-bb58-8feace81431d/","/pulp/api/v3/content/rpm/packages/89160b1c-9f0c-461b-ad92-4e9d1f49d02d/","/pulp/api/v3/content/rpm/packages/8b3ca359-2631-42dc-a1fd-4d31cd2e3ea7/","/pulp/api/v3/content/rpm/packages/98fb6079-11b3-4ec4-9fb6-13e466cc8e5e/","/pulp/api/v3/content/rpm/packages/a1951362-3f81-412d-a129-6da89711a836/","/pulp/api/v3/content/rpm/packages/a562ba07-8080-4ed7-9be9-21a4cf01a9ae/","/pulp/api/v3/content/rpm/packages/a708d1cf-9e20-46d4-abfc-07950a8e352c/","/pulp/api/v3/content/rpm/packages/ae344a41-baf1-41f2-b93a-83f67324ff13/","/pulp/api/v3/content/rpm/packages/aee67379-d66b-44e7-a6c4-a2b7ab254a63/","/pulp/api/v3/content/rpm/packages/bb3f3588-db36-4aeb-89d5-70329bc01b90/","/pulp/api/v3/content/rpm/packages/bda56f12-9114-42c6-aa61-21c1c75dd741/","/pulp/api/v3/content/rpm/packages/cf6314f9-c4ed-44fd-8a00-c74f28203c4a/","/pulp/api/v3/content/rpm/packages/d4ab21ee-6c1f-4304-abe5-40e292556706/","/pulp/api/v3/content/rpm/packages/d5943c51-be60-48e2-b090-5a81feafb548/","/pulp/api/v3/content/rpm/packages/f4408e80-0096-4e39-b7cd-5730e4f01157/","/pulp/api/v3/content/rpm/advisories/937fc1d0-a227-4691-a05b-e55a01b6ab3f/","/pulp/api/v3/content/rpm/advisories/52bd7427-7f95-4b7b-af17-46677ea08077/","/pulp/api/v3/content/rpm/advisories/73694a78-178b-4c1d-98ee-0755e62e4ce3/","/pulp/api/v3/content/rpm/advisories/149d10f7-818a-4c44-8de1-7a2c61011d86/"]}],"dependency_solving":false}
 | ~END~
 |
2020-05-08T14:49:25 [D|kat|] HTTP response body ~BEGIN~
 | {"task":"/pulp/api/v3/tasks/703f6300-57d1-4de5-a721-822543c2e9e5/"}
 | ~END~
 |

| Data: #

2020-05-08T14:49:27 [D|kat|] Calling API: RpmCopyApi.copy_content ...
2020-05-08T14:49:27 [D|kat|] HTTP request body param ~BEGIN~
 | {"config":[{"source_repo_version":"/pulp/api/v3/repositories/rpm/rpm/0c72a809-e5b5-43c0-b687-8049f8005753/versions/1/","dest_repo":"/pulp/api/v3/repositories/rpm/rpm/d0326c7c-8767-4951-80be-595b54aca32f/","dest_base_version":0,"content":["/pulp/api/v3/content/rpm/packages/56c938cd-1c82-42d4-bea1-ce24e828217b/"]}],"dependency_solving":false}
 | ~END~
 |
2020-05-08T14:49:27 [D|kat|] HTTP response body ~BEGIN~
 | {"task":"/pulp/api/v3/tasks/b872eb19-6a93-4183-b196-c7de76586f05/"}
 | ~END~
 |
 | Data: #

"versions": [
    {
      "component": "pulpcore",
      "version": "3.4.0.dev0"
    },
    {
      "component": "pulp_2to3_migration",
      "version": "0.2.0b2"
    },
    {
      "component": "pulp_rpm",
      "version": "3.4.0b1.dev0"
    },
    {
      "component": "pulp_file",
      "version": "0.4.0.dev0"
    },
    {
      "component": "pulp_container",
      "version": "1.4.0.dev0"
    }
  ]

pulp_2to3_migration_client (0.2.0b2)
pulp_ansible_client (0.2.0b13.dev01588633276)
pulp_container_client (1.2.0)
pulp_file_client (0.2.0)
pulp_rpm_client (3.3.0)
pulpcore_client (3.2.1)

Actions

Copy link

Updated by dkliban@redhat.com over 4 years ago

Description updated (diff)

Actions

Copy link

Updated by dalley over 4 years ago

Status changed from NEW to ASSIGNED
Assignee set to dalley

Actions

Copy link

Updated by dalley over 4 years ago

@jdjeffers, would you be able to apply this patch to your Pulp installation and test it?

--- a/pulp_rpm/app/tasks/copy.py
+++ b/pulp_rpm/app/tasks/copy.py
@@ -82,7 +82,8 @@ def copy_content(config, dependency_solving):
         source_repo_version = RepositoryVersion.objects.get(pk=entry["source_repo_version"])
         dest_repo = RpmRepository.objects.get(pk=entry["dest_repo"])
 
-        if entry.get("dest_base_version"):
+        dest_version_provided = bool(entry.get("dest_base_version"))
+        if dest_version_provided:
             dest_repo_version = RepositoryVersion.objects.get(pk=entry["dest_base_version"])
         else:
             dest_repo_version = dest_repo.latest_version()
@@ -96,7 +97,8 @@ def copy_content(config, dependency_solving):
             content_to_copy = source_repo_version.content.filter(content_filter)
             content_to_copy |= find_children_of_content(content_to_copy, source_repo_version)
 
-            with dest_repo.new_version() as new_version:
+            base_version = dest_repo_version if dest_version_provided else None
+            with dest_repo.new_version(base_version=base_version) as new_version:
                 new_version.add_content(content_to_copy)
 
             continue
@@ -129,5 +131,6 @@ def copy_content(config, dependency_solving):
         for from_repo, units in content_to_copy.items():
             src_repo_version = libsolv_repo_names[from_repo]
             dest_repo_version = repo_mapping[src_repo_version]
-            with dest_repo_version.repository.new_version() as new_version:
+            base_version = dest_repo_version if dest_version_provided else None
+            with dest_repo_version.repository.new_version(base_version=base_version) as new_version:
                 new_version.add_content(Content.objects.filter(pk__in=units))

If not, can you provide instructions on exactly how I should set up my repos when I attempt to reproduce it? As in, are there more repository versions present than just the ones mentioned, and which remote url was used, etc.

Actions

Copy link

Updated by jdjeffers over 4 years ago

I can test that out today, thank you.

Actions

Copy link

Updated by jdjeffers over 4 years ago

In the mean time, the repository URL used was https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/

The steps in the scenario,

Create 2 repos with that upstream URL. Sync the first repo ("source repo"), obsevere all units present.

Copy all units from source repo into the 2nd repo ("target repo"). Observer that the target repo has all the same units, and that the latest version for both repositories is now /versions/1/.

Copy a single unit from source repo into target repo, specifying the target repo base version is 0.

The copy completes, but there is no new version (target repo latest version is still /versions/1/, and the content units are all present.

Actions

Copy link

Updated by jdjeffers over 4 years ago

Initial testing of the suggested patch didn't seem to resolve the issue.

As a workaround (without the patch) deleting all content from the target repo (and getting a new version), seemed to work, if that helps with the scheduling this issue.

Actions

Copy link