Issue #6519: pulp_rpm RpmCopy api - copy method fails to copy all rpm packages with dependency solving 'on' - RPM Support - Pulp

Actions

Send by e-mail Copy link

Issue #6519

closed

pulp_rpm RpmCopy api - copy method fails to copy all rpm packages with dependency solving 'on'

Added by jdjeffers over 4 years ago. Updated over 4 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

ggainey

Sprint/Milestone:

Pulp RPM 3.3.1

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version:

Platform Release:

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Sprint:

Sprint 72

Quarter:

Description

While using the copy api with a yum repo using https://jlsherrill.fedorapeople.org/fake-repos/needed-errata/ as the upstream URL, only 31 of 32 rpm packages were copied into the target repository.

There are 2 dummy packages by the name of 'walrus'. The 'downlevel' rpm of 'walrus' was not copied.

Actions

Copy link

Updated by daviddavis over 4 years ago

Project changed from Pulp to RPM Support
Subject changed from pulp_rpm RpmCopy api - copy method fails to copy all rpm packages with depeendency solving 'on' to pulp_rpm RpmCopy api - copy method fails to copy all rpm packages with dependency solving 'on'

Actions

Copy link

Updated by jdjeffers over 4 years ago

Addition information - this is the data JSON provided to the copy api:

{
  "config": [
    {
      "source_repo_version": "/pulp/api/v3/repositories/rpm/rpm/bbe8309a-4dff-416a-870e-03b2ecd63143/versions/1/",
      "dest_repo": "/pulp/api/v3/repositories/rpm/rpm/514763db-7be8-4e7a-88df-b55daccafd51/",
      "content": [
        "/pulp/api/v3/content/rpm/packages/129908cf-8822-45e1-90f9-e50a0f1f4edc/",
        "/pulp/api/v3/content/rpm/packages/1bbeaf26-45ac-4972-822c-8d3d13f0f93e/",
        "/pulp/api/v3/content/rpm/packages/1d75f794-e9ef-454a-aae7-f7088bb35cb6/",
        "/pulp/api/v3/content/rpm/packages/1f1edd14-0c88-4617-9a1c-5bc5ed4a4d43/",
        "/pulp/api/v3/content/rpm/packages/27ed7731-3e6a-451c-8d9f-082b29992a3d/",
        "/pulp/api/v3/content/rpm/packages/37f8a1da-a798-4ef1-9647-7320585a2f3b/",
        "/pulp/api/v3/content/rpm/packages/41ea27eb-f12b-40f0-9b2a-f22a097e83c1/",
        "/pulp/api/v3/content/rpm/packages/4829aabb-c568-443b-8224-0addd385eb8b/",
        "/pulp/api/v3/content/rpm/packages/5d1ee916-65e8-40b5-a427-20737470dd45/",
        "/pulp/api/v3/content/rpm/packages/616ac763-8110-44de-9906-bc2e5f8436f0/",
        "/pulp/api/v3/content/rpm/packages/6e54e885-715a-4355-904b-0a39fb8c0673/",
        "/pulp/api/v3/content/rpm/packages/783acf38-6858-415b-b204-ff3251645ae4/",
        "/pulp/api/v3/content/rpm/packages/78f1a2a7-67e5-4af1-ac28-110f592cfdaf/",
        "/pulp/api/v3/content/rpm/packages/7ba5ae27-63ae-48c4-8102-0dc62c4ba423/",
        "/pulp/api/v3/content/rpm/packages/8465db44-8c9b-4491-b78b-087d17da9abb/",
        "/pulp/api/v3/content/rpm/packages/90a7b5a1-1730-46b9-83c3-a94b9e11c993/",
        "/pulp/api/v3/content/rpm/packages/9612b028-ec14-4ec8-a2b0-1fd2ad0bde74/",
        "/pulp/api/v3/content/rpm/packages/9a70c1ce-f0f9-4d29-914e-fbd4318fd38a/",
        "/pulp/api/v3/content/rpm/packages/a07b18f9-384b-4dd7-99d5-db0715fc0d8d/",
        "/pulp/api/v3/content/rpm/packages/a15fdb2a-9f15-4411-9961-7cb14d0023ef/",
        "/pulp/api/v3/content/rpm/packages/b30f35f9-d8d1-40a2-94aa-eeb10d4574b2/",
        "/pulp/api/v3/content/rpm/packages/bc8d47a6-d5b5-4173-b7e8-f680b7dfddcf/",
        "/pulp/api/v3/content/rpm/packages/c096bfa5-e0e0-451b-9e9e-476283734391/",
        "/pulp/api/v3/content/rpm/packages/c83856a0-c726-470b-825e-065c80e19b7d/",
        "/pulp/api/v3/content/rpm/packages/cc7b6aa2-01d1-4400-ba4b-3e41f07f47c6/",
        "/pulp/api/v3/content/rpm/packages/cf18314e-ddd5-44a7-a35a-c53144913997/",
        "/pulp/api/v3/content/rpm/packages/d9a11d6b-cd8c-464c-93dd-928a88d3989f/",
        "/pulp/api/v3/content/rpm/packages/da391b20-0940-49ce-b207-33f823349374/",
        "/pulp/api/v3/content/rpm/packages/dfa4daef-cb76-4c49-a325-b14ff4a5c42b/",
        "/pulp/api/v3/content/rpm/packages/e33690bb-e85b-46b6-b8ca-bc26cf474013/",
        "/pulp/api/v3/content/rpm/packages/e632229b-73b7-4317-a91c-3d2e9dfca632/",
        "/pulp/api/v3/content/rpm/packages/f734fd87-89af-4475-b2ed-00cd064a67bd/",
        "/pulp/api/v3/content/rpm/advisories/c4fdc73a-0260-40b1-880c-26b138b41f6f/",
        "/pulp/api/v3/content/rpm/advisories/7f1d0bbf-461f-41a9-a8f8-fb4f2fc3c424/",
        "/pulp/api/v3/content/rpm/advisories/abad23a4-5fda-4bdc-a632-5616909836ce/",
        "/pulp/api/v3/content/rpm/advisories/db1ccbed-2ff3-4c76-98ea-9d759a2b8d71/"
      ]
    }
  ],
  "dependency_solving": true
}

Actions

Copy link

Updated by ttereshc over 4 years ago

Triaged changed from No to Yes
Sprint set to Sprint 71

Actions

Copy link

Updated by ggainey over 4 years ago

Status changed from NEW to ASSIGNED
Assignee set to ggainey

From IRC discussion:

<ttereshc> dalley, ggainey, we have a dep solving issue https://pulp.plan.io/issues/6519 which needs to be fixed in the upcoming sprint (preferably in the first half, to leave some time for release). IIRC, dalley is still the only knowledge carrier for the dep solving and maybe ggainey investigated/fixed something in pulp2 related to dep solving. So I think we are in need to find someone apart from dalley who can solve dep solving issues and I think this one is
<ttereshc> a good candidate for that. I wonder if ggainey is interested or if someone else is willing to try it out. For the vacation enablement purposes (I think rchan calls it that way), we need to spread the knowledge :)
<ggainey> haven't we seen this before?
<ttereshc> it was filed couple of weeks ago
<ggainey> depsolv finds the "most appropriate" rpm and thus we fail to find the 'downlevel' one? it's ringing a faint bell
<ttereshc> define "before"
<ggainey> like, months ago
<ttereshc> maybe, was it pulp2 or pulp3
<ggainey> i'm cudgeling my branemeetz
<ggainey> pulp2
<ggainey> iirc
<ttereshc> ok, so this is pulp3
<ttereshc> so maybe it's a simialr problem
<ttereshc> ggainey, do I read it correctly that you are interested in solving that? :P
<ggainey> heh - sure
<ggainey> i'm already digging thru old notes to see if I can remember exactly what was happening in The Before Times
<dalley> ggainey, I can tell you where the problem likely is, or at least where to start looking
<ggainey> dalley: yeah - let's write things down in the issue
<ttereshc> ggainey, dalley,  it's not urgent yet and can be done next week, I just don't want you to context switch immediately.
<ttereshc> ggainey, thanks for agreeing to work on it, dalley - thanks for not solving it yourself but spread the knowledge :)
<ggainey> heheh
<ggainey> dalley: could this be related to https://bugzilla.redhat.com/show_bug.cgi?id=1784165 ? (because I think that's the bug I'm thinking of)
<ggainey> from the related issue : "This works, until we get into the situation described here, where we have a unit (chimpanzee) that has a dependency (walrus) that we haven't seen yet, and which is in the source repo. Depsolving in isolation results in chimpanzee pulling in the most up-to-date version(s) of its dependencies that the Solver can find, resulting in walrus-5.21 being pulled in from the repo."
<ggainey> hrm
<dalley> ttereshc, ggainey I'm about 90% sure the issue can be fixed by just deleting this "else" :) https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/depsolving.py#L841
<ggainey> may not be the exact problem - but it's def the area/context
<ttereshc> you are both troubles :)
<ttereshc> I'm going back to the migraiton plugin work
<dalley> ggainey, to be clear, delete the else -- and always do passthrough[repo].add(unit.pk)
<ggainey> dalley: while I have No Idea if that's the answer, that code is *definitely* the place I was thinking of looking
<ggainey> ahhh, interesting
<ggainey> a'ight, drop that in the issue and I'll pick it up and start testing
<dalley> basically, all units that were passed in, should come out the other side (be passed through) -- we shouldn't make a special case for just the units that are not dependency-solved
<ggainey> yeah zacly
<ggainey> "special cases" just make for "special bugz" :)

Actions

Copy link