Issue #1432
closedRevert changes in depsolve feature for RPM copy back to the 2.7 and earlier behavior
Description
In Pulp 2.7 and earlier when copying rpms between repos with depsolving enabled you would receive only the 'latest' copy of each dependency. In the mongoengine conversion that was changed to not limit the depsolving to the latest version of a dependency but all versions. The new behavior causes more units to be copied than before and yum will likely only use the latest one anyway so this is not as good. It is also a behavioral change.
This issue tracks the bringing back of the RPM depsolve behavior prior to the 2.8.0 release.
Updated by bmbouter almost 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to bmbouter
Updated by bmbouter almost 8 years ago
- Subject changed from Release note missing regarding copy depsolve feature modification not copying latest only to Revert changes in depsolve feature for RPM copy
- Description updated (diff)
- Status changed from ASSIGNED to NEW
- Assignee deleted (
bmbouter) - Triaged changed from Yes to No
- Tags deleted (
Documentation)
Updated by bmbouter almost 8 years ago
- Subject changed from Revert changes in depsolve feature for RPM copy to Revert changes in depsolve feature for RPM copy back to the 2.7 and earlier behavior
Updated by bmbouter almost 8 years ago
- Priority changed from Normal to Low
- Severity changed from 2. Medium to 1. Low
Updated by mhrivnak almost 8 years ago
The code that did this in 2.7 can be found starting here: https://github.com/pulp/pulp_rpm/blob/2.7-release/plugins/pulp_rpm/plugins/importers/yum/depsolve.py#L315
Updated by semyers almost 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to semyers
Updated by semyers almost 8 years ago
This case was covered by old unit tests that are currently being skipped. I've retrofit them to work again under mongoengine, and they indicate that this bug is not currently affecting pulp. The right honorable Mr. Bouterse suggested doing a manual end-to-end test covering this case, which I did.
I tried a couple different scenarios with the same result, but the simplest one to set up uses our tried-and-true zoo repo. After syncing the repo to make the package metadata easy to inspect, here's what I found...
Inside the zoo repo are three packages that are relevant to this case. First is walrus, which has two versions available:
> db.units_rpm.find({'name': 'walrus'}, {'name': 1, 'version': 1, '_id': 0})
{ "name" : "walrus", "version" : "0.71" }
{ "name" : "walrus", "version" : "5.21" }
Second is chimpanzee, which requires walrus and squirrel (null values in requires field list removed for brevity):
> db.units_rpm.find({'name': 'chimpanzee'}, {'name': 1, 'requires': 1, '_id': 0})
{ "name" : "chimpanzee", "requires" : [ { "name" : "squirrel" }, { "name" : "walrus" } ] }
Additionally, both versions of walrus are associated with the zoo repo.
> db.units_rpm.find({'name': {$in: ['walrus', 'chimpanzee']}}, {'name': 1})
{ "_id" : "dec1060f-06d3-474c-973b-3caa593d6214", "name" : "chimpanzee" }
{ "_id" : "568462b1-afc0-491f-b780-a251e684ba75", "name" : "walrus" }
{ "_id" : "98936883-fb1d-4ba4-be49-6b593e2811e9", "name" : "walrus" }
> db.repo_content_units.find({'repo_id': 'zoo', 'unit_id': {$in: ['dec1060f-06d3-474c-973b-3caa593d6214', '568462b1-afc0-491f-b780-a251e684ba75', '98936883-fb1d-4ba4-be49-6b593e2811e9']}}).count()
3
If I understand this bug correctly, after creating a new repo and copying chimpanzee over from zoo with dependency solving enabled, both walrus units would be copied. But here's what I see:
(pulp_rpm)[vagrant@dev pulp_rpm]$ pulp-admin rpm repo create --repo-id new
Successfully created repository [new]
(pulp_rpm)[vagrant@dev pulp_rpm]$ pulp-admin rpm repo copy rpm --from-repo-id zoo --to-repo-id new --str-eq="name=chimpanzee" --recursive
<snip>
Copied:
camel-0.1-1-noarch
chimpanzee-0.21-1-noarch
fox-1.1-2-noarch
squirrel-0.1-1-noarch
walrus-5.21-1-noarch
As seen above, chimpanzee also depends on squirrel, which depends on fox and camel. walrus 0.71 depends on whale, but walrus 5.21 does not, so all dependencies are accounted for with only the latest version of each package being copied. Here are chimapanzee's potential dependencies for reference (again, null values removed from the release for brevity):
> db.units_rpm.find({'name': {$in: ['squirrel', 'walrus', 'camel', 'fox']}}, {'name': 1, 'requires': 1, 'version': 1, '_id': 0})
{ "name" : "squirrel", "version" : "0.1", "requires" : [ { "name" : "camel" }, { "name" : "fox" } ] }
{ "name" : "camel", "version" : "0.1", "requires" : [ ] }
{ "name" : "fox", "version" : "1.1", "requires" : [ ] }
{ "name" : "walrus", "version" : "0.71", "requires" : [ { "name" : "whale" } ] }
{ "name" : "walrus", "version" : "5.21", "requires" : [ ] }
I believe that this bug is NOTABUG. The last commit to touch the code in question was a hash that might be familiar to pulp team
members by now, 944a05f7, so I don't have a ready explanation for the behavior that mhrivnak observed.
Here are the steps I took if someone wants to play along in the home game:
pulp-admin rpm repo create --repo-id zoo --feed https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
pulp-admin rpm repo sync run --repo-id zoo
pulp-admin rpm repo create --repo-id new
pulp-admin rpm repo copy rpm --from-repo-id zoo --to-repo-id new --str-eq="name=chimpanzee"
Updated by bmbouter almost 8 years ago
@smyers, great job testing this! It would appear that this is not a valid bug. I think we should get an ack from mhrivnak and then close it as NOTABUG.
Updated by semyers almost 8 years ago
Excellent, I'll follow up with mhrivnak.
In the meantime, I'm going through the tests that I revived and cleaning them up, double-checking their sanity, making sure they're useful, etc. I'll ref this issue when that goes up.
Updated by mhrivnak almost 8 years ago
Interesting! I will take a look. This is somewhat surprising, but a happy surprise.
The commit that I thought should have changed the behavior is this one: https://github.com/pulp/pulp_rpm/commit/d97b5e46adc49c8b87c67e1955a5c8204fe94dce
That work happened over a weekend in a bit of a blur, so it's possible I ended up figuring out how to preserve the prior behavior without much trouble, but forgot. :)
Updated by mhrivnak almost 8 years ago
- Assignee changed from semyers to mhrivnak
Updated by mhrivnak almost 8 years ago
- Status changed from ASSIGNED to CLOSED - NOTABUG
Agreed, NOTABUG. Apparently I did a better job with the conversion than I remembered. :) Sorry for the false alarm, and thanks for the help!
modernize yum depsolve tests
minor tweaks to depsolve and depsolve tests to function better in a mongoengine world
re #1432 https://pulp.plan.io/issues/1432