Project

Profile

Help

Issue #4693

Module Streams not copying correctly with recursive and recursive_conservative

Added by paji@redhat.com 6 months ago. Updated 28 days ago.

Status:
NEW
Priority:
Low
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Severity:
3. High
Version:
2.19.0
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

  1. Create and sync the following yum repo (Source) -> https://partha.fedorapeople.org/test-repos/pteradactyl/
  2. Create another repo Dest which will serve as the destination repo
  3. Go to mongo and pick up a uuid for the pteradactly:2 module stream. This stream will be copied from source to dest .
  4. run the following command
    https://<fqdn>/pulp/api/v2/repositories/Dest/actions/associate/: {"source_repo_id":"Source","criteria":{"type_ids":["modulemd"],"filters":{"association":{"unit_id":{"$in":[<$MODULE UUID>]}}}},"override_config":{"recursive":true}}: {"content_type"=>"application/json", "accept"=>"application/json"}
    
  5. pulp-admin rpm repo list. Check for the number of module mds copied over by the above call.
  6. notice that with recursive set to true all the pteradactyl module streams gets copied over, instead of just pteradactly:2 and packages belonging to that
  7. Behavior is similar for recursive conservative

Related issues

Related to RPM Support - Issue #4962: Modules are incorrectly copied when their artifacts shadow the names of non-modular RPM dependencies NEW Actions
Copied to RPM Support - Test #5320: Module Streams not copying correctly with recursive and recursive_conservative ASSIGNED Actions

History

#1 Updated by dkliban@redhat.com 6 months ago

  • Version set to 2.19.0
  • Tags Pulp 2 added

#2 Updated by ttereshc 6 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 52

#3 Updated by ttereshc 6 months ago

Very-nice-to-have in 2.19.1 but not critical.

#4 Updated by dkliban@redhat.com 6 months ago

  • Platform Release set to 2.19.1

#5 Updated by dalley 6 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley

#6 Updated by dkliban@redhat.com 6 months ago

  • Sprint/Milestone set to 2.19.1

#7 Updated by rchan 5 months ago

  • Sprint changed from Sprint 52 to Sprint 53

#8 Updated by dkliban@redhat.com 5 months ago

  • Sprint/Milestone deleted (2.19.1)
  • Platform Release deleted (2.19.1)

#9 Updated by dalley 5 months ago

I've been looking into this for a while and the behavior here is definitely wrong in multiple different ways, which is making it extra tricky to figure out where the problems are coming from.

Recursive copy by name, stream filter

[vagrant@pulp2 ~]$ pulp-admin rpm repo copy module --str-eq="name=pteradactyl" --str-eq="stream=2" --from-repo-id=test --to-repo-id=test3 --recursive
This command may be exited via ctrl+c without affecting the request.

[\]
Running...

Copied:
 modulemd:
  pteradactyl-1-20180730223407-deadbeef-noarch
  pteradactyl-1-20180730223408-deadbeef-noarch
  pteradactyl-2-20180730223407-deadbeef-noarch
  pteradactyl-2-20180730223408-deadbeef-noarch
  pteradactyl-3-20180730333407-deadbeef-noarch
  pteradactyl-3-20180730333408-deadbeef-noarch
  pteradactyl-4-20180740444407-deadbeef-noarch
  pteradactyl-4-20180740444408-deadbeef-noarch
 rpm:
  pteradactyl-2.0-1-noarch
  pteradactyl-2.1-1-noarch
  pteradactyl-4.1-1-noarch

Recursive copy by unit ID

[vagrant@pulp2 ~]$ pulp-admin rpm repo copy module --str-eq="_id=a0ed500f-9efe-4e23-a3c6-72c9e1a0f53c" --from-repo-id=test --to-repo-id=test3 --recursive
This command may be exited via ctrl+c without affecting the request.

[\]
Running...

Copied:
 modulemd:
  pteradactyl-1-20180730223407-deadbeef-noarch
  pteradactyl-1-20180730223408-deadbeef-noarch
  pteradactyl-2-20180730223407-deadbeef-noarch
  pteradactyl-2-20180730223408-deadbeef-noarch
  pteradactyl-3-20180730333407-deadbeef-noarch
  pteradactyl-3-20180730333408-deadbeef-noarch
  pteradactyl-4-20180740444407-deadbeef-noarch
  pteradactyl-4-20180740444408-deadbeef-noarch
 rpm:
  pteradactyl-2.1-1-noarch
  pteradactyl-4.1-1-noarch

Conservative Recursive copy by name, stream filter

[vagrant@pulp2 ~]$ pulp-admin rpm repo copy module --str-eq="name=pteradactyl" --str-eq="stream=2" --from-repo-id=test --to-repo-id=test3 --recursive-conservative
This command may be exited via ctrl+c without affecting the request.

Running...

Copied:
 modulemd:
  pteradactyl-1-20180730223407-deadbeef-noarch
  pteradactyl-1-20180730223408-deadbeef-noarch
  pteradactyl-2-20180730223407-deadbeef-noarch
  pteradactyl-2-20180730223408-deadbeef-noarch
  pteradactyl-3-20180730333407-deadbeef-noarch
  pteradactyl-3-20180730333408-deadbeef-noarch
  pteradactyl-4-20180740444407-deadbeef-noarch
  pteradactyl-4-20180740444408-deadbeef-noarch
 rpm:
  pteradactyl-2.0-1-noarch
  pteradactyl-2.1-1-noarch

Conservative Recursive copy by unit ID

[vagrant@pulp2 ~]$ pulp-admin rpm repo copy module --str-eq="_id=a0ed500f-9efe-4e23-a3c6-72c9e1a0f53c" --from-repo-id=test --to-repo-id=test3 --recursive-conservative
This command may be exited via ctrl+c without affecting the request.

[\]
Running...

Copied:
 modulemd:
  pteradactyl-1-20180730223407-deadbeef-noarch
  pteradactyl-1-20180730223408-deadbeef-noarch
  pteradactyl-2-20180730223407-deadbeef-noarch
  pteradactyl-2-20180730223408-deadbeef-noarch
  pteradactyl-3-20180730333407-deadbeef-noarch
  pteradactyl-3-20180730333408-deadbeef-noarch
  pteradactyl-4-20180740444407-deadbeef-noarch
  pteradactyl-4-20180740444408-deadbeef-noarch
 rpm:
  pteradactyl-2.1-1-noarch

So:

  • Copying one module will copy all modules sharing the same name, regardless of the stream or version, and regardless of whether you filter by unit ID or using metadata fields.
  • The actual RPMs being copied do actually seem to be the RPMs for the modules that we do want copied, but I'm not sure that's a good thing. The RPMs should ALWAYS be copied along with their modules, so even if the modules are copied incorrectly, we should see their RPMs also being copied so that at least the repository will always be complete and valid. But that isn't happening.
  • Additionally in recursive (non-conservative) mode it's also copying the most recent RPM from any of the modules (4.1.1) in addition to the RPMs it should be copying, which should not be happening

Considering how messed up this is, I think I'm going to start working with QE to get these behaviors more rigorously tested before I do any more stepping through in a debugger. I'm also going to bump the priority and severity.

#10 Updated by dalley 5 months ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High

#11 Updated by dalley 5 months ago

  • Platform Release set to 2.20.0

#12 Updated by amacdona@redhat.com 4 months ago

  • Sprint changed from Sprint 53 to Sprint 54

#13 Updated by dalley 4 months ago

After further investigation I've found that some of the metadata on the pteradactyl RPMs used by this test repo is either invalid, or at least very atypical.

I'm going to try recreating this test repo in a more standard way and see if we still have any issues with Pulp.

#14 Updated by dalley 4 months ago

Update: The form of "provides" in the pterodactyl test repo is indeed very uncommon, but it is apparently possible and valid. I was able to find a single module in the entire Fedora Module repo against which this bug could be reproduced.

(pulp) [vagrant@pulp2 devel]$ pulp-admin rpm repo copy module --str-eq="name=openmpi" --str-eq="stream=2.1" --from-repo-id=module1 --to-repo-id=test2 --recursive-conservative
This command may be exited via ctrl+c without affecting the request.

[-]
Running...

Copied:
  modulemd: 3
  rpm: 138

One module stream selected, 3 modules copied. With that said, it's probably OK to knock the priority of this back down.

#15 Updated by dalley 4 months ago

  • Priority changed from High to Normal
  • Severity changed from 3. High to 2. Medium

#16 Updated by dalley 4 months ago

Other bugs, maybe related, maybe not :(

(pulp) [vagrant@pulp2 devel]$ pulp-admin rpm repo copy module --str-eq="name=nodejs" --str-eq="stream=10" --from-repo-id=module1 --to-repo-id=test2 --recursive-conservative
This command may be exited via ctrl+c without affecting the request.

[-]
Running...

Copied:
 modulemd:
  nodejs-10-3020190424132100-a5b0195c-x86_64
  nodejs-12-3020190516000906-a5b0195c-x86_64
 rpm:
  c-ares-1.15.0-3.module_f30+3988+b4cb7ecc-x86_64
  http-parser-2.9.2-1.module_f30+4047+77c83006-x86_64
  http-parser-devel-2.9.2-1.module_f30+4047+77c83006-x86_64
  libnghttp2-1.38.0-1.module_f30+4047+77c83006-x86_64
  libnghttp2-devel-1.38.0-1.module_f30+4047+77c83006-x86_64
  libuv-1.28.0-1.module_f30+4047+77c83006-x86_64
  libuv-devel-1.28.0-1.module_f30+4047+77c83006-x86_64
  libuv-static-1.28.0-1.module_f30+4047+77c83006-x86_64
  nghttp2-1.38.0-1.module_f30+4047+77c83006-x86_64
  nodejs-10.15.3-2.module_f30+4047+77c83006-x86_64
  nodejs-devel-10.15.3-2.module_f30+4047+77c83006-x86_64
  nodejs-docs-10.15.3-2.module_f30+4047+77c83006-noarch
  nodejs-libs-10.15.3-2.module_f30+4047+77c83006-x86_64
  npm-6.4.1-1.10.15.3.2.module_f30+4047+77c83006-x86_64
  v8-devel-6.8.275.32-2.module_f30+4047+77c83006-x86_64

Note: NodeJS 10 was copied, only nodejs 10 RPMs were copied, but the nodejs 12 stream module was additionally copied. However there is also a nodejs 11 stream which was not copied. So unlike Pterodactyl, in this instance all of the modulemds were not copied regardless of stream.

#17 Updated by dalley 4 months ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High

Now that it's confirmed that these bugs are not confined to that one package, raising the priority again :angry_face:

#18 Updated by bherring 4 months ago

  • Copied to Test #4922: Module Streams not copying correctly with recursive and recursive_conservative added

#19 Updated by dalley 4 months ago

Updated notes and observations here, it is unweildy to track it all in in the comments: https://etherpad.net/p/pulp-modularity-issues

I'm unsure whether we should split this issue into multiple ones or not because it's difficult to know what the boundaries between the different defects are without knowing exactly why they're happening. There are certainly multiple different problems though.

#20 Updated by dalley 4 months ago

  • Related to Issue #4962: Modules are incorrectly copied when their artifacts shadow the names of non-modular RPM dependencies added

#21 Updated by ttereshc 4 months ago

  • Sprint changed from Sprint 54 to Sprint 55

#23 Updated by bherring 4 months ago

  • Copied to deleted (Test #4922: Module Streams not copying correctly with recursive and recursive_conservative )

#24 Updated by rchan 4 months ago

Fixing bz link

#25 Updated by ttereshc 4 months ago

  • Sprint/Milestone set to 2.20.0

#26 Updated by ttereshc 4 months ago

  • Sprint/Milestone deleted (2.20.0)
  • Platform Release deleted (2.20.0)

#27 Updated by dkliban@redhat.com 3 months ago

  • Sprint changed from Sprint 55 to Sprint 56

#28 Updated by rchan 2 months ago

  • Sprint changed from Sprint 56 to Sprint 57

#29 Updated by dalley 2 months ago

When using this PR, almost everything previously mentioned is fixed, and all the serious problems previously mentioned are fixed.

https://github.com/pulp/pulp_rpm/pull/1400

The remaining issue is that when copying one stream from pteradactyl, all streams get copied -- but unlike before, all of the artifacts for all streams are copied correctly, so an incomplete repo is not being created in that circumstance like it was previously. Additionally my "in the wild" example of this behavior now works fine.

Given that this example repo is atypical and created by a random tool off github, I think it's safe to de-prioritize for now, as long as you're OK with that Partha. Is this blocking anything?

#30 Updated by dalley about 2 months ago

  • Platform Release set to 2.21.0

#31 Updated by bherring about 2 months ago

  • Copied to Test #5320: Module Streams not copying correctly with recursive and recursive_conservative added

#32 Updated by rchan about 2 months ago

  • Sprint changed from Sprint 57 to Sprint 58

#33 Updated by dalley about 1 month ago

  • Status changed from ASSIGNED to NEW
  • Priority changed from High to Low
  • Platform Release deleted (2.21.0)

As far as I can tell, it is just this test fixture that has this problem, I am no longer able to reproduce it with any modules in an actual Fedora release even though I was able to with one of them previously.

As far as bugs go, since it's now copying (exclusively) too much and not too little, we can deprioritize this as it will not result in a broken repo if it does occur.

#34 Updated by rchan 28 days ago

  • Sprint deleted (Sprint 58)

Please register to edit this issue

Also available in: Atom PDF