Project

Profile

Help

Issue #7549

Need a dry run option for import/export api

Added by paji@redhat.com about 1 month ago. Updated 15 days ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Quarter:
Q4-2020

Description

Katello checks things like

  • Is the import path exists
  • Does it have the right permissions

It would be ideal if pulp provided a dry-run operation for import and export operations. Katello can then use that to figure out the correct course of action and not create stuff if its broken,

History

#1 Updated by fao89 about 1 month ago

  • Triaged changed from No to Yes
  • Quarter set to Q4-2020

#2 Updated by ggainey 15 days ago

Discussion from IRC today:

<bmbouter> partha jsherrill do you know the answer to this quesiton about how katello sets pulp settings? https://pulp.plan.io/issues/7623#note-3
<daviddavis> bmbouter: we talked about this already and they're set by the foreman installer. that doesn't provide a way to read the settings unfortunately.
<jsherrill> bmbouter: an alternative might be to allow katello to submit a path to the import/export api and for it to tell us if its valid or not
<jsherrill> per the configuration
<jsherrill> kinda like a mini-dry run, but without having to create all the needed repositories beforehand
<partha> bmbouter: jsherrill I thought we just update /etc/pulp/settings.py?
<partha> bmbouter: well you'll have to be picky on what settings you expose
<jsherrill> partha: what do you think of my alternative ?  maybe it could also check if the export is actually at the path, and readable.  
<daviddavis> fwiw, we've talked about adding a dry run action before 
<jsherrill> yeah, its kinda a dry run-lite 
<jsherrill> we don't want to have to provide the full details, because that has some implications iirc
<daviddavis> well we'll probably have a dry run endpoint that does everything and then some params or something that can narrow the scope of the dry run to certain checks
<jsherrill> we really just want to check the validity and readability of the export data
<jsherrill> that sounds good
<daviddavis> ggainey: fyi ^
<ggainey> daviddavis: yup, was just reading - pretty much what I had in mind for dry-run, yus
<daviddavis> if partha agrees maybe someone can file an issue
<daviddavis> I got a meeting
<ggainey> don't we have one?
<ggainey> I think we have one
<partha> depends on the dry-run .. Do I need to pass it a repo mapping and stuff 
<ggainey> one sec
<jsherrill> partha: right, thats what i was saying, we need an option to not pass that
<ggainey> partha: jsherrill daviddavis : add thoughts to https://pulp.plan.io/issues/7549 please
<daviddavis> yea, my thought is that there are params to control the scope of the dry run
<daviddavis> so if you just want to validate the file, path, etc you can do that
<ggainey> or, does it just do everything, and not-fail at the first problem?
<partha> jturel: so if pulp implements ^ dry-run we could remove the accessible permissions checking and all that
<daviddavis> ggainey: that would work too. it would return like areport or something
<jturel> partha: that's a big huge +1 from me
<daviddavis> although some of the later checks it might not be able to perform though if the earlier checks fail
<ggainey> partha: you could call dry-run, you'd still have to respond to the (various) errors
<ggainey> daviddavis: sure
<ggainey> a;;" http://yum.theforeman.org/pulpcore/3.7/el8/x86_64/ is a reasonable place to find our RPM builds, yes? I pointed a user at it, and now I'm worried that there's a better location
<ggainey> all, even
* ggainey looks suspiciously at fingers
<partha> ggainey: yes it depends on the scope. If pulp is going to require us to provice repo-mapping for the dry run 
<partha> it might get harder
<ggainey> partha: /shrug I'd just write it to note "absence of mapping" as Just One More Error
<ggainey> I'd think decusions would be summat like "the place you want me to read from exists. I have permissions to read from it. the thing you want me to read exists. The thing you want me to read passes validation. you gave me a repo-mapping. the repo-mapping makes sense"
<ggainey> decisions
<ggainey> jeebus, I cannot type today
<partha> ggainey: daviddavis ok at the simplest level how about a general can "pulp read this dir"
<ggainey> that's part of "I have permissions to read from it"
<ggainey> I am pretty averse to having a pulp-api be a passthru to 'os' calls tho :)
<partha> "available_for_import"
<partha> :)

#3 Updated by paji@redhat.com 15 days ago

In short

  • Make repo-mapping optional
  • Generate a report along the lines of
could read export = PASS
export is valid = PASS
mapping is valid = FAIL

#4 Updated by daviddavis 15 days ago

  • Tags Katello added

#5 Updated by daviddavis 15 days ago

My initial thought was that dry run would do an entire actual import/export but without actually changing any data or creating an export (respectively). This could be very expensive and invasive for users who don't want it (eg import would have to combine chunks and parse it to do a real dry run). So one idea I had is to have parameter(s) to limit the scope of the dry run.

Please register to edit this issue

Also available in: Atom PDF