Story #3180
closed
As a user, I can mirror Galaxy roles to a local repo
Status:
CLOSED - CURRENTRELEASE
Description
This is designed to develop the code that will find roles on Galaxy and sync them.
1. Find roles through https://galaxy.ansible.com/api/v1/roles/
2. Sync associated repos using python-git. Save them as a bare repo into /var/lib/pulp/shared/pulp_ansible/<repo_name>
3. Create a content unit for each role as a unit. Note a unit contains a version number, so when syncing a repo with 10 versions, you'll get 10 content units for that role sync.
- Blocked by Story #3179: As an user, I can view information for a role in Pulp added
- Blocks Story #3181: As a user, I have a roles API for published distribution base paths added
- Groomed changed from No to Yes
- Sprint Candidate changed from No to Yes
- Sprint/Milestone changed from 0.1 Release to 52
- Sprint/Milestone changed from 52 to 53
- Status changed from NEW to ASSIGNED
- Assignee set to daviddavis
I'm wondering how to best deal with the pagination of the Ansible Galaxy API. By default the page size is 10 which right now is about 1463 for all roles. That means we'll have to do 1463 requests to pull down info for all galaxy roles. I tried bumping up the page size to 1000 (ie https://galaxy.ansible.com/api/v1/roles/?page_size=1000) but then the time of a request slows down quite a bit (to 10-20s). So timeouts become a potential problem.
What about somewhere in the middle, like 100? That would reduce the query count by an order of magnitude and still keep the times to < 10s. If not that, then I would be in favor of just doing a lot of requests with the normal pagination size of 10.
- Sprint/Milestone changed from 53 to 54
We need to also be careful in case the Galaxy API has any rate limiting in place.
- Sprint/Milestone changed from 54 to 56
- Sprint/Milestone deleted (
56)
- Status changed from ASSIGNED to MODIFIED
- Sprint/Milestone set to 0.1 Release
- Blocks deleted (Story #3181: As a user, I have a roles API for published distribution base paths)
- Blocked by deleted (Story #3179: As an user, I can view information for a role in Pulp)
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Also available in: Atom
PDF
.
As a user, I can mirror Galaxy roles to a local repo
fixes #3180 https://pulp.plan.io/issues/3180