Project

Profile

Help

Story #6729

As user, I can sync SUSE repos which require auth-token

Added by abrix about 1 year ago. Updated 10 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
No
Tags:
Sprint:
Sprint 75
Quarter:

Description

Hi,

with Pulp 3 the possibility of mirroring a SUSE or any other repository which requires an auth-token, does not seem to be available anymore as it was possible with Pulp 2 [1].

There's also no reference in the API documentation [2] to the use of an auth-token in the remotes.

Best regards,

Alex

403 error during repo sync:

{
    "created_resources": [],
    "error": {
        "description": "403, message='Forbidden', url=URL('https://updates.suse.com/SUSE/Products/SLE-SERVER/12-SP4/x86_64/product/.treeinfo')",
        "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 884, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 664, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 670, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 125, in synchronize\n    treeinfo = get_treeinfo_data(remote)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py\", line 24, in get_treeinfo_data\n    result = downloader.fetch()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/download/base.py\", line 154, in fetch\n    return done.pop().result()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/download/base.py\", line 221, in run\n    return await self._run(extra_data=extra_data)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/backoff/_async.py\", line 133, in retry\n    ret = await target(*args, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/download/http.py\", line 185, in _run\n    response.raise_for_status()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/client_reqrep.py\", line 946, in raise_for_status\n    headers=self.headers)\n"
    },
    "finished_at": "2020-05-14T06:50:14.111981Z",
    "name": "pulp_rpm.app.tasks.synchronizing.synchronize",
    "progress_reports": [],
    "pulp_created": "2020-05-14T06:50:13.791631Z",
    "pulp_href": "/pulp/api/v3/tasks/ce86f248-93d3-4da7-b6ef-49e00693497e/",
    "reserved_resources_record": [
        "/pulp/api/v3/remotes/rpm/rpm/5adfc022-9992-4e09-8d31-454a41f0f805/",
        "/pulp/api/v3/repositories/rpm/rpm/6193f212-d07a-43de-acc4-c3bb331c372b/"
    ],
    "started_at": "2020-05-14T06:50:13.898656Z",
    "state": "failed",
    "worker": "/pulp/api/v3/workers/85fb61f8-f74e-4f3a-a95d-07f8caa24c7d/"
}

[1] [[https://pulpproject.org/2015/12/09/syncing-and-publishing-suse-repositories-with-pulp/]]

[2] [[https://pulp-rpm.readthedocs.io/en/latest/restapi.html#operation/remotes_rpm_rpm_create]]

Proposed solution:

  • Create new RpmRemote and its serializer with one extra field for a auth token
  • Create new downloader (and downloader chooser) which can use newly created remotes with auth-token

Related issues

Related to Migration Plugin - Story #6927: As a user, I can migrate Remotes for SLES reposCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 3f84d87b View on GitHub
Added by ppicka 11 months ago

SLES support with authentication token

Added support for SLES repository with use of authentication token. Also updated docs how to use it.

closes: #6729 https://pulp.plan.io/issues/6729

[nocoverage]

History

#1 Updated by ttereshc about 1 year ago

  • Tracker changed from Issue to Story
  • Subject changed from Pulp 3 remotes rpm auth-token missing to As user, I can sync SUSE repos which require auth-token
  • % Done set to 0
  • Tags deleted (Pulp 3 RPM blocker)

#2 Updated by dkliban@redhat.com about 1 year ago

Here is the story for when this feature was added to Pulp 2[0]. It does seem like this should be a separate Remote type that will use a custom downloader.

[0] https://pulp.plan.io/issues/1358

#3 Updated by abrix about 1 year ago

I tried as suggested in issue 1358 [1] the following query via repositories

http --body PUT http://localhost:24817/pulp/api/v3/repositories/rpm/rpm/d465894f-c31a-4e31-8841-c89f09920895/ name=upstream-sles-12sp4-x86_64-pool importer_config:='{"query_auth_token": "SECRET"}'
{
    "importer_config": [
        "Unexpected field"
    ]
}

and more applicable in remotes (IMHO) but with the same result

http --body PUT http://localhost:24817"/pulp/api/v3/remotes/rpm/rpm/795524ff-6ee6-444d-bd49-8eb60938c368/" name=upstream-sles-12sp4-x86_64-pool importer_config:='{"query_auth_token": "SECRET"}' url="https://updates.suse.com/SUSE/Products/SLE-SERVER/12-SP4/x86_64/product"
{
    "importer_config": [
        "Unexpected field"
    ]
}

[1] https://pulp.plan.io/issues/1358#note-8

#4 Updated by ppicka 12 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ppicka

#5 Updated by ppicka 12 months ago

  • Description updated (diff)

#6 Updated by ppicka 12 months ago

  • Description updated (diff)

#7 Updated by dkliban@redhat.com 12 months ago

  • Groomed changed from No to Yes
  • Sprint set to Sprint 73

#8 Updated by rchan 12 months ago

  • Sprint changed from Sprint 73 to Sprint 74

#9 Updated by pulpbot 12 months ago

  • Status changed from ASSIGNED to POST

#10 Updated by ipanova@redhat.com 11 months ago

  • Related to Story #6927: As a user, I can migrate Remotes for SLES repos added

#11 Updated by rchan 11 months ago

  • Sprint changed from Sprint 74 to Sprint 75

#12 Updated by ppicka 11 months ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#13 Updated by ttereshc 10 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

#14 Updated by ttereshc 10 months ago

  • Sprint/Milestone set to Pulp RPM 3.5.0

Please register to edit this issue

Also available in: Atom PDF