Story #6729: As user, I can sync SUSE repos which require auth-token - RPM Support - Pulp

Actions

Send by e-mail Copy link

Story #6729

closed

As user, I can sync SUSE repos which require auth-token

Added by abrix over 4 years ago. Updated over 4 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

ppicka

Sprint/Milestone:

Pulp RPM 3.5.0

Start date:

Due date:

% Done:

100%

Estimated time:

Platform Release:

Groomed:

Yes

Sprint Candidate:

Tags:

Sprint:

Sprint 75

Quarter:

Description

Hi,

with Pulp 3 the possibility of mirroring a SUSE or any other repository which requires an auth-token, does not seem to be available anymore as it was possible with Pulp 2 [1].

There's also no reference in the API documentation [2] to the use of an auth-token in the remotes.

Best regards,

Alex

403 error during repo sync:

{
    "created_resources": [],
    "error": {
        "description": "403, message='Forbidden', url=URL('https://updates.suse.com/SUSE/Products/SLE-SERVER/12-SP4/x86_64/product/.treeinfo')",
        "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 884, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 664, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 670, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 125, in synchronize\n    treeinfo = get_treeinfo_data(remote)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py\", line 24, in get_treeinfo_data\n    result = downloader.fetch()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/download/base.py\", line 154, in fetch\n    return done.pop().result()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/download/base.py\", line 221, in run\n    return await self._run(extra_data=extra_data)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/backoff/_async.py\", line 133, in retry\n    ret = await target(*args, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/download/http.py\", line 185, in _run\n    response.raise_for_status()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/client_reqrep.py\", line 946, in raise_for_status\n    headers=self.headers)\n"
    },
    "finished_at": "2020-05-14T06:50:14.111981Z",
    "name": "pulp_rpm.app.tasks.synchronizing.synchronize",
    "progress_reports": [],
    "pulp_created": "2020-05-14T06:50:13.791631Z",
    "pulp_href": "/pulp/api/v3/tasks/ce86f248-93d3-4da7-b6ef-49e00693497e/",
    "reserved_resources_record": [
        "/pulp/api/v3/remotes/rpm/rpm/5adfc022-9992-4e09-8d31-454a41f0f805/",
        "/pulp/api/v3/repositories/rpm/rpm/6193f212-d07a-43de-acc4-c3bb331c372b/"
    ],
    "started_at": "2020-05-14T06:50:13.898656Z",
    "state": "failed",
    "worker": "/pulp/api/v3/workers/85fb61f8-f74e-4f3a-a95d-07f8caa24c7d/"
}

[1] [[https://pulpproject.org/2015/12/09/syncing-and-publishing-suse-repositories-with-pulp/]]

[2] [[https://pulp-rpm.readthedocs.io/en/latest/restapi.html#operation/remotes_rpm_rpm_create]]

Proposed solution:

Create new RpmRemote and its serializer with one extra field for a auth token
Create new downloader (and downloader chooser) which can use newly created remotes with auth-token

Related issues

Actions

Copy link

Updated by ttereshc over 4 years ago

Tracker changed from Issue to Story
Subject changed from Pulp 3 remotes rpm auth-token missing to As user, I can sync SUSE repos which require auth-token
% Done set to 0
Tags deleted (~~Pulp 3 RPM blocker~~)

Actions

Copy link

Updated by dkliban@redhat.com over 4 years ago

Here is the story for when this feature was added to Pulp 2[0]. It does seem like this should be a separate Remote type that will use a custom downloader.

[0] https://pulp.plan.io/issues/1358

Actions

Copy link

Updated by abrix over 4 years ago

I tried as suggested in issue 1358 [1] the following query via repositories

http --body PUT http://localhost:24817/pulp/api/v3/repositories/rpm/rpm/d465894f-c31a-4e31-8841-c89f09920895/ name=upstream-sles-12sp4-x86_64-pool importer_config:='{"query_auth_token": "SECRET"}'
{
    "importer_config": [
        "Unexpected field"
    ]
}

and more applicable in remotes (IMHO) but with the same result

http --body PUT http://localhost:24817"/pulp/api/v3/remotes/rpm/rpm/795524ff-6ee6-444d-bd49-8eb60938c368/" name=upstream-sles-12sp4-x86_64-pool importer_config:='{"query_auth_token": "SECRET"}' url="https://updates.suse.com/SUSE/Products/SLE-SERVER/12-SP4/x86_64/product"
{
    "importer_config": [
        "Unexpected field"
    ]
}

[1] https://pulp.plan.io/issues/1358#note-8

Actions

Copy link