Project

Profile

Help

Issue #8689

closed

adding custom connection headers to debian remote causes repository syncs to always fail

Added by simon_t over 1 year ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 97
Quarter:

Description

pulpcore version: 3.12.1 pulp_deb version: 2.11.1

In an attempt to resolve ongoing sync failures custom headers were added to a debian remote:

http patch https://pulphost.domain.local/pulp/api/v3/remotes/deb/apt/18600e5b-4914-4bbd-ae2a-bc0488c30e07/ headers:='[{"Connection": "keep-alive"}]'

Resulting in the following configuration which matches the format described in the pulp_deb API documentation:

{
    "architectures": "amd64",
    "ca_cert": null,
    "client_cert": null,
    "components": "main contrib non-free",
    "connect_timeout": null,
    "distributions": "bullseye",
    "download_concurrency": 10,
    "gpgkey": null,
    "headers": [
        {
            "Connection": "keep-alive"
        }
    ],
    "ignore_missing_package_indices": true,
    "name": "remote_debian_bullseye",
    "policy": "immediate",
    "proxy_url": null,
    "pulp_created": "2021-04-29T11:17:38.824675Z",
    "pulp_href": "/pulp/api/v3/remotes/deb/apt/18600e5b-4914-4bbd-ae2a-bc0488c30e07/",
    "pulp_labels": {},
    "pulp_last_updated": "2021-04-30T06:39:24.047081Z",
    "rate_limit": null,
    "sock_connect_timeout": null,
    "sock_read_timeout": null,
    "sync_installer": false,
    "sync_sources": false,
    "sync_udebs": false,
    "tls_validation": true,
    "total_timeout": null,
    "url": "http://debian.mirror.digitalpacific.com.au/debian/"
}

All subsequent attempts to synchronise a repository with the remote above fail immediately with the following error:

Apr 30 06:39:46 pulphost.domain.local rq[22865]: TypeError: the JSON object must be str, bytes or bytearray, not 'list'
Actions #1

Updated by simon_t over 1 year ago

Full log traceback:

Apr 30 06:39:44 pulphost.domain.local gunicorn[23232]: pulp [fb5f8febe0cf4fc88bc1857f87f4054c]:  - admin [30/Apr/2021:06:39:44 +0000] "POST /pulp/api/v3/repositories/deb/apt/dbd1bbf7-a5ef-46b6-922a-0ddf62b92ebc/sync/ HTTP/1.0" 202 67 "-" "HTTPie/0.9.8"
Apr 30 06:39:44 pulphost.domain.local rq[22865]: pulp [None]: rq.worker:INFO: 22865@pulphost.domain.local: bda07a1f-7f81-4f50-9914-ae453df2ca26
Apr 30 06:39:44 pulphost.domain.local rq[23117]: pulp [fb5f8febe0cf4fc88bc1857f87f4054c]: rq.worker:INFO: resource-manager: Job OK (02c00f37-c7a7-47d6-b5d0-ee7f09b4069e)
Apr 30 06:39:46 pulphost.domain.local rq[22865]: pulp [fb5f8febe0cf4fc88bc1857f87f4054c]: pulp_deb.app.tasks.synchronizing:INFO: Downloading Release file for distribution: "bullseye"
Apr 30 06:39:46 pulphost.domain.local rq[22865]: pulp [fb5f8febe0cf4fc88bc1857f87f4054c]: rq.worker:ERROR: Traceback (most recent call last):
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 307, in download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return self._download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]: AttributeError: 'AptRemote' object has no attribute '_download_factory'
Apr 30 06:39:46 pulphost.domain.local rq[22865]: During handling of the above exception, another exception occurred:
Apr 30 06:39:46 pulphost.domain.local rq[22865]: Traceback (most recent call last):
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/worker.py", line 1008, in perform_job
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     rv = job.perform()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/job.py", line 706, in perform
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._result = self._execute()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/job.py", line 729, in _execute
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     result = self.func(*self.args, **self.kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 122, in synchronize
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 149, in create
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     loop.run_until_complete(pipeline)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return future.result()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await asyncio.gather(*futures)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await self.run()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 174, in run
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     pb.done += task.result()  # download_count
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 200, in _handle_content_unit
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await asyncio.gather(*downloaders_for_content)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 135, in download
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await super().download()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/models.py", line 87, in download
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     downloader = self.remote.get_downloader(url=self.url, **validation_kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 371, in get_downloader
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return self.download_factory.build(url, **kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 309, in download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._download_factory = DownloaderFactory(self)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/download/factory.py", line 88, in __init__
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._session = self._make_aiohttp_session_from_remote()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/download/factory.py", line 129, in _make_aiohttp_session_from_remote
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     headers.update(json.loads(self._remote.headers))
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/lib/python3.6/json/__init__.py", line 348, in loads
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     'not {!r}'.format(s.__class__.__name__))
Apr 30 06:39:46 pulphost.domain.local rq[22865]: TypeError: the JSON object must be str, bytes or bytearray, not 'list'
Apr 30 06:39:46 pulphost.domain.local rq[22865]: Traceback (most recent call last):
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 307, in download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return self._download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]: AttributeError: 'AptRemote' object has no attribute '_download_factory'
Apr 30 06:39:46 pulphost.domain.local rq[22865]: During handling of the above exception, another exception occurred:
Apr 30 06:39:46 pulphost.domain.local rq[22865]: Traceback (most recent call last):
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/worker.py", line 1008, in perform_job
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     rv = job.perform()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/job.py", line 706, in perform
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._result = self._execute()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/job.py", line 729, in _execute
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     result = self.func(*self.args, **self.kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 122, in synchronize
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 149, in create
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     loop.run_until_complete(pipeline)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return future.result()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await asyncio.gather(*futures)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await self.run()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 174, in run
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     pb.done += task.result()  # download_count
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 200, in _handle_content_unit
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await asyncio.gather(*downloaders_for_content)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 135, in download
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await super().download()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/models.py", line 87, in download
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     downloader = self.remote.get_downloader(url=self.url, **validation_kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 371, in get_downloader
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return self.download_factory.build(url, **kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 309, in download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._download_factory = DownloaderFactory(self)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/download/factory.py", line 88, in __init__
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._session = self._make_aiohttp_session_from_remote()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/download/factory.py", line 129, in _make_aiohttp_session_from_remote
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     headers.update(json.loads(self._remote.headers))
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/lib/python3.6/json/__init__.py", line 348, in loads
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     'not {!r}'.format(s.__class__.__name__))
Apr 30 06:39:46 pulphost.domain.local rq[22865]: TypeError: the JSON object must be str, bytes or bytearray, not 'list'
Actions #2

Updated by quba42 over 1 year ago

I think this part of the remote and API are entirely inherited from pulpcore, so this might actually be a pulpcore issue. (Or else an issue with how pulp_deb interacts with pulpcore).

I will try to ask around for people who know more about it then myself.

Actions #3

Updated by bmbouter over 1 year ago

  • Project changed from Debian Support to Pulp

Moving to pulpcore since the issue very likely in the DownloaderFactory area.

Actions #5

Updated by dkliban@redhat.com over 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dkliban@redhat.com
  • Sprint set to Sprint 96
Actions #6

Updated by pulpbot over 1 year ago

  • Status changed from ASSIGNED to POST
Actions #7

Updated by fao89 over 1 year ago

  • Triaged changed from No to Yes
  • Sprint changed from Sprint 96 to Sprint 97

Added by dkliban@redhat.com over 1 year ago

Revision 22993ffa

Fixes ability to specify custom headers on a Remote

The Remotes APIs accepts a list of headers to be sent with each request during sync. However, the headers were not being properly passed to aiohttp. This patch fixes how the headers are passed to aiohttp.

This patch also ensures that any User-Agent headers configured on a remote are appended to the User-Agent string generated by pulp.

fixes: #8689 https://pulp.plan.io/issues/8689

Actions #8

Updated by dkliban@redhat.com over 1 year ago

  • Status changed from POST to MODIFIED
Actions #9

Updated by dalley over 1 year ago

  • Sprint/Milestone set to 3.13.0
Actions #10

Updated by pulpbot over 1 year ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF