Project

Profile

Help

Issue #8689

adding custom connection headers to debian remote causes repository syncs to always fail

Added by simon_t 3 months ago. Updated 2 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 97
Quarter:

Description

pulpcore version: 3.12.1 pulp_deb version: 2.11.1

In an attempt to resolve ongoing sync failures custom headers were added to a debian remote:

http patch https://pulphost.domain.local/pulp/api/v3/remotes/deb/apt/18600e5b-4914-4bbd-ae2a-bc0488c30e07/ headers:='[{"Connection": "keep-alive"}]'

Resulting in the following configuration which matches the format described in the pulp_deb API documentation:

{
    "architectures": "amd64",
    "ca_cert": null,
    "client_cert": null,
    "components": "main contrib non-free",
    "connect_timeout": null,
    "distributions": "bullseye",
    "download_concurrency": 10,
    "gpgkey": null,
    "headers": [
        {
            "Connection": "keep-alive"
        }
    ],
    "ignore_missing_package_indices": true,
    "name": "remote_debian_bullseye",
    "policy": "immediate",
    "proxy_url": null,
    "pulp_created": "2021-04-29T11:17:38.824675Z",
    "pulp_href": "/pulp/api/v3/remotes/deb/apt/18600e5b-4914-4bbd-ae2a-bc0488c30e07/",
    "pulp_labels": {},
    "pulp_last_updated": "2021-04-30T06:39:24.047081Z",
    "rate_limit": null,
    "sock_connect_timeout": null,
    "sock_read_timeout": null,
    "sync_installer": false,
    "sync_sources": false,
    "sync_udebs": false,
    "tls_validation": true,
    "total_timeout": null,
    "url": "http://debian.mirror.digitalpacific.com.au/debian/"
}

All subsequent attempts to synchronise a repository with the remote above fail immediately with the following error:

Apr 30 06:39:46 pulphost.domain.local rq[22865]: TypeError: the JSON object must be str, bytes or bytearray, not 'list'

Associated revisions

Revision 22993ffa View on GitHub
Added by dkliban@redhat.com 2 months ago

Fixes ability to specify custom headers on a Remote

The Remotes APIs accepts a list of headers to be sent with each request during sync. However, the headers were not being properly passed to aiohttp. This patch fixes how the headers are passed to aiohttp.

This patch also ensures that any User-Agent headers configured on a remote are appended to the User-Agent string generated by pulp.

fixes: #8689 https://pulp.plan.io/issues/8689

History

#1 Updated by simon_t 3 months ago

Full log traceback:

Apr 30 06:39:44 pulphost.domain.local gunicorn[23232]: pulp [fb5f8febe0cf4fc88bc1857f87f4054c]:  - admin [30/Apr/2021:06:39:44 +0000] "POST /pulp/api/v3/repositories/deb/apt/dbd1bbf7-a5ef-46b6-922a-0ddf62b92ebc/sync/ HTTP/1.0" 202 67 "-" "HTTPie/0.9.8"
Apr 30 06:39:44 pulphost.domain.local rq[22865]: pulp [None]: rq.worker:INFO: 22865@pulphost.domain.local: bda07a1f-7f81-4f50-9914-ae453df2ca26
Apr 30 06:39:44 pulphost.domain.local rq[23117]: pulp [fb5f8febe0cf4fc88bc1857f87f4054c]: rq.worker:INFO: resource-manager: Job OK (02c00f37-c7a7-47d6-b5d0-ee7f09b4069e)
Apr 30 06:39:46 pulphost.domain.local rq[22865]: pulp [fb5f8febe0cf4fc88bc1857f87f4054c]: pulp_deb.app.tasks.synchronizing:INFO: Downloading Release file for distribution: "bullseye"
Apr 30 06:39:46 pulphost.domain.local rq[22865]: pulp [fb5f8febe0cf4fc88bc1857f87f4054c]: rq.worker:ERROR: Traceback (most recent call last):
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 307, in download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return self._download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]: AttributeError: 'AptRemote' object has no attribute '_download_factory'
Apr 30 06:39:46 pulphost.domain.local rq[22865]: During handling of the above exception, another exception occurred:
Apr 30 06:39:46 pulphost.domain.local rq[22865]: Traceback (most recent call last):
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/worker.py", line 1008, in perform_job
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     rv = job.perform()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/job.py", line 706, in perform
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._result = self._execute()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/job.py", line 729, in _execute
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     result = self.func(*self.args, **self.kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 122, in synchronize
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 149, in create
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     loop.run_until_complete(pipeline)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return future.result()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await asyncio.gather(*futures)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await self.run()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 174, in run
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     pb.done += task.result()  # download_count
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 200, in _handle_content_unit
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await asyncio.gather(*downloaders_for_content)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 135, in download
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await super().download()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/models.py", line 87, in download
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     downloader = self.remote.get_downloader(url=self.url, **validation_kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 371, in get_downloader
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return self.download_factory.build(url, **kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 309, in download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._download_factory = DownloaderFactory(self)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/download/factory.py", line 88, in __init__
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._session = self._make_aiohttp_session_from_remote()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/download/factory.py", line 129, in _make_aiohttp_session_from_remote
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     headers.update(json.loads(self._remote.headers))
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/lib/python3.6/json/__init__.py", line 348, in loads
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     'not {!r}'.format(s.__class__.__name__))
Apr 30 06:39:46 pulphost.domain.local rq[22865]: TypeError: the JSON object must be str, bytes or bytearray, not 'list'
Apr 30 06:39:46 pulphost.domain.local rq[22865]: Traceback (most recent call last):
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 307, in download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return self._download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]: AttributeError: 'AptRemote' object has no attribute '_download_factory'
Apr 30 06:39:46 pulphost.domain.local rq[22865]: During handling of the above exception, another exception occurred:
Apr 30 06:39:46 pulphost.domain.local rq[22865]: Traceback (most recent call last):
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/worker.py", line 1008, in perform_job
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     rv = job.perform()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/job.py", line 706, in perform
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._result = self._execute()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/rq/job.py", line 729, in _execute
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     result = self.func(*self.args, **self.kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 122, in synchronize
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     DebDeclarativeVersion(first_stage, repository, mirror=mirror).create()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 149, in create
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     loop.run_until_complete(pipeline)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return future.result()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await asyncio.gather(*futures)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await self.run()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 174, in run
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     pb.done += task.result()  # download_count
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 200, in _handle_content_unit
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await asyncio.gather(*downloaders_for_content)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py", line 135, in download
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     await super().download()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/plugin/stages/models.py", line 87, in download
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     downloader = self.remote.get_downloader(url=self.url, **validation_kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 371, in get_downloader
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     return self.download_factory.build(url, **kwargs)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 309, in download_factory
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._download_factory = DownloaderFactory(self)
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/download/factory.py", line 88, in __init__
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     self._session = self._make_aiohttp_session_from_remote()
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/download/factory.py", line 129, in _make_aiohttp_session_from_remote
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     headers.update(json.loads(self._remote.headers))
Apr 30 06:39:46 pulphost.domain.local rq[22865]:   File "/usr/lib/python3.6/json/__init__.py", line 348, in loads
Apr 30 06:39:46 pulphost.domain.local rq[22865]:     'not {!r}'.format(s.__class__.__name__))
Apr 30 06:39:46 pulphost.domain.local rq[22865]: TypeError: the JSON object must be str, bytes or bytearray, not 'list'

#2 Updated by quba42 3 months ago

I think this part of the remote and API are entirely inherited from pulpcore, so this might actually be a pulpcore issue. (Or else an issue with how pulp_deb interacts with pulpcore).

I will try to ask around for people who know more about it then myself.

#3 Updated by bmbouter 3 months ago

  • Project changed from Debian Support to Pulp

Moving to pulpcore since the issue very likely in the DownloaderFactory area.

#5 Updated by dkliban@redhat.com 3 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dkliban@redhat.com
  • Sprint set to Sprint 96

#6 Updated by pulpbot 3 months ago

  • Status changed from ASSIGNED to POST

#7 Updated by fao89 2 months ago

  • Triaged changed from No to Yes
  • Sprint changed from Sprint 96 to Sprint 97

#8 Updated by dkliban@redhat.com 2 months ago

  • Status changed from POST to MODIFIED

#9 Updated by dalley 2 months ago

  • Sprint/Milestone set to 3.13.0

#10 Updated by pulpbot 2 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF