RPM Support

Hello pulp upstream,

we are regularly mirroring various CentOS stream repos, but recently "sync" tasks started to fail with following traceback:

May  5 12:20:22 pulp32 rq[2083851]: Traceback (most recent call last):
May  5 12:20:22 pulp32 rq[2083851]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/rq/worker.py", line 1008, in perform_job
May  5 12:20:22 pulp32 rq[2083851]:    rv = job.perform()
May  5 12:20:22 pulp32 rq[2083851]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/rq/job.py", line 706, in perform
May  5 12:20:22 pulp32 rq[2083851]:    self._result = self._execute()
May  5 12:20:22 pulp32 rq[2083851]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/rq/job.py", line 729, in _execute
May  5 12:20:22 pulp32 rq[2083851]:    result = self.func(*self.args, **self.kwargs)
May  5 12:20:22 pulp32 rq[2083851]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 255, in synchronize
May  5 12:20:22 pulp32 rq[2083851]:    dv.create()
May  5 12:20:22 pulp32 rq[2083851]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 149, in create
May  5 12:20:22 pulp32 rq[2083851]:    loop.run_until_complete(pipeline)
May  5 12:20:22 pulp32 rq[2083851]:  File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May  5 12:20:22 pulp32 rq[2083851]:    return future.result()
May  5 12:20:22 pulp32 rq[2083851]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May  5 12:20:22 pulp32 rq[2083851]:    await asyncio.gather(*futures)
May  5 12:20:22 pulp32 rq[2083851]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May  5 12:20:22 pulp32 rq[2083851]:    await self.run()
May  5 12:20:22 pulp32 rq[2083851]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 439, in run
May  5 12:20:22 pulp32 rq[2083851]:    await self.parse_modules_metadata()
May  5 12:20:22 pulp32 rq[2083851]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 495, in parse_modules_metadata
May  5 12:20:22 pulp32 rq[2083851]:    modules_metadata_parser.parse()
May  5 12:20:22 pulp32 rq[2083851]:  File "/opt/gdc/pulp3/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 747, in parse
May  5 12:20:22 pulp32 rq[2083851]:    content = moduleyaml.read()
May  5 12:20:22 pulp32 rq[2083851]:  File "/usr/lib64/python3.6/codecs.py", line 321, in decode
May  5 12:20:22 pulp32 rq[2083851]:    (result, consumed) = self._buffer_decode(data, self.errors, final)
May  5 12:20:22 pulp32 rq[2083851]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 0: invalid start byte

We are using following versions:

[root@pulp32:~] pulp status
{
  "versions": [
    {
      "component": "core",
      "version": "3.12.1"
    },
    {
      "component": "rpm",
      "version": "3.10.0"
    },
    {
      "component": "file",
      "version": "1.7.0"
    }

Remote configuration:

[root@pulp32:~] pulp rpm remote show --name c8s-baseos-remote
{
  "pulp_href": "/pulp/api/v3/remotes/rpm/rpm/dc58e56e-4a00-47f1-af0d-a9b6786af51c/",
  "pulp_created": "2021-05-05T02:44:02.345456Z",
  "name": "c8s-baseos-remote",
  "url": "http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/",
  "ca_cert": null,
  "client_cert": null,
  "tls_validation": true,
  "proxy_url": null,
  "pulp_labels": {},
  "pulp_last_updated": "2021-05-05T02:44:02.345474Z",
  "download_concurrency": 10,
  "policy": "immediate",
  "total_timeout": null,
  "connect_timeout": null,
  "sock_connect_timeout": null,
  "sock_read_timeout": null,
  "headers": null,
  "rate_limit": null,
  "sles_auth_token": null
}

How to reproduce:

[root@pulp32:~] pulp rpm repository sync --name c8s-baseos --remote c8s-baseos-remote
Started background task /pulp/api/v3/tasks/e50f608f-9bf7-48a9-ad6a-03efb34014d6/
.Error: Task /pulp/api/v3/tasks/e50f608f-9bf7-48a9-ad6a-03efb34014d6/ failed: ''utf-8' codec can't decode byte 0xfd in position 0: invalid start byte'

Please let me know if you need any more details. Thank you in advance!