Issue #7019
closedFailure Syncing Against Remote Repository / Update docs
Description
When attempting to sync against a remote repository (created using Pulp2), I get the below information reported in the task.
{'child_tasks': [], 'created_resources': [], 'error': {u'description': u'unsupported Unicode escape sequence\nLINE 1: ...to RRD-databases.', 'http://www.pnp4nagios.org/\', '[["Christ...\n ^\nDETAIL: Unicode escape values cannot be used for code point values above 007F when the server encoding is not UTF8.\nCONTEXT: JSON data, line 1: ...ttings\n- fix npcd.8 man page prefix install"], [...\n', u'traceback': u' File "/opt/bats/lib/python3.8/site-packages/rq/worker.py", line 886, in perform_job\n rv = job.perform()\n File "/opt/bats/lib/python3.8/site-packages/rq/job.py", line 664, in perform\n self._result = self._execute()\n File "/opt/bats/lib/python3.8/site-packages/rq/job.py", line 670, in _execute\n return self.func(*self.args, **self.kwargs)\n File "/opt/bats/lib/python3.8/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 167, in synchronize\n dv.create()\n File "/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create\n loop.run_until_complete(pipeline)\n File "/opt/bats/lib/python3.8/asyncio/base_events.py", line 608, in run_until_complete\n return future.result()\n File "/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline\n await asyncio.gather(*futures)\n File "/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/stages/api.py", line 43, in call\n await self.run()\n File "/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/stages/content_stages.py", line 102, in run\n d_content.content.save()\n File "/opt/bats/lib/python3.8/site-packages/pulpcore/app/models/base.py", line 110, in save\n return super().save(*args, **kwargs)\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/base.py", line 740, in save\n self.save_base(using=using, force_insert=force_insert,\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/base.py", line 777, in save_base\n updated = self._save_table(\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/base.py", line 870, in _save_table\n result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/base.py", line 907, in _do_insert\n return manager._insert([self], fields=fields, return_id=update_pk,\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/manager.py", line 82, in manager_method\n return getattr(self.get_queryset(), name)(*args, **kwargs)\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/query.py", line 1186, in _insert\n return query.get_compiler(using=using).execute_sql(return_id)\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1375, in execute_sql\n cursor.execute(sql, params)\n File "/opt/bats/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute\n return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)\n File "/opt/bats/lib/python3.8/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers\n return executor(sql, params, many, context)\n File "/opt/bats/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute\n return self.cursor.execute(sql, params)\n File "/opt/bats/lib/python3.8/site-packages/django/db/utils.py", line 89, in exit\n raise dj_exc_value.with_traceback(traceback) from exc_value\n File "/opt/bats/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute\n return self.cursor.execute(sql, params)\n'}, 'finished_at': datetime.datetime(2020, 6, 19, 16, 7, 54, 506158, tzinfo=tzutc()), 'name': 'pulp_rpm.app.tasks.synchronizing.synchronize', 'parent_task': None, 'progress_reports': [{'code': 'downloading.metadata', 'done': 5, 'message': 'Downloading Metadata Files', 'state': 'completed', 'suffix': None, 'total': None}, {'code': 'downloading.artifacts', 'done': 651, 'message': 'Downloading Artifacts', 'state': 'completed', 'suffix': None, 'total': None}, {'code': 'associating.content', 'done': 501, 'message': 'Associating Content', 'state': 'canceled', 'suffix': None, 'total': None}, {'code': 'parsing.comps', 'done': 0, 'message': 'Parsed Comps', 'state': 'completed', 'suffix': None, 'total': 0}, {'code': 'parsing.advisories', 'done': 0, 'message': 'Parsed Advisories', 'state': 'completed', 'suffix': None, 'total': 0}, {'code': 'parsing.packages', 'done': 651, 'message': 'Parsed Packages', 'state': 'completed', 'suffix': None, 'total': 651}], 'pulp_created': datetime.datetime(2020, 6, 19, 16, 5, 33, 890149, tzinfo=tzutc()), 'pulp_href': '/pulp/api/v3/tasks/c6071362-d19b-443e-b1ea-cbb2f047cef3/', 'reserved_resources_record': [u'/pulp/api/v3/remotes/rpm/rpm/b97c095d-6d6b-4a87-a00f-26a23082e678/', u'/pulp/api/v3/repositories/rpm/rpm/a43c50fc-c02c-4fa1-8626-e2ff039896bb/'], 'started_at': datetime.datetime(2020, 6, 19, 16, 5, 33, 991107, tzinfo=tzutc()), 'state': 'failed', 'task_group': None, 'worker': '/pulp/api/v3/workers/7bc8adcf-4f58-47b7-bced-ebfa96125431/'}
I may be able to narrow down the package that is causing the issue, however, it would be helpful if that information was more readily available (maybe I'm looking in the wrong place).
This is using version 3.4.1 for both pulpcore and pulp-rpm, I can't see that version listed in the options below.
Proposed solution (as in comment bellow): to update documentation that psql require UTF8 encoding.
Updated by wibbit over 4 years ago
This was due to the Postgres DB not using UTF8 for it's encoding.
Once that was set, things started working as expected.
It would probably be a good idea to update https://docs.pulpproject.org/installation/instructions.html#postgresql to state that UTF8 encoding of the DB is required.
Updated by ppicka over 4 years ago
- Project changed from RPM Support to Pulp
- Subject changed from Failure Syncing Against Remote Repository to Failure Syncing Against Remote Repository / Update docs
- Description updated (diff)
Updated by daviddavis over 4 years ago
- Triaged changed from No to Yes
- Tags Documentation added
Updated by bmbouter over 3 years ago
- Priority changed from Normal to High
- Quarter set to Q2-2021
Updated by ipanova@redhat.com over 3 years ago
- Status changed from NEW to ASSIGNED
Added by mcorr over 3 years ago
Updated by mcorr over 3 years ago
- Status changed from ASSIGNED to MODIFIED
Applied in changeset pulpcore|813c11f133e878fb37fae4a137beb88f28686b25.
Added by mcorr over 3 years ago
Revision 3859f74f | View on GitHub
Chunk postgres section into headers
fixes #7019
Updated by mcorr over 3 years ago
- Status changed from ASSIGNED to MODIFIED
Applied in changeset pulpcore|3859f74f72d922cf6ee2ced0d493306559aed603.
Updated by pulpbot over 3 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Add line about utf-8 requirement
fixes #7019