Project

Profile

Help

Issue #7019

closed

Failure Syncing Against Remote Repository / Update docs

Added by wibbit over 4 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Documentation
Sprint:
Quarter:
Q2-2021

Description

When attempting to sync against a remote repository (created using Pulp2), I get the below information reported in the task.

{'child_tasks': [], 'created_resources': [], 'error': {u'description': u'unsupported Unicode escape sequence\nLINE 1: ...to RRD-databases.', 'http://www.pnp4nagios.org/\', '[["Christ...\n ^\nDETAIL: Unicode escape values cannot be used for code point values above 007F when the server encoding is not UTF8.\nCONTEXT: JSON data, line 1: ...ttings\n- fix npcd.8 man page prefix install"], [...\n', u'traceback': u' File "/opt/bats/lib/python3.8/site-packages/rq/worker.py", line 886, in perform_job\n rv = job.perform()\n File "/opt/bats/lib/python3.8/site-packages/rq/job.py", line 664, in perform\n self._result = self._execute()\n File "/opt/bats/lib/python3.8/site-packages/rq/job.py", line 670, in _execute\n return self.func(*self.args, **self.kwargs)\n File "/opt/bats/lib/python3.8/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 167, in synchronize\n dv.create()\n File "/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create\n loop.run_until_complete(pipeline)\n File "/opt/bats/lib/python3.8/asyncio/base_events.py", line 608, in run_until_complete\n return future.result()\n File "/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline\n await asyncio.gather(*futures)\n File "/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/stages/api.py", line 43, in call\n await self.run()\n File "/opt/bats/lib/python3.8/site-packages/pulpcore/plugin/stages/content_stages.py", line 102, in run\n d_content.content.save()\n File "/opt/bats/lib/python3.8/site-packages/pulpcore/app/models/base.py", line 110, in save\n return super().save(*args, **kwargs)\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/base.py", line 740, in save\n self.save_base(using=using, force_insert=force_insert,\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/base.py", line 777, in save_base\n updated = self._save_table(\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/base.py", line 870, in _save_table\n result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/base.py", line 907, in _do_insert\n return manager._insert([self], fields=fields, return_id=update_pk,\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/manager.py", line 82, in manager_method\n return getattr(self.get_queryset(), name)(*args, **kwargs)\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/query.py", line 1186, in _insert\n return query.get_compiler(using=using).execute_sql(return_id)\n File "/opt/bats/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1375, in execute_sql\n cursor.execute(sql, params)\n File "/opt/bats/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute\n return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)\n File "/opt/bats/lib/python3.8/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers\n return executor(sql, params, many, context)\n File "/opt/bats/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute\n return self.cursor.execute(sql, params)\n File "/opt/bats/lib/python3.8/site-packages/django/db/utils.py", line 89, in exit\n raise dj_exc_value.with_traceback(traceback) from exc_value\n File "/opt/bats/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute\n return self.cursor.execute(sql, params)\n'}, 'finished_at': datetime.datetime(2020, 6, 19, 16, 7, 54, 506158, tzinfo=tzutc()), 'name': 'pulp_rpm.app.tasks.synchronizing.synchronize', 'parent_task': None, 'progress_reports': [{'code': 'downloading.metadata', 'done': 5, 'message': 'Downloading Metadata Files', 'state': 'completed', 'suffix': None, 'total': None}, {'code': 'downloading.artifacts', 'done': 651, 'message': 'Downloading Artifacts', 'state': 'completed', 'suffix': None, 'total': None}, {'code': 'associating.content', 'done': 501, 'message': 'Associating Content', 'state': 'canceled', 'suffix': None, 'total': None}, {'code': 'parsing.comps', 'done': 0, 'message': 'Parsed Comps', 'state': 'completed', 'suffix': None, 'total': 0}, {'code': 'parsing.advisories', 'done': 0, 'message': 'Parsed Advisories', 'state': 'completed', 'suffix': None, 'total': 0}, {'code': 'parsing.packages', 'done': 651, 'message': 'Parsed Packages', 'state': 'completed', 'suffix': None, 'total': 651}], 'pulp_created': datetime.datetime(2020, 6, 19, 16, 5, 33, 890149, tzinfo=tzutc()), 'pulp_href': '/pulp/api/v3/tasks/c6071362-d19b-443e-b1ea-cbb2f047cef3/', 'reserved_resources_record': [u'/pulp/api/v3/remotes/rpm/rpm/b97c095d-6d6b-4a87-a00f-26a23082e678/', u'/pulp/api/v3/repositories/rpm/rpm/a43c50fc-c02c-4fa1-8626-e2ff039896bb/'], 'started_at': datetime.datetime(2020, 6, 19, 16, 5, 33, 991107, tzinfo=tzutc()), 'state': 'failed', 'task_group': None, 'worker': '/pulp/api/v3/workers/7bc8adcf-4f58-47b7-bced-ebfa96125431/'}

I may be able to narrow down the package that is causing the issue, however, it would be helpful if that information was more readily available (maybe I'm looking in the wrong place).

This is using version 3.4.1 for both pulpcore and pulp-rpm, I can't see that version listed in the options below.

Proposed solution (as in comment bellow): to update documentation that psql require UTF8 encoding.

Actions #1

Updated by wibbit over 4 years ago

This was due to the Postgres DB not using UTF8 for it's encoding.

Once that was set, things started working as expected.

It would probably be a good idea to update https://docs.pulpproject.org/installation/instructions.html#postgresql to state that UTF8 encoding of the DB is required.

Actions #2

Updated by ppicka over 4 years ago

  • Project changed from RPM Support to Pulp
  • Subject changed from Failure Syncing Against Remote Repository to Failure Syncing Against Remote Repository / Update docs
  • Description updated (diff)
Actions #3

Updated by daviddavis over 4 years ago

  • Triaged changed from No to Yes
  • Tags Documentation added
Actions #4

Updated by bmbouter over 3 years ago

  • Priority changed from Normal to High
  • Quarter set to Q2-2021
Actions #5

Updated by mcorr over 3 years ago

  • Assignee set to mcorr
Actions #6

Updated by ipanova@redhat.com over 3 years ago

  • Status changed from NEW to ASSIGNED

Added by mcorr over 3 years ago

Revision 813c11f1 | View on GitHub

Add line about utf-8 requirement

fixes #7019

Actions #7

Updated by mcorr over 3 years ago

  • Status changed from ASSIGNED to MODIFIED
Actions #8

Updated by bmbouter over 3 years ago

  • Status changed from MODIFIED to ASSIGNED

Added by mcorr over 3 years ago

Revision 3859f74f | View on GitHub

Chunk postgres section into headers

fixes #7019

Actions #9

Updated by mcorr over 3 years ago

  • Status changed from ASSIGNED to MODIFIED
Actions #10

Updated by dalley over 3 years ago

  • Sprint/Milestone set to 3.13.0
Actions #11

Updated by pulpbot over 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF