Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #2622

closed

Sync fails when non-ASCII characters are present in primary.xml

Added by ttereshc over 7 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
ttereshc
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.12.2
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 16
Quarter:

Description

Before this change https://github.com/pulp/pulp_rpm/pull/1028, utf-8 decoding for primary.xml snippets happened inside the change_location_tag.
After that change decoding happens outside of it but only for upload case, so sync case has to be adjusted to this change or decoding should be moved back to change_location_tag.

To reproduce, create and sync EPEL repo:

pulp-admin rpm repo create --repo-id epel7 --relative-url epel7 --feed https://dl.fedoraproject.org/pub/epel/7/x86_64/ --download-policy on_demand
pulp-admin rpm repo sync  run --repo-id epel7

Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072) 'ascii' codec can't decode byte 0xc3 in position 884: ordinal not in range(128)
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072) Traceback (most recent call last):
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 269, in run
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)     self.update_content(metadata_files, url)
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 549, in update_c
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)     self.download_rpms(metadata_files, rpms_to_download, url)
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 780, in download
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)     unit = self.add_rpm_unit(metadata_files, unit)
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 702, in add_rpm_
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)     metadata_files.add_repodata(unit)
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/repomd/metadata.py", line 372, 
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)     model.repodata['primary'] = change_location_tag(raw_xml, model.filename)
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)   File "/home/user1/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/parse/rpm.py", line 75, in chan
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072)     return first_portion + location + end_portion
Mar 07 11:01:39 dev pulp[16378]: pulp_rpm.plugins.importers.yum.sync:ERROR: (16378-91072) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 884: ordinal not in range(128)
Mar 07 11:01:39 dev pulp[16378]: pulp.server.async.tasks:INFO: Task failed : [b8b85b93-dd63-487f-8b40-ce5384bdd9a8]
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072) Task pulp.server.managers.repo.sync.sync[b8b85b93-dd63-487f-8b40-ce5384bdd9a8] raised unexpected: PulpExecutionException('I
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072) Traceback (most recent call last):
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072)     R = retval = fun(*args, **kwargs)
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072)   File "/home/user1/devel/pulp/server/pulp/server/async/tasks.py", line 506, in __call__
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072)     return super(Task, self).__call__(*args, **kwargs)
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072)   File "/home/user1/devel/pulp/server/pulp/server/async/tasks.py", line 107, in __call__
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072)     return super(PulpTask, self).__call__(*args, **kwargs)
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072)     return self.run(*args, **kwargs)
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072)   File "/home/user1/devel/pulp/server/pulp/server/controllers/repository.py", line 820, in sync
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Mar 07 11:01:39 dev pulp[16314]: celery.worker.job:ERROR: (16314-91072) PulpExecutionException: Importer indicated a failed response

Related issues

Related to RPM Support - Issue #1903: RPM import traceback (non-utf-8 metadata slipping through)CLOSED - CURRENTRELEASEjortel@redhat.comActions
Actions
Send by e-mail Copy link

Also available in: Atom PDF