Issue #2346
closed
repoview decode errors
Description
Hello all,
I've been tracking down publishing failures due to repoview generation not being able to decode to UTF-8.
[traceback] => Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 484, in __call__
return super(Task, self).__call__(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 103, in __call__
return super(PulpTask, self).__call__(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__
return self.run(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 971, in publish
result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1038, in check_publish
result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1090, in _do_publish
publish_report = publish_repo(transfer_repo, conduit, call_config)
File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 669, in wrap_f
return f(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
return self._publisher.process_lifecycle()
File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 564, in process_lifecycle
super(PluginStep, self).process_lifecycle()
File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 161, in process_lifecycle
step.process()
File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 263, in process
self._record_failure(e, tb)
File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 394, in _record_failure
error_details['error'] = str(e)
File "/usr/lib/python2.6/site-packages/pulp/server/exceptions.py", line 96, in __str__
return msg.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 113061: ordinal not in range(128)
After some investication, I've narrowed it down to this pattern:
This command (which pulp seems to be generating):
createrepo_c -d --update --keep-all-metadata --local-sqlite -s sha256 --skip-stat /var/lib/pulp/published/yum/http/repos/mirror/scientific/6x/x86_64/os/
Followed by:
repoview -f /var/lib/pulp/published/yum/http/repos/mirror/scientific/6x/x86_64/os/
Will generate the decoding error:
sqlite3.OperationalError: Could not decode to UTF-8 column 'location_href' with text ...
However, if I run the createrepo_c without the update or keep metadata flags:
createrepo_c -d --local-sqlite -s sha256 --skip-stat /var/lib/pulp/published/yum/http/repos/mirror/scientific/6x/x86_64/os/
The repoview command works like a charm.
Any thoughts?
Files
.