Issue #814
closedUnicodeEncodeError during repository publish listing file step
Description
Dear Team,
I have managed to created new epel repository and started syncing with cdn.redhat.com
**************************************************************************
Steps I have followed to create repository
#pulp-admin rpm repo create --repo-id rhel-6 --feed-ca-cert=/etc/rhsm/ca/redhat-uep.pem --feed-cert=/etc/pki/entitlement/7281296287748190629.pem --feed-key=/etc/pki/entitlement/7281296287748190629-key.pem --feed=https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os --retain-old-count=2 --validate=true --relative-url rhel-6 --serve-http=true --serve-https=false --gpg-key=/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
#pulp-admin rpm repo update --repod-id rhel-6 --proxy-host http://proxyserver --proxy-port 8080 --proxy-user username --proxy-pass yourpassword
#pulp-admin rpm repo sync run --repo-id rhel-6
*****************************************************************************
While syncing with
https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os
URL ,I am getting this below error.
[root@pocserver ~]# pulp-admin rpm repo sync run --repo-id rhel-6
--------------------------------------------------------------------
Synchronizing Repository [rhel-6]
--------------------------------------------------------------------
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[|]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 7662/7662 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[\]
... completed
Importing package groups/categories...
[/]
... completed
Task Succeeded
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[==================================================] 100%
7662 of 7662 items
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[==================================================] 100%
2865 of 2865 items
... completed
Publishing Comps file
[==================================================] 100%
212 of 212 items
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Publishing files to web
[\]
... completed
Writing Listings File
[\]
... failed
'ascii' codec can't encode character u'\xa0' in position 6: ordinal not in
range(128)
Task Failed
'ascii' codec can't encode character u'\xa0' in position 6: ordinal not in
range(128)
[root@pocserver ~]#
**************************************Locale output******************
Here I have mentioned the locale command output...
[root@pocserver ~]# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
**************************
Files
Updated by rbarlow over 8 years ago
- Subject changed from Repository sync issue - Writing Listings File error to UnicodeEncodeError during repository publish listing file step
- Category deleted (
22)
Here is the traceback:
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 316, in __call__
return super(Task, self).__call__(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__
return self.run(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 98, in publish
transfer_repo, conduit, call_config)
File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish
publish_report = publish_repo(transfer_repo, conduit, call_config)
File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 471, in wrap_f
return f(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo
return self._publisher.publish()
File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish
return self.process_lifecycle()
File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle
super(PluginStep, self).process_lifecycle()
File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle
step.process()
File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 179, in process
self._process_block()
File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 214, in _process_block
self.process_main()
self.process_main()
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 313, in process_main
util.generate_listing_files(self.root_dir, self.target_dir)
File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/util.py", line 181, in generate_listing_files
listing_handle.write('\n'.join(directories))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 6: ordinal not in range(128)
Updated by cduryee over 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to cduryee
Updated by cduryee over 8 years ago
- Severity changed from 3. High to 2. Medium
steps to repro:
open vi, type "rhel-6" and then enter a "non-breaking space" before and after the string. To enter one, type Ctrl-K <space> <space>. Save this as a new file.
Then, cat this file and save to the clip board.
Create a repo but for relative-url, use the string you just created.
When publishing this repo, you should get the stacktrace listed above. Pulp should do a check when parsing to ensure that the relative-url is ascii (same as the check for repo-id).
Workaround: ensure that spaces surrounding relative url are "regular" spaces and not non-breaking spaces. I suspect this can happen when copying/pasting from a web page or other source.
Updated by cduryee over 8 years ago
- Status changed from ASSIGNED to NEW
moving issue back to NEW state
Updated by bmbouter over 4 years ago
- Status changed from NEW to CLOSED - WONTFIX
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
Added by Mike DePaulo almost 2 years ago
Added by Mike DePaulo almost 2 years ago
Drop support for upgrading from RPM packages prior to 3.6.
Fixes: #814
Drop support for upgrading from RPM packages prior to 3.6.
Fixes: #814