Project

Profile

Help

Issue #814

closed

UnicodeEncodeError during repository publish listing file step

Added by bvijaycom about 9 years ago. Updated about 5 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.5
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Easy Fix, Pulp 2
Sprint:
Quarter:

Description

Dear Team,

I have managed to created new epel repository and started syncing with cdn.redhat.com

**************************************************************************

Steps I have followed to create repository

#pulp-admin rpm repo create --repo-id rhel-6 --feed-ca-cert=/etc/rhsm/ca/redhat-uep.pem --feed-cert=/etc/pki/entitlement/7281296287748190629.pem --feed-key=/etc/pki/entitlement/7281296287748190629-key.pem --feed=https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os --retain-old-count=2 --validate=true --relative-url rhel-6 --serve-http=true --serve-https=false --gpg-key=/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

#pulp-admin rpm repo update --repod-id rhel-6 --proxy-host http://proxyserver --proxy-port 8080 --proxy-user username --proxy-pass yourpassword

#pulp-admin rpm repo sync run --repo-id rhel-6

*****************************************************************************
While syncing with

https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os

URL ,I am getting this below error.

[root@pocserver ~]# pulp-admin rpm repo sync run --repo-id rhel-6
--------------------------------------------------------------------
Synchronizing Repository [rhel-6]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs: 7662/7662 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[\]
... completed

Importing package groups/categories...
[/]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
7662 of 7662 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
2865 of 2865 items
... completed

Publishing Comps file
[==================================================] 100%
212 of 212 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[\]
... completed

Writing Listings File
[\]
... failed
'ascii' codec can't encode character u'\xa0' in position 6: ordinal not in
range(128)

Task Failed

'ascii' codec can't encode character u'\xa0' in position 6: ordinal not in
range(128)

[root@pocserver ~]#

**************************************Locale output******************

Here I have mentioned the locale command output...

[root@pocserver ~]# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

**************************


Files

pulp-file-list-error.txt (11.8 KB) pulp-file-list-error.txt bvijaycom, 03/25/2015 04:02 PM
Actions #1

Updated by rbarlow about 9 years ago

  • Subject changed from Repository sync issue - Writing Listings File error to UnicodeEncodeError during repository publish listing file step
  • Category deleted (22)

Here is the traceback:

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 316, in __call__
    return super(Task, self).__call__(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 98, in publish
    transfer_repo, conduit, call_config)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish
    publish_report = publish_repo(transfer_repo, conduit, call_config)
  File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 471, in wrap_f
    return f(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo
    return self._publisher.publish()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish
    return self.process_lifecycle()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle
    super(PluginStep, self).process_lifecycle()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle
    step.process()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 179, in process
    self._process_block()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 214, in _process_block
    self.process_main()
    self.process_main()
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 313, in process_main
    util.generate_listing_files(self.root_dir, self.target_dir)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/util.py", line 181, in generate_listing_files
    listing_handle.write('\n'.join(directories))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 6: ordinal not in range(128)
Actions #2

Updated by cduryee about 9 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to cduryee
Actions #3

Updated by cduryee about 9 years ago

  • Severity changed from 3. High to 2. Medium

steps to repro:

open vi, type "rhel-6" and then enter a "non-breaking space" before and after the string. To enter one, type Ctrl-K <space> <space>. Save this as a new file.

Then, cat this file and save to the clip board.

Create a repo but for relative-url, use the string you just created.

When publishing this repo, you should get the stacktrace listed above. Pulp should do a check when parsing to ensure that the relative-url is ascii (same as the check for repo-id).

Workaround: ensure that spaces surrounding relative url are "regular" spaces and not non-breaking spaces. I suspect this can happen when copying/pasting from a web page or other source.

Actions #4

Updated by cduryee about 9 years ago

  • Status changed from ASSIGNED to NEW

moving issue back to NEW state

Actions #5

Updated by cduryee about 9 years ago

  • Assignee deleted (cduryee)
Actions #6

Updated by mhrivnak about 9 years ago

  • Triaged changed from No to Yes
Actions #7

Updated by bmbouter about 5 years ago

  • Status changed from NEW to CLOSED - WONTFIX

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.

Actions #8

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added

Added by Mike DePaulo over 2 years ago

Revision 2036abf1 | View on GitHub

Drop support for upgrading from RPM packages prior to 3.6.

Fixes: #814

Added by Mike DePaulo over 2 years ago

Revision 2036abf1 | View on GitHub

Drop support for upgrading from RPM packages prior to 3.6.

Fixes: #814

Also available in: Atom PDF