Project

Profile

Help

Issue #814

UnicodeEncodeError during repository publish listing file step

Added by bvijaycom over 6 years ago. Updated over 2 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.5
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Easy Fix, Pulp 2
Sprint:
Quarter:

Description

Dear Team,

I have managed to created new epel repository and started syncing with cdn.redhat.com

**************************************************************************

Steps I have followed to create repository

#pulp-admin rpm repo create --repo-id rhel-6 --feed-ca-cert=/etc/rhsm/ca/redhat-uep.pem --feed-cert=/etc/pki/entitlement/7281296287748190629.pem --feed-key=/etc/pki/entitlement/7281296287748190629-key.pem --feed=https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os --retain-old-count=2 --validate=true --relative-url rhel-6 --serve-http=true --serve-https=false --gpg-key=/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

#pulp-admin rpm repo update --repod-id rhel-6 --proxy-host http://proxyserver --proxy-port 8080 --proxy-user username --proxy-pass yourpassword

#pulp-admin rpm repo sync run --repo-id rhel-6

*****************************************************************************
While syncing with

https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os

URL ,I am getting this below error.

[root@pocserver ~]# pulp-admin rpm repo sync run --repo-id rhel-6
--------------------------------------------------------------------
Synchronizing Repository [rhel-6]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs: 7662/7662 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[\]
... completed

Importing package groups/categories...
[/]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
7662 of 7662 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
2865 of 2865 items
... completed

Publishing Comps file
[==================================================] 100%
212 of 212 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[\]
... completed

Writing Listings File
[\]
... failed
'ascii' codec can't encode character u'\xa0' in position 6: ordinal not in
range(128)

Task Failed

'ascii' codec can't encode character u'\xa0' in position 6: ordinal not in
range(128)

[root@pocserver ~]#

**************************************Locale output******************

Here I have mentioned the locale command output...

[root@pocserver ~]# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

**************************

pulp-file-list-error.txt (11.8 KB) pulp-file-list-error.txt bvijaycom, 03/25/2015 04:02 PM

History

#1 Updated by rbarlow over 6 years ago

  • Subject changed from Repository sync issue - Writing Listings File error to UnicodeEncodeError during repository publish listing file step
  • Category deleted (22)

Here is the traceback:

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 316, in __call__
    return super(Task, self).__call__(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 98, in publish
    transfer_repo, conduit, call_config)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish
    publish_report = publish_repo(transfer_repo, conduit, call_config)
  File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 471, in wrap_f
    return f(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo
    return self._publisher.publish()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish
    return self.process_lifecycle()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle
    super(PluginStep, self).process_lifecycle()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle
    step.process()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 179, in process
    self._process_block()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 214, in _process_block
    self.process_main()
    self.process_main()
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 313, in process_main
    util.generate_listing_files(self.root_dir, self.target_dir)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/util.py", line 181, in generate_listing_files
    listing_handle.write('\n'.join(directories))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 6: ordinal not in range(128)

#2 Updated by cduryee over 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to cduryee

#3 Updated by cduryee over 6 years ago

  • Severity changed from 3. High to 2. Medium

steps to repro:

open vi, type "rhel-6" and then enter a "non-breaking space" before and after the string. To enter one, type Ctrl-K <space> <space>. Save this as a new file.

Then, cat this file and save to the clip board.

Create a repo but for relative-url, use the string you just created.

When publishing this repo, you should get the stacktrace listed above. Pulp should do a check when parsing to ensure that the relative-url is ascii (same as the check for repo-id).

Workaround: ensure that spaces surrounding relative url are "regular" spaces and not non-breaking spaces. I suspect this can happen when copying/pasting from a web page or other source.

#4 Updated by cduryee over 6 years ago

  • Status changed from ASSIGNED to NEW

moving issue back to NEW state

#5 Updated by cduryee over 6 years ago

  • Assignee deleted (cduryee)

#6 Updated by mhrivnak over 6 years ago

  • Triaged changed from No to Yes

#7 Updated by bmbouter over 2 years ago

  • Status changed from NEW to CLOSED - WONTFIX

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.

#8 Updated by bmbouter over 2 years ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF