Project

Profile

Help

Download (24.7 KB)

Issue #1903 ยป Issue #11490 - 2016-05-16T13_39_37Z.eml

darkfader, 05/16/2016 03:39 PM

 
Return-Path: <florian.heigl@gmail.com>
Received: from mi012.mc1.hosteurope.de ([80.237.138.243]) by wp245.webpack.hosteurope.de running ExIM with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) id 1b2Ik1-0008Cv-Au; Mon, 16 May 2016 15:39:25 +0200
Received: from mail-ig0-f174.google.com ([209.85.213.174]) by mx0.webpack.hosteurope.de (mi012.mc1.hosteurope.de) with esmtps (TLSv1.2:AES128-GCM-SHA256:128) id 1b2Ijz-0005xa-8h for dropbox+pulp+c71e+pulp@plan.io; Mon, 16 May 2016 15:39:25 +0200
Received: by mail-ig0-f174.google.com with SMTP id qe5so49612588igc.1 for <dropbox+pulp+c71e+pulp@plan.io>; Mon, 16 May 2016 06:39:23 -0700
Received: by 10.64.121.162 with HTTP; Mon, 16 May 2016 06:39:22 -0700
Received: by 10.64.121.162 with HTTP; Mon, 16 May 2016 06:39:22 -0700
Date: Mon, 16 May 2016 15:39:22 +0200
From: Florian Heigl <florian.heigl@gmail.com>
To: Pulp <dropbox+pulp+c71e+pulp@plan.io>
Message-ID: <CAFivhP=6bd0qUtZdYk4rn3yc0_PjJEpAhGGGtaS_SjoKJpAr+Q@mail.gmail.com>
In-Reply-To: <redmine.journal-11489.20160516132618.656f8c8197aadcf7@plan.io>
References: <redmine.issue-1903.20160509152844@plan.io>
<redmine.journal-11489.20160516132618.656f8c8197aadcf7@plan.io>
Subject: Re: [Pulp - Issue #1903] RPM import traceback (non-utf-8 metadata
slipping through)
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary=047d7b10c88389ffeb0532f5c0e9;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Delivery-date: Mon, 16 May 2016 15:39:25 +0200
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to;
bh=Y9H9cqjCMPCqSMV34vUbG3anYMVeDgvIW6d8kR76A64=;
b=pZHS89r6tSZqX6Zpdp7Ggpqq0akfuBseEkqg93MYvp1lKFpOvoexU0bdfzAVtIcJdv
vl/MMC1QhkIAmM9Z1oJChheqhKoStw3wuLfryZ3eRgSO4uBh1iGC8VeECyRrrWu6A92w
OA2669hTDft6ChoktdQdBaS58jBAp0UfKs5ieeeJOGJiW1xTcsvwZIVGmWxAO16EA1zO
1kyYRE4BJd6Z5I3WNZ8JEBfySu5gZ3fiQ5z8AKv31kr5s8YN18n+XzRTdFs04haDY2Vp
dJ0FD0cdvVLxSzrjstHlh0lENBaY8iPVCIpHM9UF9d6uFFTqFjkUeYVBqy4EodG5573b drVQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net;
s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date
:message-id:subject:from:to; bh=Y9H9cqjCMPCqSMV34vUbG3anYMVeDgvIW6d8kR76A64=;
b=j7dum8XXyv+zrcvwZ1z/s5Yo4LbpyWhvtKeTqHZ07vBs9jK2y+Vz26xqYXC+QmhUVK
6oP5cUsjbm1GvJNkR4Qiv997vXkh6CAaNltFfNCNp4NQYSEqcCGn80SYqb1UMmzSC2j0
8F+KKRl6LCO+6RsoiUa/ItK6TR8i+Ctm3GmILW1L3fStlgP0H0Sc07x4wRtmZR4VY69b
G0jSlwndAVFT11JSZlrdLTns4/VnT4IJbxg45IVv47bxQA4r976z5dJMtcFE/bMyChxt
R4LmqUwXKiMGET/RntSE2/JkeUA2exgd9YdytjQx2J7VMhebR7YDL3etkdwoDU8crF4t pw8Q==
X-Gm-Message-State: AOPr4FUtmjKqaR1+S8TvBd4fZHrKvzIvg+rA4LA3izQ0Dmm0vE+KoX+AJVcsolJeqTbjsAo1cj2v3kbkDx73Og==
X-Received: by 10.50.22.41 with SMTP id a9mr10355193igf.45.1463405962196; Mon,
16 May 2016 06:39:22 -0700 (PDT)
X-HE-Spam-Level: +
X-HE-Spam-Score: 1.8
X-HE-Spam-Report: Content analysis details: (1.8 points) pts rule name
description ---- ----------------------
-------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL:
Sender listed at http://www.dnswl.org/, low trust [209.85.213.174 listed in
list.dnswl.org] 2.5 RCVD_IN_SORBS_HTTP RBL: SORBS: sender is open HTTP proxy
server [209.85.213.174 listed in dnsbl.sorbs.net] 0.0 FREEMAIL_FROM Sender
email is commonly abused enduser mail provider (florian.heigl[at]gmail.com)
0.1 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has
a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has
at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or
DK signature, not necessarily valid
X-HE-SPF: PASSED
Envelope-to: dropbox+pulp+c71e+pulp@plan.io


--047d7b10c88389ffeb0532f5c0e9
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: quoted-printable

I'll currently traveling but will write something about 'sensible' as soo=
n
as I can.
Basically, You can pick between not correct or useless to the main purpos=
e.

Florian
Am 16.05.2016 3:26 nachm. schrieb "Pulp" <dropbox+pulp+c71e+pulp@plan.io>=
:

> --- Please write your response above this line ---
> Issue #1903 has been updated by rbarlow.
>
>
> The attached patch is dangerous. It's a dirty hack to hide an error
> message. The only reason it "works" (and I use that term loosely becaus=
e it
> doesn't actually work, it just hides the error message) is that latin-1=

> uses the full 8 bits, and so no codepoint can raise an error since they=
are
> all "valid". The most populous continent of the world does not (and can=

> not) use latin-1. RPMs encoded from those languages will result in garb=
age
> data throughout the RPM. If Pulp's fields are not valuable to the point=

> that we are willing to fill them with garbage data intentionally, then =
why
> not just remove the fields?
>
> The sensible approach is to raise an error message for RPMs that do not=

> conform to the standard (UTF-8, which includes ASCII). This allows the =
user
> to take appropriate action empowered with a correct error message. Rath=
er
> than walking away believing that Pulp is garbage and it ate their data,=

> they can instead solve the problem or bring the matter to the vendor of=
the
> RPM so they can solve it.
> ------------------------------
> Issue #1903: RPM import traceback (non-utf-8 metadata slipping through)=

> <https://pulp.plan.io/issues/1903?pn=3D1#change-11489>
>
> - Author: darkfader
> - Status: NEW
> - Priority: Normal
> - Assignee:
> - Category:
> - Sprint/Milestone:
> - Severity: 2. Medium
> - Version: 2.8.2
> - Target Platform Release:
> - OS: CentOS 7
> - Backwards Incompatible: No
> - Triaged: No
> - Tags:
> - QA Contact:
> - Complexity:
>
> Hi,
>
> I'm getting a "special" problem with a few RPMs.
> Those RPMs have finnish alphabet characters in their author's names in =
the
> %description metadata.
>
> Authors:
> --------
> Juha Yrj<F6>l<E4> <jyrjola@cc.hut.fi>
> Antti Tapaninen <aet@cc.hut.fi>
> Timo Ter<E4>s <timo.teras@iki.fi>
> Olaf Kirch <okir@suse.de>
> %files
>
> When you run them through iconv from latin to utf-8 it all magically
> clears up:
>
> iconv -f iso8859-1 -t utf-8 opensc.spec | grep -A6 Authors
> Authors:
> --------
> Juha Yrj=C3=B6l=C3=A4 <jyrjola@cc.hut.fi>
> Antti Tapaninen <aet@cc.hut.fi>
> Timo Ter=C3=A4s <timo.teras@iki.fi>
> Olaf Kirch <okir@suse.de>
> %files
>
> So my impression is that the SPEC file has utf-8 content (yay) but is i=
n
> fact stored as latin1 (NOES).
>
> Trying to upload such a package of them triggers a traceback.
>
> We try to mirror all packages in all versions we need, so a re-issue of=

> the package doesn't really solve matters.
> (Example to clarify this: Let's assume they're on one of the OS DVD's a=
nd
> we want to mirror them 1:1)
>
> An example package is opensc-0.11.6-5.27.1.x86_64.rpm from SLES11SP2:
> It can apparently be obtained via
>
> http://mirror.mes.edu.cu/SLES_11_SP2/CD1/suse/x86_64/opensc-0.11.6-5.27=
.1.x86_64.rpm
>
> md5: 0c463515b28998ac9966400e5d14588d
>
> The traceback looks like this:
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) unexpected e=
rror
> occurred importing uploaded file
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) Traceback (m=
ost
> recent call last):
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) File
> "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/upload=
.py",
> line 118, in upload
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912)
> handlers[type_id](repo, type_id, unit_key, metadata, file_path, conduit=
,
> config)
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) File
> "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/upload=
.py",
> line 390, in _handle_package
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912)
> unit.save_and_import_content(file_path)
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) File
> "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", li=
ne
> 802, in save_and_import_content
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) self.save()
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) File
> "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 324, i=
n
> save
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) object_id =3D=

> collection.save(doc, **write_concern)
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) File
> "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 2180, =
in
> save
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) check_keys,
> False, manipulate, write_concern)
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) File
> "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 709, i=
n
> _update
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912)
> codec_options=3Dself.codec_options).copy()
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) File
> "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 216, in comm=
and
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912)
> self._raise_connection_failure(error)
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) File
> "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 343, in
> _raise_connection_failure
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) raise error
> May 06 14:49:18 myhost pulp[27466]:
> pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912)
> InvalidStringData: strings in documents must be valid UTF-8: 'OpenSC
> provides a set of libraries and utilities to access smart cards.\nIt ma=
inly
> focuses on cards that support cryptographic operations. It\nfacilitates=

> their use in security applications such as mail
> encryption,\nauthentication, and digital signature. OpenSC implements t=
he
> PKCS#11\nAPI. Applications supporting this API, such as Mozilla Firefox=

> and\nThunderbird, can use it. OpenSC implements the PKCS#15 standard
> and\naims to be compatible with every software that does so, too.\n\nBe=
fore
> purchasing any cards, please read carefully documentation
> in\n/usr/share/doc/packages/opensc/wiki/index.html - only some cards
> are\nsupported. Not only card type matters, but also card version, card=

> OS\nversion and preloaded applet. Only subset of possible operations ma=
y
> be\nsupported for your card. Card initialization may require third
> party\nproprietary software.\n\n\n\nAuthors:\n--------\n Juha Yrj\xf6l\=
xe4 <
> jyrjola@cc.hut.fi>\n Antti Tapaninen <aet@cc.hut.fi>\n Timo Ter\xe4s <
> timo.teras@iki.fi>\n Olaf Kirch <okir@suse.de>'
> May 06 14:49:18 myhost pulp[27466]: py.warnings:WARNING: (27466-26912)
> /usr/lib/python2.7/site-packages/mongoengine/document.py:367:
> DeprecationWarning: update is deprecated. Use replace_one, update_one o=
r
> update_many instead.
> May 06 14:49:18 myhost pulp[27466]: py.warnings:WARNING: (27466-26912)
> upsert=3Dupsert, **write_concern)
> May 06 14:49:18 myhost pulp[27466]: py.warnings:WARNING: (27466-26912)
>
> I've been hinted to look at string_to_unicode(data) in
> pulp_rpm/yum_plugin/util.py which in my test runs all but two lines thr=
ough
> the utf-8 path and the two with the 'umlauts' fail there and (supposedl=
y)
> run through the iso-8859-1 path.
>
> That seems to work but where later, when running self.save() it seems
> there are demons.
>
> I'm attaching the extracted SPEC, too.
>
> Since the issue occurs basically with some random RPMs I don't have
> influence on I'd love to help improve the input handling here so it
> generally survives them.
> Files opensc.spec
> <https://pulp.plan.io/attachments/download/240/opensc.spec> (22.2 KB)
> 1903.patch <https://pulp.plan.io/attachments/download/242/1903.patch>
> (1.13 KB)
> ------------------------------
>
> You have received this notification because you have either subscribed =
to
> or are involved in a project on Pulp Planio.
> To change your notification preferences, please click here:
> https://pulp.plan.io/my/account?tour=3Dmail_preferences
>
> This notification was cheerfully delivered by <https://plan.io/>
> [image: Planio] <https://plan.io/>
>

--047d7b10c88389ffeb0532f5c0e9
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<p dir=3D"ltr">I&#39;ll currently traveling but will write something abou=
t &#39;sensible&#39; as soon as I can.<br>
Basically, You can pick between not correct or useless to the main purpos=
e.</p>
<p dir=3D"ltr">Florian</p>
<div class=3D"gmail_quote">Am 16.05.2016 3:26 nachm. schrieb &quot;Pulp&q=
uot; &lt;<a href=3D"mailto:dropbox%2Bpulp%2Bc71e%2Bpulp@plan.io">dropbox+=
pulp+c71e+pulp@plan.io</a>&gt;:<br type=3D"attribution"><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pa=
dding-left:1ex">



<div>
<table width=3D"100%">
<tr><td style=3D"font-family:MarketWeb,Verdana,sans-serif;font-size:0.8em=
;text-align:center;width:100%;color:#d7d7d7"><p>--- Please write your res=
ponse above this line ---</p></td></tr>
<tr><td>Issue #1903 has been updated by rbarlow.

<ul>
</ul>

<p>The attached patch is dangerous. It&#39;s a dirty hack to hide an erro=
r message. The only reason it &quot;works&quot; (and I use that term loos=
ely because it doesn&#39;t actually work, it just hides the error message=
) is that latin-1 uses the full 8 bits, and so no codepoint can raise an =
error since they are all &quot;valid&quot;. The most populous continent o=
f the world does not (and can not) use latin-1. RPMs encoded from those l=
anguages will result in garbage data throughout the RPM. If Pulp&#39;s fi=
elds are not valuable to the point that we are willing to fill them with =
garbage data intentionally, then why not just remove the fields?</p>


<p>The sensible approach is to raise an error message for RPMs that do n=
ot conform to the standard (UTF-8, which includes ASCII). This allows the=
user to take appropriate action empowered with a correct error message. =
Rather than walking away believing that Pulp is garbage and it ate their =
data, they can instead solve the problem or bring the matter to the vendo=
r of the RPM so they can solve it.</p>
<hr>
<h1><a href=3D"https://pulp.plan.io/issues/1903?pn=3D1#change-11489" targ=
et=3D"_blank">Issue #1903: RPM import traceback (non-utf-8 metadata slipp=
ing through)</a></h1>

<ul><li>Author: darkfader</li>
<li>Status: NEW</li>
<li>Priority: Normal</li>
<li>Assignee: </li>
<li>Category: </li>
<li>Sprint/Milestone: </li>
<li>Severity: 2. Medium</li>
<li>Version: 2.8.2</li>
<li>Target Platform Release: </li>
<li>OS: CentOS 7</li>
<li>Backwards Incompatible: No</li>
<li>Triaged: No</li>
<li>Tags: </li>
<li>QA Contact: </li>
<li>Complexity: </li></ul>

<p>Hi,</p>


<p>I&#39;m getting a &quot;special&quot; problem with a few RPMs.<br>Tho=
se RPMs have finnish alphabet characters in their author&#39;s names in t=
he %description metadata.</p>


<p><code>Authors:<br>--------<br> Juha Yrj&lt;F6&gt;l&lt;E4&gt; &lt;<=
a href=3D"mailto:jyrjola@cc.hut.fi" target=3D"_blank">jyrjola@cc.hut.fi</=
a>&gt;<br> Antti Tapaninen &lt;<a href=3D"mailto:aet@cc.hut.fi" target=
=3D"_blank">aet@cc.hut.fi</a>&gt;<br> Timo Ter&lt;E4&gt;s &lt;<a href=3D=
"mailto:timo.teras@iki.fi" target=3D"_blank">timo.teras@iki.fi</a>&gt;<br=
> Olaf Kirch &lt;<a href=3D"mailto:okir@suse.de" target=3D"_blank">oki=
r@suse.de</a>&gt;<br>%files</code></p>


<p>When you run them through iconv from latin to utf-8 it all magically =
clears up:</p>


<p><code>iconv -f iso8859-1 -t utf-8 opensc.spec | grep -A6 Authors<br>A=
uthors:<br>--------<br> Juha Yrj=C3=B6l=C3=A4 &lt;<a href=3D"mailto:jy=
rjola@cc.hut.fi" target=3D"_blank">jyrjola@cc.hut.fi</a>&gt;<br> Antti=
Tapaninen &lt;<a href=3D"mailto:aet@cc.hut.fi" target=3D"_blank">aet@cc.=
hut.fi</a>&gt;<br> Timo Ter=C3=A4s &lt;<a href=3D"mailto:timo.teras@ik=
i.fi" target=3D"_blank">timo.teras@iki.fi</a>&gt;<br> Olaf Kirch &lt;<=
a href=3D"mailto:okir@suse.de" target=3D"_blank">okir@suse.de</a>&gt;<br>=
%files</code></p>


<p>So my impression is that the SPEC file has utf-8 content (yay) but is=
in fact stored as latin1 (NOES).</p>


<p>Trying to upload such a package of them triggers a traceback.</p>


<p>We try to mirror all packages in all versions we need, so a re-issue =
of the package doesn&#39;t really solve matters.<br>(Example to clarify t=
his: Let&#39;s assume they&#39;re on one of the OS DVD&#39;s and we want =
to mirror them 1:1)</p>


<p>An example package is opensc-0.11.6-5.27.1.x86_64.rpm from SLES11SP2:=
<br>It can apparently be obtained via <br><a href=3D"http://mirror.mes.ed=
u.cu/SLES_11_SP2/CD1/suse/x86_64/opensc-0.11.6-5.27.1.x86_64.rpm" target=3D=
"_blank">http://mirror.mes.edu.cu/SLES_11_SP2/CD1/suse/x86_64/opensc-0.11=
.6-5.27.1.x86_64.rpm</a></p>


<p>md5: 0c463515b28998ac9966400e5d14588d</p>


<p>The traceback looks like this:<br><code>May 06 14:49:18 myhost pulp[2=
7466]: pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) unexpec=
ted error occurred importing uploaded file<br>May 06 14:49:18 myhost pulp=
[27466]: pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) Trace=
back (most recent call last):<br>May 06 14:49:18 myhost pulp[27466]: pulp=
_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) File &quot;/usr/=
lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/upload.py&quot=
;, line 118, in upload<br>May 06 14:49:18 myhost pulp[27466]: pulp_rpm.pl=
ugins.importers.yum.upload:ERROR: (27466-26912) handlers[type_id](rep=
o, type_id, unit_key, metadata, file_path, conduit, config)<br>May 06 14:=
49:18 myhost pulp[27466]: pulp_rpm.plugins.importers.yum.upload:ERROR: (2=
7466-26912) File &quot;/usr/lib/python2.7/site-packages/pulp_rpm/plugin=
s/importers/yum/upload.py&quot;, line 390, in _handle_package<br>May 06 1=
4:49:18 myhost pulp[27466]: pulp_rpm.plugins.importers.yum.upload:ERROR: =
(27466-26912) unit.save_and_import_content(file_path)<br>May 06 14:49=
:18 myhost pulp[27466]: pulp_rpm.plugins.importers.yum.upload:ERROR: (274=
66-26912) File &quot;/usr/lib/python2.7/site-packages/pulp/server/db/mo=
del/__init__.py&quot;, line 802, in save_and_import_content<br>May 06 14:=
49:18 myhost pulp[27466]: pulp_rpm.plugins.importers.yum.upload:ERROR: (2=
7466-26912) self.save()<br>May 06 14:49:18 myhost pulp[27466]: pulp_r=
pm.plugins.importers.yum.upload:ERROR: (27466-26912) File &quot;/usr/li=
b/python2.7/site-packages/mongoengine/document.py&quot;, line 324, in sav=
e<br>May 06 14:49:18 myhost pulp[27466]: pulp_rpm.plugins.importers.yum.u=
pload:ERROR: (27466-26912) object_id =3D collection.save(doc, **write=
_concern)<br>May 06 14:49:18 myhost pulp[27466]: pulp_rpm.plugins.importe=
rs.yum.upload:ERROR: (27466-26912) File &quot;/usr/lib64/python2.7/site=
-packages/pymongo/collection.py&quot;, line 2180, in save<br>May 06 14:49=
:18 myhost pulp[27466]: pulp_rpm.plugins.importers.yum.upload:ERROR: (274=
66-26912) check_keys, False, manipulate, write_concern)<br>May 06 14:=
49:18 myhost pulp[27466]: pulp_rpm.plugins.importers.yum.upload:ERROR: (2=
7466-26912) File &quot;/usr/lib64/python2.7/site-packages/pymongo/colle=
ction.py&quot;, line 709, in _update<br>May 06 14:49:18 myhost pulp[27466=
]: pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) codec_o=
ptions=3Dself.codec_options).copy()<br>May 06 14:49:18 myhost pulp[27466]=
: pulp_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) File &quot=
;/usr/lib64/python2.7/site-packages/pymongo/pool.py&quot;, line 216, in c=
ommand<br>May 06 14:49:18 myhost pulp[27466]: pulp_rpm.plugins.importers.=
yum.upload:ERROR: (27466-26912) self._raise_connection_failure(error)=
<br>May 06 14:49:18 myhost pulp[27466]: pulp_rpm.plugins.importers.yum.up=
load:ERROR: (27466-26912) File &quot;/usr/lib64/python2.7/site-packages=
/pymongo/pool.py&quot;, line 343, in _raise_connection_failure<br>May 06 =
14:49:18 myhost pulp[27466]: pulp_rpm.plugins.importers.yum.upload:ERROR:=
(27466-26912) raise error<br>May 06 14:49:18 myhost pulp[27466]: pul=
p_rpm.plugins.importers.yum.upload:ERROR: (27466-26912) InvalidStringData=
: strings in documents must be valid UTF-8: &#39;OpenSC provides a set of=
libraries and utilities to access smart cards.\nIt mainly focuses on car=
ds that support cryptographic operations. It\nfacilitates their use in se=
curity applications such as mail encryption,\nauthentication, and digital=
signature. OpenSC implements the PKCS#11\nAPI. Applications supporting t=
his API, such as Mozilla Firefox and\nThunderbird, can use it. OpenSC imp=
lements the PKCS#15 standard and\naims to be compatible with every softwa=
re that does so, too.\n\nBefore purchasing any cards, please read careful=
ly documentation in\n/usr/share/doc/packages/opensc/wiki/index.html - onl=
y some cards are\nsupported. Not only card type matters, but also card ve=
rsion, card OS\nversion and preloaded applet. Only subset of possible ope=
rations may be\nsupported for your card. Card initialization may require =
third party\nproprietary software.\n\n\n\nAuthors:\n--------\n Juha Yr=
j\xf6l\xe4 &lt;<a href=3D"mailto:jyrjola@cc.hut.fi" target=3D"_blank">jyr=
jola@cc.hut.fi</a>&gt;\n Antti Tapaninen &lt;<a href=3D"mailto:aet@cc.=
hut.fi" target=3D"_blank">aet@cc.hut.fi</a>&gt;\n Timo Ter\xe4s &lt;<a=
href=3D"mailto:timo.teras@iki.fi" target=3D"_blank">timo.teras@iki.fi</a=
>&gt;\n Olaf Kirch &lt;<a href=3D"mailto:okir@suse.de" target=3D"_blan=
k">okir@suse.de</a>&gt;&#39;<br>May 06 14:49:18 myhost pulp[27466]: py.wa=
rnings:WARNING: (27466-26912) /usr/lib/python2.7/site-packages/mongoengin=
e/document.py:367: DeprecationWarning: update is deprecated. Use replace_=
one, update_one or update_many instead.<br>May 06 14:49:18 myhost pulp[27=
466]: py.warnings:WARNING: (27466-26912) upsert=3Dupsert, **write_conce=
rn)<br>May 06 14:49:18 myhost pulp[27466]: py.warnings:WARNING: (27466-26=
912)</code></p>


<p>I&#39;ve been hinted to look at string_to_unicode(data) in pulp_rpm/=
yum_plugin/util.py which in my test runs all but two lines through the ut=
f-8 path and the two with the &#39;umlauts&#39; fail there and (supposedl=
y) run through the iso-8859-1 path.</p>


<p>That seems to work but where later, when running self.save() it seems=
there are demons.</p>


<p>I&#39;m attaching the extracted SPEC, too.</p>


<p>Since the issue occurs basically with some random RPMs I don&#39;t ha=
ve influence on I&#39;d love to help improve the input handling here so i=
t generally survives them.</p>

<fieldset><legend>Files</legend>
<a href=3D"https://pulp.plan.io/attachments/download/240/opensc.spec"=
target=3D"_blank">opensc.spec</a>
(22.2 KB)<br>
<a href=3D"https://pulp.plan.io/attachments/download/242/1903.patch" =
target=3D"_blank">1903.patch</a>
(1.13 KB)<br>
</fieldset>



</td></tr>
<tr><td style=3D"font-size:0.8em;width:100%"><hr><p>You have received thi=
s notification because you have either subscribed to or are involved in a=
project on Pulp Planio.<br>To change your notification preferences, plea=
se click here: <a href=3D"https://pulp.plan.io/my/account?tour=3Dmail_pre=
ferences" target=3D"_blank">https://pulp.plan.io/my/account?tour=3Dmail_p=
references</a></p><td></td></td></tr>
<tr><td style=3D"font-family:MarketWeb,Verdana,sans-serif;font-size:1.2em=
;text-align:center;width:100%;color:#d7d7d7"><br><div><a href=3D"https://=
plan.io/" style=3D"color:#d7d7d7;text-decoration:none" target=3D"_blank">=
This notification was cheerfully delivered by</a></div><td></td></td></tr=
>
<tr><td style=3D"text-align:center;width:100%"><a href=3D"https://plan.io=
/" title=3D"Planio" target=3D"_blank"><img src=3D"https://assets.plan.io/=
images/planio_logo_gray_204x50.png" height=3D"25" width=3D"102" alt=3D"Pl=
anio" style=3D"vertical-align:middle"></a></td></tr>
</table>
</div>

</blockquote></div>

--047d7b10c88389ffeb0532f5c0e9--
    (1-1/1)