Debian Support

Working on reproducing this locally.

... and I ran out of space. Is there any chance you can reproduce this on a smaller repository?

So far I've only found ubuntu universe, and jessie-main. I assume jessie-main is by far smaller, yet you'll still need a sizeable disk. I think it happens on wheezy-main as well, but again. Still somewhat large.

Update: I've pinned the package for jessie-main to be "aspell-is". I had the hope of reproducing this error in full by creating an empty repo and upload only the broken package. However, while it does not publish the package, the error message is far less clear:

Feb 24 18:06:42 pulp01vp.office.noris.de pulp[2204]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[8e15464b-c90f-452c-8b54-18cb68bc7b1f]
Feb 24 18:06:42 pulp01vp.office.noris.de pulp[2302]: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[71c030f1-4ca5-44f9-baa1-ca95c3b69cc3]
Feb 24 18:06:42 pulp01vp.office.noris.de pulp[2204]: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[8e15464b-c90f-452c-8b54-18cb68bc7b1f] succeeded in 0.0136467898265s: None
Feb 24 18:06:42 pulp01vp.office.noris.de pulp[2302]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[f45eb252-5f80-4e85-abae-b8f07022ebef]
Feb 24 18:06:42 pulp01vp.office.noris.de pulp[2302]: celery.worker.job:INFO: Task pulp.server.managers.repo.publish.publish[71c030f1-4ca5-44f9-baa1-ca95c3b69cc3] succeeded in 0.0346576478332s: {'exception': None, 'repo_id': u'aspell-test', 'traceback': None, 'started': '2018-02-24T17:06:42Z', '_ns':...
Feb 24 18:06:42 pulp01vp.office.noris.de pulp[2302]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[f45eb252-5f80-4e85-abae-b8f07022ebef] succeeded in 0.0030136150308s: None

(And yes, this is everything that shows up in the logs when trying to publish the single broken package.)

Awesome, I have a reproducer.

Downloading this:

http://ubuntu-master.mirror.tudos.de/ubuntu/pool/universe/a/aspell-is/aspell-is_0.51-0-4_all.deb

And then running:

python -c "import sys; from debpkgr import debpkg; debpkg.DebPkg.from_file(sys.argv[1])" aspell-is_0.51-0-4_all.deb

On first look, the fix is really ugly.

If the package is really old, the filenames may be encoded with ISO-8859-1.

I could try to open the file as utf-8 and fall back to iso-8859-1 if I get a UnicodeDecodeError.

Unfortunately, debian.debfile.DebPart.get_file wraps the file in a codecs.EncodedFile object, but fails to specify the output encoding as UTF-8, so I end up with a bunch of strings in the original encoding.

Subclassing DebFile and overwriting get_file may be the way to go, since I don't know how to provide patches to upstream python-debian

The command to test the issue needs to be:

python -c "import sys; from debpkgr import debpkg; debpkg.DebPkg.from_file(sys.argv[1])" aspell-is_0.51-0-4_all.deb

mihai.ibanescu@gmail.com wrote:

Awesome, I have a reproducer.

Downloading this:

http://ubuntu-master.mirror.tudos.de/ubuntu/pool/universe/a/aspell-is/aspell-is_0.51-0-4_all.deb

And then running:

python -c "import sys; from debpkgr import debpkg; debpkg.DebPkg.from_file(sys.argv[1])" ~/Downloads/aspell-is_0.51-0-4_all.deb

I'm not 100% sure, but the following file in aspell looks pretty bad:

rw-r--r- root/root 72 2004-03-10 12:23 ./usr/lib/aspell/\355slenska.alias

That is indeed the problem.

It is an old package, that has filenames with a non-UTF8 encoding. That breaks python-debian.

I had a look at newer versions of python-debian and it doesn't fix the issue. Therefore, do you know how to fix / add a workaround for python-debpkgr?

BTW, the git repo for python-debian can be found here: https://anonscm.debian.org/gitweb/?p=pkg-python-debian/python-debian.git

I had started working on a fix, but realized it's ugly, and backed away from it. And then other things became priorities.

I will try to get back to it today/tomorrow.

Merge request for upstream python-debian: https://salsa.debian.org/python-debian-team/python-debian/merge_requests/4

It may be just me, but I haven't managed to get either of those patches working so far (by either I mean the first encoding_issue.patch, as well as deb822.patch + debpkgr.patch), both fail with the same issue.

debpkgr 1.1.0 should include sbernhard's workaround until python-debian merges the fix for python2 dump().