Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #647

closed

rpm repo publish fails with "Incorrect length of data produced" error

Added by bryce.pier@capella.edu about 9 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
bcourt
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.5
Platform Release:
2.6.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Description of problem:
Prior to and after upgrading from 2.5.0 to 2.5.1 (and now also after applying the 2.5.2-0.0.beta packages), I've been unable to publish both feed-url and upload rpm repos. The issue appears to affect all of my repos.

Version-Release number of selected component (if applicable):
2.5.0
2.5.1
2.5.2-0.0.beta

How reproducible:
Always

Steps to Reproduce:
1. pulp-admin rpm repo publish run --repo-id=<repo name>
OR
pulp-admin rpm repo sync run --repo-id=<repo-name>
2.
3.

Actual results:

  1. pulp-admin rpm repo publish run --repo-id=rhel-6-x86_64-os
    --------------------------------------------------------------------
    Publishing Repository [rhel-6-x86_64-os]
    --------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[====================== ] 45%
6605 of 14365 items
... failed
Incorrect length of data produced

Task Failed

Incorrect length of data produced

  1. pulp-admin rpm repo sync run --repo-id=rhel-6-x86_64-os
    --------------------------------------------------------------------
    Synchronizing Repository [rhel-6-x86_64-os]
    --------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... completed

Downloading repository content...
[==================================================] 100%
RPMs: 113/113 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[\]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[====================== ] 45%
6643 of 14478 items
... failed
Incorrect length of data produced

Task Failed

Incorrect length of data produced

Expected results:
Successfully publishing repos for use via yum.

Additional info:

Current package version (although this was failing prior to upgrading to this version):
yum list installed |grep pulp
grinder.noarch 0.1.16-1.el6 @pulp-v2-stable
m2crypto.x86_64 0.21.1.pulp-8.el6 @pulp-v2-stable
mod_wsgi.x86_64 3.4-2.pulp.el6 @pulp-v2-stable
pulp-admin-client.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
pulp-puppet-admin-extensions.noarch
2.5.2-0.0.beta.el6 @pulp-v2-beta
pulp-puppet-plugins.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
pulp-rpm-admin-extensions.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
pulp-rpm-plugins.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
pulp-selinux.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
pulp-server.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
python-amqp.noarch 1.4.5-1.el6 @pulp-v2-stable
python-anyjson.noarch 0.3.3-4.el6 @pulp-v2-stable
python-billiard.x86_64 1:3.3.0.17-1.el6 @pulp-v2-stable
python-celery.noarch 3.1.11-1.el6 @pulp-v2-stable
python-gofer.noarch 1.3.1-1.el6 @pulp-v2-stable
python-gofer-qpid.noarch 1.3.1-1.el6 @pulp-v2-stable
python-isodate.noarch 0.5.0-4.pulp.el6 @pulp-v2-stable
python-kombu.noarch 1:3.0.15-12.pulp.el6 @pulp-v2-stable
python-nectar.noarch 1.3.1-1.el6 @pulp-v2-stable
python-pulp-bindings.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
python-pulp-client-lib.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
python-pulp-common.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
python-pulp-puppet-common.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
python-pulp-rpm-common.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta
python-qpid.noarch 0.26-2.el6 @pulp-v2-stable
python-qpid-common.noarch 0.26-2.el6 @pulp-v2-stable
python-qpid-qmf.x86_64 0.26-2.el6 @pulp-v2-stable
python-requests.noarch 2.2.1-1.el6 @pulp-v2-stable
python-semantic-version.noarch 2.2.0-3.el6 @pulp-v2-stable
qpid-cpp-client.x86_64 0.26-9.el6 @pulp-v2-beta
qpid-cpp-server.x86_64 0.26-9.el6 @pulp-v2-beta
qpid-cpp-server-store.x86_64 0.26-9.el6 @pulp-v2-beta
qpid-qmf.x86_64 0.26-2.el6 @pulp-v2-stable

Trace from /var/log/messages:
Dec 22 14:57:51 dvpuap02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._reserve_resource[831267ce-98eb-41c3-9dad-0b0e32035ec0]
Dec 22 14:57:51 dvpuap02 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._reserve_resource[831267ce-98eb-41c3-9dad-0b0e32035ec0] succeeded in 0.031
6150639992s: u''
Dec 22 14:57:52 dvpuap02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[7003450d-2b88-45a5-b05b-f5a92cfbba67]
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: Exception caught from plugin during publish for repo [epel6]
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: Traceback (most recent call last):
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 1
26, in do_publish
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config)
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 470, in wra
p_f
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return f(*args, **kwargs)
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor
.py", line 143, in publish_repo
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return self._publisher.publish()
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return self.process_lifecycle()
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: super(PluginStep, self).process_lifecycle()
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: step.process()
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 185, in process
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self.finalize()
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self.other_context.checksum)
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: content = file_handle.read()
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 212, in read
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self._read(readsize)
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 284, in _read
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self._read_eof()
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 306, in _read_eof
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: raise IOError, "Incorrect length of data produced"
Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: IOError: Incorrect length of data produced
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: Task pulp.server.managers.repo.publish.publish[7003450d-2b88-45a5-b05b-f5a92cfbba67] raised unexpected: IOError('Incorrect length of data produced',)
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: Traceback (most recent call last):
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: R = retval = fun(*args, **kwargs)
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 315, incall_
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: return super(Task, self).__call__(*args, **kwargs)
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in protected_call
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: return self.run(*args, **kwargs)
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 98, in publish
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: transfer_repo, conduit, call_config)
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config)
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 470, in wrap_f
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: return f(*args, **kwargs)
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: return self._publisher.publish()
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: return self.process_lifecycle()
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: super(PluginStep, self).process_lifecycle()
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: step.process()
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 185, in process
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: self.finalize()
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: self.other_context.checksum)
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: content = file_handle.read()
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 212, in read
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: self._read(readsize)
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 284, in _read
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: self._read_eof()
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 306, in _read_eof
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: raise IOError, "Incorrect length of data produced"
Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: IOError: Incorrect length of data produced

+ This bug was cloned from Bugzilla Bug #1176698 +

Related issues

Blocks RPM Support - Issue #512: Pulp cannot sync an EPEL 5 repository that was published by PulpCLOSED - CURRENTRELEASEmhrivnakActions

Added by jcline@redhat.com over 9 years ago

Revision 6117c1ac | View on GitHub

1158945 - Pulp can now publish RPM packages with descriptions containing unicode characters.

Added by bcourt over 9 years ago

Revision 5bbb8b4b | View on GitHub

Ensure we support Python 2.6 when encoding unicode

Actions
Copy link
 #1

Updated by mhrivnak about 9 years ago

Please also share the traceback that occurs during sync.

Is this reproducible by syncing a publicly-available repository? Let us know which so we can try to reproduce it here.

+ This comment was cloned from Bugzilla #1176698 comment 1 +

Actions
Copy link
 #2

Updated by mhrivnak about 9 years ago

For reference, this is the call in pulp's publish workflow that results in an exception from python's gzip module:

https://github.com/pulp/pulp_rpm/blob/pulp-rpm-2.5.2-0.0.beta/plugins/pulp_rpm/plugins/distributors/yum/metadata/repomd.py#L115

+ This comment was cloned from Bugzilla #1176698 comment 2 +

Actions
Copy link
 #3

Updated by mhrivnak about 9 years ago

***** Bug 1177627 has been marked as a duplicate of this bug. ***

+ This comment was cloned from Bugzilla #1176698 comment 3 +

Actions
Copy link
 #4

Updated by bryce.pier@capella.edu about 9 years ago

The publish error occurs on several of my 17 repos. I thought it was all of them as the first 4 I attempted all failed. The following are failing:

https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os
https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/optional/os
http://public-yum.oracle.com/repo/OracleLinux/OL5/latest/x86_64
https://cdn.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os
http://mirror.us.leaseweb.net/epel/6/x86_64/
http://mirror.us.leaseweb.net/epel/5/x86_64/

These are publishing correctly:
https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/rhscl/1/os
https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/debug
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/optional/os
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhscl/1/os
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/debug
http://dl.iuscommunity.org/pub/ius/stable/Redhat/6/x86_64/
http://mirror.pnl.gov/epel/7/x86_64/
https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/scalablefilesystem/os
http://public-yum.oracle.com/repo/OracleLinux/OL6/latest/x86_64
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os

A couple of items of note:
1. The publish failures always happen on the same package for each repo (e.g. for epel6 at 75 of 19440 items, for rhel6-optional at 54 of 8104 items)
2. I have a repo called rhel-6-x86_64-os-physical which is a point in time copy of the rhel6 os repo just prior to 6.6 being released. That repo is publishing fine but the rhel6 os repo is not.

Here is the traceback from the sync of the rhel6 os repo:
Jan 2 11:48:06 dvpuap02 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading 1 RPMs.
Jan 2 11:51:06 dvpuap02 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Jan 2 11:52:33 dvpuap02 pulp: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: The content container reported: {'downloads': {'___/primary/___': {'tot
al_failed': 0, 'total_succeeded': 1}}, 'total_sources': 0} for base URL: https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os/
Jan 2 11:52:33 dvpuap02 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading additional units.
Jan 2 11:52:33 dvpuap02 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Jan 2 11:52:34 dvpuap02 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Jan 2 11:52:49 dvpuap02 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Sync complete.
Jan 2 11:52:49 dvpuap02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._reserve_resource[fc8b7a5f-5911-43ae-ad11-4e2661791ede]
Jan 2 11:52:49 dvpuap02 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._reserve_resource[fc8b7a5f-5911-43ae-ad11-4e2661791ede] succeeded in 0.0
31124184s: u''
Jan 2 11:52:50 dvpuap02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[f0a6ad0e-2fe6-4eaa-838c-465b91db9d37]
Jan 2 11:52:50 dvpuap02 pulp: celery.worker.job:INFO: Task pulp.server.tasks.repository.sync_with_auto_publish[4b8189f1-6c87-40a1-97f2-3176bab09b76] succeeded in 1773.61550857s: <pulp.server.async.tasks.TaskResult object at 0x7fb75803eb50>
Jan 2 11:52:58 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting
Jan 2 11:53:09 dvpuap02 rsyslogd-2177: imuxsock lost 352 messages from pid 2344 due to rate-limiting
Jan 2 11:53:09 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting
Jan 2 11:53:15 dvpuap02 rsyslogd-2177: imuxsock lost 130 messages from pid 2344 due to rate-limiting
Jan 2 11:53:17 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting
Jan 2 11:53:21 dvpuap02 rsyslogd-2177: imuxsock lost 872 messages from pid 2344 due to rate-limiting
Jan 2 11:53:22 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting
Jan 2 11:53:27 dvpuap02 rsyslogd-2177: imuxsock lost 853 messages from pid 2344 due to rate-limiting
Jan 2 11:53:27 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting
Jan 2 11:53:33 dvpuap02 rsyslogd-2177: imuxsock lost 974 messages from pid 2344 due to rate-limiting
Jan 2 11:53:42 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting
Jan 2 11:53:45 dvpuap02 rsyslogd-2177: imuxsock lost 68 messages from pid 2344 due to rate-limiting
Jan 2 11:54:44 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting
Jan 2 11:54:51 dvpuap02 rsyslogd-2177: imuxsock lost 50 messages from pid 2344 due to rate-limiting
Jan 2 11:54:52 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting
Jan 2 11:54:59 dvpuap02 rsyslogd-2177: imuxsock lost 78 messages from pid 2344 due to rate-limiting
Jan 2 11:55:00 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting
Jan 2 11:55:05 dvpuap02 rsyslogd-2177: imuxsock lost 481 messages from pid 2344 due to rate-limiting
Jan 2 11:55:08 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting
Jan 2 11:55:14 dvpuap02 rsyslogd-2177: imuxsock lost 172 messages from pid 2344 due to rate-limiting
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: Exception caught from plugin during publish for repo [rhel-6-x86_64-os]
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: Traceback (most recent call last):
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in do_publish
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config)
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 470, in wrap_f
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return f(*args, **kwargs)
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return self._publisher.publish()
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return self.process_lifecycle()
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: super(PluginStep, self).process_lifecycle()
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: step.process()
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 185, in process
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self.finalize()
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self.other_context.checksum)
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: content = file_handle.read()
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 212, in read
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self._read(readsize)
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 284, in _read
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self._read_eof()
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 306, in _read_eof
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: raise IOError, "Incorrect length of data produced"
Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: IOError: Incorrect length of data produced
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: Task pulp.server.managers.repo.publish.publish[f0a6ad0e-2fe6-4eaa-838c-465b91db9d37] raised unexpected: IOError('Incorrect length of data produced',)
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: Traceback (most recent call last):
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: R = retval = fun(*args, **kwargs)
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 315, incall_
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: return super(Task, self).__call__(*args, **kwargs)
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in protected_call
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: return self.run(*args, **kwargs)
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 98, in publish
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: transfer_repo, conduit, call_config)
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config)
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 470, in wrap_f
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: return f(*args, **kwargs)
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: return self._publisher.publish()
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: return self.process_lifecycle()
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: super(PluginStep, self).process_lifecycle()
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: step.process()
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 185, in process
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: self.finalize()
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: self.other_context.checksum)
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: content = file_handle.read()
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 212, in read
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: self._read(readsize)
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 284, in _read
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: self._read_eof()
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 306, in _read_eof
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: raise IOError, "Incorrect length of data produced"
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: IOError: Incorrect length of data produced
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[8c30c2b7-e142-4550-9914-dc1145cbd8f8]
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_release_resource[0e332bf8-5a6a-4331-b6f7-66c24b85f6a0] succeeded in 0.0150966489996s: None
Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[8c30c2b7-e142-4550-9914-dc1145cbd8f8] succeeded in 0.042722529s: None

+ This comment was cloned from Bugzilla #1176698 comment 4 +

Actions
Copy link
 #5

Updated by jiri.tyr@gmail.com about 9 years ago

I think this issue it's related to the bug #1158945. The proposed patch (https://github.com/pulp/pulp_rpm/pull/605) fixes the problem but on CentOS 6.5 you have to modify the patch to use encode('utf-8') instead of encode(encoding='utf-8').

+ This comment was cloned from Bugzilla #1176698 comment 5 +

Actions
Copy link
 #6

Updated by pthomas@redhat.com about 9 years ago

I was able sync and publish rhel6 repo from cdn. Pasting the results. Please let us know if we are missing something.

[root@ibm-x3250m4-02 ~]# pulp-admin rpm repo create --repo-id rhel6 --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.6/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1391032266_4771.crt --feed-key CDN/1391032266_4771.key
Successfully created repository [rhel6]

[root@ibm-x3250m4-02 ~]#
[root@ibm-x3250m4-02 ~]#
[root@ibm-x3250m4-02 ~]# pulp-admin rpm repo sync run --repo-id rhel6
--------------------------------------------------------------------
Synchronizing Repository [rhel6]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs: 14472/14472 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[\]
... completed

Importing package groups/categories...
[|]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
14472 of 14472 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
2818 of 2818 items
... completed

Publishing Comps file
[==================================================] 100%
212 of 212 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[|]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

[root@ibm-x3250m4-02 ~]#

+ This comment was cloned from Bugzilla #1176698 comment 6 +

Actions
Copy link
 #8

Updated by cduryee about 9 years ago

This was merged to 2.6-dev which aligns to 2.6.1. If you need this fix in 2.6.0, please create a PR on 2.6-testing, and set the bug's target release back to 2.6.0.

Setting target release to 2.6.1.

+ This comment was cloned from Bugzilla #1176698 comment 8 +

Actions
Copy link
 #9

Updated by igulina@redhat.com about 9 years ago

rpm -qa pulp-server

pulp-server-2.6.0-0.4.beta.el6.noarch

pulp-admin rpm repo create --repo-id epel6_1 --feed http://mirror.karneval.cz/pub/linux/fedora/epel/6/x86_64/

Successfully created repository [epel6_1]

pulp-admin rpm repo sync run --repo-id epel6_1

--------------------------------------------------------------------
Synchronizing Repository [epel6_1]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs: 11178/11178 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[\]
... completed

Importing package groups/categories...
[|]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[ ] 0%
1 of 11178 items
... failed
encode() takes no keyword arguments

Task Failed

encode() takes no keyword arguments

+ This comment was cloned from Bugzilla #1176698 comment 9 +

Actions
Copy link
 #10

Updated by cduryee about 9 years ago

pulp 2.6.0 beta 5

+ This comment was cloned from Bugzilla #1176698 comment 10 +

Actions
Copy link
 #11

Updated by igulina@redhat.com about 9 years ago

after update to pulp-server-2.6.0-0.5.beta.el6.noarch

pulp-admin rpm repo sync run --repo-id epel6_1

--------------------------------------------------------------------
Synchronizing Repository [epel6_1]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... completed

Downloading repository content...
[==================================================] 100%
RPMs: 1/1 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[|]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
11178 of 11178 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
3635 of 3635 items
... completed

Publishing Comps file
[==================================================] 100%
211 of 211 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[|]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

pulp-admin rpm repo publish run --repo-id epel6_1

--------------------------------------------------------------------
Publishing Repository [epel6_1]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Copying files
[|]
... completed

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[/]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
3635 of 3635 items
... completed

Publishing Comps file
[==================================================] 100%
211 of 211 items
... completed

Publishing Metadata.
[\]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[/]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

pulp-admin rpm repo list

--------------------------------------------------------------------
RPM Repositories
--------------------------------------------------------------------

Id: epel6_1
Display Name: epel6_1
Description: None
Content Unit Counts:
Erratum: 3635
Package Category: 3
Package Group: 208
Rpm: 11178
Yum Repo Metadata File: 1

ls -l /var/lib/pulp/published/yum/master/yum_distributor/

total 4
drwxr-xr-x. 3 apache apache 4096 Jan 16 22:51 epel6_1

I also checked this http://mirror.us.leaseweb.net/epel/5/x86_64/ from commnet#4

pulp-admin rpm repo create --repo-id epel5 --feed http://mirror.us.leaseweb.net/epel/5/x86_64/

Successfully created repository [epel5]

pulp-admin rpm repo sync run --repo-id epel5

--------------------------------------------------------------------
Synchronizing Repository [epel5]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[/]
... completed

Downloading repository content...
[==================================================] 100%
RPMs: 6848/6848 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[\]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
6848 of 6848 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
1998 of 1998 items
... completed

Publishing Comps file
[==================================================] 100%
41 of 41 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[\]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

pulp-admin rpm repo publish run --repo-id epel5

--------------------------------------------------------------------
Publishing Repository [epel5]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Copying files
[\]
... completed

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[\]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
1998 of 1998 items
... completed

Publishing Comps file
[==================================================] 100%
41 of 41 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[|]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

pulp-admin rpm repo list

--------------------------------------------------------------------
RPM Repositories
--------------------------------------------------------------------

Id: epel6_1
Display Name: epel6_1
Description: None
Content Unit Counts:
Erratum: 3635
Package Category: 3
Package Group: 208
Rpm: 11178
Yum Repo Metadata File: 1

Id: epel5
Display Name: epel5
Description: None
Content Unit Counts:
Erratum: 1998
Package Category: 5
Package Group: 36
Rpm: 6848
Yum Repo Metadata File: 1

ls -l /var/lib/pulp/published/yum/master/yum_distributor/

total 8
drwxr-xr-x. 3 apache apache 4096 Jan 16 23:11 epel5
drwxr-xr-x. 3 apache apache 4096 Jan 16 22:51 epel6_1

+ This comment was cloned from Bugzilla #1176698 comment 11 +

Actions
Copy link
 #12

Updated by jbaird@follett.com about 9 years ago

I'm having the same issue with:

pulp-server-2.5.1-1.el7.noarch

$ pulp-admin rpm repo publish run --repo-id=test
--------------------------------------------------------------------
Publishing Repository [test]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[== ] 4%
1 of 25 items
... failed
Incorrect length of data produced

Task Failed

Incorrect length of data produced

Traceback:

Jan 20 07:19:51 fc-d1pulp02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._reserve_resource[5492606b-de1f-4b9f-9d85-17b422d611f7]
Jan 20 07:19:51 fc-d1pulp02 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._reserve_resource[5492606b-de1f-4b9f-9d85-17b422d611f7] succeeded in 0.0307118759956s: u''
Jan 20 07:19:51 fc-d1pulp02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[9bb85e7f-d035-478a-86b8-61c417239d9b]
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: Exception caught from plugin during publish for repo [test]
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: Traceback (most recent call last):
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 126, in do_publish
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config)
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 470, in wrap_f
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: return f(*args, **kwargs)
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: return self._publisher.publish()
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: return self.process_lifecycle()
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: super(PluginStep, self).process_lifecycle()
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: step.process()
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 185, in process
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: self.finalize()
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: self.other_context.checksum)
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: content = file_handle.read()
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.7/gzip.py", line 254, in read
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: self._read(readsize)
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.7/gzip.py", line 325, in _read
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: self._read_eof()
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.7/gzip.py", line 349, in _read_eof
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: raise IOError, "Incorrect length of data produced"
Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: IOError: Incorrect length of data produced
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: Task pulp.server.managers.repo.publish.publish[9bb85e7f-d035-478a-86b8-61c417239d9b] raised unexpected: IOError('Incorrect length of data produced',)
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: Traceback (most recent call last):
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: R = retval = fun(*args, **kwargs)
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 315, incall_
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: return super(Task, self).__call__(*args, **kwargs)
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: return self.run(*args, **kwargs)
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 98, in publish
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: transfer_repo, conduit, call_config)
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config)
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 470, in wrap_f
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: return f(*args, **kwargs)
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: return self._publisher.publish()
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: return self.process_lifecycle()
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: super(PluginStep, self).process_lifecycle()
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: step.process()
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 185, in process
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: self.finalize()
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: self.other_context.checksum)
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: content = file_handle.read()
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.7/gzip.py", line 254, in read
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: self._read(readsize)
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.7/gzip.py", line 325, in _read
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: self._read_eof()
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.7/gzip.py", line 349, in _read_eof
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: raise IOError, "Incorrect length of data produced"
Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: IOError: Incorrect length of data produced

Is there a proposed fix for 2.5.1?

+ This comment was cloned from Bugzilla #1176698 comment 12 +

Actions
Copy link
 #13

Updated by jiri.tyr@gmail.com about 9 years ago

(In reply to Josh Baird from comment #12)

I'm having the same issue with:

pulp-server-2.5.1-1.el7.noarch

Is there a proposed fix for 2.5.1?

Yes, apply these two patches:
https://github.com/jeremycline/pulp_rpm/commit/6117c1ac703f4f759bd1aaa5da9a9ca8559495af.patch
https://github.com/barnabycourt/pulp_rpm/commit/5bbb8b4be838ff4f4be9549bd1986efca5c23ce6.patch

+ This comment was cloned from Bugzilla #1176698 comment 13 +

Actions
Copy link
 #14

Updated by jbaird@follett.com about 9 years ago

This doesn't appear to apply cleanly to 2.5.1.

+ This comment was cloned from Bugzilla #1176698 comment 14 +

Actions
Copy link
 #15

Updated by jiri.tyr@gmail.com about 9 years ago

(In reply to Josh Baird from comment #14)

This doesn't appear to apply cleanly to 2.5.1.

Then patch it manually. You only need the "if" conditions.

+ This comment was cloned from Bugzilla #1176698 comment 15 +

Actions
Copy link
 #16

Updated by bryce.pier@capella.edu about 9 years ago

I can confirm that 2.6.0-0.5.beta fixes my issue and all of my repos now publish successfully.

+ This comment was cloned from Bugzilla #1176698 comment 16 +

Actions
Copy link
 #17

Updated by dhaval.joshi@nomura.com about 9 years ago

This is still an issues with 2.5.1 after applying patch, how do i resolve this in 2.5.1..? this is bit critical.. it only does not upload some RPM..not sure what's the reason

+ This comment was cloned from Bugzilla #1176698 comment 17 +

Actions
Copy link
 #18

Updated by dhaval.joshi@nomura.com about 9 years ago

please ignore, it does work, tested it.

+ This comment was cloned from Bugzilla #1176698 comment 18 +

Actions
Copy link
 #19

Updated by bmbouter about 9 years ago

  • Blocks Issue #512: Pulp cannot sync an EPEL 5 repository that was published by Pulp added
Actions
Copy link
 #20

Updated by bmbouter about 9 years ago

  • Severity changed from High to 3. High
Actions
Copy link
 #21

Updated by rbarlow almost 9 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE
Actions
Copy link
 #25

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added
Actions
Send by e-mail Copy link

Also available in: Atom PDF