Issue #2966
Updated by kdelee@redhat.com over 7 years ago
When attempting to sync a docker repo on Pulp 2.14 beta (see below for detailed version info),
a worker dies while syncing and the sync is abandoned.
Versions of pulp packages:
<pre>
# rpm -qa | grep pulp
python-pulp-bindings-2.14.0-0.1.beta.fc25.noarch
pulp-ostree-admin-extensions-1.3.0-0.3.rc.git.1.3a770cd.fc25.noarch
python-pulp-docker-common-3.0.0-0.1.beta.fc25.noarch
pulp-deb-plugins-1.5.2-0.2.beta.fc25.noarch
python-pulp-deb-common-1.5.2-0.2.beta.fc25.noarch
pulp-puppet-admin-extensions-2.14.0-0.3.rc.git.1.12bf98c.fc25.noarch
python-pulp-rpm-common-2.14.0-0.1.beta.fc25.noarch
python-pulp-python-common-2.0.1-0.1.beta.fc25.noarch
pulp-selinux-2.14.0-0.3.rc.git.1.b8a44dd.fc25.noarch
pulp-deb-admin-extensions-1.5.2-0.2.beta.fc25.noarch
python-pulp-repoauth-2.14.0-0.1.beta.fc25.noarch
pulp-puppet-plugins-2.14.0-0.3.rc.git.1.12bf98c.fc25.noarch
pulp-rpm-admin-extensions-2.14.0-0.3.rc.git.1.5746bbf.fc25.noarch
python-pulp-oid_validation-2.14.0-0.1.beta.fc25.noarch
pulp-admin-client-2.14.0-0.3.rc.git.1.b8a44dd.fc25.noarch
pulp-python-plugins-2.0.1-0.3.rc.git.1.1d5b1c8.fc25.noarch
python-pulp-client-lib-2.14.0-0.1.beta.fc25.noarch
python-pulp-streamer-2.14.0-0.1.beta.fc25.noarch
pulp-docker-plugins-3.0.0-0.3.rc.git.1.5149268.fc25.noarch
python-pulp-common-2.14.0-0.1.beta.fc25.noarch
pulp-ostree-plugins-1.3.0-0.3.rc.git.1.3a770cd.fc25.noarch
python-pulp-puppet-common-2.14.0-0.1.beta.fc25.noarch
python-pulp-ostree-common-1.3.0-0.1.beta.fc25.noarch
pulp-server-2.14.0-0.3.rc.git.1.b8a44dd.fc25.noarch
pulp-docker-admin-extensions-3.0.0-0.3.rc.git.1.5149268.fc25.noarch
pulp-python-admin-extensions-2.0.1-0.3.rc.git.1.1d5b1c8.fc25.noarch
pulp-rpm-plugins-2.14.0-0.3.rc.git.1.5746bbf.fc25.noarch
# cat /etc/redhat-release
Fedora release 25 (Twenty Five)
</pre>
Steps to recreate:
<pre>
[root@fedora25base ~]# pulp-admin docker repo create --repo-id containers --feed https://index.docker.io --upstream-name library/busybox
Repository [containers] successfully created
[root@fedora25base ~]# pulp-admin docker repo sync run --repo-id containers
+----------------------------------------------------------------------+
Synchronizing Repository [containers]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading manifests
[|]
... completed
Copying units already in pulp
[-]
... completed
Copying units already in pulp
[-]
... completed
Downloading remote files
[==================================================] 100%
57 of 57 items
... failed
Task Failed
Worker terminated abnormally while processing task
f1fcebda-3f45-44f9-8047-876ddb6e5962. Check the logs for details
</pre>
Logs:
<pre>
// REMOVED wrong logs were attached. See below for logs from same steps as above for correct repo Aug 08 15:04:18 fedora25base pulp[3115]: pulp.server.async.tasks:INFO: [0ae86c12] Task failed : [593b68ce-4750-4515-8507-f5dd74379ef1] : Worker
Aug 08 15:04:18 fedora25base pulp[3115]: py.warnings:WARNING: [0ae86c12] (3115-29216)
Aug 08 15:04:18 fedora25base pulp[3115]: py.warnings:WARNING: [0ae86c12] (3115-29216) "MongoClient opened before fork. Create MongoClient "
Aug 08 15:04:18 fedora25base pulp[3115]: py.warnings:WARNING: [0ae86c12] (3115-29216) /usr/lib64/python2.7/site-packages/pymongo/topology.py:143
Aug 08 15:04:18 fedora25base pulp[2272]: celery.worker.job:ERROR: (2272-29216) WorkerLostError: Worker exited prematurely: signal 9 (SIGKILL).
Aug 08 15:04:18 fedora25base pulp[2272]: celery.worker.job:ERROR: (2272-29216) human_status(exitcode)),
Aug 08 15:04:18 fedora25base pulp[2272]: celery.worker.job:ERROR: (2272-29216) File "/usr/lib64/python2.7/site-packages/billiard/pool.py", lin
Aug 08 15:04:18 fedora25base pulp[2272]: celery.worker.job:ERROR: (2272-29216) Traceback (most recent call last):
Aug 08 15:04:18 fedora25base pulp[2272]: celery.worker.job:ERROR: (2272-29216) Task pulp.server.managers.repo.sync.sync[593b68ce-4750-4515-8507-
Aug 08 15:04:13 fedora25base pulp[3105]: nectar.downloaders.threaded:ERROR: Skipping requests to registry-1.docker.io due to repeated connection
Aug 08 15:04:10 fedora25base pulp[3105]: nectar.downloaders.threaded:INFO: Download succeeded: https://registry-1.docker.io/v2/library/busybox/b
Aug 08 15:04:06 fedora25base pulp[3105]: nectar.downloaders.threaded:INFO: Download succeeded: https://registry-1.docker.io/v2/library/busybox/b
Aug 08 15:03:54 fedora25base pulp[3105]: nectar.downloaders.threaded:INFO: Download succeeded: https://registry-1.docker.io/v2/library/busybox/b
Aug 08 15:03:52 fedora25base pulp[3105]: nectar.downloaders.threaded:INFO: Download succeeded: https://registry-1.docker.io/v2/library/busybox/b
Aug 08 15:03:46 fedora25base pulp[3105]: nectar.downloaders.threaded:INFO: Download succeeded: https://registry-1.docker.io/v2/library/busybox/b
Aug 08 15:03:46 fedora25base pulp[3105]: nectar.downloaders.threaded:INFO: Download succeeded: https://registry-1.docker.io/v2/library/busybox/b
Aug 08 15:03:46 fedora25base pulp[3105]: nectar.downloaders.threaded:INFO: Download succeeded: https://registry-1.docker.io/v2/library/busybox/b
Aug 08 15:03:45 fedora25base pulp[3105]: nectar.downloaders.threaded:INFO: Download succeeded: https://registry-1.docker.io/v2/library/busybox/b
Aug 08 15:03:45 fedora25base pulp[3105]: nectar.downloaders.threaded:INFO: Download succeeded: https://registry-1.docker.io/v2/library/busybox/b
Aug 08 15:03:45 fedora25base pulp[3105]: nectar.downloaders.threaded:INFO: Download succeeded: https://registry-1.docker.io/v2/library/bus
</pre>
.