Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #3892

closed

Can't copy Docker tags or manifest lists between repositories

Added by Ichimonji10 over 6 years ago. Updated almost 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
bizhang
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Docker:
Platform Release:
2.17.0
Target Release - Docker:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

It's impossible to copy Docker tags or manifest lists between repositories. Attempting to do triggers an error in Pulp. Here's the relevant journalctl output when attempting to copy manifest lists:

Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168) Exception from importer [docker_importer] while importing units into repository [14053d5e-0ea9-4b13-9e56-f4c769a
0c0a6]
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168) Traceback (most recent call last):
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     units=transfer_units)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 217, in import_units
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     units_added |= set(unit_importers[type(unit)](import_conduit, unit, dest_repo.repo_obj))
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 369, in _import_manifest_list
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     for manifest in models.Manifest.objects.filter(digest__in=sorted(manifest_digests)):
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/mongoengine/queryset/base.py", line 1407, in next
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     raw_doc = self._cursor.next()
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1097, in next
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     if len(self.__data) or self._refresh():
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1019, in _refresh
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     self.__read_concern))
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 850, in __send_message
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 794, in _send_message_with_response
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     exhaust)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 805, in _reset_on_error
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     return func(*args, **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/server.py", line 108, in send_message_with_response
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     set_slave_okay, sock_info.is_mongos, use_find_cmd)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/message.py", line 275, in get_message
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     spec, self.fields, self.codec_options)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.async.tasks:INFO: [30e85096] Task failed : [30e85096-c215-4b4e-99e6-682dcbede8b1]
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168) Task pulp.server.managers.repo.unit_association.associate_from_repo[30e85096-c215-4b4e-99e6-682dcbede8b1] raised unexpected: InvalidDocument('Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>',)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168) Traceback (most recent call last):
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)     R = retval = fun(*args, **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 529, in __call__
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)     return super(Task, self).__call__(*args, **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)     return super(PulpTask, self).__call__(*args, **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)     return self.run(*args, **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 291, in associate_from_repo
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)     raise (e, None, sys.exc_info()[2])
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>

...and here's the error when attempting to copy tags between repositories:

Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168) Exception from importer [docker_importer] while importing units into repository [94d08260-7dc9-4bf0-9b9a-d23ed19
3c2c3]
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168) Traceback (most recent call last):
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     units=transfer_units)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 217, in import_units
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     units_added |= set(unit_importers[type(unit)](import_conduit, unit, dest_repo.repo_obj))
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 299, in _import_tag
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     conduit, manifest, dest_repo))
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 369, in _import_manifest_list
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     for manifest in models.Manifest.objects.filter(digest__in=sorted(manifest_digests)):
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/mongoengine/queryset/base.py", line 1407, in next
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     raw_doc = self._cursor.next()
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1097, in next
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     if len(self.__data) or self._refresh():
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1019, in _refresh
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     self.__read_concern))
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 850, in __send_message
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 794, in _send_message_with_response
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     exhaust)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 805, in _reset_on_error
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     return func(*args, **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/server.py", line 108, in send_message_with_response
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     set_slave_okay, sock_info.is_mongos, use_find_cmd)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/message.py", line 275, in get_message
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     spec, self.fields, self.codec_options)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.async.tasks:INFO: [7c6d1641] Task failed : [7c6d1641-66f4-4408-8a70-6a7d3d2b36ec]
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168) Task pulp.server.managers.repo.unit_association.associate_from_repo[7c6d1641-66f4-4408-8a70-6a7d3d2b36ec] raised unexpected: InvalidDocument('Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>',)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168) Traceback (most recent call last):
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)     R = retval = fun(*args, **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 529, in __call__
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)     return super(Task, self).__call__(*args, **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)     return super(PulpTask, self).__call__(*args, **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)     return self.run(*args, **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 291, in associate_from_repo
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)     raise (e, None, sys.exc_info()[2])
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>

The error looks nearly identical. This error occurs with the nightly build of Pulp 2 (to-be 2.17) on both RHEL 7 and Fedora 27. Here's some sample RPMs:

[root@rhel-7-pulp-2-17-nightly ~]# rpm -qa | grep pulp | sort
pulp-admin-client-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
pulp-deb-admin-extensions-1.8.0-0.1.alpha.201807310516git4df9825.el7.noarch
pulp-deb-plugins-1.8.0-0.1.alpha.201807310516git4df9825.el7.noarch
pulp-docker-admin-extensions-3.2.0-0.1.alpha.201807310503git4356228.el7.noarch
pulp-docker-plugins-3.2.0-0.1.alpha.201807310503git4356228.el7.noarch
pulp-ostree-admin-extensions-1.4.0-0.1.alpha.201807310504git2c2d534.el7.noarch
pulp-ostree-plugins-1.4.0-0.1.alpha.201807310504git2c2d534.el7.noarch
pulp-puppet-admin-extensions-2.17.0-0.1.alpha.201807310506git44a02f8.el7.noarch
pulp-puppet-plugins-2.17.0-0.1.alpha.201807310506git44a02f8.el7.noarch
pulp-puppet-tools-2.17.0-0.1.alpha.201807310506git44a02f8.el7.noarch
pulp-python-admin-extensions-2.1.0-0.1.alpha.201807310501git5e2aa35.el7.noarch
pulp-python-plugins-2.1.0-0.1.alpha.201807310501git5e2aa35.el7.noarch
pulp-rpm-admin-extensions-2.17.0-0.1.alpha.201807310508giteaff2ca.el7.noarch
pulp-rpm-plugins-2.17.0-0.1.alpha.201807310508giteaff2ca.el7.noarch
pulp-selinux-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
pulp-server-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
python-pulp-bindings-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-pulp-client-lib-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-pulp-common-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-pulp-deb-common-1.8.0-0.1.alpha.201807310516git4df9825.el7.noarch
python-pulp-docker-common-3.2.0-0.1.alpha.201807310503git4356228.el7.noarch
python-pulp-oid_validation-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-pulp-ostree-common-1.4.0-0.1.alpha.201807310504git2c2d534.el7.noarch
python-pulp-puppet-common-2.17.0-0.1.alpha.201807310506git44a02f8.el7.noarch
python-pulp-python-common-2.1.0-0.1.alpha.201807310501git5e2aa35.el7.noarch
python-pulp-repoauth-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-pulp-rpm-common-2.17.0-0.1.alpha.201807310508giteaff2ca.el7.noarch
python-pulp-streamer-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch

The easiest way to reproduce this issue is to run the automated test in Pulp 2 Tests: python -m unittest pulp_2_tests.tests.docker.api_v2.test_copy.CopyV2ContentTestCase. The error occurs when this line of code is executing, inside test_02_copy_manifest_lists:

client.post(urljoin(repo['_href'], 'actions/associate/'), {
    'criteria': {'filters': {}, 'type_ids': ['docker_manifest_list']},
    'source_repo_id': self.repo['id'],
})

It also occurs when this line of code is executing, inside test_02_copy_tags:

client.post(urljoin(repo['_href'], 'actions/associate/'), {
    'source_repo_id': self.repo['id'],
    'criteria': {'filters': {}, 'type_ids': ['docker_tag']},
})

Also see:

Related issues

Copied to Docker Support - Issue #3904: Can't upload manifest list to Docker repositoryCLOSED - CURRENTRELEASEbizhangActions
Actions
Copy link
 #1

Updated by Ichimonji10 over 6 years ago

This issue is a regression.

Actions
Copy link
 #2

Updated by Ichimonji10 over 6 years ago

  • Subject changed from Can't copy Docker manifest lists between repositories to Can't copy Docker tags or manifest lists between repositories
  • Description updated (diff)

Added by werwty over 6 years ago

Revision baeef3ae | View on GitHub

Fix issue with associating embedded manifests in manifest list

https://pulp.plan.io/issues/3892

re #3137 closes #3892

Added by werwty over 6 years ago

Revision baeef3ae | View on GitHub

Fix issue with associating embedded manifests in manifest list

https://pulp.plan.io/issues/3892

re #3137 closes #3892

Actions
Copy link
 #3

Updated by bizhang over 6 years ago

  • Status changed from NEW to POST
  • Assignee set to bizhang
Actions
Copy link
 #4

Updated by werwty over 6 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #5

Updated by ipanova@redhat.com over 6 years ago

  • Sprint/Milestone set to 2.17.0
Actions
Copy link
 #6

Updated by Ichimonji10 over 6 years ago

  • Status changed from MODIFIED to ASSIGNED

This issue still manifests itself with the latest nightly builds of 2 master (to-be 2.17) on both F27 and RHEL 7. To reproduce the issue, run the same automate tests as before with python -m unittest --verbose pulp_2_tests.tests.docker.api_v2.test_copy.CopyV2ContentTestCase. Here's a snippet from journalctl:

Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480) Exception from importer [docker_importer] while importing units into repository [159a49d7-505c-454b-8483-2061831c97a5]
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480) Traceback (most recent call last):
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     units=transfer_units)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 217, in import_units
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     units_added |= set(unit_importers[type(unit)](import_conduit, unit, dest_repo.repo_obj))
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 299, in _import_tag
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     conduit, manifest, dest_repo))
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 369, in _import_manifest_list
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     for manifest in models.Manifest.objects.filter(digest__in=sorted(manifest_digests)):
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/mongoengine/queryset/base.py", line 1407, in next
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     raw_doc = self._cursor.next()
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1097, in next
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     if len(self.__data) or self._refresh():
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1019, in _refresh
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     self.__read_concern))
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 850, in __send_message
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 794, in _send_message_with_response
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     exhaust)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 805, in _reset_on_error
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     return func(*args, **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/server.py", line 108, in send_message_with_response
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     set_slave_okay, sock_info.is_mongos, use_find_cmd)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/message.py", line 275, in get_message
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     spec, self.fields, self.codec_options)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.async.tasks:INFO: [b39c1fdd] Task failed : [b39c1fdd-4680-40c4-8150-c134937df5fb]
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480) Task pulp.server.managers.repo.unit_association.associate_from_repo[b39c1fdd-4680-40c4-8150-c134937df5fb] raised unexpected: InvalidDocument('Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>',)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480) Traceback (most recent call last):
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)     R = retval = fun(*args, **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 529, in __call__
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)     return super(Task, self).__call__(*args, **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)     return super(PulpTask, self).__call__(*args, **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)     return self.run(*args, **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 291, in associate_from_repo
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)     raise (e, None, sys.exc_info()[2])
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>

Here's the RPMs on a sample test VM:

[root@rhel-7-pulp-2-17-nightly ~]# rpm -qa | grep -i 'solv\|pulp' | sort
libsolv-0.6.34-4.el7.x86_64
pulp-admin-client-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
pulp-deb-admin-extensions-1.8.0-0.1.alpha.201808010517git4df9825.el7.noarch
pulp-deb-plugins-1.8.0-0.1.alpha.201808010517git4df9825.el7.noarch
pulp-docker-admin-extensions-3.2.0-0.1.alpha.201808010504git4356228.el7.noarch
pulp-docker-plugins-3.2.0-0.1.alpha.201808010504git4356228.el7.noarch
pulp-ostree-admin-extensions-1.4.0-0.1.alpha.201808010506git2c2d534.el7.noarch
pulp-ostree-plugins-1.4.0-0.1.alpha.201808010506git2c2d534.el7.noarch
pulp-puppet-admin-extensions-2.17.0-0.1.alpha.201808010507git44a02f8.el7.noarch
pulp-puppet-plugins-2.17.0-0.1.alpha.201808010507git44a02f8.el7.noarch
pulp-puppet-tools-2.17.0-0.1.alpha.201808010507git44a02f8.el7.noarch
pulp-python-admin-extensions-2.1.0-0.1.alpha.201808010503git5e2aa35.el7.noarch
pulp-python-plugins-2.1.0-0.1.alpha.201808010503git5e2aa35.el7.noarch
pulp-rpm-admin-extensions-2.17.0-0.1.alpha.201808010509git421caf4.el7.noarch
pulp-rpm-plugins-2.17.0-0.1.alpha.201808010509git421caf4.el7.noarch
pulp-selinux-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
pulp-server-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python2-solv-0.6.34-4.el7.x86_64
python-isodate-0.5.0-4.pulp.el7.noarch
python-pulp-bindings-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python-pulp-client-lib-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python-pulp-common-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python-pulp-deb-common-1.8.0-0.1.alpha.201808010517git4df9825.el7.noarch
python-pulp-docker-common-3.2.0-0.1.alpha.201808010504git4356228.el7.noarch
python-pulp-oid_validation-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python-pulp-ostree-common-1.4.0-0.1.alpha.201808010506git2c2d534.el7.noarch
python-pulp-puppet-common-2.17.0-0.1.alpha.201808010507git44a02f8.el7.noarch
python-pulp-python-common-2.1.0-0.1.alpha.201808010503git5e2aa35.el7.noarch
python-pulp-repoauth-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python-pulp-rpm-common-2.17.0-0.1.alpha.201808010509git421caf4.el7.noarch
python-pulp-streamer-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
Actions
Copy link
 #7

Updated by bizhang over 6 years ago

  • Status changed from ASSIGNED to MODIFIED

looks like the nightly packages haven't pulled in the latest commit (hash is 4356228 instead of fd8d4e7)
the nightlies didn't build this morning, so the change never made it into the latest rpm
https://repos.fedorapeople.org/repos/pulp/pulp/testing/automation/2.17/stage/fedora-27/x86_64/

`python -m unittest --verbose pulp_2_tests.tests.docker.api_v2.test_copy.CopyV2ContentTestCase` with pulp_docker 2-master branch succeeds without an error

Actions
Copy link
 #8

Updated by Ichimonji10 over 6 years ago

  • Copied to Issue #3904: Can't upload manifest list to Docker repository added
Actions
Copy link
 #9

Updated by ipanova@redhat.com over 6 years ago

  • Platform Release set to 2.17.0
Actions
Copy link
 #10

Updated by ipanova@redhat.com over 6 years ago

  • Platform Release changed from 2.17.0 to master
Actions
Copy link
 #11

Updated by ipanova@redhat.com over 6 years ago

  • Platform Release deleted (master)
Actions
Copy link
 #12

Updated by ipanova@redhat.com over 6 years ago

  • Platform Release set to 2.17.0
Actions
Copy link
 #13

Updated by ipanova@redhat.com over 6 years ago

  • Project changed from Crane to Docker Support
Actions
Copy link
 #14

Updated by ipanova@redhat.com over 6 years ago

  • Status changed from MODIFIED to 5
Actions
Copy link
 #15

Updated by ipanova@redhat.com over 6 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions
Copy link
 #16

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added
Actions
Copy link
 #17

Updated by ttereshc almost 3 years ago

  • Sprint/Milestone deleted (2.17.0)
Actions
Send by e-mail Copy link

Also available in: Atom PDF