Project

Profile

Help

Issue #3892

Can't copy Docker tags or manifest lists between repositories

Added by Ichimonji10 11 months ago. Updated 2 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Severity:
2. Medium
Version - Docker:
Platform Release:
2.17.0
Blocks Release:
Target Release - Docker:
OS:
Backwards Incompatible:
No
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

It's impossible to copy Docker tags or manifest lists between repositories. Attempting to do triggers an error in Pulp. Here's the relevant journalctl output when attempting to copy manifest lists:

Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168) Exception from importer [docker_importer] while importing units into repository [14053d5e-0ea9-4b13-9e56-f4c769a
0c0a6]
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168) Traceback (most recent call last):
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     units=transfer_units)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 217, in import_units
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     units_added |= set(unit_importers[type(unit)](import_conduit, unit, dest_repo.repo_obj))
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 369, in _import_manifest_list
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     for manifest in models.Manifest.objects.filter(digest__in=sorted(manifest_digests)):
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/mongoengine/queryset/base.py", line 1407, in next
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     raw_doc = self._cursor.next()
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1097, in next
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     if len(self.__data) or self._refresh():
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1019, in _refresh
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     self.__read_concern))
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 850, in __send_message
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 794, in _send_message_with_response
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     exhaust)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 805, in _reset_on_error
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     return func(*args, **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/server.py", line 108, in send_message_with_response
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     set_slave_okay, sock_info.is_mongos, use_find_cmd)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/message.py", line 275, in get_message
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168)     spec, self.fields, self.codec_options)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [30e85096] (1877-39168) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.async.tasks:INFO: [30e85096] Task failed : [30e85096-c215-4b4e-99e6-682dcbede8b1]
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168) Task pulp.server.managers.repo.unit_association.associate_from_repo[30e85096-c215-4b4e-99e6-682dcbede8b1] raised unexpected: InvalidDocument('Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>',)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168) Traceback (most recent call last):
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)     R = retval = fun(*args, **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 529, in __call__
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)     return super(Task, self).__call__(*args, **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)     return super(PulpTask, self).__call__(*args, **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)     return self.run(*args, **kwargs)
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 291, in associate_from_repo
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168)     raise (e, None, sys.exc_info()[2])
Jul 31 16:09:17 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [30e85096] (1877-39168) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>

...and here's the error when attempting to copy tags between repositories:

Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168) Exception from importer [docker_importer] while importing units into repository [94d08260-7dc9-4bf0-9b9a-d23ed19
3c2c3]
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168) Traceback (most recent call last):
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     units=transfer_units)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 217, in import_units
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     units_added |= set(unit_importers[type(unit)](import_conduit, unit, dest_repo.repo_obj))
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 299, in _import_tag
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     conduit, manifest, dest_repo))
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 369, in _import_manifest_list
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     for manifest in models.Manifest.objects.filter(digest__in=sorted(manifest_digests)):
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/mongoengine/queryset/base.py", line 1407, in next
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     raw_doc = self._cursor.next()
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1097, in next
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     if len(self.__data) or self._refresh():
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1019, in _refresh
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     self.__read_concern))
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 850, in __send_message
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 794, in _send_message_with_response
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     exhaust)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 805, in _reset_on_error
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     return func(*args, **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/server.py", line 108, in send_message_with_response
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     set_slave_okay, sock_info.is_mongos, use_find_cmd)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib64/python2.7/site-packages/pymongo/message.py", line 275, in get_message
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168)     spec, self.fields, self.codec_options)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.managers.repo.unit_association:ERROR: [7c6d1641] (1877-39168) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: pulp.server.async.tasks:INFO: [7c6d1641] Task failed : [7c6d1641-66f4-4408-8a70-6a7d3d2b36ec]
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168) Task pulp.server.managers.repo.unit_association.associate_from_repo[7c6d1641-66f4-4408-8a70-6a7d3d2b36ec] raised unexpected: InvalidDocument('Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>',)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168) Traceback (most recent call last):
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)     R = retval = fun(*args, **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 529, in __call__
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)     return super(Task, self).__call__(*args, **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)     return super(PulpTask, self).__call__(*args, **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)     return self.run(*args, **kwargs)
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 291, in associate_from_repo
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168)     raise (e, None, sys.exc_info()[2])
Jul 31 17:02:35 rhel-7-pulp-2-17-nightly pulp[1877]: celery.app.trace:ERROR: [7c6d1641] (1877-39168) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>

The error looks nearly identical. This error occurs with the nightly build of Pulp 2 (to-be 2.17) on both RHEL 7 and Fedora 27. Here's some sample RPMs:

[root@rhel-7-pulp-2-17-nightly ~]# rpm -qa | grep pulp | sort
pulp-admin-client-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
pulp-deb-admin-extensions-1.8.0-0.1.alpha.201807310516git4df9825.el7.noarch
pulp-deb-plugins-1.8.0-0.1.alpha.201807310516git4df9825.el7.noarch
pulp-docker-admin-extensions-3.2.0-0.1.alpha.201807310503git4356228.el7.noarch
pulp-docker-plugins-3.2.0-0.1.alpha.201807310503git4356228.el7.noarch
pulp-ostree-admin-extensions-1.4.0-0.1.alpha.201807310504git2c2d534.el7.noarch
pulp-ostree-plugins-1.4.0-0.1.alpha.201807310504git2c2d534.el7.noarch
pulp-puppet-admin-extensions-2.17.0-0.1.alpha.201807310506git44a02f8.el7.noarch
pulp-puppet-plugins-2.17.0-0.1.alpha.201807310506git44a02f8.el7.noarch
pulp-puppet-tools-2.17.0-0.1.alpha.201807310506git44a02f8.el7.noarch
pulp-python-admin-extensions-2.1.0-0.1.alpha.201807310501git5e2aa35.el7.noarch
pulp-python-plugins-2.1.0-0.1.alpha.201807310501git5e2aa35.el7.noarch
pulp-rpm-admin-extensions-2.17.0-0.1.alpha.201807310508giteaff2ca.el7.noarch
pulp-rpm-plugins-2.17.0-0.1.alpha.201807310508giteaff2ca.el7.noarch
pulp-selinux-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
pulp-server-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
python-pulp-bindings-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-pulp-client-lib-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-pulp-common-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-pulp-deb-common-1.8.0-0.1.alpha.201807310516git4df9825.el7.noarch
python-pulp-docker-common-3.2.0-0.1.alpha.201807310503git4356228.el7.noarch
python-pulp-oid_validation-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-pulp-ostree-common-1.4.0-0.1.alpha.201807310504git2c2d534.el7.noarch
python-pulp-puppet-common-2.17.0-0.1.alpha.201807310506git44a02f8.el7.noarch
python-pulp-python-common-2.1.0-0.1.alpha.201807310501git5e2aa35.el7.noarch
python-pulp-repoauth-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch
python-pulp-rpm-common-2.17.0-0.1.alpha.201807310508giteaff2ca.el7.noarch
python-pulp-streamer-2.17.0-0.1.alpha.201807310510git0697ade.el7.noarch

The easiest way to reproduce this issue is to run the automated test in Pulp 2 Tests: python -m unittest pulp_2_tests.tests.docker.api_v2.test_copy.CopyV2ContentTestCase. The error occurs when this line of code is executing, inside test_02_copy_manifest_lists:

client.post(urljoin(repo['_href'], 'actions/associate/'), {
    'criteria': {'filters': {}, 'type_ids': ['docker_manifest_list']},
    'source_repo_id': self.repo['id'],
})

It also occurs when this line of code is executing, inside test_02_copy_tags:

client.post(urljoin(repo['_href'], 'actions/associate/'), {
    'source_repo_id': self.repo['id'],
    'criteria': {'filters': {}, 'type_ids': ['docker_tag']},
})

Also see:


Related issues

Copied to Docker Support - Issue #3904: Can't upload manifest list to Docker repository CLOSED - CURRENTRELEASE Actions

Associated revisions

Revision baeef3ae View on GitHub
Added by werwty 11 months ago

Fix issue with associating embedded manifests in manifest list

https://pulp.plan.io/issues/3892

re #3137
closes #3892

Revision baeef3ae View on GitHub
Added by werwty 11 months ago

Fix issue with associating embedded manifests in manifest list

https://pulp.plan.io/issues/3892

re #3137
closes #3892

History

#1 Updated by Ichimonji10 11 months ago

This issue is a regression.

#2 Updated by Ichimonji10 11 months ago

  • Subject changed from Can't copy Docker manifest lists between repositories to Can't copy Docker tags or manifest lists between repositories
  • Description updated (diff)

#3 Updated by bizhang 11 months ago

  • Status changed from NEW to POST
  • Assignee set to bizhang

#4 Updated by werwty 11 months ago

  • Status changed from POST to MODIFIED

#5 Updated by ipanova@redhat.com 11 months ago

  • Sprint/Milestone set to 2.17.0

#6 Updated by Ichimonji10 11 months ago

  • Status changed from MODIFIED to ASSIGNED

This issue still manifests itself with the latest nightly builds of 2 master (to-be 2.17) on both F27 and RHEL 7. To reproduce the issue, run the same automate tests as before with python -m unittest --verbose pulp_2_tests.tests.docker.api_v2.test_copy.CopyV2ContentTestCase. Here's a snippet from journalctl:

Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480) Exception from importer [docker_importer] while importing units into repository [159a49d7-505c-454b-8483-2061831c97a5]
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480) Traceback (most recent call last):
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     units=transfer_units)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 217, in import_units
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     units_added |= set(unit_importers[type(unit)](import_conduit, unit, dest_repo.repo_obj))
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 299, in _import_tag
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     conduit, manifest, dest_repo))
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py", line 369, in _import_manifest_list
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     for manifest in models.Manifest.objects.filter(digest__in=sorted(manifest_digests)):
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/mongoengine/queryset/base.py", line 1407, in next
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     raw_doc = self._cursor.next()
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1097, in next
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     if len(self.__data) or self._refresh():
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1019, in _refresh
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     self.__read_concern))
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 850, in __send_message
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 794, in _send_message_with_response
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     exhaust)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/mongo_client.py", line 805, in _reset_on_error
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     return func(*args, **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/server.py", line 108, in send_message_with_response
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     set_slave_okay, sock_info.is_mongos, use_find_cmd)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib64/python2.7/site-packages/pymongo/message.py", line 275, in get_message
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480)     spec, self.fields, self.codec_options)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.managers.repo.unit_association:ERROR: [b39c1fdd] (1846-64480) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: pulp.server.async.tasks:INFO: [b39c1fdd] Task failed : [b39c1fdd-4680-40c4-8150-c134937df5fb]
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480) Task pulp.server.managers.repo.unit_association.associate_from_repo[b39c1fdd-4680-40c4-8150-c134937df5fb] raised unexpected: InvalidDocument('Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>',)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480) Traceback (most recent call last):
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)     R = retval = fun(*args, **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 529, in __call__
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)     return super(Task, self).__call__(*args, **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)     return super(PulpTask, self).__call__(*args, **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)     return self.run(*args, **kwargs)
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 291, in associate_from_repo
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480)     raise (e, None, sys.exc_info()[2])
Aug 02 14:00:21 rhel-7-pulp-2-17-nightly pulp[1846]: celery.app.trace:ERROR: [b39c1fdd] (1846-64480) InvalidDocument: Cannot encode object: <EmbeddedManifest: EmbeddedManifest object>

Here's the RPMs on a sample test VM:

[root@rhel-7-pulp-2-17-nightly ~]# rpm -qa | grep -i 'solv\|pulp' | sort
libsolv-0.6.34-4.el7.x86_64
pulp-admin-client-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
pulp-deb-admin-extensions-1.8.0-0.1.alpha.201808010517git4df9825.el7.noarch
pulp-deb-plugins-1.8.0-0.1.alpha.201808010517git4df9825.el7.noarch
pulp-docker-admin-extensions-3.2.0-0.1.alpha.201808010504git4356228.el7.noarch
pulp-docker-plugins-3.2.0-0.1.alpha.201808010504git4356228.el7.noarch
pulp-ostree-admin-extensions-1.4.0-0.1.alpha.201808010506git2c2d534.el7.noarch
pulp-ostree-plugins-1.4.0-0.1.alpha.201808010506git2c2d534.el7.noarch
pulp-puppet-admin-extensions-2.17.0-0.1.alpha.201808010507git44a02f8.el7.noarch
pulp-puppet-plugins-2.17.0-0.1.alpha.201808010507git44a02f8.el7.noarch
pulp-puppet-tools-2.17.0-0.1.alpha.201808010507git44a02f8.el7.noarch
pulp-python-admin-extensions-2.1.0-0.1.alpha.201808010503git5e2aa35.el7.noarch
pulp-python-plugins-2.1.0-0.1.alpha.201808010503git5e2aa35.el7.noarch
pulp-rpm-admin-extensions-2.17.0-0.1.alpha.201808010509git421caf4.el7.noarch
pulp-rpm-plugins-2.17.0-0.1.alpha.201808010509git421caf4.el7.noarch
pulp-selinux-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
pulp-server-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python2-solv-0.6.34-4.el7.x86_64
python-isodate-0.5.0-4.pulp.el7.noarch
python-pulp-bindings-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python-pulp-client-lib-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python-pulp-common-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python-pulp-deb-common-1.8.0-0.1.alpha.201808010517git4df9825.el7.noarch
python-pulp-docker-common-3.2.0-0.1.alpha.201808010504git4356228.el7.noarch
python-pulp-oid_validation-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python-pulp-ostree-common-1.4.0-0.1.alpha.201808010506git2c2d534.el7.noarch
python-pulp-puppet-common-2.17.0-0.1.alpha.201808010507git44a02f8.el7.noarch
python-pulp-python-common-2.1.0-0.1.alpha.201808010503git5e2aa35.el7.noarch
python-pulp-repoauth-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch
python-pulp-rpm-common-2.17.0-0.1.alpha.201808010509git421caf4.el7.noarch
python-pulp-streamer-2.17.0-0.1.alpha.201808010512git0697ade.el7.noarch

#7 Updated by bizhang 11 months ago

  • Status changed from ASSIGNED to MODIFIED

looks like the nightly packages haven't pulled in the latest commit (hash is 4356228 instead of fd8d4e7)
the nightlies didn't build this morning, so the change never made it into the latest rpm
https://repos.fedorapeople.org/repos/pulp/pulp/testing/automation/2.17/stage/fedora-27/x86_64/

`python -m unittest --verbose pulp_2_tests.tests.docker.api_v2.test_copy.CopyV2ContentTestCase` with pulp_docker 2-master branch succeeds without an error

#8 Updated by Ichimonji10 11 months ago

  • Copied to Issue #3904: Can't upload manifest list to Docker repository added

#9 Updated by ipanova@redhat.com 11 months ago

  • Platform Release set to 2.17.0

#10 Updated by ipanova@redhat.com 11 months ago

  • Platform Release changed from 2.17.0 to master

#11 Updated by ipanova@redhat.com 11 months ago

  • Platform Release deleted (master)

#12 Updated by ipanova@redhat.com 11 months ago

  • Platform Release set to 2.17.0

#13 Updated by ipanova@redhat.com 11 months ago

  • Project changed from Crane to Docker Support

#14 Updated by ipanova@redhat.com 10 months ago

  • Status changed from MODIFIED to ON_QA

#15 Updated by ipanova@redhat.com 10 months ago

  • Status changed from ON_QA to CLOSED - CURRENTRELEASE

#16 Updated by bmbouter 2 months ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF