Project

Profile

Help

Issue #1733

closed

Pulp search call raising serialization issues

Added by paji@redhat.com about 8 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.8.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

What I have

$ rpm -qa|grep pulp
python-pulp-ostree-common-1.1.0-0.5.beta.git.5.59218a9.el7.noarch
python-pulp-oid_validation-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-nodes-admin-extensions-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
python-pulp-streamer-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-client-1.0-1.noarch
python-pulp-common-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
python-pulp-docker-common-2.0.0-0.5.beta.git.30.8fbfb2e.el7.noarch
pulp-server-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-consumer-client-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-rpm-consumer-extensions-2.8.0-0.5.beta.git.50.28185d2.el7.noarch
pulp-docker-admin-extensions-2.0.0-0.5.beta.git.30.8fbfb2e.el7.noarch
pulp-nodes-parent-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-python-plugins-1.1.0-0.2.beta.git.10.9827327.el7.noarch
python-pulp-python-common-1.1.0-0.2.beta.git.10.9827327.el7.noarch
rubygem-smart_proxy_pulp-1.2.0-1.el7.noarch
python-pulp-repoauth-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-admin-client-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-puppet-consumer-extensions-2.8.0-0.5.beta.git.6.5894bd0.el7.noarch
pulp-ostree-admin-extensions-1.1.0-0.5.beta.git.5.59218a9.el7.noarch
pulp-puppet-admin-extensions-2.8.0-0.5.beta.git.6.5894bd0.el7.noarch
pulp-puppet-plugins-2.8.0-0.5.beta.git.6.5894bd0.el7.noarch
pulp-selinux-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-katello-1.0-1.el7.noarch
python-pulp-bindings-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
python-pulp-rpm-common-2.8.0-0.5.beta.git.50.28185d2.el7.noarch
python-pulp-client-lib-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-rpm-plugins-2.8.0-0.5.beta.git.50.28185d2.el7.noarch
pulp-python-admin-extensions-1.1.0-0.2.beta.git.10.9827327.el7.noarch
pulp-rpm-admin-extensions-2.8.0-0.5.beta.git.50.28185d2.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
pulp-docker-plugins-2.0.0-0.5.beta.git.30.8fbfb2e.el7.noarch
python-kombu-3.0.33-2.pulp.el7.noarch
python-pulp-puppet-common-2.8.0-0.5.beta.git.6.5894bd0.el7.noarch
pulp-nodes-common-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-nodes-consumer-extensions-2.8.0-0.5.beta.git.125.1d7e5c5.el7.noarch
pulp-ostree-plugins-1.1.0-0.5.beta.git.5.59218a9.el7.noarch

Rest call I ran

RestClient.post "https://katello-yoda.example.com/pulp/api/v2/content/units/ostree/search/", "{\"criteria\":{}}", "Accept"=>"*/*; q=0.5, application/xml", "Accept-Encoding"=>"gzip, deflate", "Content-Length"=>"15", "accept"=>"application/json", "content_type"=>"application/json"

trace I got


Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: Unhandled Exception
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584) datetime.datetime(2016, 2, 23, 0, 23, 32, 547000) is not JSON serializable
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584) Traceback (most recent call last):
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     response = wrapped_callback(request, *callback_args, **callback_kwargs)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     return self.dispatch(request, *args, **kwargs)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     return handler(request, *args, **kwargs)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py", line 241, in _auth_decorator
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     return _verify_auth(self, operation, super_user_only, method, *args, **kwargs)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py", line 195, in _verify_auth
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     value = method(self, *args, **kwargs)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/util.py", line 130, in wrapper
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     return func(*args, **kwargs)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/search.py", line 127, in post
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     return self._generate_response(query, options, *args, **kwargs)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/search.py", line 195, in _generate_response
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     *args, **kwargs))
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/util.py", line 52, in generate_json_response
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     json_obj = json.dumps(content, default=default)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib64/python2.7/json/__init__.py", line 243, in dumps
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     return _default_encoder.encode(obj)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     chunks = self.iterencode(o, _one_shot=True)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     return _iterencode(o, 0)
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)   File "/usr/lib64/python2.7/json/encoder.py", line 184, in default
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584)     raise TypeError(repr(o) + " is not JSON serializable")
Mar  1 17:13:42 katello-yoda pulp: pulp.server.webservices.middleware.exception:ERROR: (2077-87584) TypeError: datetime.datetime(2016, 2, 23, 0, 23, 32, 547000) is not JSON serializable

Further investigation content it was trying to serialize in /usr/lib/python2.7/site-packages/pulp/server/webservices/views/util.py(52)generate_json_response() (https://github.com/pulp/pulp/blob/master/server/pulp/server/webservices/views/util.py#L52)

-> json_obj = json.dumps(content, default=default)
(Pdb) content
[{u'_storage_path': u'/var/lib/pulp/content/shared/ostree/f3769796a028aff68cf0e680a3e84bd236833ad48c2700f1e593ea584e5e6abf/links/f90f3a90-b210-42b5-8773-8b447ad667a0', '_href': '/pulp/api/v2/content/units/ostree/f90f3a90-b210-42b5-8773-8b447ad667a0/', u'_last_updated': '2016-02-23T00:23:32Z', u'remote_id': u'174b6356ebe246768ac1600709cfefc6589a43503397a9e7589111d4da9188f6', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'rhel-atomic-host/7/x86_64/standard', u'_created': datetime.datetime(2016, 2, 23, 0, 23, 32, 547000), u'commit': u'04019d7e50b22c19b717e657b58c0e33333eb92448fb99fe478c922d6e8bbe95', u'_id': u'f90f3a90-b210-42b5-8773-8b447ad667a0', 'children': {}, u'metadata': {u'version': u'7.2.2-1', u'rpmostree-inputhash': u'9f94ff173c413420ce2a21eee338482f97d619054f5d6b80a5e90745620e4430'}}, {u'_storage_path': u'/var/lib/pulp/content/shared/ostree/34f4a6fe63bffbe1afcb3a80ba0cf4f1e1d3e4e59e28f3e6c336fc29df54bf57/links/b359ac09-bdbe-4e86-8cf7-e907294deb01', '_href': '/pulp/api/v2/content/units/ostree/b359ac09-bdbe-4e86-8cf7-e907294deb01/', u'_last_updated': '2016-02-29T21:37:53Z', u'remote_id': u'8cb7a64cef67a71052af69d61a5f5de8dca2224faef02ce28ed0d48e43a3d9be', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'fedora-atomic/f22/x86_64/docker-host', u'_created': datetime.datetime(2016, 2, 29, 21, 37, 53, 234000), u'commit': u'da8d1176c068506905777a0bf8b452e9f0477947005fe4a346ad04273950bd5f', u'_id': u'b359ac09-bdbe-4e86-8cf7-e907294deb01', 'children': {}, u'metadata': {u'version': u'22.236'}}, {u'_storage_path': u'/var/lib/pulp/content/shared/ostree/34f4a6fe63bffbe1afcb3a80ba0cf4f1e1d3e4e59e28f3e6c336fc29df54bf57/links/785947cd-4fa5-4170-82e8-9784f084be38', '_href': '/pulp/api/v2/content/units/ostree/785947cd-4fa5-4170-82e8-9784f084be38/', u'_last_updated': '2016-02-29T21:37:53Z', u'remote_id': u'8cb7a64cef67a71052af69d61a5f5de8dca2224faef02ce28ed0d48e43a3d9be', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'fedora-atomic/f22/x86_64/testing/docker-host', u'_created': datetime.datetime(2016, 2, 29, 21, 37, 53, 238000), u'commit': u'37447667bdf911d656326dced2485f680fe0a10288c4ce5b07a3eb2433d3b988', u'_id': u'785947cd-4fa5-4170-82e8-9784f084be38', 'children': {}, u'metadata': {u'version': u'22.155', u'rpmostree-inputhash': u'769ffd023cdd143e91dec6481c2efcf4bf3764dc39c5d0cd1499907a2306dcdd'}}, {u'_storage_path': u'/var/lib/pulp/content/shared/ostree/34f4a6fe63bffbe1afcb3a80ba0cf4f1e1d3e4e59e28f3e6c336fc29df54bf57/links/7f36b3dc-3abe-43c7-8cf1-22ad9f730751', '_href': '/pulp/api/v2/content/units/ostree/7f36b3dc-3abe-43c7-8cf1-22ad9f730751/', u'_last_updated': '2016-02-29T21:37:53Z', u'remote_id': u'8cb7a64cef67a71052af69d61a5f5de8dca2224faef02ce28ed0d48e43a3d9be', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'fedora-atomic/f22/x86_64/docker-host', u'_created': datetime.datetime(2016, 2, 29, 21, 37, 53, 260000), u'commit': u'7e82c9ae39e0f91e27f459ed1d57273142b8e868586ec88035de085d7b7920c8', u'_id': u'7f36b3dc-3abe-43c7-8cf1-22ad9f730751', 'children': {}, u'metadata': {u'version': u'22.248'}}, {u'_storage_path': u'/var/lib/pulp/content/shared/ostree/34f4a6fe63bffbe1afcb3a80ba0cf4f1e1d3e4e59e28f3e6c336fc29df54bf57/links/af66e2ee-1d18-4066-916f-8e3acd9fa2a0', '_href': '/pulp/api/v2/content/units/ostree/af66e2ee-1d18-4066-916f-8e3acd9fa2a0/', u'_last_updated': '2016-02-29T21:37:53Z', u'remote_id': u'8cb7a64cef67a71052af69d61a5f5de8dca2224faef02ce28ed0d48e43a3d9be', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'fedora-atomic/f22/x86_64/testing/docker-host', u'_created': datetime.datetime(2016, 2, 29, 21, 37, 53, 262000), u'commit': u'cac8ce342a4477e0fb5947c7db96835124f088e1e8df35d1591478705a1e6e50', u'_id': u'af66e2ee-1d18-4066-916f-8e3acd9fa2a0', 'children': {}, u'metadata': {u'version': u'22.164', u'rpmostree-inputhash': u'609f92be153e49945dd9f826966be724011e6e6c627b6606211e557e6907d1be'}}, {u'_storage_path': u'/var/lib/pulp/content/shared/ostree/34f4a6fe63bffbe1afcb3a80ba0cf4f1e1d3e4e59e28f3e6c336fc29df54bf57/links/ffcd942f-d5b4-4459-a67d-78b2f03c648b', '_href': '/pulp/api/v2/content/units/ostree/ffcd942f-d5b4-4459-a67d-78b2f03c648b/', u'_last_updated': '2016-02-29T21:37:53Z', u'remote_id': u'8cb7a64cef67a71052af69d61a5f5de8dca2224faef02ce28ed0d48e43a3d9be', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'fedora-atomic/f22/x86_64/docker-host', u'_created': datetime.datetime(2016, 2, 29, 21, 37, 53, 264000), u'commit': u'ae8bd035f998d773ee63bbafce84af8688e3f14054b2094c5ad2f4649dad8af1', u'_id': u'ffcd942f-d5b4-4459-a67d-78b2f03c648b', 'children': {}, u'metadata': {u'version': u'22.229'}}, {u'_storage_path': u'/var/lib/pulp/content/shared/ostree/34f4a6fe63bffbe1afcb3a80ba0cf4f1e1d3e4e59e28f3e6c336fc29df54bf57/links/11712625-d85a-484e-9791-2be382c6008c', '_href': '/pulp/api/v2/content/units/ostree/11712625-d85a-484e-9791-2be382c6008c/', u'_last_updated': '2016-02-29T21:37:53Z', u'remote_id': u'8cb7a64cef67a71052af69d61a5f5de8dca2224faef02ce28ed0d48e43a3d9be', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'fedora-atomic/f22/x86_64/testing/docker-host', u'_created': datetime.datetime(2016, 2, 29, 21, 37, 53, 267000), u'commit': u'ff4b625a75cb8edbe2ccf1f984fd9d7fa31f931a79d715fb657768e5b9ddb9a5', u'_id': u'11712625-d85a-484e-9791-2be382c6008c', 'children': {}, u'metadata': {u'version': u'22.150', u'rpmostree-inputhash': u'598f61fc23e9af68ca608471a24b3806b1e1a65df2068411cc23a8d81b69b709'}}, {u'_storage_path': u'/var/lib/pulp/content/shared/ostree/34f4a6fe63bffbe1afcb3a80ba0cf4f1e1d3e4e59e28f3e6c336fc29df54bf57/links/9a3534ea-08a3-49ba-ba4a-be31ed6895a3', '_href': '/pulp/api/v2/content/units/ostree/9a3534ea-08a3-49ba-ba4a-be31ed6895a3/', u'_last_updated': '2016-02-29T23:04:43Z', u'remote_id': u'8cb7a64cef67a71052af69d61a5f5de8dca2224faef02ce28ed0d48e43a3d9be', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'fedora-atomic/f22/x86_64/docker-host', u'_created': datetime.datetime(2016, 2, 29, 23, 4, 43, 817000), u'commit': u'd9c8bde9d39334f72124f8b22c402f859ab1dcb37a4f7c946c6576224809a9fd', u'_id': u'9a3534ea-08a3-49ba-ba4a-be31ed6895a3', 'children': {}, u'metadata': {u'version': u'22.249'}}, {u'_storage_path': u'/var/lib/pulp/content/shared/ostree/48868e83e4a50500d16c6adac1dfa68bb50d72ec87311aa4be87f01796293958/links/327c2157-fbef-40ff-8ba8-52f3d1501841', '_href': '/pulp/api/v2/content/units/ostree/327c2157-fbef-40ff-8ba8-52f3d1501841/', u'_last_updated': '2016-03-01T00:07:24Z', u'remote_id': u'2c5b6e07fe47d0b626a417d5dbb855a4aa8c991177e930213a3a80514d3e1a14', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'fedora-atomic/f23/x86_64/docker-host', u'_created': datetime.datetime(2016, 3, 1, 0, 7, 24, 521000), u'commit': u'954bdbeebebfa87b625d9d7bd78c81400bdd6756fcc3205987970af4b64eb678', u'_id': u'327c2157-fbef-40ff-8ba8-52f3d1501841', 'children': {}, u'metadata': {u'version': u'23.74', u'rpmostree-inputhash': u'cefa153b7f2ec7f2ee2cc6f4b2bb18ac34ef5128cf4a56cbcb6998e8a35b63ca'}}, {u'_storage_path': u'/var/lib/pulp/content/shared/ostree/48868e83e4a50500d16c6adac1dfa68bb50d72ec87311aa4be87f01796293958/links/7e2a244d-3b74-4647-a9fc-6868f177c135', '_href': '/pulp/api/v2/content/units/ostree/7e2a244d-3b74-4647-a9fc-6868f177c135/', u'_last_updated': '2016-03-01T00:07:24Z', u'remote_id': u'2c5b6e07fe47d0b626a417d5dbb855a4aa8c991177e930213a3a80514d3e1a14', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'fedora-atomic/f23/x86_64/testing/docker-host', u'_created': datetime.datetime(2016, 3, 1, 0, 7, 24, 530000), u'commit': u'2c515ce0327df9cec4288490c8d0275c311bab72d24faebf6d6f9c4d6dd8aa8b', u'_id': u'7e2a244d-3b74-4647-a9fc-6868f177c135', 'children': {}, u'metadata': {u'version': u'23.78', u'rpmostree-inputhash': u'71871d96a49212899335587be9af33354151b927f4d8c8cb8e58729b036afda9'}}, {u'_storage_path': u'/var/lib/pulp/content/shared/ostree/34f4a6fe63bffbe1afcb3a80ba0cf4f1e1d3e4e59e28f3e6c336fc29df54bf57/links/fc7c6b5c-7e70-4bdc-a6d5-b709a7e7ad84', '_href': '/pulp/api/v2/content/units/ostree/fc7c6b5c-7e70-4bdc-a6d5-b709a7e7ad84/', u'_last_updated': '2016-03-01T00:09:51Z', u'remote_id': u'8cb7a64cef67a71052af69d61a5f5de8dca2224faef02ce28ed0d48e43a3d9be', u'_content_type_id': u'ostree', u'pulp_user_metadata': {}, u'branch': u'fedora-atomic/f22/x86_64/testing/docker-host', u'_created': datetime.datetime(2016, 3, 1, 0, 9, 51, 124000), u'commit': u'64d595fb1159e2e267e7d574c37420c125b46e77e53eb59ce4bd3a19a271830d', u'_id': u'fc7c6b5c-7e70-4bdc-a6d5-b709a7e7ad84', 'children': {}, u'metadata': {u'version': u'22.165', u'rpmostree-inputhash': u'adb48464bccaf5aa93483acf8d3e0113464f1210be44f6895dd61a37bbc5cfe5'}}]

Last but not the least the Actual error


(Pdb) json.dumps(content, default=default)
*** TypeError: datetime.datetime(2016, 2, 23, 0, 23, 32, 547000) is not JSON serializable

So basically json.dumps doesnt know how to handle date time for type_id "ostree"

Also available in: Atom PDF