Issue #1040: Search with Get does not work - Pulp

Actions

Send by e-mail Copy link

Issue #1040

closed

Search with Get does not work

Added by ipanova@redhat.com over 9 years ago. Updated over 4 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

High

Assignee:

amacdona@redhat.com

Category:

Sprint/Milestone:

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version:

2.7.0 Beta

Platform Release:

2.7.0

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Easy Fix, Pulp 2

Sprint:

Quarter:

Description

While trying to reproduce issue1038, found that request of search with GET does not work:

$ curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/repositories/search/?limit=1'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1479  100  1479    0     0   5479      0 --:--:-- --:--:-- --:--:--  5498
{
    "_href": "/pulp/api/v2/repositories/search/?limit=1",
    "error_message": "expected string or buffer",
    "exception": [
        "TypeError: expected string or buffer\n"
    ],
    "http_request_method": "GET",
    "http_status": 500,
    "traceback": [
        "  File \"/usr/lib/python2.7/site-packages/django/core/handlers/base.py\", line 112, in get_response\n    response = wrapped_callback(request, *callback_args, **callback_kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/django/views/generic/base.py\", line 69, in view\n    return self.dispatch(request, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/django/views/generic/base.py\", line 87, in dispatch\n    return handler(request, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py\", line 237, in _auth_decorator\n    return _verify_auth(self, operation, super_user_only, method, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py\", line 191, in _verify_auth\n    value = method(self, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/pulp/server/webservices/views/search.py\", line 82, in get\n    query['filters'] = json.loads(request.GET.get('filters'))\n",
        "  File \"/usr/lib64/python2.7/json/__init__.py\", line 338, in loads\n    return _default_decoder.decode(s)\n",
        "  File \"/usr/lib64/python2.7/json/decoder.py\", line 365, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n"
    ]
}

$ curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/consumers/search/'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1468  100  1468    0     0   5736      0 --:--:-- --:--:-- --:--:--  5734
{
    "_href": "/pulp/api/v2/consumers/search/",
    "error_message": "expected string or buffer",
    "exception": [
        "TypeError: expected string or buffer\n"
    ],
    "http_request_method": "GET",
    "http_status": 500,
    "traceback": [
        "  File \"/usr/lib/python2.7/site-packages/django/core/handlers/base.py\", line 112, in get_response\n    response = wrapped_callback(request, *callback_args, **callback_kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/django/views/generic/base.py\", line 69, in view\n    return self.dispatch(request, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/django/views/generic/base.py\", line 87, in dispatch\n    return handler(request, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py\", line 237, in _auth_decorator\n    return _verify_auth(self, operation, super_user_only, method, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py\", line 191, in _verify_auth\n    value = method(self, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/pulp/server/webservices/views/search.py\", line 82, in get\n    query['filters'] = json.loads(request.GET.get('filters'))\n",
        "  File \"/usr/lib64/python2.7/json/__init__.py\", line 338, in loads\n    return _default_decoder.decode(s)\n",
        "  File \"/usr/lib64/python2.7/json/decoder.py\", line 365, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n"
    ]
}

$ curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/content/units/rpm/search/'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1476  100  1476    0     0   5808      0 --:--:-- --:--:-- --:--:--  5833
{
    "_href": "/pulp/api/v2/content/units/rpm/search/",
    "error_message": "expected string or buffer",
    "exception": [
        "TypeError: expected string or buffer\n"
    ],
    "http_request_method": "GET",
    "http_status": 500,
    "traceback": [
        "  File \"/usr/lib/python2.7/site-packages/django/core/handlers/base.py\", line 112, in get_response\n    response = wrapped_callback(request, *callback_args, **callback_kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/django/views/generic/base.py\", line 69, in view\n    return self.dispatch(request, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/django/views/generic/base.py\", line 87, in dispatch\n    return handler(request, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py\", line 237, in _auth_decorator\n    return _verify_auth(self, operation, super_user_only, method, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py\", line 191, in _verify_auth\n    value = method(self, *args, **kwargs)\n",
        "  File \"/usr/lib/python2.7/site-packages/pulp/server/webservices/views/search.py\", line 82, in get\n    query['filters'] = json.loads(request.GET.get('filters'))\n",
        "  File \"/usr/lib64/python2.7/json/__init__.py\", line 338, in loads\n    return _default_decoder.decode(s)\n",
        "  File \"/usr/lib64/python2.7/json/decoder.py\", line 365, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n"
    ]
}

Related issues

Actions

Copy link

Updated by amacdona@redhat.com over 9 years ago

Tags Easy Fix added

Apparently, GET searches did not work at all previously, so while it may be a regression, I don't think this is a new regression to 2.7.

I looked into this a little bit and I see two problems.

First is that json.loads() does not work with None. https://github.com/pulp/pulp/blob/master/server/pulp/server/webservices/views/search.py#L82
The simplest way to fix this would be to enforce that `filters` not be an optional argument. This seems reasonable to me.

The second issue is is that the QueryDict object does not cast to a dict the way I would expect. https://github.com/pulp/pulp/blob/master/server/pulp/server/webservices/views/search.py#L63

qd = QueryDict('a=b')
qd.get('a')

Out: u'b'

dict_args = dict(qd)
dict_args.get('a')

Out[14]: [u'b']

The InvalidValue on limit seen here is due to the fact that the limit is accidentally converted into a list.

Actions

Copy link

Updated by rbarlow over 9 years ago

Is duplicate of Issue #312: It is impossible to use filters with GET search requests, which makes the name of the REST view a bit of a lie added

Actions

Copy link

Updated by ipanova@redhat.com over 9 years ago

@rbarlow, i did not dig into the issue much, but i am not sure it is a dup, here in general you are not able to make search with GET, with or without filters.

Actions

Copy link

Updated by rbarlow over 9 years ago

On 06/09/2015 10:51 AM, Pulp wrote:

@rbarlow, i did not dig into the issue much, but i am not sure it is a
dup, here in general you are not able to make search with GET, with or
without filters.

Ah OK, in that case I switch them to just be related. Thanks for the
clarification!

--
Randy Barlow

Actions

Copy link

Updated by rbarlow over 9 years ago

Is duplicate of deleted (Issue #312: It is impossible to use filters with GET search requests, which makes the name of the REST view a bit of a lie)

Actions

Copy link

Updated by rbarlow over 9 years ago

Related to Issue #312: It is impossible to use filters with GET search requests, which makes the name of the REST view a bit of a lie added

Actions

Copy link

Updated by mhrivnak over 9 years ago

Priority changed from Normal to High
Triaged changed from No to Yes

Actions

Copy link

Updated by amacdona@redhat.com over 9 years ago

Status changed from NEW to ASSIGNED
Assignee set to amacdona@redhat.com
Platform Release set to 2.7.0

Actions

Copy link

Updated by amacdona@redhat.com over 9 years ago

Status changed from ASSIGNED to POST

https://github.com/pulp/pulp/pull/1919

Actions

Copy link

#10

Updated by amacdona@redhat.com over 9 years ago

Status changed from POST to MODIFIED

Moved to this PR

https://github.com/pulp/pulp/pull/1923

Actions

Copy link

#11

Updated by dkliban@redhat.com over 9 years ago

Status changed from MODIFIED to 5

Actions

Copy link

#13

Updated by igulina@redhat.com over 9 years ago

Status changed from 5 to 6

>> rpm -qa pulp-server
pulp-server-2.7.0-0.4.beta.el7.noarch

>> curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/repositories/search/?limit=1'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   291  100   291    0     0    544      0 --:--:-- --:--:-- --:--:--   561
[
    {
        "_href": "/pulp/api/v2/repositories/zoopark/",
        "_id": {
            "$oid": "559be5fb77b2943159a2a435"
        },
        "_ns": "repos",
        "content_unit_counts": {},
        "description": null,
        "display_name": "zoopark",
        "id": "zoopark",
        "last_unit_added": null,
        "last_unit_removed": null,
        "notes": {
            "_repo-type": "rpm-repo"
        }
    }
]

curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/repositories/search/'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   576  100   576    0     0   1494      0 --:--:-- --:--:-- --:--:--  1496
[
    {
        "_href": "/pulp/api/v2/repositories/zoopark/",
        "_id": {
            "$oid": "559be5fb77b2943159a2a435"
        },
        "_ns": "repos",
        "content_unit_counts": {},
        "description": null,
        "display_name": "zoopark",
        "id": "zoopark",
        "last_unit_added": null,
        "last_unit_removed": null,
        "notes": {
            "_repo-type": "rpm-repo"
        }
    },
    {
        "_href": "/pulp/api/v2/repositories/panda/",
        "_id": {
            "$oid": "559d00cf77b29431586958cc"
        },
        "_ns": "repos",
        "content_unit_counts": {},
        "description": null,
        "display_name": "panda",
        "id": "panda",
        "last_unit_added": null,
        "last_unit_removed": null,
        "notes": {
            "_repo-type": "rpm-repo"
        }
    }
]

>>  curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/repository/search/'
<h1>Not Found</h1><p>The requested URL /pulp/api/v2/repository/search/ was not found on this server.</p>

>> curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/consumers/search/'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     2  100     2    0     0      3      0 --:--:-- --:--:-- --:--:--     3
[]

>> pulp-consumer -u admin -p admin register --consumer-id pupyrka
Consumer [pupyrka] successfully registered

[root@ip-10-74-193-205 ec2-user]# curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/consumers/search/'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   684  100   684    0     0   1338      0 --:--:-- --:--:-- --:--:--  1341
[
    {
        "_href": "/pulp/api/v2/consumers/pupyrka/",
        "_id": {
            "$oid": "559d06de77b29431586958d3"
        },
        "_ns": "consumers",
        "capabilities": {},
        "description": null,
        "display_name": "pupyrka",
        "id": "pupyrka",
        "notes": {},
        "rsa_pub": "-----BEGIN PUBLIC KEY-----\nXXX//\n-----END PUBLIC KEY-----\n"
    }
]
>> pulp-consumer -u admin -p admin unregister
Consumer [pupyrka] successfully unregistered

>> curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/consumers/search/'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     2  100     2    0     0      3      0 --:--:-- --:--:-- --:--:--     3
[]

>> curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/consumers/search/?limit=1'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     2  100     2    0     0      3      0 --:--:-- --:--:-- --:--:--     3
[]

>> curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/content/units/search/'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     2  100     2    0     0      3      0 --:--:-- --:--:-- --:--:--     3
[]

Actions

Copy link

#14