Project

Profile

Help

Issue #312

It is impossible to use filters with GET search requests, which makes the name of the REST view a bit of a lie

Added by rbarlow over 6 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.7.2
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Easy Fix, Pulp 2
Sprint:
Quarter:

Description

In converting our search API to Django, I found that it is impossible to search with filters in a GET request, despite the documentation saying it is possible[0].

$ http --json -a admin:admin --verify=no GET https://localhost/pulp/api/v2/tasks/search/?filters=\{\"state\"\:\"running\"\}\&field=task_id\&limit=1 WebFrameworkSwitch:webpy
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
HTTP/1.1 400 Bad Request
Connection: close
Content-Encoding: utf-8
Content-Length: 378
Content-Type: application/json
Date: Wed, 25 Feb 2015 16:46:32 GMT
Server: Apache/2.4.6 (Red Hat) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

{
    "_href": "/pulp/api/v2/tasks/search/", 
    "error": {
        "code": "PLP0015", 
        "data": {
            "properties": "['filters']", 
            "property_names": [
                "filters"
            ]
        }, 
        "description": "Invalid properties: ['filters']", 
        "sub_errors": []
    }, 
    "error_message": "Invalid properties: ['filters']", 
    "exception": null, 
    "http_request_method": "GET", 
    "http_status": 400, 
    "property_names": [
        "filters"
    ], 
    "traceback": null
}

I expect this behavior instead:

$ http --json -a admin:admin --verify=no GET https://localhost/pulp/api/v2/tasks/search/?filters=\{\"task_id\"\:\"e29c1188-892c-44d6-ba3a-2742beb40423\"\}\&field=task_id WebFrameworkSwitch:webpy
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 449
Content-Type: application/json
Date: Wed, 25 Feb 2015 17:02:09 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.6 (Red Hat) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

[
    {
        "_href": "/pulp/api/v2/tasks/e29c1188-892c-44d6-ba3a-2742beb40423/", 
        "_id": {
            "$oid": "54ed4356a568cca370f63bdc"
        }, 
        "_ns": "task_status", 
        "error": null, 
        "exception": null, 
        "finish_time": null, 
        "id": "54ed4356a568cca370f63bdc", 
        "progress_report": {}, 
        "result": null, 
        "spawned_tasks": [], 
        "start_time": null, 
        "state": "waiting", 
        "tags": [], 
        "task_id": "e29c1188-892c-44d6-ba3a-2742beb40423", 
        "task_type": null, 
        "traceback": null, 
        "worker_name": null
    }
]

I dug into the code for a bit and determined that the get handler was passing the "filters" parameter as a string instead of as a dictionary to the Criteria object, which is not valid. It is the Criteria object that raises the error message "Invalid properties: ['filters']". The fix is a one liner! All we need to do is to insert this into the SearchView's get() handler before it hands the query to the Criteria.from_client_input() method:

query['filters'] = json.loads(query['filters'])

I considered fixing this along with my conversion to Django, but in the interest of making a pure conversion, I've decided to handle this with a bug report and separate pull request instead.

In my opinion, this is important to fix. It is not very RESTful to search via a POST method, but currently it is impossible to search via a GET method, which is the intended method for read operations.

[0] http://pulp.readthedocs.org/en/latest/dev-guide/conventions/criteria.html#search-api


Related issues

Related to Pulp - Issue #1040: Search with Get does not workCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Blocks Pulp - Issue #1332: in Search API, restritcting fields does not work with MongoengineCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 8297b6fb View on GitHub
Added by Austin Macdonald over 6 years ago

fix filters field so GET searches work

closes #312

Revision 8297b6fb View on GitHub
Added by Austin Macdonald over 6 years ago

fix filters field so GET searches work

closes #312

Revision f25ebc96 View on GitHub
Added by Austin Macdonald almost 6 years ago

refactor the search class to simplify use of GET and POST searches

closes: #312

Revision f25ebc96 View on GitHub
Added by Austin Macdonald almost 6 years ago

refactor the search class to simplify use of GET and POST searches

closes: #312

History

#1 Updated by bmbouter over 6 years ago

  • Triaged changed from No to Yes

#2 Updated by rbarlow over 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to rbarlow

#3 Updated by bmbouter over 6 years ago

  • Severity changed from High to 3. High

#4 Updated by amacdona@redhat.com over 6 years ago

  • Status changed from ASSIGNED to POST
  • Assignee changed from rbarlow to amacdona@redhat.com

#5 Updated by amacdona@redhat.com over 6 years ago

  • Status changed from POST to MODIFIED

Switched PR's to merge to 2.7-dev rather than master.

https://github.com/pulp/pulp/pull/1790

#6 Updated by dkliban@redhat.com over 6 years ago

  • Platform Release set to 2.7.0

#7 Updated by rbarlow over 6 years ago

  • Has duplicate Issue #1040: Search with Get does not work added

#8 Updated by rbarlow over 6 years ago

  • Has duplicate deleted (Issue #1040: Search with Get does not work)

#9 Updated by rbarlow over 6 years ago

  • Related to Issue #1040: Search with Get does not work added

#10 Updated by dkliban@redhat.com over 6 years ago

  • Status changed from MODIFIED to 5

#11 Updated by igulina@redhat.com over 6 years ago

  • Status changed from 5 to ASSIGNED
>> rpm -qa pulp-server
pulp-server-2.7.0-0.4.beta.el7.noarch

Operations:  publish
Resources:   zoopark (repository)
State:       Successful
Start Time:  2015-07-08T11:53:38Z
Finish Time: 2015-07-08T11:53:39Z
Task Id:     c9b1f1b3-0293-4470-bbc9-293a3965a901

>> http --json -a admin:admin --verify=no GET https://localhost/pulp/api/v2/tasks/search/?filters=\{\"task_id\"\:\"c9b1f1b3-0293-4470-bbc9-293a3965a901\"\}\&field=task_id WebFrameworkSwitch:webpy
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
HTTP/1.1 400 BAD REQUEST
Connection: close
Content-Encoding: utf-8
Content-Length: 458
Content-Type: application/json
Date: Wed, 08 Jul 2015 11:54:59 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

{
    "_href": "/pulp/api/v2/tasks/search/?filters=%7B%22task_id%22:%22c9b1f1b3-0293-4470-bbc9-293a3965a901%22%7D&field=task_id", 
    "error": {
        "code": "PLP0015", 
        "data": {
            "properties": "['fields']", 
            "property_names": [
                "fields"
            ]
        }, 
        "description": "Invalid properties: ['fields']", 
        "sub_errors": []
    }, 
    "error_message": "Invalid properties: ['fields']", 
    "exception": null, 
    "http_request_method": "GET", 
    "http_status": 400, 
    "property_names": [
        "fields"
    ], 
    "traceback": null
}

>>  http --json -a admin:admin --verify=no GET https://localhost/pulp/api/v2/tasks/search/?filters=\{\"state\"\:\"running\"\}\&field=task_id\&limit=1 WebFrameworkSwitch:webpy
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
HTTP/1.1 400 BAD REQUEST
Connection: close
Content-Encoding: utf-8
Content-Length: 435
Content-Type: application/json
Date: Wed, 08 Jul 2015 11:51:40 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

{
    "_href": "/pulp/api/v2/tasks/search/?filters=%7B%22state%22:%22running%22%7D&field=task_id&limit=1", 
    "error": {
        "code": "PLP0015", 
        "data": {
            "properties": "['fields']", 
            "property_names": [
                "fields"
            ]
        }, 
        "description": "Invalid properties: ['fields']", 
        "sub_errors": []
    }, 
    "error_message": "Invalid properties: ['fields']", 
    "exception": null, 
    "http_request_method": "GET", 
    "http_status": 400, 
    "property_names": [
        "fields"
    ], 
    "traceback": null
}

The same with curl:

curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/repositories/search/?field=id&field=display_name&limit=20'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   417  100   417    0     0    812      0 --:--:-- --:--:-- --:--:--   812
{
    "_href": "/pulp/api/v2/repositories/search/?field=id&field=display_name&limit=20",
    "error": {
        "code": "PLP0015",
        "data": {
            "properties": "['fields']",
            "property_names": [
                "fields"
            ]
        },
        "description": "Invalid properties: ['fields']",
        "sub_errors": []
    },
    "error_message": "Invalid properties: ['fields']",
    "exception": null,
    "http_request_method": "GET",
    "http_status": 400,
    "property_names": [
        "fields"
    ],
    "traceback": null
}

#12 Updated by amacdona@redhat.com over 6 years ago

  • Status changed from ASSIGNED to POST

Good catch @igulina

https://github.com/pulp/pulp/pull/1975

The original fix for this corrected how the 'filters' field is parsed, but the same thing needed to be done for filters, fields, and sort.

#13 Updated by amacdona@redhat.com about 6 years ago

  • Status changed from POST to MODIFIED

#14 Updated by dkliban@redhat.com about 6 years ago

  • Status changed from MODIFIED to 5

#15 Updated by pthomas@redhat.com almost 6 years ago

  • Status changed from 5 to ASSIGNED

This seem to be broken in 2.7

[root@ibm-x3550m3-06 ~]# rpm -qa pulp-server
pulp-server-2.7.0-0.7.beta.el7.noarch
[root@ibm-x3550m3-06 ~]# 
root@ibm-x3550m3-06 ~]# 
[root@ibm-x3550m3-06 ~]# http --json -a admin:admin --verify=no GET https://localhost/pulp/api/v2/tasks/search/?filters=\{\"state\"\:\"running\"\}\&field=task_id\&limit=1 WebFrameworkSwitch:webpy
/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
HTTP/1.1 400 BAD REQUEST
Connection: close
Content-Length: 435
Content-Type: application/json; charset=utf-8
Date: Wed, 21 Oct 2015 14:43:27 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

{
    "_href": "/pulp/api/v2/tasks/search/?filters=%7B%22state%22:%22running%22%7D&field=task_id&limit=1", 
    "error": {
        "code": "PLP0015", 
        "data": {
            "properties": "['fields']", 
            "property_names": [
                "fields"
            ]
        }, 
        "description": "Invalid properties: ['fields']", 
        "sub_errors": []
    }, 
    "error_message": "Invalid properties: ['fields']", 
    "exception": null, 
    "http_request_method": "GET", 
    "http_status": 400, 
    "property_names": [
        "fields"
    ], 
    "traceback": null
}
[root@ibm-x3550m3-06 ~]# 
[root@ibm-x3550m3-06 ~]# curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/repositories/search/?field=id&field=display_name&limit=20'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   417  100   417    0     0   1012      0 --:--:-- --:--:-- --:--:--  1014
{
    "_href": "/pulp/api/v2/repositories/search/?field=id&field=display_name&limit=20",
    "error": {
        "code": "PLP0015",
        "data": {
            "properties": "['fields']",
            "property_names": [
                "fields"
            ]
        },
        "description": "Invalid properties: ['fields']",
        "sub_errors": []
    },
    "error_message": "Invalid properties: ['fields']",
    "exception": null,
    "http_request_method": "GET",
    "http_status": 400,
    "property_names": [
        "fields"
    ],
    "traceback": null
}

#16 Updated by amacdona@redhat.com almost 6 years ago

  • Status changed from ASSIGNED to POST

#17 Updated by amacdona@redhat.com almost 6 years ago

  • Blocks Issue #1332: in Search API, restritcting fields does not work with Mongoengine added

#18 Updated by amacdona@redhat.com almost 6 years ago

  • Status changed from POST to MODIFIED
  • Platform Release changed from 2.7.0 to 2.7.1

#19 Updated by amacdona@redhat.com almost 6 years ago

  • Platform Release changed from 2.7.1 to 2.7.2

#20 Updated by dkliban@redhat.com over 5 years ago

  • Status changed from MODIFIED to 5

#21 Updated by dkliban@redhat.com over 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#22 Updated by bmbouter over 2 years ago

  • Tags Pulp 2 added

#23 Updated by bmbouter over 1 year ago

  • Category deleted (14)

We are removing the 'API' category per open floor discussion June 16, 2020.

Please register to edit this issue

Also available in: Atom PDF