Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #312

closed

It is impossible to use filters with GET search requests, which makes the name of the REST view a bit of a lie

Added by rbarlow about 9 years ago. Updated almost 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
amacdona@redhat.com
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.7.2
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Easy Fix, Pulp 2
Sprint:
Quarter:

Description

In converting our search API to Django, I found that it is impossible to search with filters in a GET request, despite the documentation saying it is possible[0].

$ http --json -a admin:admin --verify=no GET https://localhost/pulp/api/v2/tasks/search/?filters=\{\"state\"\:\"running\"\}\&field=task_id\&limit=1 WebFrameworkSwitch:webpy
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
HTTP/1.1 400 Bad Request
Connection: close
Content-Encoding: utf-8
Content-Length: 378
Content-Type: application/json
Date: Wed, 25 Feb 2015 16:46:32 GMT
Server: Apache/2.4.6 (Red Hat) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

{
    "_href": "/pulp/api/v2/tasks/search/", 
    "error": {
        "code": "PLP0015", 
        "data": {
            "properties": "['filters']", 
            "property_names": [
                "filters"
            ]
        }, 
        "description": "Invalid properties: ['filters']", 
        "sub_errors": []
    }, 
    "error_message": "Invalid properties: ['filters']", 
    "exception": null, 
    "http_request_method": "GET", 
    "http_status": 400, 
    "property_names": [
        "filters"
    ], 
    "traceback": null
}

I expect this behavior instead:

$ http --json -a admin:admin --verify=no GET https://localhost/pulp/api/v2/tasks/search/?filters=\{\"task_id\"\:\"e29c1188-892c-44d6-ba3a-2742beb40423\"\}\&field=task_id WebFrameworkSwitch:webpy
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 449
Content-Type: application/json
Date: Wed, 25 Feb 2015 17:02:09 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.6 (Red Hat) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

[
    {
        "_href": "/pulp/api/v2/tasks/e29c1188-892c-44d6-ba3a-2742beb40423/", 
        "_id": {
            "$oid": "54ed4356a568cca370f63bdc"
        }, 
        "_ns": "task_status", 
        "error": null, 
        "exception": null, 
        "finish_time": null, 
        "id": "54ed4356a568cca370f63bdc", 
        "progress_report": {}, 
        "result": null, 
        "spawned_tasks": [], 
        "start_time": null, 
        "state": "waiting", 
        "tags": [], 
        "task_id": "e29c1188-892c-44d6-ba3a-2742beb40423", 
        "task_type": null, 
        "traceback": null, 
        "worker_name": null
    }
]

I dug into the code for a bit and determined that the get handler was passing the "filters" parameter as a string instead of as a dictionary to the Criteria object, which is not valid. It is the Criteria object that raises the error message "Invalid properties: ['filters']". The fix is a one liner! All we need to do is to insert this into the SearchView's get() handler before it hands the query to the Criteria.from_client_input() method:

query['filters'] = json.loads(query['filters'])

I considered fixing this along with my conversion to Django, but in the interest of making a pure conversion, I've decided to handle this with a bug report and separate pull request instead.

In my opinion, this is important to fix. It is not very RESTful to search via a POST method, but currently it is impossible to search via a GET method, which is the intended method for read operations.

[0] http://pulp.readthedocs.org/en/latest/dev-guide/conventions/criteria.html#search-api

Related issues

Related to Pulp - Issue #1040: Search with Get does not workCLOSED - CURRENTRELEASEamacdona@redhat.comActions
Blocks Pulp - Issue #1332: in Search API, restritcting fields does not work with MongoengineCLOSED - CURRENTRELEASEamacdona@redhat.comActions
Actions
Copy link
 #1

Updated by bmbouter about 9 years ago

  • Triaged changed from No to Yes
Actions
Copy link
 #2

Updated by rbarlow about 9 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to rbarlow
Actions
Copy link
 #3

Updated by bmbouter about 9 years ago

  • Severity changed from High to 3. High
Actions
Copy link
 #4

Updated by amacdona@redhat.com about 9 years ago

  • Status changed from ASSIGNED to POST
  • Assignee changed from rbarlow to amacdona@redhat.com

Added by Austin Macdonald about 9 years ago

Revision 8297b6fb | View on GitHub

fix filters field so GET searches work

closes #312

Added by Austin Macdonald about 9 years ago

Revision 8297b6fb | View on GitHub

fix filters field so GET searches work

closes #312

Actions
Copy link
 #5

Updated by amacdona@redhat.com about 9 years ago

  • Status changed from POST to MODIFIED

Switched PR's to merge to 2.7-dev rather than master.

https://github.com/pulp/pulp/pull/1790

Actions
Copy link
 #6

Updated by dkliban@redhat.com almost 9 years ago

  • Platform Release set to 2.7.0
Actions
Copy link
 #7

Updated by rbarlow almost 9 years ago

  • Has duplicate Issue #1040: Search with Get does not work added
Actions
Copy link
 #8

Updated by rbarlow almost 9 years ago

  • Has duplicate deleted (Issue #1040: Search with Get does not work)
Actions
Copy link
 #9

Updated by rbarlow almost 9 years ago

  • Related to Issue #1040: Search with Get does not work added
Actions
Copy link
 #10

Updated by dkliban@redhat.com almost 9 years ago

  • Status changed from MODIFIED to 5
Actions
Copy link
 #11

Updated by igulina@redhat.com almost 9 years ago

  • Status changed from 5 to ASSIGNED
>> rpm -qa pulp-server
pulp-server-2.7.0-0.4.beta.el7.noarch

Operations:  publish
Resources:   zoopark (repository)
State:       Successful
Start Time:  2015-07-08T11:53:38Z
Finish Time: 2015-07-08T11:53:39Z
Task Id:     c9b1f1b3-0293-4470-bbc9-293a3965a901

>> http --json -a admin:admin --verify=no GET https://localhost/pulp/api/v2/tasks/search/?filters=\{\"task_id\"\:\"c9b1f1b3-0293-4470-bbc9-293a3965a901\"\}\&field=task_id WebFrameworkSwitch:webpy
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
HTTP/1.1 400 BAD REQUEST
Connection: close
Content-Encoding: utf-8
Content-Length: 458
Content-Type: application/json
Date: Wed, 08 Jul 2015 11:54:59 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

{
    "_href": "/pulp/api/v2/tasks/search/?filters=%7B%22task_id%22:%22c9b1f1b3-0293-4470-bbc9-293a3965a901%22%7D&field=task_id", 
    "error": {
        "code": "PLP0015", 
        "data": {
            "properties": "['fields']", 
            "property_names": [
                "fields"
            ]
        }, 
        "description": "Invalid properties: ['fields']", 
        "sub_errors": []
    }, 
    "error_message": "Invalid properties: ['fields']", 
    "exception": null, 
    "http_request_method": "GET", 
    "http_status": 400, 
    "property_names": [
        "fields"
    ], 
    "traceback": null
}

>>  http --json -a admin:admin --verify=no GET https://localhost/pulp/api/v2/tasks/search/?filters=\{\"state\"\:\"running\"\}\&field=task_id\&limit=1 WebFrameworkSwitch:webpy
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)
HTTP/1.1 400 BAD REQUEST
Connection: close
Content-Encoding: utf-8
Content-Length: 435
Content-Type: application/json
Date: Wed, 08 Jul 2015 11:51:40 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

{
    "_href": "/pulp/api/v2/tasks/search/?filters=%7B%22state%22:%22running%22%7D&field=task_id&limit=1", 
    "error": {
        "code": "PLP0015", 
        "data": {
            "properties": "['fields']", 
            "property_names": [
                "fields"
            ]
        }, 
        "description": "Invalid properties: ['fields']", 
        "sub_errors": []
    }, 
    "error_message": "Invalid properties: ['fields']", 
    "exception": null, 
    "http_request_method": "GET", 
    "http_status": 400, 
    "property_names": [
        "fields"
    ], 
    "traceback": null
}

The same with curl:

curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/repositories/search/?field=id&field=display_name&limit=20'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   417  100   417    0     0    812      0 --:--:-- --:--:-- --:--:--   812
{
    "_href": "/pulp/api/v2/repositories/search/?field=id&field=display_name&limit=20",
    "error": {
        "code": "PLP0015",
        "data": {
            "properties": "['fields']",
            "property_names": [
                "fields"
            ]
        },
        "description": "Invalid properties: ['fields']",
        "sub_errors": []
    },
    "error_message": "Invalid properties: ['fields']",
    "exception": null,
    "http_request_method": "GET",
    "http_status": 400,
    "property_names": [
        "fields"
    ],
    "traceback": null
}
Actions
Copy link
 #12

Updated by amacdona@redhat.com almost 9 years ago

  • Status changed from ASSIGNED to POST

Good catch @igulina

https://github.com/pulp/pulp/pull/1975

The original fix for this corrected how the 'filters' field is parsed, but the same thing needed to be done for filters, fields, and sort.

Actions
Copy link
 #13

Updated by amacdona@redhat.com over 8 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #14

Updated by dkliban@redhat.com over 8 years ago

  • Status changed from MODIFIED to 5
Actions
Copy link
 #15

Updated by pthomas@redhat.com over 8 years ago

  • Status changed from 5 to ASSIGNED

This seem to be broken in 2.7

[root@ibm-x3550m3-06 ~]# rpm -qa pulp-server
pulp-server-2.7.0-0.7.beta.el7.noarch
[root@ibm-x3550m3-06 ~]# 

root@ibm-x3550m3-06 ~]# 
[root@ibm-x3550m3-06 ~]# http --json -a admin:admin --verify=no GET https://localhost/pulp/api/v2/tasks/search/?filters=\{\"state\"\:\"running\"\}\&field=task_id\&limit=1 WebFrameworkSwitch:webpy
/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
HTTP/1.1 400 BAD REQUEST
Connection: close
Content-Length: 435
Content-Type: application/json; charset=utf-8
Date: Wed, 21 Oct 2015 14:43:27 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

{
    "_href": "/pulp/api/v2/tasks/search/?filters=%7B%22state%22:%22running%22%7D&field=task_id&limit=1", 
    "error": {
        "code": "PLP0015", 
        "data": {
            "properties": "['fields']", 
            "property_names": [
                "fields"
            ]
        }, 
        "description": "Invalid properties: ['fields']", 
        "sub_errors": []
    }, 
    "error_message": "Invalid properties: ['fields']", 
    "exception": null, 
    "http_request_method": "GET", 
    "http_status": 400, 
    "property_names": [
        "fields"
    ], 
    "traceback": null
}

[root@ibm-x3550m3-06 ~]# 
[root@ibm-x3550m3-06 ~]# curl  -H "Accept: application/json" -X GET -k -u admin:admin 'https://localhost/pulp/api/v2/repositories/search/?field=id&field=display_name&limit=20'| python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   417  100   417    0     0   1012      0 --:--:-- --:--:-- --:--:--  1014
{
    "_href": "/pulp/api/v2/repositories/search/?field=id&field=display_name&limit=20",
    "error": {
        "code": "PLP0015",
        "data": {
            "properties": "['fields']",
            "property_names": [
                "fields"
            ]
        },
        "description": "Invalid properties: ['fields']",
        "sub_errors": []
    },
    "error_message": "Invalid properties: ['fields']",
    "exception": null,
    "http_request_method": "GET",
    "http_status": 400,
    "property_names": [
        "fields"
    ],
    "traceback": null
}
Actions
Copy link
 #16

Updated by amacdona@redhat.com over 8 years ago

  • Status changed from ASSIGNED to POST

Added by Austin Macdonald over 8 years ago

Revision f25ebc96 | View on GitHub

refactor the search class to simplify use of GET and POST searches

closes: #312

Added by Austin Macdonald over 8 years ago

Revision f25ebc96 | View on GitHub

refactor the search class to simplify use of GET and POST searches

closes: #312

Actions
Copy link
 #17

Updated by amacdona@redhat.com over 8 years ago

  • Blocks Issue #1332: in Search API, restritcting fields does not work with Mongoengine added
Actions
Copy link
 #18

Updated by amacdona@redhat.com over 8 years ago

  • Status changed from POST to MODIFIED
  • Platform Release changed from 2.7.0 to 2.7.1
Actions
Copy link
 #19

Updated by amacdona@redhat.com over 8 years ago

  • Platform Release changed from 2.7.1 to 2.7.2
Actions
Copy link
 #20

Updated by dkliban@redhat.com about 8 years ago

  • Status changed from MODIFIED to 5
Actions
Copy link
 #21

Updated by dkliban@redhat.com about 8 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions
Copy link
 #22

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added
Actions
Copy link
 #23

Updated by bmbouter almost 4 years ago

  • Category deleted (14)

We are removing the 'API' category per open floor discussion June 16, 2020.

Actions
Send by e-mail Copy link

Also available in: Atom PDF