Issue #1527: package group/category/distribution search not working - Pulp

Actions

Send by e-mail Copy link

Issue #1527

closed

package group/category/distribution search not working

Added by paji@redhat.com almost 9 years ago. Updated over 4 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

High

Assignee:

semyers

Category:

Sprint/Milestone:

Start date:

Due date:

Estimated time:

Severity:

3. High

Version:

Master

Platform Release:

2.8.0

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Pulp 2

Sprint:

Quarter:

Description

I can reproduce this both on API and via pulp-admin

1) Get the zoo5 repo from -> https://github.com/Katello/runcible/tree/master/test/fixtures/repositories/zoo5 (we just need the repo.) Steps below will assume file:///var/www/repositories/zoo5 is the destination
2)

$ pulp-admin rpm repo create --repo-id=ideate --feed=file:///var/www/repositories/zoo5
$ pulp-admin rpm repo sync run --repo-id=ideate
$ pulp-admin rpm repo content group --repo-id=ideate

Conditional Package Names: 
Default Package Names:     
Id:                        mammal
......
Conditional Package Names: 
Default Package Names:     
Id:                        bird
......

$ pulp-admin rpm repo content group --repo-id=ideate --fields=id
<BLANK>
$ pulp-admin -vv rpm repo content group --repo-id=ideate --in='id=bird,mammal'
<BLANK>

Above 2 commands are blanks for different reasons. 
--fields=id  => I am assuming is a client side thing since -vv tells me 2 responses with no "id" column
--in='id=bird,mammal' => Is a server side thing since -vv should [] as server response.

Related issues

Actions

Copy link

Updated by paji@redhat.com almost 9 years ago

Copied to Issue #1534: Errata search not working added

Actions

Copy link

Updated by paji@redhat.com almost 9 years ago

Subject changed from package group search not working to package group/category/distribution search not working

Actions

Copy link

Updated by mhrivnak almost 9 years ago

Priority changed from Normal to High
Severity changed from 2. Medium to 3. High
Platform Release set to 2.8.0
Triaged changed from No to Yes

Actions

Copy link

Updated by mhrivnak almost 9 years ago

Related to Issue #1533: package group/category unassociate not working added

Actions

Copy link

Updated by semyers almost 9 years ago

Status changed from NEW to ASSIGNED
Assignee set to semyers

Actions

Copy link

Updated by semyers almost 9 years ago

I assumed this was a serializer issue, but it isn't, really. The serializer is working alright, but the search view is (reasonably) filtering out all the fields that aren't requested, and isn't aware that when the user asks for the id field, one of the fields we really want in that response is the unit_id field. We need to effectively unserialize the search request...

/me gets back to hacking

Actions

Copy link

Updated by semyers almost 9 years ago

Status changed from ASSIGNED to POST

https://github.com/pulp/pulp/pull/2341

Added by semyers almost 9 years ago

Revision 8d462ca8 | View on GitHub

unit association query uses model serializers

When building mongo queries, UnitAssociationQuery wasn't translating the incoming fields to their proper mongo fields via the model serializer.

To start, this change makes "public" the serializer translate methods that provide useful functionality when incoming criteria get broken down into their actual mongo queries.

Then, those public methods are used by UnitAssociationQuery to translate model fields before building queries to mongo.

https://pulp.plan.io/issues/1527 fixes #1527

https://pulp.plan.io/issues/1533 re #1533

https://pulp.plan.io/issues/1534 fixes #1534

https://pulp.plan.io/issues/1535 re #1535

Added by semyers almost 9 years ago

Revision 8d462ca8 | View on GitHub

unit association query uses model serializers

When building mongo queries, UnitAssociationQuery wasn't translating the incoming fields to their proper mongo fields via the model serializer.

To start, this change makes "public" the serializer translate methods that provide useful functionality when incoming criteria get broken down into their actual mongo queries.

Then, those public methods are used by UnitAssociationQuery to translate model fields before building queries to mongo.

https://pulp.plan.io/issues/1527 fixes #1527

https://pulp.plan.io/issues/1533 re #1533

https://pulp.plan.io/issues/1534 fixes #1534

https://pulp.plan.io/issues/1535 re #1535

Actions

Copy link

Updated by semyers almost 9 years ago

Status changed from POST to MODIFIED
% Done changed from 0 to 100

Applied in changeset pulp|8d462ca81bb1c39d8403a3fe00013628c97b8b6c.

Actions

Copy link

Updated by dkliban@redhat.com over 8 years ago

Status changed from MODIFIED to 5

Actions

Copy link

#10

Updated by placko over 8 years ago

Status changed from 5 to 6

$ pulp-admin status
+----------------------------------------------------------------------+
                          Status of the server
+----------------------------------------------------------------------+

Api Version:           2
Database Connection:
  Connected: True
Known Workers:
  _id:            scheduler@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:24Z
  _id:            reserved_resource_worker-0@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:56Z
  _id:            reserved_resource_worker-1@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:58Z
  _id:            resource_manager@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:58Z
Messaging Connection:
  Connected: True
Versions:
  Platform Version: 2.8.0b6

$ pulp-admin rpm repo create --repo-id=ideate --feed=file:///var/www/repositories/zoo5
Successfully created repository [ideate]

$ pulp-admin rpm repo sync run --repo-id=ideate
+----------------------------------------------------------------------+
                   Synchronizing Repository [ideate]
+----------------------------------------------------------------------+

===================================== Snip ====================================
===============================================================================

Task Succeeded

$ pulp-admin rpm repo content group --repo-id=ideate
Conditional Package Names:
Default Package Names:
Id:                        bird
Mandatory Package Names:   penguin
Name:                      bird
Optional Package Names:
User Visible:              True

Conditional Package Names:
Default Package Names:
Id:                        mammal
Mandatory Package Names:   elephant,giraffe,cheetah,lion,monkey,penguin,squirrel
                           ,walrus, penguin
Name:                      mammal
Optional Package Names:
User Visible:              True

$ pulp-admin rpm repo content group --repo-id=ideate --fields=id
Id:                        bird

Id:                        mammal

$ pulp-admin -vv rpm repo content group --repo-id=ideate --fields=id
2016-03-03 07:44:15,705 - DEBUG - sending POST request to /pulp/api/v2/repositories/ideate/search/units/
2016-03-03 07:44:15,786 - INFO - POST request to /pulp/api/v2/repositories/ideate/search/units/ with parameters {"criteria": {"fields": {"unit": ["id"]}, "type_ids": ["package_group"], "filters": {"unit": {}}}}
2016-03-03 07:44:15,786 - INFO - Response status : 200

2016-03-03 07:44:15,787 - INFO - Response body :
 [
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf0"
    },
    "metadata": {
      "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
      "id": "bird",
      "_content_type_id": "package_group"
    }
  },
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf1"
    },
    "metadata": {
      "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
      "id": "mammal",
      "_content_type_id": "package_group"
    }
  }
]

Id:                        bird

Id:                        mammal

$ pulp-admin -vv rpm repo content group --repo-id=ideate --in='id=bird,mammal'
2016-03-03 07:44:23,111 - DEBUG - sending POST request to /pulp/api/v2/repositories/ideate/search/units/
2016-03-03 07:44:23,223 - INFO - POST request to /pulp/api/v2/repositories/ideate/search/units/ with parameters {"criteria": {"type_ids": ["package_group"], "filters": {"unit": {"id": {"$in": ["bird", "mammal"]}}}}}
2016-03-03 07:44:23,223 - INFO - Response status : 200

2016-03-03 07:44:23,223 - INFO - Response body :
 [
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf0"
    },
    "metadata": {
      "mandatory_package_names": [
        "penguin"
      ],
      "repo_id": "ideate",
      "name": "bird",
      "default": true,
      "_ns": "units_package_group",
      "_last_updated": 1457008996,
      "display_order": 1024,
      "user_visible": true,
      "translated_name": {},
      "conditional_package_names": [],
      "translated_description": {},
      "optional_package_names": [],
      "default_package_names": [],
      "_content_type_id": "package_group",
      "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
      "id": "bird",
      "pulp_user_metadata": {}
    }
  },
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf1"
    },
    "metadata": {
      "mandatory_package_names": [
        "elephant,giraffe,cheetah,lion,monkey,penguin,squirrel,walrus",
        "penguin"
      ],
      "repo_id": "ideate",
      "name": "mammal",
      "default": true,
      "_ns": "units_package_group",
      "_last_updated": 1457008996,
      "display_order": 1024,
      "user_visible": true,
      "translated_name": {},
      "conditional_package_names": [],
      "translated_description": {},
      "optional_package_names": [],
      "default_package_names": [],
      "_content_type_id": "package_group",
      "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
      "id": "mammal",
      "pulp_user_metadata": {}
    }
  }
]

Conditional Package Names:
Default Package Names:
Id:                        bird
Mandatory Package Names:   penguin
Name:                      bird
Optional Package Names:
User Visible:              True

Conditional Package Names:
Default Package Names:
Id:                        mammal
Mandatory Package Names:   elephant,giraffe,cheetah,lion,monkey,penguin,squirrel
                           ,walrus, penguin
Name:                      mammal
Optional Package Names:
User Visible:              True

$ http POST https://host/pulp/api/v2/repositories/ideate/search/units/ --verify=$VIRTUAL_ENV/.config/pulp_smash/pulp-cert.crt -vvv --auth=admin:admin <<< '{"criteria": {"fields": {"unit": ["id"]}, "type_ids": ["package_group"], "filters": {"unit": {}}}}'
POST /pulp/api/v2/repositories/ideate/search/units/ HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Authorization: Basic YWRtaW46YWRtaW4=
Connection: keep-alive
Content-Length: 99
Content-Type: application/json
Host: host
User-Agent: HTTPie/0.9.2

{
    "criteria": {
        "fields": {
            "unit": [
                "id"
            ]
        },
        "filters": {
            "unit": {}
        },
        "type_ids": [
            "package_group"
        ]
    }
}

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 670
Content-Type: application/json; charset=utf-8
Date: Thu, 03 Mar 2016 12:48:01 GMT
Keep-Alive: timeout=5, max=10000
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

[
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf0"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
            "id": "bird"
        },
        "repo_id": "ideate",
        "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    },
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf1"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
            "id": "mammal"
        },
        "repo_id": "ideate",
        "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    }
]

$ http POST https://host/pulp/api/v2/repositories/ideate/search/units/ --verify=cert.crt -vvv --auth=admin:admin <<< '{"criteria": {"type_ids": ["package_group"], "filters": {"unit": {"id": {"$in": ["bird", "mammal"]}}}}}'
POST /pulp/api/v2/repositories/ideate/search/units/ HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Authorization: Basic YWRtaW46YWRtaW4=
Connection: keep-alive
Content-Length: 104
Content-Type: application/json
Host: host
User-Agent: HTTPie/0.9.2

{
    "criteria": {
        "filters": {
            "unit": {
                "id": {
                    "$in": [
                        "bird",
                        "mammal"
                    ]
                }
            }
        },
        "type_ids": [
            "package_group"
        ]
    }
}

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 1474
Content-Type: application/json; charset=utf-8
Date: Thu, 03 Mar 2016 12:49:35 GMT
Keep-Alive: timeout=5, max=10000
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

[
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf0"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
            "_last_updated": 1457008996,
            "_ns": "units_package_group",
            "conditional_package_names": [],
            "default": true,
            "default_package_names": [],
            "display_order": 1024,
            "id": "bird",
            "mandatory_package_names": [
                "penguin"
            ],
            "name": "bird",
            "optional_package_names": [],
            "pulp_user_metadata": {},
            "repo_id": "ideate",
            "translated_description": {},
            "translated_name": {},
            "user_visible": true
        },
        "repo_id": "ideate",
        "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    },
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf1"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
            "_last_updated": 1457008996,
            "_ns": "units_package_group",
            "conditional_package_names": [],
            "default": true,
            "default_package_names": [],
            "display_order": 1024,
            "id": "mammal",
            "mandatory_package_names": [
                "elephant,giraffe,cheetah,lion,monkey,penguin,squirrel,walrus",
                "penguin"
            ],
            "name": "mammal",
            "optional_package_names": [],
            "pulp_user_metadata": {},
            "repo_id": "ideate",
            "translated_description": {},
            "translated_name": {},
            "user_visible": true
        },
        "repo_id": "ideate",
        "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    }
]

Actions

Copy link

#11