Project

Profile

Help

Issue #1527

package group/category/distribution search not working

Added by paji@redhat.com almost 5 years ago. Updated 4 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Master
Platform Release:
2.8.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

I can reproduce this both on API and via pulp-admin

1) Get the zoo5 repo from -> https://github.com/Katello/runcible/tree/master/test/fixtures/repositories/zoo5 (we just need the repo.) Steps below will assume file:///var/www/repositories/zoo5 is the destination
2)

$ pulp-admin rpm repo create --repo-id=ideate --feed=file:///var/www/repositories/zoo5
$ pulp-admin rpm repo sync run --repo-id=ideate
$ pulp-admin rpm repo content group --repo-id=ideate

Conditional Package Names: 
Default Package Names:     
Id:                        mammal
......
Conditional Package Names: 
Default Package Names:     
Id:                        bird
......

$ pulp-admin rpm repo content group --repo-id=ideate --fields=id
<BLANK>
$ pulp-admin -vv rpm repo content group --repo-id=ideate --in='id=bird,mammal'
<BLANK>

Above 2 commands are blanks for different reasons. 
--fields=id  => I am assuming is a client side thing since -vv tells me 2 responses with no "id" column
--in='id=bird,mammal' => Is a server side thing since -vv should [] as server response.

Related issues

Related to Pulp - Issue #1533: package group/category unassociate not workingCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to Pulp - Issue #1534: Errata search not workingCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 8d462ca8 View on GitHub
Added by semyers almost 5 years ago

unit association query uses model serializers

When building mongo queries, UnitAssociationQuery wasn't translating the incoming fields to their proper mongo fields via the model serializer.

To start, this change makes "public" the serializer translate methods that provide useful functionality when incoming criteria get broken down into their actual mongo queries.

Then, those public methods are used by UnitAssociationQuery to translate model fields before building queries to mongo.

https://pulp.plan.io/issues/1527 fixes #1527

https://pulp.plan.io/issues/1533 re #1533

https://pulp.plan.io/issues/1534 fixes #1534

https://pulp.plan.io/issues/1535 re #1535

Revision 8d462ca8 View on GitHub
Added by semyers almost 5 years ago

unit association query uses model serializers

When building mongo queries, UnitAssociationQuery wasn't translating the incoming fields to their proper mongo fields via the model serializer.

To start, this change makes "public" the serializer translate methods that provide useful functionality when incoming criteria get broken down into their actual mongo queries.

Then, those public methods are used by UnitAssociationQuery to translate model fields before building queries to mongo.

https://pulp.plan.io/issues/1527 fixes #1527

https://pulp.plan.io/issues/1533 re #1533

https://pulp.plan.io/issues/1534 fixes #1534

https://pulp.plan.io/issues/1535 re #1535

History

#1 Updated by paji@redhat.com almost 5 years ago

  • Copied to Issue #1534: Errata search not working added

#2 Updated by paji@redhat.com almost 5 years ago

  • Subject changed from package group search not working to package group/category/distribution search not working

#3 Updated by mhrivnak almost 5 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Platform Release set to 2.8.0
  • Triaged changed from No to Yes

#4 Updated by mhrivnak almost 5 years ago

  • Related to Issue #1533: package group/category unassociate not working added

#5 Updated by semyers almost 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to semyers

#6 Updated by semyers almost 5 years ago

I assumed this was a serializer issue, but it isn't, really. The serializer is working alright, but the search view is (reasonably) filtering out all the fields that aren't requested, and isn't aware that when the user asks for the id field, one of the fields we really want in that response is the unit_id field. We need to effectively unserialize the search request...

/me gets back to hacking

#7 Updated by semyers almost 5 years ago

  • Status changed from ASSIGNED to POST

#8 Updated by semyers almost 5 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#9 Updated by dkliban@redhat.com over 4 years ago

  • Status changed from MODIFIED to 5

#10 Updated by placko over 4 years ago

  • Status changed from 5 to 6
$ pulp-admin status
+----------------------------------------------------------------------+
                          Status of the server
+----------------------------------------------------------------------+

Api Version:           2
Database Connection:
  Connected: True
Known Workers:
  _id:            scheduler@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:24Z
  _id:            reserved_resource_worker-0@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:56Z
  _id:            reserved_resource_worker-1@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:58Z
  _id:            resource_manager@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:58Z
Messaging Connection:
  Connected: True
Versions:
  Platform Version: 2.8.0b6

$ pulp-admin rpm repo create --repo-id=ideate --feed=file:///var/www/repositories/zoo5
Successfully created repository [ideate]

$ pulp-admin rpm repo sync run --repo-id=ideate
+----------------------------------------------------------------------+
                   Synchronizing Repository [ideate]
+----------------------------------------------------------------------+

===================================== Snip ====================================
===============================================================================

Task Succeeded

$ pulp-admin rpm repo content group --repo-id=ideate
Conditional Package Names:
Default Package Names:
Id:                        bird
Mandatory Package Names:   penguin
Name:                      bird
Optional Package Names:
User Visible:              True

Conditional Package Names:
Default Package Names:
Id:                        mammal
Mandatory Package Names:   elephant,giraffe,cheetah,lion,monkey,penguin,squirrel
                           ,walrus, penguin
Name:                      mammal
Optional Package Names:
User Visible:              True

$ pulp-admin rpm repo content group --repo-id=ideate --fields=id
Id:                        bird

Id:                        mammal

$ pulp-admin -vv rpm repo content group --repo-id=ideate --fields=id
2016-03-03 07:44:15,705 - DEBUG - sending POST request to /pulp/api/v2/repositories/ideate/search/units/
2016-03-03 07:44:15,786 - INFO - POST request to /pulp/api/v2/repositories/ideate/search/units/ with parameters {"criteria": {"fields": {"unit": ["id"]}, "type_ids": ["package_group"], "filters": {"unit": {}}}}
2016-03-03 07:44:15,786 - INFO - Response status : 200

2016-03-03 07:44:15,787 - INFO - Response body :
 [
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf0"
    },
    "metadata": {
      "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
      "id": "bird",
      "_content_type_id": "package_group"
    }
  },
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf1"
    },
    "metadata": {
      "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
      "id": "mammal",
      "_content_type_id": "package_group"
    }
  }
]

Id:                        bird

Id:                        mammal

$ pulp-admin -vv rpm repo content group --repo-id=ideate --in='id=bird,mammal'
2016-03-03 07:44:23,111 - DEBUG - sending POST request to /pulp/api/v2/repositories/ideate/search/units/
2016-03-03 07:44:23,223 - INFO - POST request to /pulp/api/v2/repositories/ideate/search/units/ with parameters {"criteria": {"type_ids": ["package_group"], "filters": {"unit": {"id": {"$in": ["bird", "mammal"]}}}}}
2016-03-03 07:44:23,223 - INFO - Response status : 200

2016-03-03 07:44:23,223 - INFO - Response body :
 [
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf0"
    },
    "metadata": {
      "mandatory_package_names": [
        "penguin"
      ],
      "repo_id": "ideate",
      "name": "bird",
      "default": true,
      "_ns": "units_package_group",
      "_last_updated": 1457008996,
      "display_order": 1024,
      "user_visible": true,
      "translated_name": {},
      "conditional_package_names": [],
      "translated_description": {},
      "optional_package_names": [],
      "default_package_names": [],
      "_content_type_id": "package_group",
      "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
      "id": "bird",
      "pulp_user_metadata": {}
    }
  },
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf1"
    },
    "metadata": {
      "mandatory_package_names": [
        "elephant,giraffe,cheetah,lion,monkey,penguin,squirrel,walrus",
        "penguin"
      ],
      "repo_id": "ideate",
      "name": "mammal",
      "default": true,
      "_ns": "units_package_group",
      "_last_updated": 1457008996,
      "display_order": 1024,
      "user_visible": true,
      "translated_name": {},
      "conditional_package_names": [],
      "translated_description": {},
      "optional_package_names": [],
      "default_package_names": [],
      "_content_type_id": "package_group",
      "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
      "id": "mammal",
      "pulp_user_metadata": {}
    }
  }
]

Conditional Package Names:
Default Package Names:
Id:                        bird
Mandatory Package Names:   penguin
Name:                      bird
Optional Package Names:
User Visible:              True

Conditional Package Names:
Default Package Names:
Id:                        mammal
Mandatory Package Names:   elephant,giraffe,cheetah,lion,monkey,penguin,squirrel
                           ,walrus, penguin
Name:                      mammal
Optional Package Names:
User Visible:              True

$ http POST https://host/pulp/api/v2/repositories/ideate/search/units/ --verify=$VIRTUAL_ENV/.config/pulp_smash/pulp-cert.crt -vvv --auth=admin:admin <<< '{"criteria": {"fields": {"unit": ["id"]}, "type_ids": ["package_group"], "filters": {"unit": {}}}}'
POST /pulp/api/v2/repositories/ideate/search/units/ HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Authorization: Basic YWRtaW46YWRtaW4=
Connection: keep-alive
Content-Length: 99
Content-Type: application/json
Host: host
User-Agent: HTTPie/0.9.2

{
    "criteria": {
        "fields": {
            "unit": [
                "id"
            ]
        },
        "filters": {
            "unit": {}
        },
        "type_ids": [
            "package_group"
        ]
    }
}

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 670
Content-Type: application/json; charset=utf-8
Date: Thu, 03 Mar 2016 12:48:01 GMT
Keep-Alive: timeout=5, max=10000
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

[
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf0"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
            "id": "bird"
        },
        "repo_id": "ideate",
        "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    },
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf1"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
            "id": "mammal"
        },
        "repo_id": "ideate",
        "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    }
]

$ http POST https://host/pulp/api/v2/repositories/ideate/search/units/ --verify=cert.crt -vvv --auth=admin:admin <<< '{"criteria": {"type_ids": ["package_group"], "filters": {"unit": {"id": {"$in": ["bird", "mammal"]}}}}}'
POST /pulp/api/v2/repositories/ideate/search/units/ HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Authorization: Basic YWRtaW46YWRtaW4=
Connection: keep-alive
Content-Length: 104
Content-Type: application/json
Host: host
User-Agent: HTTPie/0.9.2

{
    "criteria": {
        "filters": {
            "unit": {
                "id": {
                    "$in": [
                        "bird",
                        "mammal"
                    ]
                }
            }
        },
        "type_ids": [
            "package_group"
        ]
    }
}

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 1474
Content-Type: application/json; charset=utf-8
Date: Thu, 03 Mar 2016 12:49:35 GMT
Keep-Alive: timeout=5, max=10000
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

[
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf0"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
            "_last_updated": 1457008996,
            "_ns": "units_package_group",
            "conditional_package_names": [],
            "default": true,
            "default_package_names": [],
            "display_order": 1024,
            "id": "bird",
            "mandatory_package_names": [
                "penguin"
            ],
            "name": "bird",
            "optional_package_names": [],
            "pulp_user_metadata": {},
            "repo_id": "ideate",
            "translated_description": {},
            "translated_name": {},
            "user_visible": true
        },
        "repo_id": "ideate",
        "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    },
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf1"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
            "_last_updated": 1457008996,
            "_ns": "units_package_group",
            "conditional_package_names": [],
            "default": true,
            "default_package_names": [],
            "display_order": 1024,
            "id": "mammal",
            "mandatory_package_names": [
                "elephant,giraffe,cheetah,lion,monkey,penguin,squirrel,walrus",
                "penguin"
            ],
            "name": "mammal",
            "optional_package_names": [],
            "pulp_user_metadata": {},
            "repo_id": "ideate",
            "translated_description": {},
            "translated_name": {},
            "user_visible": true
        },
        "repo_id": "ideate",
        "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    }
]

#11 Updated by dkliban@redhat.com over 4 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE

#13 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

#14 Updated by bmbouter 4 months ago

  • Category deleted (14)

We are removing the 'API' category per open floor discussion June 16, 2020.

Please register to edit this issue

Also available in: Atom PDF