Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #1527

closed

package group/category/distribution search not working

Added by paji@redhat.com over 8 years ago. Updated almost 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
semyers
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Master
Platform Release:
2.8.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

I can reproduce this both on API and via pulp-admin

1) Get the zoo5 repo from -> https://github.com/Katello/runcible/tree/master/test/fixtures/repositories/zoo5 (we just need the repo.) Steps below will assume file:///var/www/repositories/zoo5 is the destination
2)

$ pulp-admin rpm repo create --repo-id=ideate --feed=file:///var/www/repositories/zoo5
$ pulp-admin rpm repo sync run --repo-id=ideate
$ pulp-admin rpm repo content group --repo-id=ideate

Conditional Package Names: 
Default Package Names:     
Id:                        mammal
......
Conditional Package Names: 
Default Package Names:     
Id:                        bird
......

$ pulp-admin rpm repo content group --repo-id=ideate --fields=id
<BLANK>
$ pulp-admin -vv rpm repo content group --repo-id=ideate --in='id=bird,mammal'
<BLANK>

Above 2 commands are blanks for different reasons. 
--fields=id  => I am assuming is a client side thing since -vv tells me 2 responses with no "id" column
--in='id=bird,mammal' => Is a server side thing since -vv should [] as server response.

Related issues

Related to Pulp - Issue #1533: package group/category unassociate not workingCLOSED - CURRENTRELEASEsemyersActions
Copied to Pulp - Issue #1534: Errata search not workingCLOSED - CURRENTRELEASEsemyersActions
Actions
Copy link
 #1

Updated by paji@redhat.com over 8 years ago

  • Copied to Issue #1534: Errata search not working added
Actions
Copy link
 #2

Updated by paji@redhat.com over 8 years ago

  • Subject changed from package group search not working to package group/category/distribution search not working
Actions
Copy link
 #3

Updated by mhrivnak about 8 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Platform Release set to 2.8.0
  • Triaged changed from No to Yes
Actions
Copy link
 #4

Updated by mhrivnak about 8 years ago

  • Related to Issue #1533: package group/category unassociate not working added
Actions
Copy link
 #5

Updated by semyers about 8 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to semyers
Actions
Copy link
 #6

Updated by semyers about 8 years ago

I assumed this was a serializer issue, but it isn't, really. The serializer is working alright, but the search view is (reasonably) filtering out all the fields that aren't requested, and isn't aware that when the user asks for the id field, one of the fields we really want in that response is the unit_id field. We need to effectively unserialize the search request...

/me gets back to hacking

Actions
Copy link
 #7

Updated by semyers about 8 years ago

  • Status changed from ASSIGNED to POST

Added by semyers about 8 years ago

Revision 8d462ca8 | View on GitHub

unit association query uses model serializers

When building mongo queries, UnitAssociationQuery wasn't translating the incoming fields to their proper mongo fields via the model serializer.

To start, this change makes "public" the serializer translate methods that provide useful functionality when incoming criteria get broken down into their actual mongo queries.

Then, those public methods are used by UnitAssociationQuery to translate model fields before building queries to mongo.

https://pulp.plan.io/issues/1527 fixes #1527

https://pulp.plan.io/issues/1533 re #1533

https://pulp.plan.io/issues/1534 fixes #1534

https://pulp.plan.io/issues/1535 re #1535

Added by semyers about 8 years ago

Revision 8d462ca8 | View on GitHub

unit association query uses model serializers

When building mongo queries, UnitAssociationQuery wasn't translating the incoming fields to their proper mongo fields via the model serializer.

To start, this change makes "public" the serializer translate methods that provide useful functionality when incoming criteria get broken down into their actual mongo queries.

Then, those public methods are used by UnitAssociationQuery to translate model fields before building queries to mongo.

https://pulp.plan.io/issues/1527 fixes #1527

https://pulp.plan.io/issues/1533 re #1533

https://pulp.plan.io/issues/1534 fixes #1534

https://pulp.plan.io/issues/1535 re #1535

Actions
Copy link
 #8

Updated by semyers about 8 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions
Copy link
 #9

Updated by dkliban@redhat.com about 8 years ago

  • Status changed from MODIFIED to 5
Actions
Copy link
 #10

Updated by placko about 8 years ago

  • Status changed from 5 to 6
$ pulp-admin status
+----------------------------------------------------------------------+
                          Status of the server
+----------------------------------------------------------------------+

Api Version:           2
Database Connection:
  Connected: True
Known Workers:
  _id:            scheduler@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:24Z
  _id:            reserved_resource_worker-0@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:56Z
  _id:            reserved_resource_worker-1@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:58Z
  _id:            resource_manager@host.internal
  _ns:            workers
  Last Heartbeat: 2016-03-03T10:34:58Z
Messaging Connection:
  Connected: True
Versions:
  Platform Version: 2.8.0b6

$ pulp-admin rpm repo create --repo-id=ideate --feed=file:///var/www/repositories/zoo5
Successfully created repository [ideate]

$ pulp-admin rpm repo sync run --repo-id=ideate
+----------------------------------------------------------------------+
                   Synchronizing Repository [ideate]
+----------------------------------------------------------------------+

===================================== Snip ====================================
===============================================================================

Task Succeeded

$ pulp-admin rpm repo content group --repo-id=ideate
Conditional Package Names:
Default Package Names:
Id:                        bird
Mandatory Package Names:   penguin
Name:                      bird
Optional Package Names:
User Visible:              True

Conditional Package Names:
Default Package Names:
Id:                        mammal
Mandatory Package Names:   elephant,giraffe,cheetah,lion,monkey,penguin,squirrel
                           ,walrus, penguin
Name:                      mammal
Optional Package Names:
User Visible:              True

$ pulp-admin rpm repo content group --repo-id=ideate --fields=id
Id:                        bird

Id:                        mammal

$ pulp-admin -vv rpm repo content group --repo-id=ideate --fields=id
2016-03-03 07:44:15,705 - DEBUG - sending POST request to /pulp/api/v2/repositories/ideate/search/units/
2016-03-03 07:44:15,786 - INFO - POST request to /pulp/api/v2/repositories/ideate/search/units/ with parameters {"criteria": {"fields": {"unit": ["id"]}, "type_ids": ["package_group"], "filters": {"unit": {}}}}
2016-03-03 07:44:15,786 - INFO - Response status : 200

2016-03-03 07:44:15,787 - INFO - Response body :
 [
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf0"
    },
    "metadata": {
      "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
      "id": "bird",
      "_content_type_id": "package_group"
    }
  },
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf1"
    },
    "metadata": {
      "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
      "id": "mammal",
      "_content_type_id": "package_group"
    }
  }
]

Id:                        bird

Id:                        mammal

$ pulp-admin -vv rpm repo content group --repo-id=ideate --in='id=bird,mammal'
2016-03-03 07:44:23,111 - DEBUG - sending POST request to /pulp/api/v2/repositories/ideate/search/units/
2016-03-03 07:44:23,223 - INFO - POST request to /pulp/api/v2/repositories/ideate/search/units/ with parameters {"criteria": {"type_ids": ["package_group"], "filters": {"unit": {"id": {"$in": ["bird", "mammal"]}}}}}
2016-03-03 07:44:23,223 - INFO - Response status : 200

2016-03-03 07:44:23,223 - INFO - Response body :
 [
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf0"
    },
    "metadata": {
      "mandatory_package_names": [
        "penguin"
      ],
      "repo_id": "ideate",
      "name": "bird",
      "default": true,
      "_ns": "units_package_group",
      "_last_updated": 1457008996,
      "display_order": 1024,
      "user_visible": true,
      "translated_name": {},
      "conditional_package_names": [],
      "translated_description": {},
      "optional_package_names": [],
      "default_package_names": [],
      "_content_type_id": "package_group",
      "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
      "id": "bird",
      "pulp_user_metadata": {}
    }
  },
  {
    "updated": "2016-03-03T12:43:16Z",
    "repo_id": "ideate",
    "created": "2016-03-03T12:43:16Z",
    "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
    "unit_type_id": "package_group",
    "_id": {
      "$oid": "56d83164cbba263a6aedbbf1"
    },
    "metadata": {
      "mandatory_package_names": [
        "elephant,giraffe,cheetah,lion,monkey,penguin,squirrel,walrus",
        "penguin"
      ],
      "repo_id": "ideate",
      "name": "mammal",
      "default": true,
      "_ns": "units_package_group",
      "_last_updated": 1457008996,
      "display_order": 1024,
      "user_visible": true,
      "translated_name": {},
      "conditional_package_names": [],
      "translated_description": {},
      "optional_package_names": [],
      "default_package_names": [],
      "_content_type_id": "package_group",
      "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
      "id": "mammal",
      "pulp_user_metadata": {}
    }
  }
]

Conditional Package Names:
Default Package Names:
Id:                        bird
Mandatory Package Names:   penguin
Name:                      bird
Optional Package Names:
User Visible:              True

Conditional Package Names:
Default Package Names:
Id:                        mammal
Mandatory Package Names:   elephant,giraffe,cheetah,lion,monkey,penguin,squirrel
                           ,walrus, penguin
Name:                      mammal
Optional Package Names:
User Visible:              True

$ http POST https://host/pulp/api/v2/repositories/ideate/search/units/ --verify=$VIRTUAL_ENV/.config/pulp_smash/pulp-cert.crt -vvv --auth=admin:admin <<< '{"criteria": {"fields": {"unit": ["id"]}, "type_ids": ["package_group"], "filters": {"unit": {}}}}'
POST /pulp/api/v2/repositories/ideate/search/units/ HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Authorization: Basic YWRtaW46YWRtaW4=
Connection: keep-alive
Content-Length: 99
Content-Type: application/json
Host: host
User-Agent: HTTPie/0.9.2

{
    "criteria": {
        "fields": {
            "unit": [
                "id"
            ]
        },
        "filters": {
            "unit": {}
        },
        "type_ids": [
            "package_group"
        ]
    }
}

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 670
Content-Type: application/json; charset=utf-8
Date: Thu, 03 Mar 2016 12:48:01 GMT
Keep-Alive: timeout=5, max=10000
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

[
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf0"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
            "id": "bird"
        },
        "repo_id": "ideate",
        "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    },
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf1"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
            "id": "mammal"
        },
        "repo_id": "ideate",
        "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    }
]

$ http POST https://host/pulp/api/v2/repositories/ideate/search/units/ --verify=cert.crt -vvv --auth=admin:admin <<< '{"criteria": {"type_ids": ["package_group"], "filters": {"unit": {"id": {"$in": ["bird", "mammal"]}}}}}'
POST /pulp/api/v2/repositories/ideate/search/units/ HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Authorization: Basic YWRtaW46YWRtaW4=
Connection: keep-alive
Content-Length: 104
Content-Type: application/json
Host: host
User-Agent: HTTPie/0.9.2

{
    "criteria": {
        "filters": {
            "unit": {
                "id": {
                    "$in": [
                        "bird",
                        "mammal"
                    ]
                }
            }
        },
        "type_ids": [
            "package_group"
        ]
    }
}

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 1474
Content-Type: application/json; charset=utf-8
Date: Thu, 03 Mar 2016 12:49:35 GMT
Keep-Alive: timeout=5, max=10000
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5

[
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf0"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "8255ae11-898d-492f-91c3-ca8cda733068",
            "_last_updated": 1457008996,
            "_ns": "units_package_group",
            "conditional_package_names": [],
            "default": true,
            "default_package_names": [],
            "display_order": 1024,
            "id": "bird",
            "mandatory_package_names": [
                "penguin"
            ],
            "name": "bird",
            "optional_package_names": [],
            "pulp_user_metadata": {},
            "repo_id": "ideate",
            "translated_description": {},
            "translated_name": {},
            "user_visible": true
        },
        "repo_id": "ideate",
        "unit_id": "8255ae11-898d-492f-91c3-ca8cda733068",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    },
    {
        "_id": {
            "$oid": "56d83164cbba263a6aedbbf1"
        },
        "created": "2016-03-03T12:43:16Z",
        "metadata": {
            "_content_type_id": "package_group",
            "_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
            "_last_updated": 1457008996,
            "_ns": "units_package_group",
            "conditional_package_names": [],
            "default": true,
            "default_package_names": [],
            "display_order": 1024,
            "id": "mammal",
            "mandatory_package_names": [
                "elephant,giraffe,cheetah,lion,monkey,penguin,squirrel,walrus",
                "penguin"
            ],
            "name": "mammal",
            "optional_package_names": [],
            "pulp_user_metadata": {},
            "repo_id": "ideate",
            "translated_description": {},
            "translated_name": {},
            "user_visible": true
        },
        "repo_id": "ideate",
        "unit_id": "e7b6cbec-1013-413f-aee7-fb8e017d6411",
        "unit_type_id": "package_group",
        "updated": "2016-03-03T12:43:16Z"
    }
]
Actions
Copy link
 #11

Updated by dkliban@redhat.com about 8 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE
Actions
Copy link
 #13

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added
Actions
Copy link
 #14

Updated by bmbouter almost 4 years ago

  • Category deleted (14)

We are removing the 'API' category per open floor discussion June 16, 2020.

Actions
Send by e-mail Copy link

Also available in: Atom PDF