Project

Profile

Help

Issue #1865

Error searching rpm repositories by fields using dot notation

Added by alanoe over 4 years ago. Updated over 1 year ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.8.2
Platform Release:
OS:
RHEL 7
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

when I run 'pulp-admin rpm repo search --fields notes.A' , I get the following error:
Invalid properties: ['Field: <notes.A> does not exist on objects in the <repos> collection']

I'm sure that the attribute A exists in notes dict.
This is confirmed by a find() command with the same query in the mongo shell:

$ use pulp_database
$ db.repos.find({}, {'notes.A': 1})

This pulp-admin command works with pulp/pulp-admin 2.5.3 and MongoDB 2.4.9 installed on RHEL 6.

History

#1 Updated by bmbouter over 4 years ago

I agree this is an issue; I've reproduced it. Here are some details:

In my vanilla Vagrant environment there are 4 repos. The rpm repo looks like this from db.repos.find()

{ 
        "_id" : ObjectId("57220e7ce779890a1bd0d311"),
        "repo_id" : "zoo",
        "notes" : {
                "_repo-type" : "rpm-repo"
        },
        "scratchpad" : {

        },
        "content_unit_counts" : {

        },
        "_ns" : "repos"
}

In there we should be able to search on the field notes._repo-type I'm able to also do that in my db:

> db.repos.find({}, {'notes._repo-type': 1})
{ "_id" : ObjectId("57220e7be779890a1c1e2bfb"), "notes" : { "_repo-type" : "docker-repo" } }
{ "_id" : ObjectId("57220e7ce779890a1d75f848"), "notes" : { "_repo-type" : "docker-repo" } }
{ "_id" : ObjectId("57220e7ce779890a1bd0d311"), "notes" : { "_repo-type" : "rpm-repo" } }
{ "_id" : ObjectId("57220e7de779890a1d75f84c"), "notes" : { "_repo-type" : "puppet-repo" } }
{ "_id" : ObjectId("57220e7de779890a1c1e2bff"), "notes" : { "_repo-type" : "PYTHON" } }

I then try to do the same with pulp-admin:

pulp-admin -vv rpm repo search --fields notes._repo-type

Which returns the error message:

Invalid properties: ['Field: <notes._repo-type> does not exist on objects in the
<repos> collection']

The above pulp-admin command is producing this API request:

POST request to /pulp/api/v2/repositories/search/ with parameters {"criteria": {"sort": null, "fields": ["notes._repo-type"], "limit": null, "filters": {"notes._repo-type": "rpm-repo"}, "skip": null}}

I can have httpie perform the same request:

http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/search/ criteria:='{"sort": null, "fields": ["notes._repo-type"], "limit": null, "filters": {"notes._repo-type": "rpm-repo"}, "skip": null}'

This simplified version also fails with the same error:

http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/search/ criteria:='{"fields": ["notes._repo-type"], "filters": {"notes._repo-type": "rpm-repo"}}'

Both of these work as expected indicating that the filters section is working correctly.

http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/search/ criteria:='{"fields": ["notes"], "filters": {"notes._repo-type": "rpm-repo"}}'
http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/search/ criteria:='{"filters": {"notes._repo-type": "rpm-repo"}}'

The bottom one corresponds with the pulp-admin command which also works as expected:

pulp-admin -vv rpm repo search --filters '{"notes._repo-type": "rpm-repo"}'

The above command produces the following API call:

POST request to /pulp/api/v2/repositories/search/ with parameters {"criteria": {"sort": null, "fields": null, "limit": null, "filters": {"notes._repo-type": "rpm-repo"}, "skip": null}}

#2 Updated by bmbouter over 4 years ago

I see two issues here.

1. The bug as reported is legitimate and is an error on the server side.

2. The specification of --fields causes --filters to also be set. This does not make sense to me. Perhaps that's expected? I would like other input on if this should be filed as a separate pulp bug against pulp-admin in pulp_rpm?

#3 Updated by amacdona@redhat.com over 4 years ago

The offending line is https://github.com/pulp/pulp/blob/6d25a13e993efc5f447a29ea8e8490dc1d812b74/server/pulp/server/webservices/views/serializers/__init__.py#L301

What is happening here that we are checking to see if the field being passed needs to be translated to match the internal db representation. This line assumes that the field passed is a legitimate field, but it fails to take "dot notation" into account. The correct behavior would be to split the field names on "." and translate only the field name, not the inner name. (We are not using translate to change the key name within a DictField.)

#4 Updated by dkliban@redhat.com over 4 years ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High
  • Platform Release set to 2.8.4
  • Triaged changed from No to Yes

#5 Updated by semyers over 4 years ago

  • Status changed from NEW to POST
  • Assignee set to alanoe

alanoe submitted a PR for review (thanks!)
https://github.com/pulp/pulp/pull/2562

#6 Updated by semyers over 4 years ago

  • Platform Release changed from 2.8.4 to 2.8.5

#7 Updated by semyers over 4 years ago

  • Platform Release changed from 2.8.5 to 2.8.6

#8 Updated by semyers about 4 years ago

  • Platform Release deleted (2.8.6)

#9 Updated by dalley almost 3 years ago

  • Status changed from POST to NEW
  • Priority changed from High to Normal

Since this PR still has unaddressed issues and has been untouched for a year, it has been closed.

Hopefully it can be picked back up again later.

#10 Updated by dalley almost 3 years ago

  • Assignee deleted (alanoe)

#11 Updated by bmbouter over 1 year ago

  • Status changed from NEW to CLOSED - WONTFIX

#12 Updated by bmbouter over 1 year ago

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.

#13 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF