Issue #1865
closedError searching rpm repositories by fields using dot notation
Added by alanoe over 8 years ago. Updated over 5 years ago.
Description
when I run 'pulp-admin rpm repo search --fields notes.A' , I get the following error:
Invalid properties: ['Field: <notes.A> does not exist on objects in the <repos> collection']
I'm sure that the attribute A exists in notes dict.
This is confirmed by a find() command with the same query in the mongo shell:
$ use pulp_database
$ db.repos.find({}, {'notes.A': 1})
This pulp-admin command works with pulp/pulp-admin 2.5.3 and MongoDB 2.4.9 installed on RHEL 6.
Updated by bmbouter over 8 years ago
I agree this is an issue; I've reproduced it. Here are some details:
In my vanilla Vagrant environment there are 4 repos. The rpm repo looks like this from db.repos.find()
{
"_id" : ObjectId("57220e7ce779890a1bd0d311"),
"repo_id" : "zoo",
"notes" : {
"_repo-type" : "rpm-repo"
},
"scratchpad" : {
},
"content_unit_counts" : {
},
"_ns" : "repos"
}
In there we should be able to search on the field notes._repo-type I'm able to also do that in my db:
> db.repos.find({}, {'notes._repo-type': 1})
{ "_id" : ObjectId("57220e7be779890a1c1e2bfb"), "notes" : { "_repo-type" : "docker-repo" } }
{ "_id" : ObjectId("57220e7ce779890a1d75f848"), "notes" : { "_repo-type" : "docker-repo" } }
{ "_id" : ObjectId("57220e7ce779890a1bd0d311"), "notes" : { "_repo-type" : "rpm-repo" } }
{ "_id" : ObjectId("57220e7de779890a1d75f84c"), "notes" : { "_repo-type" : "puppet-repo" } }
{ "_id" : ObjectId("57220e7de779890a1c1e2bff"), "notes" : { "_repo-type" : "PYTHON" } }
I then try to do the same with pulp-admin:
pulp-admin -vv rpm repo search --fields notes._repo-type
Which returns the error message:
Invalid properties: ['Field: <notes._repo-type> does not exist on objects in the
<repos> collection']
The above pulp-admin command is producing this API request:
POST request to /pulp/api/v2/repositories/search/ with parameters {"criteria": {"sort": null, "fields": ["notes._repo-type"], "limit": null, "filters": {"notes._repo-type": "rpm-repo"}, "skip": null}}
I can have httpie perform the same request:
http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/search/ criteria:='{"sort": null, "fields": ["notes._repo-type"], "limit": null, "filters": {"notes._repo-type": "rpm-repo"}, "skip": null}'
This simplified version also fails with the same error:
http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/search/ criteria:='{"fields": ["notes._repo-type"], "filters": {"notes._repo-type": "rpm-repo"}}'
Both of these work as expected indicating that the filters section is working correctly.
http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/search/ criteria:='{"fields": ["notes"], "filters": {"notes._repo-type": "rpm-repo"}}'
http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/search/ criteria:='{"filters": {"notes._repo-type": "rpm-repo"}}'
The bottom one corresponds with the pulp-admin command which also works as expected:
pulp-admin -vv rpm repo search --filters '{"notes._repo-type": "rpm-repo"}'
The above command produces the following API call:
POST request to /pulp/api/v2/repositories/search/ with parameters {"criteria": {"sort": null, "fields": null, "limit": null, "filters": {"notes._repo-type": "rpm-repo"}, "skip": null}}
Updated by bmbouter over 8 years ago
I see two issues here.
1. The bug as reported is legitimate and is an error on the server side.
2. The specification of --fields causes --filters to also be set. This does not make sense to me. Perhaps that's expected? I would like other input on if this should be filed as a separate pulp bug against pulp-admin in pulp_rpm?
Updated by amacdona@redhat.com over 8 years ago
The offending line is https://github.com/pulp/pulp/blob/6d25a13e993efc5f447a29ea8e8490dc1d812b74/server/pulp/server/webservices/views/serializers/__init__.py#L301
What is happening here that we are checking to see if the field being passed needs to be translated to match the internal db representation. This line assumes that the field passed is a legitimate field, but it fails to take "dot notation" into account. The correct behavior would be to split the field names on "." and translate only the field name, not the inner name. (We are not using translate to change the key name within a DictField.)
Updated by dkliban@redhat.com over 8 years ago
- Priority changed from Normal to High
- Severity changed from 2. Medium to 3. High
- Platform Release set to 2.8.4
- Triaged changed from No to Yes
Updated by semyers over 8 years ago
- Status changed from NEW to POST
- Assignee set to alanoe
alanoe submitted a PR for review (thanks!)
https://github.com/pulp/pulp/pull/2562
Updated by semyers over 8 years ago
- Platform Release changed from 2.8.4 to 2.8.5
Updated by semyers over 8 years ago
- Platform Release changed from 2.8.5 to 2.8.6
Updated by dalley about 7 years ago
- Status changed from POST to NEW
- Priority changed from High to Normal
Since this PR still has unaddressed issues and has been untouched for a year, it has been closed.
Hopefully it can be picked back up again later.
Updated by bmbouter over 5 years ago
- Status changed from NEW to CLOSED - WONTFIX
Updated by bmbouter over 5 years ago
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.