Project

Profile

Help

Story #214

As a user, I can retrieve a list of repos published with distributor X since Y using the API

Added by dgregor@redhat.com over 6 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
2.7.0
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
April 2015
Quarter:

Description

++ This bug was initially created as a clone of Bugzilla Bug #1055750 ++

Description of problem:

I would like to get a list of all repos that have been published with the yum distributor in the last 24 hours. It's possible to make this query one repo at a time, but that is not practical for a server with many repos. Please add an API that takes the following inputs:
* distributor_type_id (e.g. yum_distributor)
* since_date
* until_date (optional)

and returns a list of repositories like the /pulp/api/v2/repositories/search/ API. In fact, it would be great if /pulp/api/v2/repositories/search/ could be extended to take the above fields as additional search criteria.

Associated revisions

Revision ee7b4664 View on GitHub
Added by jortel@redhat.com over 6 years ago

ref #214, support distributor search.

Revision ee7b4664 View on GitHub
Added by jortel@redhat.com over 6 years ago

ref #214, support distributor search.

Revision 717758aa View on GitHub
Added by jortel@redhat.com over 6 years ago

Merge pull request #1835 from jortel/issue-214

ref #214, support distributor search.

Revision 717758aa View on GitHub
Added by jortel@redhat.com over 6 years ago

Merge pull request #1835 from jortel/issue-214

ref #214, support distributor search.

Revision d33f9b02 View on GitHub
Added by jortel@redhat.com over 6 years ago

ref #214, add criteria search operator.

Revision d33f9b02 View on GitHub
Added by jortel@redhat.com over 6 years ago

ref #214, add criteria search operator.

History

#1 Updated by mhrivnak over 6 years ago

  • Tags Sprint Candidate added

#2 Updated by jortel@redhat.com over 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jortel@redhat.com

#3 Updated by dgregor@redhat.com over 6 years ago

  • Description updated (diff)
  • Private changed from Yes to No

#4 Updated by bmbouter over 6 years ago

  • Subject changed from [RFE] Add API for "list of repos published with distributor X since Y" to As a user, I can retrieve a list of repos published with distributor X since Y using the API

#5 Updated by jortel@redhat.com over 6 years ago

After consideration, I think that augmenting the repository search for this will introduce some strange semantics from the REST perspective. You'd be searching on last_published which is a distributor attribute, not a repository attribute. I'm thinking that adding a new REST endpoint for searching distributor resources would satisfy this use case and provide a basis for future API improvements.

The URL would be:

/v2/distributors/search

and would support standard Pulp criteria searches. For this use case, the criteria would be: distributor_type_id = YUM and last_published >= 24 hours ago. The result set would be a list of distributors which includes the repo_id.

Dennis, would this be satisfactory?

#6 Updated by dgregor@redhat.com over 6 years ago

Yes, that would do the trick. As part of this, would you also change the last_published data type as described in the earlier comment?

#7 Updated by jortel@redhat.com over 6 years ago

wrote:

Yes, that would do the trick. As part of this, would you also change the last_published data type as described in the earlier comment?

Yes. The storage type will be changed to support the search but will still be returned as an iso8601 string.

#8 Updated by jortel@redhat.com over 6 years ago

  • Status changed from ASSIGNED to POST

#9 Updated by jortel@redhat.com over 6 years ago

  • Status changed from POST to MODIFIED

#10 Updated by bmbouter over 6 years ago

  • Sprint Candidate set to Yes
  • Tags deleted (Sprint Candidate)

#11 Updated by jortel@redhat.com over 6 years ago

  • Sprint/Milestone set to 15
  • Groomed set to No

#12 Updated by jortel@redhat.com over 6 years ago

  • Groomed changed from No to Yes

#13 Updated by dkliban@redhat.com over 6 years ago

  • Platform Release set to 2.7.0

#14 Updated by jortel@redhat.com over 6 years ago

  • Status changed from MODIFIED to POST

https://github.com/pulp/pulp/pull/1856

Essential to support this use case.

#15 Updated by jortel@redhat.com over 6 years ago

Testing:
1. create a repository.
2. search for distributors that have published since NOW.
3. publish your repository
4. search again and see the distributor listed.
5. change the query as need to be satisfied.

Example query:

curl -i -H "Accept: application/json" -H "WebFrameworkSwitch: django" -X POST -k -u admin:admin  "https://localhost/pulp/api/v2/distributorsearch/" -d '{"criteria":{"filters":{"last_publish":{"$gt":{"$date":"2013-12-01T14:35:00Z"}}}}}'
HTTP/1.1 200 OK
Date: Tue, 12 May 2015 19:21:28 GMT
Server: Apache/2.4.10 (Fedora) OpenSSL/1.0.1k-fips mod_wsgi/4.3.2 Python/2.7.8
Content-Length: 348
Content-Type: application/json

[{"repo_id": "el7", "_ns": "repo_distributors", "last_publish": "2015-04-28T18:19:01Z", "auto_publish": null, "scheduled_publishes": [], "distributor_type_id": "ostree_web_distributor", "scratchpad": null, "_id": {"$oid": "553fcf11e138234656dbec38"}, "config": {"relative_path": "/opt/content/ostree/el7"}, "id": "ostree_web_distributor_name_cli"}]

#16 Updated by jortel@redhat.com over 6 years ago

  • Status changed from POST to MODIFIED

#17 Updated by dkliban@redhat.com over 6 years ago

  • Status changed from MODIFIED to 5

#18 Updated by igulina@redhat.com over 6 years ago

>> rpm -qa pulp-server
pulp-server-2.7.0-0.4.beta.el7.noarch
>> pulp-admin rpm repo list
+----------------------------------------------------------------------+
                            RPM Repositories
+----------------------------------------------------------------------+
>> pulp-admin rpm repo create --repo-id zoopark --feed https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
Successfully created repository [zoopark]

>> date
Tue Jul  7 10:46:02 EDT 2015

>> curl -i -H "Accept: application/json" -H "WebFrameworkSwitch: django" -X POS -k -u admin:admin  "https://localhost/pulp/api/v2/distributors/search/" -d '{"criteria":{"filters":{"last_publish":{"$gt":{"$date":"2015-07-07T10:40:00Z"}}}}}'
HTTP/1.1 200 OK
Date: Tue, 07 Jul 2015 14:47:27 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
Content-Length: 2
Content-Type: application/json

[]

>> pulp-admin rpm repo publish run --repo-id zoopark
+----------------------------------------------------------------------+
                    Publishing Repository [zoopark]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[-]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[-]
... completed

Publishing Comps file
[-]
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

>> curl -i -H "Accept: application/json" -H "WebFrameworkSwitch: django" -X POST -k -u admin:admin  "https://localhost/pulp/api/v2/distributors/search/" -d '{"criteria":{"filters":{"last_publish":{"$gt":{"$date":"2015-07-07T10:40:00Z"}}}}}'
HTTP/1.1 200 OK
Date: Tue, 07 Jul 2015 14:49:31 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
Content-Length: 367
Content-Type: application/json

[{"repo_id": "zoopark", "_ns": "repo_distributors", "last_publish": "2015-07-07T14:49:24Z", "auto_publish": true, "scheduled_publishes": [], "distributor_type_id": "yum_distributor", "scratchpad": null, "_id": {"$oid": "559be5fb77b2943159a2a437"}, "config": {"http": false, "https": true, "relative_url": "/repos/pulp/pulp/demo_repos/zoo/"}, "id": "yum_distributor"}]

>> curl -i -H "Accept: application/json" -H "WebFrameworkSwitch: django" -X POST -k -u admin:admin  "https://localhost/pulp/api/v2/distributors/search/" -d '{"criteria":{"filters":{"last_publish":{"$lt":{"$date":"2013-07-07T10:40:00Z"}}}}}'
HTTP/1.1 200 OK
Date: Tue, 07 Jul 2015 14:51:13 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
Content-Length: 2
Content-Type: application/json

[]

>> pulp-admin rpm repo publish run --repo-id panda
+----------------------------------------------------------------------+
                     Publishing Repository [panda]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[-]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[-]
... completed

Publishing Comps file
[-]
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

>> curl -i -H "Accept: application/json" -H "WebFrameworkSwitch: django" -X POST -k -u admin:admin  "https://localhost/pulp/api/v2/distributors/search/" -d '{"criteria":{"filters":{"last_publish":{"$gt":{"$date":"2013-07-07T10:40:00Z"}}}}}'
HTTP/1.1 200 OK
Date: Tue, 07 Jul 2015 14:52:22 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
Content-Length: 705
Content-Type: application/json

[{"repo_id": "zoopark", "_ns": "repo_distributors", "last_publish": "2015-07-07T14:49:24Z", "auto_publish": true, "scheduled_publishes": [], "distributor_type_id": "yum_distributor", "scratchpad": null, "_id": {"$oid": "559be5fb77b2943159a2a437"}, "config": {"http": false, "https": true, "relative_url": "/repos/pulp/pulp/demo_repos/zoo/"}, "id": "yum_distributor"}, {"repo_id": "panda", "_ns": "repo_distributors", "last_publish": "2015-07-07T14:52:15Z", "auto_publish": true, "scheduled_publishes": [], "distributor_type_id": "yum_distributor", "scratchpad": null, "_id": {"$oid": "559be79277b2943159a2a441"}, "config": {"http": false, "https": true, "relative_url": "panda"}, "id": "yum_distributor"}]

#19 Updated by igulina@redhat.com over 6 years ago

  • Status changed from 5 to ASSIGNED

Everything works fine. However a response body is different from the one in docs [1]
On pulp 2.7.0-0.4 I have a response like:

{
 "repo_id": "zoopark", 
 "_ns": "repo_distributors",
 "last_publish": "2015-07-07T14:49:24Z", 
 "auto_publish": true, 
 "scheduled_publishes": [], 
 "distributor_type_id": "yum_distributor",
 "scratchpad": null,
 "_id": {"$oid": "559be5fb77b2943159a2a437"}, 
 "config": {
        "http": false, 
        "https": true, 
        "relative_url": "/repos/pulp/pulp/demo_repos/zoo/"
            }, 
 "id": "yum_distributor"
}

[1] https://github.com/pulp/pulp/blob/master/docs/dev-guide/integration/rest-api/repo/retrieval.rst#advanced-search-for-distributors

#20 Updated by jortel@redhat.com about 6 years ago

Can you be more specific about that is incorrect about the documentation? The only difference I see is that in the documentation sample, the query results in a list of (2) ostree distributors (instead of yum).

#21 Updated by jortel@redhat.com about 6 years ago

  • Status changed from ASSIGNED to 5

#22 Updated by rbarlow over 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#23 Updated by bmbouter over 3 years ago

  • Sprint set to April 2015

#24 Updated by bmbouter over 3 years ago

  • Sprint/Milestone deleted (15)

#25 Updated by bmbouter over 2 years ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF