Project

Profile

Help

Story #214

closed

As a user, I can retrieve a list of repos published with distributor X since Y using the API

Added by dgregor@redhat.com about 9 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
2.7.0
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
April 2015
Quarter:

Description

++ This bug was initially created as a clone of Bugzilla Bug #1055750 ++

Description of problem:

I would like to get a list of all repos that have been published with the yum distributor in the last 24 hours. It's possible to make this query one repo at a time, but that is not practical for a server with many repos. Please add an API that takes the following inputs:
* distributor_type_id (e.g. yum_distributor)
* since_date
* until_date (optional)

and returns a list of repositories like the /pulp/api/v2/repositories/search/ API. In fact, it would be great if /pulp/api/v2/repositories/search/ could be extended to take the above fields as additional search criteria.

Actions #1

Updated by mhrivnak almost 9 years ago

  • Tags Sprint Candidate added
Actions #2

Updated by jortel@redhat.com almost 9 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jortel@redhat.com
Actions #3

Updated by dgregor@redhat.com almost 9 years ago

  • Description updated (diff)
  • Private changed from Yes to No
Actions #4

Updated by bmbouter almost 9 years ago

  • Subject changed from [RFE] Add API for "list of repos published with distributor X since Y" to As a user, I can retrieve a list of repos published with distributor X since Y using the API
Actions #5

Updated by jortel@redhat.com almost 9 years ago

After consideration, I think that augmenting the repository search for this will introduce some strange semantics from the REST perspective. You'd be searching on last_published which is a distributor attribute, not a repository attribute. I'm thinking that adding a new REST endpoint for searching distributor resources would satisfy this use case and provide a basis for future API improvements.

The URL would be:

/v2/distributors/search

and would support standard Pulp criteria searches. For this use case, the criteria would be: distributor_type_id = YUM and last_published >= 24 hours ago. The result set would be a list of distributors which includes the repo_id.

Dennis, would this be satisfactory?

Actions #6

Updated by dgregor@redhat.com almost 9 years ago

Yes, that would do the trick. As part of this, would you also change the last_published data type as described in the earlier comment?

Added by jortel@redhat.com almost 9 years ago

Revision ee7b4664 | View on GitHub

ref #214, support distributor search.

Added by jortel@redhat.com almost 9 years ago

Revision ee7b4664 | View on GitHub

ref #214, support distributor search.

Actions #7

Updated by jortel@redhat.com almost 9 years ago

wrote:

Yes, that would do the trick. As part of this, would you also change the last_published data type as described in the earlier comment?

Yes. The storage type will be changed to support the search but will still be returned as an iso8601 string.

Actions #8

Updated by jortel@redhat.com almost 9 years ago

  • Status changed from ASSIGNED to POST

Added by jortel@redhat.com almost 9 years ago

Revision 717758aa | View on GitHub

Merge pull request #1835 from jortel/issue-214

ref #214, support distributor search.

Added by jortel@redhat.com almost 9 years ago

Revision 717758aa | View on GitHub

Merge pull request #1835 from jortel/issue-214

ref #214, support distributor search.

Actions #9

Updated by jortel@redhat.com almost 9 years ago

  • Status changed from POST to MODIFIED
Actions #10

Updated by bmbouter almost 9 years ago

  • Sprint Candidate set to Yes
  • Tags deleted (Sprint Candidate)
Actions #11

Updated by jortel@redhat.com almost 9 years ago

  • Sprint/Milestone set to 15
  • Groomed set to No
Actions #12

Updated by jortel@redhat.com almost 9 years ago

  • Groomed changed from No to Yes
Actions #13

Updated by dkliban@redhat.com almost 9 years ago

  • Platform Release set to 2.7.0
Actions #14

Updated by jortel@redhat.com almost 9 years ago

  • Status changed from MODIFIED to POST

https://github.com/pulp/pulp/pull/1856

Essential to support this use case.

Actions #15

Updated by jortel@redhat.com almost 9 years ago

Testing:
1. create a repository.
2. search for distributors that have published since NOW.
3. publish your repository
4. search again and see the distributor listed.
5. change the query as need to be satisfied.

Example query:

curl -i -H "Accept: application/json" -H "WebFrameworkSwitch: django" -X POST -k -u admin:admin  "https://localhost/pulp/api/v2/distributorsearch/" -d '{"criteria":{"filters":{"last_publish":{"$gt":{"$date":"2013-12-01T14:35:00Z"}}}}}'
HTTP/1.1 200 OK
Date: Tue, 12 May 2015 19:21:28 GMT
Server: Apache/2.4.10 (Fedora) OpenSSL/1.0.1k-fips mod_wsgi/4.3.2 Python/2.7.8
Content-Length: 348
Content-Type: application/json

[{"repo_id": "el7", "_ns": "repo_distributors", "last_publish": "2015-04-28T18:19:01Z", "auto_publish": null, "scheduled_publishes": [], "distributor_type_id": "ostree_web_distributor", "scratchpad": null, "_id": {"$oid": "553fcf11e138234656dbec38"}, "config": {"relative_path": "/opt/content/ostree/el7"}, "id": "ostree_web_distributor_name_cli"}]
Actions #16

Updated by jortel@redhat.com almost 9 years ago

  • Status changed from POST to MODIFIED

Added by jortel@redhat.com almost 9 years ago

Revision d33f9b02 | View on GitHub

ref #214, add criteria search operator.

Added by jortel@redhat.com almost 9 years ago

Revision d33f9b02 | View on GitHub

ref #214, add criteria search operator.

Actions #17

Updated by dkliban@redhat.com almost 9 years ago

  • Status changed from MODIFIED to 5
Actions #18

Updated by igulina@redhat.com over 8 years ago

>> rpm -qa pulp-server
pulp-server-2.7.0-0.4.beta.el7.noarch
>> pulp-admin rpm repo list
+----------------------------------------------------------------------+
                            RPM Repositories
+----------------------------------------------------------------------+
>> pulp-admin rpm repo create --repo-id zoopark --feed https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
Successfully created repository [zoopark]

>> date
Tue Jul  7 10:46:02 EDT 2015

>> curl -i -H "Accept: application/json" -H "WebFrameworkSwitch: django" -X POS -k -u admin:admin  "https://localhost/pulp/api/v2/distributors/search/" -d '{"criteria":{"filters":{"last_publish":{"$gt":{"$date":"2015-07-07T10:40:00Z"}}}}}'
HTTP/1.1 200 OK
Date: Tue, 07 Jul 2015 14:47:27 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
Content-Length: 2
Content-Type: application/json

[]

>> pulp-admin rpm repo publish run --repo-id zoopark
+----------------------------------------------------------------------+
                    Publishing Repository [zoopark]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[-]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[-]
... completed

Publishing Comps file
[-]
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

>> curl -i -H "Accept: application/json" -H "WebFrameworkSwitch: django" -X POST -k -u admin:admin  "https://localhost/pulp/api/v2/distributors/search/" -d '{"criteria":{"filters":{"last_publish":{"$gt":{"$date":"2015-07-07T10:40:00Z"}}}}}'
HTTP/1.1 200 OK
Date: Tue, 07 Jul 2015 14:49:31 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
Content-Length: 367
Content-Type: application/json

[{"repo_id": "zoopark", "_ns": "repo_distributors", "last_publish": "2015-07-07T14:49:24Z", "auto_publish": true, "scheduled_publishes": [], "distributor_type_id": "yum_distributor", "scratchpad": null, "_id": {"$oid": "559be5fb77b2943159a2a437"}, "config": {"http": false, "https": true, "relative_url": "/repos/pulp/pulp/demo_repos/zoo/"}, "id": "yum_distributor"}]

>> curl -i -H "Accept: application/json" -H "WebFrameworkSwitch: django" -X POST -k -u admin:admin  "https://localhost/pulp/api/v2/distributors/search/" -d '{"criteria":{"filters":{"last_publish":{"$lt":{"$date":"2013-07-07T10:40:00Z"}}}}}'
HTTP/1.1 200 OK
Date: Tue, 07 Jul 2015 14:51:13 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
Content-Length: 2
Content-Type: application/json

[]

>> pulp-admin rpm repo publish run --repo-id panda
+----------------------------------------------------------------------+
                     Publishing Repository [panda]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[-]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[-]
... completed

Publishing Comps file
[-]
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

>> curl -i -H "Accept: application/json" -H "WebFrameworkSwitch: django" -X POST -k -u admin:admin  "https://localhost/pulp/api/v2/distributors/search/" -d '{"criteria":{"filters":{"last_publish":{"$gt":{"$date":"2013-07-07T10:40:00Z"}}}}}'
HTTP/1.1 200 OK
Date: Tue, 07 Jul 2015 14:52:22 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
Content-Length: 705
Content-Type: application/json

[{"repo_id": "zoopark", "_ns": "repo_distributors", "last_publish": "2015-07-07T14:49:24Z", "auto_publish": true, "scheduled_publishes": [], "distributor_type_id": "yum_distributor", "scratchpad": null, "_id": {"$oid": "559be5fb77b2943159a2a437"}, "config": {"http": false, "https": true, "relative_url": "/repos/pulp/pulp/demo_repos/zoo/"}, "id": "yum_distributor"}, {"repo_id": "panda", "_ns": "repo_distributors", "last_publish": "2015-07-07T14:52:15Z", "auto_publish": true, "scheduled_publishes": [], "distributor_type_id": "yum_distributor", "scratchpad": null, "_id": {"$oid": "559be79277b2943159a2a441"}, "config": {"http": false, "https": true, "relative_url": "panda"}, "id": "yum_distributor"}]
Actions #19

Updated by igulina@redhat.com over 8 years ago

  • Status changed from 5 to ASSIGNED

Everything works fine. However a response body is different from the one in docs [1]
On pulp 2.7.0-0.4 I have a response like:

{
 "repo_id": "zoopark", 
 "_ns": "repo_distributors",
 "last_publish": "2015-07-07T14:49:24Z", 
 "auto_publish": true, 
 "scheduled_publishes": [], 
 "distributor_type_id": "yum_distributor",
 "scratchpad": null,
 "_id": {"$oid": "559be5fb77b2943159a2a437"}, 
 "config": {
        "http": false, 
        "https": true, 
        "relative_url": "/repos/pulp/pulp/demo_repos/zoo/"
            }, 
 "id": "yum_distributor"
}

[1] https://github.com/pulp/pulp/blob/master/docs/dev-guide/integration/rest-api/repo/retrieval.rst#advanced-search-for-distributors

Actions #20

Updated by jortel@redhat.com over 8 years ago

Can you be more specific about that is incorrect about the documentation? The only difference I see is that in the documentation sample, the query results in a list of (2) ostree distributors (instead of yum).

Actions #21

Updated by jortel@redhat.com over 8 years ago

  • Status changed from ASSIGNED to 5
Actions #22

Updated by rbarlow about 8 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #23

Updated by bmbouter about 6 years ago

  • Sprint set to April 2015
Actions #24

Updated by bmbouter about 6 years ago

  • Sprint/Milestone deleted (15)
Actions #25

Updated by bmbouter almost 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF