Project

Profile

Help

Task #1873

Plan REST API for 3.0

Added by mhrivnak over 3 years ago. Updated 8 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Platform Release:
Blocks Release:
Backwards Incompatible:
Yes
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 9

Description

Primarily due to the search API being tied to mongodb, and that pulp 3.0 will not use mongodb, pulp 3.0 will require a new API for search. Search is pervasive throughout the REST API, so a huge number of endpoints will have to change.

While we're at it, there are a number of API changes that have been desired for some time, and this will be a rare opportunity to act on them.

Ideally we will be able to auto-generate most or all of the API and its docs.

requests.txt (40.6 KB) amacdona@redhat.com, 08/11/2016 03:53 PM requests.txt

Checklist


Subtasks

Task #2175: Make DRF tools that can represent our master/detail relationships via the APICLOSED - CURRENTRELEASEsemyers

Actions
Task #2238: Make DRF tools that can represent our generic models via the APICLOSED - CURRENTRELEASEsemyers

Actions
Task #2298: Add serializer for the repository models.MODIFIEDamacdona@redhat.com

Actions

Related issues

Related to Pulp - Task #1803: Plan replacement of mongodb with postgres CLOSED - CURRENTRELEASE Actions
Related to Pulp - Task #1874: Plan User/Auth system for 3.0 CLOSED - DUPLICATE Actions
Duplicated by Pulp - Task #2091: Create a plan for the REST API in 3.0 CLOSED - DUPLICATE Actions

Associated revisions

Revision d03d73c9 View on GitHub
Added by semyers about 3 years ago

REST API initial documentation and base classes

closes #1873
https://pulp.plan.io/issues/1873

Revision d03d73c9 View on GitHub
Added by semyers about 3 years ago

REST API initial documentation and base classes

closes #1873
https://pulp.plan.io/issues/1873

Revision d03d73c9 View on GitHub
Added by semyers about 3 years ago

REST API initial documentation and base classes

closes #1873
https://pulp.plan.io/issues/1873

History

#1 Updated by mhrivnak over 3 years ago

  • Related to Task #1803: Plan replacement of mongodb with postgres added

#2 Updated by mhrivnak over 3 years ago

  • Related to Task #1874: Plan User/Auth system for 3.0 added

#3 Updated by mhrivnak over 3 years ago

  • Sprint Candidate changed from Yes to No

#4 Updated by mhrivnak over 3 years ago

  • Sprint Candidate changed from No to Yes

#5 Updated by semyers over 3 years ago

  • Groomed changed from No to Yes

#6 Updated by semyers over 3 years ago

  • Duplicated by Task #2091: Create a plan for the REST API in 3.0 added

#7 Updated by amacdona@redhat.com over 3 years ago

Because of issues with the translation of search queries, Katello recorded this list of all the criteria searches that Runcible uses. This should be helpful in determining their search requirements.

#8 Updated by mhrivnak over 3 years ago

  • Sprint/Milestone set to 25

#9 Updated by semyers over 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to semyers

wrote:

Because of issues with the translation of search queries, Katello recorded this list of all the criteria searches that Runcible uses. This should be helpful in determining their search requirements.

I believe that most of the "search" views in pulp 2 will be managed with normal filtering in pulp 3. DRF integrates with django-filter0, and exposes a pretty robust filtering mechanism via REST URLs, so we should pull out some characteristic searches that Katello is doing and looking at how we might construct Django queries or django-filter queries based on them.

I thought I added checklist items to better represent this task; I'll correct that in a moment. In the meantime, I'll point out that implementing the functionality provided by Pulp 2 "search" is likely going to be the least difficult piece of the pulp 3 API to figure out. Coming up with a reasonable implementation to handle our master/detail relationships is likely going to require some serious thought and effort. I've done some initial work on this in the rel-pulp project, so I'd like to take this on and continue that effort.

[0]: http://www.django-rest-framework.org/api-guide/filtering/

#10 Updated by semyers over 3 years ago

  • Checklist item Lay out initial documentation for properly documenting API endpoints added
  • Checklist item Figure out how to compile endpoint documentation into a single document using DRF's features added
  • Checklist item Based on search requirements, identify model fields that we'll want to expose as filterable added
  • Checklist item Put together a demo of DRF's autogenerated API client added

Checklist updated. :)

#11 Updated by mhrivnak about 3 years ago

  • Sprint/Milestone changed from 25 to 26

#12 Updated by amacdona@redhat.com about 3 years ago

  • Checklist item Identify search requirements from stakeholders set to Done

From the requests from katello tests, I have concluded that we will not need to implement full text search, that filtering is enough. Given that modeling is still going on, we cannot "identify model fields that we'll want to expose as filterable" because those fields might not exist or might change.

https://github.com/pulp/pulp/pull/2740/ implements and explains one of each type of filter. This demonstrates that filtering is powerful enough to meet katello needs and is as complete as we can get without finishing modeling first. Once modeling is complete, it will be necessary to re-review the requests on this ticket and ensure that the appropriate fields are filterable to allow comparable filters.

#13 Updated by semyers about 3 years ago

wrote:

https://github.com/pulp/pulp/pull/2740/ implements and explains one of each type of filter.

I've grafted this PR onto my dev branch for this task (commit hash unchanged), which I'm currently preparing for review.

#14 Updated by semyers about 3 years ago

  • Checklist item Lay out initial documentation for properly documenting API endpoints set to Done
  • Checklist item Figure out how to compile endpoint documentation into a single document using DRF's features set to Done
  • Checklist item Based on search requirements, identify model fields that we'll want to expose as filterable set to Done
  • Checklist item Put together a demo of DRF's autogenerated API client set to Done

#15 Updated by semyers about 3 years ago

almost there

https://github.com/pulp/pulp/compare/3.0-dev...seandst:rest-api-startup?expand=1

I've been hacking on this branch for a while now, and there are a lot of ideas captured on it that aren't related to this issue directly. I've already broken out and created PRs for a few of these, and will continue to do that until that branch is focused on just this issue. At that point,
I'll come back to the docs with fresh eyes to clean up errors and inconsistencies and open the PR to get to POST for feedback.

#16 Updated by mhrivnak about 3 years ago

  • Sprint/Milestone changed from 26 to 27

#17 Updated by semyers about 3 years ago

  • Status changed from ASSIGNED to POST

So this is a thing now:
https://github.com/pulp/pulp/pull/2779

All aboard the comment train! Next stop: The new REST API!

#18 Updated by semyers about 3 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 67 to 100

#19 Updated by semyers about 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Calling this closed since it's in the current (nonexistent) release of 3.0. :)

#20 Updated by bmbouter over 1 year ago

  • Sprint set to Sprint 9

#21 Updated by bmbouter over 1 year ago

  • Sprint/Milestone deleted (27)

#22 Updated by bmbouter 8 months ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF