Project

Profile

Help

Task #1803

Plan replacement of mongodb with postgres

Added by Anonymous over 4 years ago. Updated 23 days ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Documentation, Pulp 2
Sprint:
Sprint 5

Related issues

Related to Pulp - Task #1872: Profile Django ORM instantiation costCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to Pulp - Task #1873: Plan REST API for 3.0CLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to Pulp - Task #1874: Plan User/Auth system for 3.0CLOSED - DUPLICATE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

History

#1 Updated by ipanova@redhat.com over 4 years ago

  • Status changed from NEW to ASSIGNED

#2 Updated by semyers over 4 years ago

I'm in the process of writing all this in more detail in an etherpad, but for now I'll outline the work so far.

pcreech and I started by basically just writing out all of models we had, and trying to organize them in a diagram.

This went poorly, because pulp has four distinct categories of models, and the tool we were using didn't make it very easy to diagram them all together. With all the pulp releasing I've been doing, and all the RHUI pcreech has been doing, all of this finally came together toward the end of last week while jortel was on PTO. Before unleashing the confusing and questionably useful diagrams on the team, I wanted to get jortel's feedback. I was not disappointed.

With his help, we've identified the aformentioned data categories: Repositories, Content Units, RBAC, Tasks. I'm writing up a doc right now that explains what collections we currently have in mongo, to which category they belong, how the currently relate to each other, and some speculation about how we can migrate the data in that category to a more relational design. Since I'll be speculating, this doc will go up in an etherpad for review and improvement by the small team focused on this at the moment (jortel, pcreech, me), before being submitted to the team at large.

Depending on how quickly I get this done and reviewed, the link to that etherpad should most likely be appearing here tomorrow.

#3 Updated by semyers over 4 years ago

  • Private changed from No to Yes

The "Relational Pulp" etherpad has been drafted and edited. It is available for internal comment at http://pulp.etherpad.corp.redhat.com/244 by the entirety of the pulp team.

#4 Updated by semyers over 4 years ago

  • Private changed from Yes to No

#5 Updated by semyers about 4 years ago

  • Related to Task #1872: Profile Django ORM instantiation cost added

#6 Updated by mhrivnak about 4 years ago

  • Related to Task #1873: Plan REST API for 3.0 added

#7 Updated by mhrivnak about 4 years ago

  • Related to Task #1874: Plan User/Auth system for 3.0 added

#8 Updated by mhrivnak about 4 years ago

  • Sprint/Milestone changed from 19 to 20

#9 Updated by semyers about 4 years ago

The migration plan is largely "ratified", in that objections to it have been (or are currently being) addressed. I feel like we've reached a point where what we don't know outweighs what we do know, and the best way I can think of to bridge the knowledge gap is to finishing modeling pulp out on postgres and start trying to use it. I'll be modifying my relational pulp project to this end over the coming days.

https://github.com/seandst/relational-pulp/

#10 Updated by mhrivnak about 4 years ago

  • Sprint/Milestone changed from 20 to 21

#11 Updated by semyers about 4 years ago

Quick update, progress is still being made. pcreech has converted the project from docker to Vagrant, which is awesome. The pulp platform models related to repos and units have been written down, and I'm currently porting RPM's units and repo-related models now so we can start looking at migrating data from nonrel-pulp to rel-pulp to see what explodes and needs to be revisited. :)

#12 Updated by mhrivnak about 4 years ago

  • Sprint/Milestone changed from 21 to 22

#13 Updated by mhrivnak about 4 years ago

  • Sprint/Milestone changed from 22 to 23

#14 Updated by semyers about 4 years ago

  • Status changed from ASSIGNED to CLOSED - CURRENTRELEASE

This is largely done (and by done, I mean now we can start to do it? :D). The rel-pulp doc and related repo are generally accepted by the team, and are now available in the pulp namespace on github:

https://github.com/pulp/relational-pulp - check out the db-translation-guide.md doc.

Most of the work was done by a cabal made up of jortel, pcreech, and myself with lots of great ideas and insights coming from all over the place, including stakeholders not on the pulp team. Thanks to all involved. More meetings will ensue, followed by more redmine tasks.

#15 Updated by bmbouter over 2 years ago

  • Sprint set to Sprint 5

#16 Updated by bmbouter over 2 years ago

  • Sprint/Milestone deleted (23)

#17 Updated by bmbouter about 1 year ago

  • Tags Pulp 2 added

#18 Updated by bmbouter 23 days ago

  • Category deleted (14)

We are removing the 'API' category per open floor discussion June 16, 2020.

Please register to edit this issue

Also available in: Atom PDF