Project

Profile

Help

Task #1803

closed

Plan replacement of mongodb with postgres

Added by Anonymous almost 8 years ago. Updated almost 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Documentation, Pulp 2
Sprint:
Sprint 5
Quarter:

Related issues

Related to Pulp - Task #1872: Profile Django ORM instantiation costCLOSED - CURRENTRELEASEsemyers

Actions
Related to Pulp - Task #1873: Plan REST API for 3.0CLOSED - CURRENTRELEASEsemyers

Actions
Related to Pulp - Task #1874: Plan User/Auth system for 3.0CLOSED - DUPLICATE

Actions
Actions #1

Updated by ipanova@redhat.com almost 8 years ago

  • Status changed from NEW to ASSIGNED
Actions #2

Updated by semyers almost 8 years ago

I'm in the process of writing all this in more detail in an etherpad, but for now I'll outline the work so far.

pcreech and I started by basically just writing out all of models we had, and trying to organize them in a diagram.

This went poorly, because pulp has four distinct categories of models, and the tool we were using didn't make it very easy to diagram them all together. With all the pulp releasing I've been doing, and all the RHUI pcreech has been doing, all of this finally came together toward the end of last week while jortel was on PTO. Before unleashing the confusing and questionably useful diagrams on the team, I wanted to get jortel's feedback. I was not disappointed.

With his help, we've identified the aformentioned data categories: Repositories, Content Units, RBAC, Tasks. I'm writing up a doc right now that explains what collections we currently have in mongo, to which category they belong, how the currently relate to each other, and some speculation about how we can migrate the data in that category to a more relational design. Since I'll be speculating, this doc will go up in an etherpad for review and improvement by the small team focused on this at the moment (jortel, pcreech, me), before being submitted to the team at large.

Depending on how quickly I get this done and reviewed, the link to that etherpad should most likely be appearing here tomorrow.

Actions #3

Updated by semyers almost 8 years ago

  • Private changed from No to Yes

The "Relational Pulp" etherpad has been drafted and edited. It is available for internal comment at http://pulp.etherpad.corp.redhat.com/244 by the entirety of the pulp team.

Actions #4

Updated by semyers almost 8 years ago

  • Private changed from Yes to No
Actions #5

Updated by semyers almost 8 years ago

  • Related to Task #1872: Profile Django ORM instantiation cost added
Actions #6

Updated by mhrivnak almost 8 years ago

  • Related to Task #1873: Plan REST API for 3.0 added
Actions #7

Updated by mhrivnak almost 8 years ago

  • Related to Task #1874: Plan User/Auth system for 3.0 added
Actions #8

Updated by mhrivnak almost 8 years ago

  • Sprint/Milestone changed from 19 to 20
Actions #9

Updated by semyers almost 8 years ago

The migration plan is largely "ratified", in that objections to it have been (or are currently being) addressed. I feel like we've reached a point where what we don't know outweighs what we do know, and the best way I can think of to bridge the knowledge gap is to finishing modeling pulp out on postgres and start trying to use it. I'll be modifying my relational pulp project to this end over the coming days.

https://github.com/seandst/relational-pulp/

Actions #10

Updated by mhrivnak almost 8 years ago

  • Sprint/Milestone changed from 20 to 21
Actions #11

Updated by semyers almost 8 years ago

Quick update, progress is still being made. pcreech has converted the project from docker to Vagrant, which is awesome. The pulp platform models related to repos and units have been written down, and I'm currently porting RPM's units and repo-related models now so we can start looking at migrating data from nonrel-pulp to rel-pulp to see what explodes and needs to be revisited. :)

Actions #12

Updated by mhrivnak almost 8 years ago

  • Sprint/Milestone changed from 21 to 22
Actions #13

Updated by mhrivnak over 7 years ago

  • Sprint/Milestone changed from 22 to 23
Actions #14

Updated by semyers over 7 years ago

  • Status changed from ASSIGNED to CLOSED - CURRENTRELEASE

This is largely done (and by done, I mean now we can start to do it? :D). The rel-pulp doc and related repo are generally accepted by the team, and are now available in the pulp namespace on github:

https://github.com/pulp/relational-pulp - check out the db-translation-guide.md doc.

Most of the work was done by a cabal made up of jortel, pcreech, and myself with lots of great ideas and insights coming from all over the place, including stakeholders not on the pulp team. Thanks to all involved. More meetings will ensue, followed by more redmine tasks.

Actions #15

Updated by bmbouter about 6 years ago

  • Sprint set to Sprint 5
Actions #16

Updated by bmbouter about 6 years ago

  • Sprint/Milestone deleted (23)
Actions #17

Updated by bmbouter almost 5 years ago

  • Tags Pulp 2 added
Actions #18

Updated by bmbouter almost 4 years ago

  • Category deleted (14)

We are removing the 'API' category per open floor discussion June 16, 2020.

Also available in: Atom PDF