Task #2086
closed
add django DB boilerplate
0%
Description
Add a python module to hold django models, so model creators know where to put them.
Consider moving the django boilerplate code up the python path. Currently, most of it lives in pulp.server.webservices, which I think is non-standard for a django app. Maybe it's fine where it is, but it deserves a little thought.
Include at least a little bit of documentation about how a developer can get started with database settings, or a link to django docs. (the current 1.4 links in settings.py are broken)
Related issues
Updated by mhrivnak over 8 years ago
- Blocks Refactor #2085: create django models for consumer/binding/applicability added
Updated by mhrivnak over 8 years ago
- Blocks Refactor #2087: create django models for task system added
Updated by mhrivnak over 8 years ago
- Blocks Refactor #2088: create django models for lazy catalog and deferred downloads added
Updated by mhrivnak over 8 years ago
- Blocks Refactor #2089: create django models for repos/importers/distributors added
Updated by semyers over 8 years ago
I have the sense that we're going to end up having to rewrite just about everything. I also think that formalizing what we refer to as the pulp "platform" might be done by namespacing pulp'such as the s platform elements (such as the django project) as 'pulp.platform'. We could then import pulp.platform.models, for example.
We've already got a decent categorization going of our models, so at least in platform I think it makes sense to split the models up into multiple modules by category (e.g. repository, contentunit, tasking, lazy, etc). Django has guidelines for doing this: https://docs.djangoproject.com/en/1.9/topics/db/models/#organizing-models-in-a-package
To get started with a database, we're going to want to take a look at the existing server.conf loading code and use that to build the DB connection string in Django's settings.py. Additionally, we might want to customize the postgres connection class to handle our existing reconnect-on-failure behavior.
Updated by mhrivnak over 8 years ago
Good thoughts.
For reconnect functionality, I think that should be tackled separately if django's out-of-the-box behavior isn't what we want. Since this issue blocks others, it's wise to keep it focused on the required changes.
I was also thinking today about the difference between pulp.plugins and pulp.server, and whether we want to tackle cleaning that up. I think a lot of it will happen naturally as stuff gets refactored. And moving code around isn't too big of a deal, so we can also do that later.
Updated by mhrivnak over 8 years ago
- Blocks Task #2092: Create django model(s) for progress reporting added
Updated by semyers over 8 years ago
mhrivnak wrote:
I was also thinking today about the difference between pulp.plugins and pulp.server, and whether we want to tackle cleaning that up. I think a lot of it will happen naturally as stuff gets refactored. And moving code around isn't too big of a deal, so we can also do that later.
Since moving code around isn't too big of a deal, I'd rather put the the Django project in a new namespace to start, so it's easier to see what work's been done during the process.
Updated by mhrivnak over 8 years ago
semyers wrote:
mhrivnak wrote:
I was also thinking today about the difference between pulp.plugins and pulp.server, and whether we want to tackle cleaning that up. I think a lot of it will happen naturally as stuff gets refactored. And moving code around isn't too big of a deal, so we can also do that later.
Since moving code around isn't too big of a deal, I'd rather put the the Django project in a new namespace to start, so it's easier to see what work's been done during the process.
I see your point. Although while moving code isn't a big deal, deciding where to move it and why can be a big deal. That probably deserves its own discussion (which I am very interested to have), but I don't think that discussion should block this task.
Updated by semyers over 8 years ago
After some more consideration, I think that adding the server.conf support to the django settings can be split out into its own task, and doesn't need to be part of the initial effort to get developers a working django env for development. Initially I think it's worthwhile to just get Vagrant working right, which means we can make reasonable assumptions about the database settings to start.
Updated by mhrivnak over 8 years ago
- Blocks Refactor #2098: create django models for content units added
Updated by semyers over 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to semyers
Updated by semyers over 8 years ago
The existing code in rel-pulp that I'm converting to work in the current pulp codebase to start 3.0-dev is, at the moment, written to be python3 only (largely print statements), which makes it difficult to install in the current pulp dev env. My proposed solution is to bring in print_function from future where needed to get our "new" Django install working in the existing platform virtualenv.
Updated by semyers over 8 years ago
- Status changed from ASSIGNED to POST
There were also some modules that moved in stdlib, so some conditional imports were used. I brought in six in hopes that it would help, but the module in question (py2 abc -> py3 collections.abc) isn't supported out of the box by six. The print statements were being used by some scripts that are, for the moment, nonfunctional in the new pulp 3 base, because the models they were based on have been culled.
I also brought over the db-translation-guide from rel-pulp, with the intention of making its location in the pulp platform project be its canonical location, with any changes made to it henceforth done using our normal peer-review process.
Changes were also necessary to the environment. Most notably, ansible now installs and configures postgres with a 'pulp' superuser. Additionally, DJANGO_SETTINGS_MODULE in bashrc points to the new django installation instead of pulp.server.webservices. Considering that the "old" django install mainly provides the REST API (which we're replacing), and mongoengine models (which we're replacing), this seems like a reasonable change.
jortel is going to be starting with these branches to implement a few good reference models that we've identified, and we're planning to start reviewing these changes presently.
There were shenanigans with the app naming, where Django by default sets an app's label to the final component of its package name if it's got a dotted name, so pulp.platform was being called "platform" by Django. The PulpPlatformConfig seen in apps.py fixes this, to make the pulp.platform app label explicitly be "pulp_platform". We could potentially make it just be "pulp" if desired. Much like the pulp.platform package itself, this bit is easy to change later if we end up wanting something different; this gives us a place to start.
devel changes:
My remotes for devel were screwed up, so I accidentally opened the 3.0-dev branch in there with this commit. Since it's in the pulp namespace I consider this to be a "you can't unring that bell" situation, so any changes made as a result of review will be done on top of this commit (e.g. no push -f to repos in the pulp namespace).
https://github.com/pulp/devel/commit/c88f4a7045a3e61d99d0d73c6133921e8a2f748e
platform changes:
correctly pushed to my fork. It's comparing against master, but once it passes review I'll just push it up as 3.0-dev in the pulp repo. :)
https://github.com/pulp/pulp/compare/master...seandst:3.0-dev
Updated by semyers over 8 years ago
- Blocks Refactor #2109: Load Django DB settings from server.conf added
Updated by semyers over 8 years ago
Opened up the server.conf checklist item as a new issue.
I plan to work with jortel to develop the developer docs for getting started, largely based on his experience getting started. :)
Updated by semyers over 8 years ago
3.0-dev branches are up in platform and devel:
https://github.com/pulp/pulp/tree/3.0-dev
https://github.com/pulp/devel/tree/3.0-dev
jortel and I are planning to add more implemented models to provide useful examples for folks picking up some of the many linked "create models" tasks, and we still need to add some basic docs about setting up the DB and getting Django to actually do stuff with the models being written.
Updated by semyers over 8 years ago
- Assignee deleted (
semyers)
I still haven't created the "minimum viable docs" :(
Since I'm going on PTO, I'm unassigning this in case someone wants to push this over the goal line
Updated by semyers over 8 years ago
This is also loosely related to #2194, in that having fixed bash aliases to manage the db would be useful when creating the "getting started" docs
Updated by bmbouter over 8 years ago
- Status changed from POST to CLOSED - CURRENTRELEASE
I'm closing this because I've moved the only remaining task "minimum viable docs" to its own redmine task as issue #2249 @smyers and I discussed this via IRC prior to doing it.
.