Project

Profile

Help

Story #3359

Anyone should have the option to install Pulp with SQLite3 as the default DB

Added by milan almost 2 years ago. Updated 8 months ago.

Status:
MODIFIED
Priority:
Normal
Assignee:
Category:
API
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Platform Release:
3.0.0
Blocks Release:
Backwards Incompatible:
No
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 33

Description

Based on the Plugin Writer's Coding Workshop Feedback, users requested SQLite3 be the default database type.

  • folks might feel strong about what additional services have to run on their (personal) systems
  • smaller production deployments out there might be OK with SQLite
  • plug-in writers don't necessarily have to run their code against Postgresql (right away)
  • to have the flexibility in choosing a DB during installation, based on target deployment environment

The deliverables are outlined in checklist items.


Checklist

Associated revisions

Revision 5575dcae View on GitHub
Added by bmbouter almost 2 years ago

Switch Pulp to use sqlite3 as default

https://pulp.plan.io/issues/3359
closes #3359

Revision 0ab78243 View on GitHub
Added by bmbouter almost 2 years ago

Enables Travis testing

This PR enables testing on both postgresql and sqlite.

Thanks to @daviddavis for the .travis file changes!

re #3359

Revision 5575dcae View on GitHub
Added by bmbouter almost 2 years ago

Switch Pulp to use sqlite3 as default

https://pulp.plan.io/issues/3359
closes #3359

Revision 0ab78243 View on GitHub
Added by bmbouter almost 2 years ago

Enables Travis testing

This PR enables testing on both postgresql and sqlite.

Thanks to @daviddavis for the .travis file changes!

re #3359

Revision 4f3cb542 View on GitHub
Added by bmbouter almost 2 years ago

Sqlite3 and some install fixes

This commit updates the database docs to identify sqlite3 as the default
database and recommend that users use Postgresql in production. It also
links to the Django docs for more info on the settings.

Additionally, it updates the source and pypi install instructions in the
following ways:
  • fixes some errors
  • renames {venv} to pulpvenv for clarity
  • adds a helper 1-liner to generate the SECRET_KEY line
  • updates the PULP_SETTINGS to use python3.6 instead of {python_version}

https://pulp.plan.io/issues/3359
re #3359

Revision 4f3cb542 View on GitHub
Added by bmbouter almost 2 years ago

Sqlite3 and some install fixes

This commit updates the database docs to identify sqlite3 as the default
database and recommend that users use Postgresql in production. It also
links to the Django docs for more info on the settings.

Additionally, it updates the source and pypi install instructions in the
following ways:
  • fixes some errors
  • renames {venv} to pulpvenv for clarity
  • adds a helper 1-liner to generate the SECRET_KEY line
  • updates the PULP_SETTINGS to use python3.6 instead of {python_version}

https://pulp.plan.io/issues/3359
re #3359

Revision fe68c0a8 View on GitHub
Added by bmbouter almost 2 years ago

Adding sqlite binary to dev env

The `dbshell` command wasn't working because it requires the binary
client. This will ensure that on dev installs at least that will work.

https://pulp.plan.io/issues/3359
re #3359

History

#1 Updated by bmbouter almost 2 years ago

  • Tags Pulp 3 added
  • Tags deleted (Dev Environment)

I removed the dev environment because the default would be for the codebase altogether which would make postgresql a soft-dependency. This would do great things for Pulp's cross-distro portability and the perception that Pulp is "too big" to install for them.

The Pulp3 plugin developers and users we talked indicated a strong preference for sqlite3 as the default db. They gave a few reasons:

1) you don't have to install something extra. In practice this took more time and more configuration. A lot of users are doing this on their personal system and feel strongly about what runs on their system so they chose to run postgresql in a container which was cool, but took more time. Several people expressed this view. One user at the booth said that postgresql is a deal-breaker.

2) Many users are small and they don't mind the slower, simpler db

3) plugin writer's only need a db to see if their code works so a simpler experience benefits them greatly.

4) Users who want postgresql can enable it easily and already want to install postgresql. The Ansible installer can too.

#2 Updated by pcreech almost 2 years ago

I know i'm not really involved in pulp's day to day anymore, but I may have some info relevant here:

When I was involved in the data modeling effort, great care was taken to write the models in a manner that would hopefully allow any of the supported django orm database providers to be used with the app in the environment users chose. If this same care has been used since, it hopefully shouldn't be to hard to just allow other django orm providers to act as the backing database.

I will say caution should probably be used though in determining which databases are supported in a production environment. Databases are fickle creatures, and each behaves differently under stress situations. For this particular issue, this link may be of help in determining the support level of sqlite databases.

https://www.sqlite.org/whentouse.html

#3 Updated by milan almost 2 years ago

  • Subject changed from As a (plug-in) developer I have the option to install Pulp with SQLite3 as the default DB to Anyone should have the option to install Pulp with SQLite3 as the default DB
  • Description updated (diff)

#4 Updated by jortel@redhat.com almost 2 years ago

Sqlite3 is a fine DB for users who want to try pulp and for plugin development environments. @pcreech made some good points and linked a good guide for users to make the proper DB choice. The issue as written describes what users want but needs clarity regarding the specific changes being proposed and what the deliverables are. +1 moving forward on this.

#5 Updated by bmbouter almost 2 years ago

Thanks for all the responses. To clarify the details of the change, I made this example PR here: https://github.com/pulp/pulp/pull/3344/files

The right db for a production install is an important choice. Users in production environments need to think about what is the right database for them. Django supports 4 databases directly and 5 more through 3rd party support. https://docs.djangoproject.com/en/1.11/ref/databases/

I expect that the Ansible installer will allow a user to choose between sqlite and postgresql in terms of what it can configure. The defaults for the Ansible roles for Pulp is also something we should figure out, but that is a separate discussion specifically for the installer.

#6 Updated by bmbouter almost 2 years ago

  • Description updated (diff)

#7 Updated by dkliban@redhat.com almost 2 years ago

  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes

#8 Updated by jortel@redhat.com almost 2 years ago

  • Sprint/Milestone set to 56

#9 Updated by bmbouter almost 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bmbouter

#10 Updated by bmbouter almost 2 years ago

  • Status changed from ASSIGNED to MODIFIED
  • % Done changed from 0 to 100

#11 Updated by bmbouter almost 2 years ago

  • Status changed from MODIFIED to POST

Pulling back to POST because another PR needs to be done which will update the docs for this change.

#12 Updated by bmbouter almost 2 years ago

  • Status changed from POST to MODIFIED

This is fully completed!

#13 Updated by bmbouter almost 2 years ago

  • Sprint set to Sprint 33

#14 Updated by bmbouter almost 2 years ago

  • Sprint/Milestone deleted (56)

#15 Updated by daviddavis 8 months ago

  • Sprint/Milestone set to 3.0

#16 Updated by bmbouter 8 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF