The vagrant environment should use Postgres as a DB
Justification is written here: https://www.redhat.com/archives/pulp-dev/2018-April/msg00204.html
Sqlite falls over very quickly when loaded. Despite the recent change to add a 20 second timeout, Pulp still frequently gets "OperationalError: database is locked" errors when handling a couple dozen to a couple hundred packages, with the liklihood of a sync failure due to db lock being almost 100% after about 300 packages.
This is immediately apparent with the Python plugin, which downloads every release package for a provided project name by default.
In practical terms, syncing the "Django" and "pulpcore" projects works about half the time, and syncing "Django", "pulpcore", and "scipy" usually fails.
This makes developing some plugins, such as the Python plugin, particularly difficult. We can keep sqlite support for very simple plugin development but we shouldn't leave it the default in our own development environment.
#5 Updated by bmbouter over 2 years ago
- Tags Dev Environment added
- Tags deleted (
Pulp 3 MVP, Pulp 3 installer)
Couldn't it be more frustrating if a workload runs well in the dev environment, but fails on Travis due to sqlite CI? I think having it fail directly in front of the developer would actually be better.
Retagging to match that this issue is specific to a Pulp3 dev environment.
#6 Updated by dalley over 2 years ago
I think it's still the other way around because it's much easier to switch from postgres to sqlite than to switch from sqlite to postgres.
And if we reach a point where our ** smoke tests are failing on sqlite due to sqlite limitations rather than logic flaws, we should seriously reconsider supporting sqlite to begin with.
#7 Updated by bmbouter over 2 years ago
Each time we've hit a problem with sqlite it's been an opportunity for us to improve Pulp's ability to run in more places. With the recent sqlite timing changes, I don't know of any current workflows that raise OperationalError. If Pulp+sqlite is broken somehow and it is unfixable that would be very useful info.
#8 Updated by firstname.lastname@example.org over 2 years ago
The problem seems to be that even though SQLite supports a better concurrency model already, Django does not take advantage of it.
The Django issue has a suggestion to use the 'connection_created' signal to make Pulp use sqlite with the WAL mode. We should probably do exactly that.
#9 Updated by bmbouter over 2 years ago
- Description updated (diff)
We need to switch back to postgreSQL. I outlined the reasons why here: https://www.redhat.com/archives/pulp-dev/2018-April/msg00204.html
Please register to edit this issue