Project

Profile

Help

Issue #3456

OperationalError: database is locked

Added by daviddavis over 1 year ago. Updated 7 months ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Severity:
2. Medium
Version:
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

Sometimes, we encounter the following error on sqlite:

Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/celery/app/trace.py", line 374, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/travis/build/pulp/pulp/pulpcore/pulpcore/tasking/tasks.py", line 293, in __call__
    return super(UserFacingTask, self).__call__(*args, **kwargs)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/celery/app/trace.py", line 629, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/pulp_file/app/tasks/synchronizing.py", line 72, in synchronize
    for report in changeset.apply():
  File "/home/travis/build/pulp/pulp/plugin/pulpcore/plugin/changeset/main.py", line 196, in apply
    for report in itertools.chain(self._apply_additions(), self._apply_removals()):
  File "/home/travis/build/pulp/pulp/plugin/pulpcore/plugin/changeset/main.py", line 155, in _apply_additions
    self._add_content(content)
  File "/home/travis/build/pulp/pulp/plugin/pulpcore/plugin/changeset/main.py", line 104, in _add_content
    self.repository_version.add_content(content.stored_model)
  File "/home/travis/build/pulp/pulp/pulpcore/pulpcore/app/models/repository.py", line 392, in add_content
    association.save()
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/models/base.py", line 808, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/models/base.py", line 838, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/models/base.py", line 905, in _save_table
    forced_update)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/models/base.py", line 955, in _do_update
    return filtered._update(values) > 0
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/models/query.py", line 664, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1204, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 899, in execute_sql
    raise original_exception
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 889, in execute_sql
    cursor.execute(sql, params)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages/django/db/backends/sqlite3/base.py", line 328, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: database is locked

More info from django:

https://docs.djangoproject.com/en/2.0/ref/databases/#database-is-locked-errors


Related issues

Related to Pulp - Task #3505: The vagrant environment should use Postgres as a DB MODIFIED Actions

Associated revisions

Revision 3537757c View on GitHub
Added by dalley over 1 year ago

Set a 20 second timeout for sqlite

closes #3456
https://pulp.plan.io/issues/3456

Revision 3537757c View on GitHub
Added by dalley over 1 year ago

Set a 20 second timeout for sqlite

closes #3456
https://pulp.plan.io/issues/3456

Revision 3537757c View on GitHub
Added by dalley over 1 year ago

Set a 20 second timeout for sqlite

closes #3456
https://pulp.plan.io/issues/3456

History

#1 Updated by amacdona@redhat.com over 1 year ago

What situations does this happen? Can it be avoided by "being gentle"?

I see 2 options:

1. Accept that developing Pulp is inherently not "lightweight" and stop claiming to support SQLite.
2. Add something like this to the contributor documentation. "SQLite should be used for lightweight development, but may fail when running concurrent tasks. If you run into this problem, it is time to switch to Postgres."

#2 Updated by dalley over 1 year ago

  • Project changed from File Support to Pulp
  • Triaged changed from No to Yes

We should try increasing the timeout first.

#3 Updated by dalley over 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley

#4 Updated by dalley over 1 year ago

  • Status changed from ASSIGNED to POST

#5 Updated by dalley over 1 year ago

  • Status changed from POST to MODIFIED

#6 Updated by dalley over 1 year ago

  • Related to Task #3505: The vagrant environment should use Postgres as a DB added

#7 Updated by dalley over 1 year ago

  • Status changed from MODIFIED to NEW

Reopening because this is still a big problem with certain plugins like the Python plugin, which downloads every release package for each project specified.

Syncing only a couple of projects from PyPI will fail due to this issue.

#8 Updated by dalley over 1 year ago

  • Assignee deleted (dalley)

#9 Updated by kersom over 1 year ago

I have been found this problem a few times when running smash tests.

[root@f27-pulp3 ~]# journalctl -xe
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return Database.Cursor.execute(self, query, params)
Apr 03 13:24:18 f27-pulp3 celery[3149]: sqlite3.OperationalError: database is locked
Apr 03 13:24:18 f27-pulp3 celery[3149]: The above exception was the direct cause of the following exception:
Apr 03 13:24:18 f27-pulp3 celery[3149]: Traceback (most recent call last):
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/celery/app/trace.py", line 374, in trace_task
Apr 03 13:24:18 f27-pulp3 celery[3149]:     R = retval = fun(*args, **kwargs)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/devel/pulp/pulpcore/pulpcore/tasking/tasks.py", line 273, in __call__
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return super().__call__(*args, **kwargs)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/celery/app/trace.py", line 629, in __protected_call__
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return self.run(*args, **kwargs)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/pulp_file/app/tasks/synchronizing.py", line 72, in synchronize
Apr 03 13:24:18 f27-pulp3 celery[3149]:     for report in changeset.apply():
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/changeset/main.py", line 196, in apply
Apr 03 13:24:18 f27-pulp3 celery[3149]:     for report in itertools.chain(self._apply_additions(), self._apply_removals()):
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/changeset/main.py", line 155, in _apply_additions
Apr 03 13:24:18 f27-pulp3 celery[3149]:     self._add_content(content)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/changeset/main.py", line 104, in _add_content
Apr 03 13:24:18 f27-pulp3 celery[3149]:     self.repository_version.add_content(content.stored_model)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/devel/pulp/pulpcore/pulpcore/app/models/repository.py", line 394, in add_content
Apr 03 13:24:18 f27-pulp3 celery[3149]:     association.save()
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 729, in save
Apr 03 13:24:18 f27-pulp3 celery[3149]:     force_update=force_update, update_fields=update_fields)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 759, in save_base
Apr 03 13:24:18 f27-pulp3 celery[3149]:     updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 823, in _save_table
Apr 03 13:24:18 f27-pulp3 celery[3149]:     forced_update)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 872, in _do_update
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return filtered._update(values) > 0
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 709, in _update
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return query.get_compiler(self.db).execute_sql(CURSOR)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
Apr 03 13:24:18 f27-pulp3 celery[3149]:     cursor = super().execute_sql(result_type)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1066, in execute_sql
Apr 03 13:24:18 f27-pulp3 celery[3149]:     cursor.execute(sql, params)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return super().execute(sql, params)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return executor(sql, params, many, context)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return self.cursor.execute(sql, params)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Apr 03 13:24:18 f27-pulp3 celery[3149]:     raise dj_exc_value.with_traceback(traceback) from exc_value
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return self.cursor.execute(sql, params)
Apr 03 13:24:18 f27-pulp3 celery[3149]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 303, in execute
Apr 03 13:24:18 f27-pulp3 celery[3149]:     return Database.Cursor.execute(self, query, params)
Apr 03 13:24:18 f27-pulp3 celery[3149]: django.db.utils.OperationalError: database is locked
Apr 03 13:25:51 f27-pulp3 audit[1501]: USER_START pid=1501 uid=0 auid=0 ses=2 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=? addr=192.168.122.1 terminal=/dev/pts/1 res=success'
Apr 03 13:25:51 f27-pulp3 audit[1501]: CRYPTO_KEY_USER pid=1501 uid=0 auid=0 ses=2 msg='op=destroy kind=server fp=SHA256:e2:23:f1:b4:55:25:b5:c5:b9:6c:91:db:28:3b:7a:15:c8:26:fd:e7:f3:ea:07:98:46:55:2b:6e:e0:05:

#10 Updated by daviddavis over 1 year ago

  • Status changed from NEW to MODIFIED
  • Assignee set to dalley

#11 Updated by dalley over 1 year ago

  • Status changed from MODIFIED to NEW

A significant percentage of our travis jobs are hitting this.

#12 Updated by bmbouter over 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee changed from dalley to bmbouter

At the meeting today we determined we want some more info on this issue. I'm taking as ASSIGNED so I can investigate and post what I find. With more info we can determine if it's possible to run Pulp on sqlite without error.

@dalley, I am hoping to work w/ you to analyze whatever I find.

#13 Updated by dkliban@redhat.com over 1 year ago

  • Sprint/Milestone set to 3.0

#14 Updated by bmbouter over 1 year ago

We aren't able to fix this issue adequately. A description of why is here: https://www.redhat.com/archives/pulp-dev/2018-April/msg00204.html

I'm moving to CLOSED - WONTFIX since we can't. We are switching to PostgreSQL: https://pulp.plan.io/issues/3606

#15 Updated by bmbouter over 1 year ago

  • Status changed from ASSIGNED to CLOSED - WONTFIX

#16 Updated by bmbouter 7 months ago

  • Tags deleted (Pulp 3, Pulp 3 MVP)

Please register to edit this issue

Also available in: Atom PDF