Project

Profile

Help

Issue #4013

closed

UpdateRecord index row size is too large

Added by ttereshc over 5 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 43
Quarter:

Description

To reproduce sync EPEL repo https://dl.fedoraproject.org/pub/epel/7/x86_64/
Index for the UpdateRecord: https://github.com/pulp/pulp_rpm/blob/cf68be94f7b758faa24948e68f5050dedc4b4d33/pulp_rpm/app/models.py#L330

Sep 14 16:19:47 pulp3.dev rq[9581]: Traceback (most recent call last):
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
Sep 14 16:19:47 pulp3.dev rq[9581]:     return self.cursor.execute(sql, params)
Sep 14 16:19:47 pulp3.dev rq[9581]: psycopg2.OperationalError: index row size 3656 exceeds maximum 2712 for index "pulp_rpm_updaterecord_errata_id_updated_date_d_cc03c752_uniq"
Sep 14 16:19:47 pulp3.dev rq[9581]: HINT:  Values larger than 1/3 of a buffer page cannot be indexed.
Sep 14 16:19:47 pulp3.dev rq[9581]: Consider a function index of an MD5 hash of the value, or use full text indexing.
Sep 14 16:19:47 pulp3.dev rq[9581]: The above exception was the direct cause of the following exception:
Sep 14 16:19:47 pulp3.dev rq[9581]: Traceback (most recent call last):
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/worker.py", line 793, in perform_job
Sep 14 16:19:47 pulp3.dev rq[9581]:     rv = job.perform()
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/job.py", line 599, in perform
Sep 14 16:19:47 pulp3.dev rq[9581]:     self._result = self._execute()
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/job.py", line 605, in _execute
Sep 14 16:19:47 pulp3.dev rq[9581]:     return self.func(*self.args, **self.kwargs)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py", line 69, in synchronize
Sep 14 16:19:47 pulp3.dev rq[9581]:     loop.run_until_complete(pipeline)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/usr/lib64/python3.6/asyncio/base_events.py", line 468, in run_until_complete
Sep 14 16:19:47 pulp3.dev rq[9581]:     return future.result()
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/stages/api.py", line 58, in create_pipeline
Sep 14 16:19:47 pulp3.dev rq[9581]:     await asyncio.gather(*futures)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/stages/content_unit_stages.py", line 152, in __call__
Sep 14 16:19:47 pulp3.dev rq[9581]:     declarative_content.content.save()
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/devel/pulp/pulpcore/pulpcore/app/models/base.py", line 83, in save
Sep 14 16:19:47 pulp3.dev rq[9581]:     return super().save(*args, **kwargs)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 718, in save
Sep 14 16:19:47 pulp3.dev rq[9581]:     force_update=force_update, update_fields=update_fields)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 748, in save_base
Sep 14 16:19:47 pulp3.dev rq[9581]:     updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 831, in _save_table
Sep 14 16:19:47 pulp3.dev rq[9581]:     result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 869, in _do_insert
Sep 14 16:19:47 pulp3.dev rq[9581]:     using=using, raw=raw)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Sep 14 16:19:47 pulp3.dev rq[9581]:     return getattr(self.get_queryset(), name)(*args, **kwargs)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1136, in _insert
Sep 14 16:19:47 pulp3.dev rq[9581]:     return query.get_compiler(using=using).execute_sql(return_id)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1289, in execute_sql
Sep 14 16:19:47 pulp3.dev rq[9581]:     cursor.execute(sql, params)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
Sep 14 16:19:47 pulp3.dev rq[9581]:     return super().execute(sql, params)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
Sep 14 16:19:47 pulp3.dev rq[9581]:     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
Sep 14 16:19:47 pulp3.dev rq[9581]:     return executor(sql, params, many, context)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
Sep 14 16:19:47 pulp3.dev rq[9581]:     return self.cursor.execute(sql, params)
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Sep 14 16:19:47 pulp3.dev rq[9581]:     raise dj_exc_value.with_traceback(traceback) from exc_value
Sep 14 16:19:47 pulp3.dev rq[9581]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
Sep 14 16:19:47 pulp3.dev rq[9581]:     return self.cursor.execute(sql, params)
Sep 14 16:19:47 pulp3.dev rq[9581]: django.db.utils.OperationalError: index row size 3656 exceeds maximum 2712 for index "pulp_rpm_updaterecord_errata_id_updated_date_d_cc03c752_uniq"
Sep 14 16:19:47 pulp3.dev rq[9581]: HINT:  Values larger than 1/3 of a buffer page cannot be indexed.
Sep 14 16:19:47 pulp3.dev rq[9581]: Consider a function index of an MD5 hash of the value, or use full text indexing.
Actions #1

Updated by daviddavis over 5 years ago

I wonder if we can just use digest.

Actions #2

Updated by ttereshc over 5 years ago

  • Sprint/Milestone set to Pulp 3 RPM MVP
  • Triaged changed from No to Yes
  • Sprint set to Sprint 43
Actions #3

Updated by ttereshc over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc

Added by ttereshc over 5 years ago

Revision 467da737 | View on GitHub

Use digest for uniqueness constraint for UpdateRecord

closes #4013 https://pulp.plan.io/issues/4013

Actions #4

Updated by ttereshc over 5 years ago

  • Status changed from ASSIGNED to POST
Actions #5

Updated by ttereshc over 5 years ago

  • Status changed from POST to MODIFIED
Actions #6

Updated by bmbouter almost 5 years ago

  • Tags deleted (Pulp 3)
Actions #7

Updated by ttereshc over 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF