Project

Profile

Help

Issue #9451

closed

Set bach_size to mitigate query result exceeding 16mb

Added by ipanova@redhat.com over 2 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 106
Quarter:

Description

pre-migration of errata fails

Starting task.
2021-08-31 15:01:37 +1000: Pre-migrating Pulp 2 erratum content (detail info) 601855/602875


pulpcore-worker-4: RuntimeWarning)
pulpcore-worker-4: pulp: rq.worker:ERROR: Traceback (most recent call last):
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
pulpcore-worker-4: rv = job.perform()
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
pulpcore-worker-4: self._result = self._execute()
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
pulpcore-worker-4: return self.func(*self.args, **self.kwargs)
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 77, in migrate_from_pulp2
pulpcore-worker-4: pre_migrate_all_content(plan)
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 70, in pre_migrate_all_content
pulpcore-worker-4: pre_migrate_content_type(content_model, mutable_type, lazy_type, premigrate_hook)
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 300, in pre_migrate_content_type
pulpcore-worker-4: record.id: record for record in mongo_content_qs.only(*mongo_fields).no_cache()
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 299, in <dictcomp>
pulpcore-worker-4: pulp2_content_by_id = {
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/mongoengine/queryset/base.py", line 1590, in __next__
pulpcore-worker-4: raw_doc = next(self._cursor)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1207, in next
pulpcore-worker-4: if len(self.__data) or self._refresh():
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1124, in _refresh
pulpcore-worker-4: self.__send_message(q)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1001, in __send_message
pulpcore-worker-4: address=self.__address)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
pulpcore-worker-4: exhaust=exhaust)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
pulpcore-worker-4: return func(session, server, sock_info, slave_ok)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
pulpcore-worker-4: unpack_res)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/server.py", line 116, in run_operation_with_response
pulpcore-worker-4: sock_info.send_message(data, max_doc_size)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/pool.py", line 711, in send_message
pulpcore-worker-4: (max_doc_size, self.max_bson_size))
pulpcore-worker-4: pymongo.errors.DocumentTooLarge: BSON document too large (32444638 bytes) - the connected server supports BSON document sizes up to 16777216 bytes.

Related issues

Copied to Migration Plugin - Backport #9482: Backport "Set bach_size to mitigate query result exceeding 16mb" to 0.11.zCLOSED - CURRENTRELEASE

Actions

Added by ipanova@redhat.com over 2 years ago

Revision dde5a63b | View on GitHub

Added batch_size to a pulp 2 query for content pre-migration of errata.

closes #9451

Added by ipanova@redhat.com over 2 years ago

Revision dde5a63b | View on GitHub

Added batch_size to a pulp 2 query for content pre-migration of errata.

closes #9451

Added by ipanova@redhat.com over 2 years ago

Revision dde5a63b | View on GitHub

Added batch_size to a pulp 2 query for content pre-migration of errata.

closes #9451

Actions #2

Updated by ipanova@redhat.com over 2 years ago

  • Status changed from POST to MODIFIED
Actions #3

Updated by ttereshc over 2 years ago

  • Copied to Backport #9482: Backport "Set bach_size to mitigate query result exceeding 16mb" to 0.11.z added
Actions #5

Updated by ttereshc over 2 years ago

  • Sprint/Milestone set to 0.15.0
Actions #6

Updated by pulpbot over 2 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF