Issue #3854: Mongo cursor times out during task pulp.server.managers.content.orphan.delete_all_orphans - Pulp

Actions

Send by e-mail Copy link

Issue #3854

closed

Mongo cursor times out during task pulp.server.managers.content.orphan.delete_all_orphans

Added by daviddavis almost 6 years ago. Updated about 5 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

Category:

Sprint/Milestone:

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version:

Platform Release:

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Easy Fix, Pulp 2

Sprint:

Sprint 40

Quarter:

Description

In #2584, we attempted to fix a cursor timeout during orphan delete but it appears to still be happening:

celery.worker.job:ERROR: (42386-18048) Task pulp.server.managers.content.orphan.delete_all_orphans[e7ace470-061d-434f-a269-24781a09cb64] raised unexpected: CursorNotFound('Cursor not found, cursor id: 190495872103',)
celery.worker.job:ERROR: (42386-18048) Traceback (most recent call last):
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
celery.worker.job:ERROR: (42386-18048)     R = retval = fun(*args, **kwargs)
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 527, in __call__
celery.worker.job:ERROR: (42386-18048)     return super(Task, self).__call__(*args, **kwargs)
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
celery.worker.job:ERROR: (42386-18048)     return super(PulpTask, self).__call__(*args, **kwargs)
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
celery.worker.job:ERROR: (42386-18048)     return self.run(*args, **kwargs)
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/orphan.py", line 173, in delete_all_orphans
celery.worker.job:ERROR: (42386-18048)     count = OrphanManager.delete_orphans_by_type(content_type_id)
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/orphan.py", line 236, in delete_orphans_by_type
celery.worker.job:ERROR: (42386-18048)     for content_unit in OrphanManager.generate_orphans_by_type(content_type_id, fields=fields):
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/content/orphan.py", line 106, in generate_orphans_by_type
celery.worker.job:ERROR: (42386-18048)     for content_unit in content_units_collection.find({}, projection=fields).batch_size(100):
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1097, in next
celery.worker.job:ERROR: (42386-18048)     if len(self.__data) or self._refresh():
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 1039, in _refresh
celery.worker.job:ERROR: (42386-18048)     self.__max_await_time_ms))
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 903, in __send_message
celery.worker.job:ERROR: (42386-18048)     codec_options=self.__codec_options)
celery.worker.job:ERROR: (42386-18048)   File "/usr/lib64/python2.7/site-packages/pymongo/helpers.py", line 114, in _unpack_response
celery.worker.job:ERROR: (42386-18048)     raise CursorNotFound(msg, 43, errobj)
celery.worker.job:ERROR: (42386-18048) CursorNotFound: Cursor not found, cursor id: 190495872103

This was on a system running no other tasks. Total was about 7650 orphans.

Related issues