Project

Profile

Help

Issue #3893

closed

pulp.server.maintenance.monthly fails if applicability collection is too large

Added by ttereshc almost 6 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.17.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 41
Quarter:

Description

Jul 31 16:04:24 pulp: celery.worker.job:INFO: Task pulp.server.maintenance.monthly.queue_monthly_maintenance[d00b6ddf-6f3a-42dc-8b72-690513c7ce5a] succeeded in 0.293885109015s: None

Jul 31 16:04:29 pulp: celery.worker.job:ERROR: (28015-45440) Task pulp.server.maintenance.monthly.monthly_maintenance[8de6e9ec-b998-4581-acf8-29d3810c0f3f] raised unexpected:
OperationFailure(u"command SON ([('distinct', u'repo_profile_applicability'), ('key', 'profile_hash')]) on namespace pulp_database.$cmd failed: exception: distinct too big, 16mb cap",)

Jul 31 16:04:29 chlp2550341a pulp: celery.worker.job:ERROR: (28015-45440) File "/usr/lib/python2.7/site-packages/pulp/server/maintenance/monthly.py", line 22, in monthly_maintenance

Actions #2

Updated by CodeHeeler almost 6 years ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 41

Added by ttereshc almost 6 years ago

Revision 8a043dbd | View on GitHub

Use aggregation pipeline to avoid BSON 16MB size limit

Results are written into a separate collection which is reused every time the cleanup is run. If there are millions of orphaned applicability profiles, it's easy to exceed 16MB.

closes #3893 https://pulp.plan.io/issues/3893

Actions #4

Updated by ttereshc almost 6 years ago

  • Status changed from ASSIGNED to MODIFIED
Actions #5

Updated by dkliban@redhat.com almost 6 years ago

  • Platform Release set to 2.17.1

Added by ttereshc almost 6 years ago

Revision cbbe5faf | View on GitHub

Use aggregation pipeline to avoid BSON 16MB size limit

Results are written into a separate collection which is reused every time the cleanup is run. If there are millions of orphaned applicability profiles, it's easy to exceed 16MB.

closes #3893 https://pulp.plan.io/issues/3893

(cherry picked from commit 8a043dbd10e418f3460e7fa3bd97651d71517050)

Actions #7

Updated by ttereshc almost 6 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #8

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF