Project

Profile

Help

Issue #3148

dupicate key error on task_status collection with mongoengine 0.14

Added by ttereshc about 4 years ago. Updated almost 3 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Reported by user s0lar.
Reproducible 100% (fwiw, pulp installation on SUSE) with mongoengine 0.14. Downgrading mongoengine to 0.11 solved the issue.
To reproduce just create a repo and trigger a sync:

pulp-admin -vv rpm repo create --repo-id legacy --serve-http true --feed http://smt/repo/SUSE/Products/SLE-Module-Legacy/12/x86_64/product/
pulp-admin rpm repo sync run --repo-id legacy
pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [0341bd74] Sync complete.
pulp: amqp:DEBUG: Start from server, version: 0.9, properties: {'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'product': 'RabbitMQ', 'copyright': 'Copyright (C) 2007-2017 Pivotal Software, Inc.', 'capabilities': {'exchange_exchange_bindings': True, 'connection.blocked': True, 'authentication_failure_close': True, 'direct_reply_to': True, 'basic.nack': True, 'per_consumer_qos': True, 'consumer_priorities': True, 'consumer_cancel_notify': True, 'publisher_confirms': True}, 'cluster_name': 'rabbit@localhost', 'platform': 'Erlang/OTP 20.0.5', 'version': '3.6.11'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: [u'en_US']
pulp: amqp:DEBUG: using channel_id: 1
pulp: amqp:DEBUG: Channel open
pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[43acade8-86eb-4613-b290-9912d03ea3f7]  
pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[5f2d1af9-6bcc-4771-988f-ef756880774d]  
pulp: pulp.server.async.tasks:DEBUG: [0341bd74] Task successful : [0341bd74-da40-4d59-9152-602381a13727]
pulp: py.warnings:WARNING: /opt/venv/pulp/lib/python2.7/site-packages/celery/app/trace.py:549: RuntimeWarning: Exception raised outside body: NotUniqueError(u'Tried to save duplicate unique keys (E11000 duplicate key error collection: pulp_database.task_status index: task_id_-1 dup key: { : "0341bd74-da40-4d59-9152-602381a13727" })',):
Traceback (most recent call last):
  File "/opt/venv/pulp/lib/python2.7/site-packages/celery/app/trace.py", line 442, in trace_task
    task_on_success(retval, uuid, args, kwargs)
  File "/opt/venv/pulp/lib/python2.7/site-packages/pulp_server-2.14.1-py2.7.egg/pulp/server/async/tasks.py", line 577, in on_success
    task_status.save()
  File "/opt/venv/pulp/lib/python2.7/site-packages/mongoengine/document.py", line 397, in save
    raise NotUniqueError(message % six.text_type(err))
NotUniqueError: Tried to save duplicate unique keys (E11000 duplicate key error collection: pulp_database.task_status index: task_id_-1 dup key: { : "0341bd74-da40-4d59-9152-602381a13727" })

  exc, exc_info.traceback)))

pulp: pulp.server.async.tasks:INFO: Task failed : [0341bd74-da40-4d59-9152-602381a13727]
pulp: celery.app.trace:ERROR: Task pulp.server.managers.repo.sync.sync[0341bd74-da40-4d59-9152-602381a13727] raised unexpected: NotUniqueError(u'Tried to save duplicate unique keys (E11000 duplicate key error collection: pulp_database.task_status index: task_id_-1 dup key: { : "0341bd74-da40-4d59-9152-602381a13727" })',)
Traceback (most recent call last):
  File "/opt/venv/pulp/lib/python2.7/site-packages/celery/app/trace.py", line 442, in trace_task
    task_on_success(retval, uuid, args, kwargs)
  File "/opt/venv/pulp/lib/python2.7/site-packages/pulp_server-2.14.1-py2.7.egg/pulp/server/async/tasks.py", line 577, in on_success
    task_status.save()
  File "/opt/venv/pulp/lib/python2.7/site-packages/mongoengine/document.py", line 397, in save
    raise NotUniqueError(message % six.text_type(err))
NotUniqueError: Tried to save duplicate unique keys (E11000 duplicate key error collection: pulp_database.task_status index: task_id_-1 dup key: { : "0341bd74-da40-4d59-9152-602381a13727" })
pulp: celery.pool:DEBUG: TaskPool: Apply <function _fast_trace_task at 0x7f78a57fbc08> (args:('pulp.server.async.tasks._release_resource', 'fbe9ffb3-58f6-4c5d-b3cf-4eebf976bee9', {'origin': 'gen70214@localhost', 'lang': 'py', 'task': 'pulp.server.async.tasks._release_resource', 'group': None, 'root_id': 'eb6506b7-b50f-408b-bc8b-df85321ee191', u'delivery_info': {u'priority': 0, u'redelivered': False, u'routing_key': u'celery@localhost', u'exchange': u'C.dq2'}, 'expires': None, u'correlation_id': 'fbe9ffb3-58f6-4c5d-b3cf-4eebf976bee9', 'retries': 0, 'timelimit': [None, None], 'argsrepr': "('0341bd74-da40-4d59-9152-602381a13727',)", 'eta': None, 'parent_id': 'eb6506b7-b50f-408b-bc8b-df85321ee191', u'reply_to': 'a8f5e663-6dc9-3001-962c-4d086d4a4753', 'id': 'fbe9ffb3-58f6-4c5d-b3cf-4eebf976bee9', 'kwargsrepr': '{}'}, u'[["0341bd74-da40-4d59-9152-602381a13727"], {}, {"chord": null, "callbacks": null, "errbacks": null, "chain": null}]', 'application/json', 'utf-8') kwargs:{})
pulp: celery.worker.request:DEBUG: Task accepted: pulp.server.async.tasks._release_resource[fbe9ffb3-58f6-4c5d-b3cf-4eebf976bee9] pid:70283
pulp: pulp.server.async.tasks:INFO: [fbe9ffb3] Task failed : [0341bd74-da40-4d59-9152-602381a13727] : Worker terminated abnormally while processing task 0341bd74-da40-4d59-9152-602381a13727.  Check the logs for details
pulp: pulp.server.async.tasks:DEBUG: [fbe9ffb3] None

pulp: celery.app.trace:ERROR: [fbe9ffb3] Task pulp.server.async.tasks._release_resource[fbe9ffb3-58f6-4c5d-b3cf-4eebf976bee9] raised unexpected: AttributeError("'NoneType' object has no attribute 'top'",)
Traceback (most recent call last):
  File "/opt/venv/pulp/lib/python2.7/site-packages/celery/app/trace.py", line 374, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/opt/venv/pulp/lib/python2.7/site-packages/pulp_server-2.14.1-py2.7.egg/pulp/server/async/tasks.py", line 107, in __call__
    return super(PulpTask, self).__call__(*args, **kwargs)
  File "/opt/venv/pulp/lib/python2.7/site-packages/celery/app/trace.py", line 629, in __protected_call__
    return self.run(*args, **kwargs)
  File "/opt/venv/pulp/lib/python2.7/site-packages/pulp_server-2.14.1-py2.7.egg/pulp/server/async/tasks.py", line 296, in _release_resource
    new_task.on_failure(exception, task_id, (), {}, MyEinfo)
  File "/opt/venv/pulp/lib/python2.7/site-packages/pulp_server-2.14.1-py2.7.egg/pulp/server/async/tasks.py", line 602, in on_failure
    if not self.request.called_directly:
  File "/opt/venv/pulp/lib/python2.7/site-packages/celery/app/task.py", line 980, in _get_request
    req = self.request_stack.top
AttributeError: 'NoneType' object has no attribute 'top'
duplicate_key_error_full.log (77.4 KB) duplicate_key_error_full.log full logs reported by s0lar related to this issue ttereshc, 11/23/2017 10:44 PM

Related issues

Related to Pulp - Task #3155: Test Pulp on Fedora 27CLOSED - WONTFIX

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Has duplicate Pulp - Issue #3687: Pulp 2 doesn't work on Fedora 27+CLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

History

#1 Updated by ttereshc about 4 years ago

I've added all logs available if needed (one sync and one publish task).

I think, it would be good to test compatibility with mongoengine 0.14 in general. IIRC, the most recent mongoengine version which has been tested with Pulp is 0.13.

#2 Updated by dalley about 4 years ago

  • Parent task set to #3155
  • Triaged changed from No to Yes

#3 Updated by dalley about 4 years ago

  • Parent task deleted (#3155)

#4 Updated by dalley about 4 years ago

  • Related to Task #3155: Test Pulp on Fedora 27 added

#5 Updated by dalley over 3 years ago

  • Has duplicate Issue #3687: Pulp 2 doesn't work on Fedora 27+ added

#6 Updated by rchan over 3 years ago

  • Status changed from NEW to CLOSED - DUPLICATE

Keeping the newer issue open (#3687) since it has a higher urgency due to impacting fedora and our nightly automation.

#7 Updated by bmbouter almost 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF