amqp-failure.log

Issue #4387 » amqp-failure.log

Failure seen on - bherring, 02/06/2019 09:10 PM

       an 29 16:51:37 localhost.localdomain systemd[1]: Stopping An AMQP message broker daemon....
       Jan 29 16:51:37 localhost.localdomain pulp[31599]: gofer.messaging.adapter.qpid.reliability:WARNING: connection aborted
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256) consumer: Connection to broker lost. Trying to re-establish the connection...
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256) Traceback (most recent call last):
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 317, in start
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     blueprint.start(self)
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     step.start(parent)
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 593, in start
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     c.loop(*c.loop_args())
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/celery/worker/loops.py", line 121, in synloop
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     connection.drain_events(timeout=2.0)
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 301, in drain_events
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     return self.transport.drain_events(self.connection, **kwargs)
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/kombu/transport/qpid.py", line 1693, in drain_events
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     receiver = self.session.next_receiver(timeout=timeout)
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "<string>", line 6, in next_receiver
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 777, in next_receiver
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     if self._ecwait(lambda: self.incoming, timeout):
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104) consumer: Connection to broker lost. Trying to re-establish the connection...
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 57, in _ecwait
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104) Traceback (most recent call last):
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     result = self._ewait(lambda: self.closed or predicate(), timeout)
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 317, in start
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     blueprint.start(self)
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 637, in _ewait
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     step.start(parent)
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     result = self.connection._ewait(lambda: self.error or predicate(), timeout)
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 593, in start
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     c.loop(*c.loop_args())
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/celery/worker/loops.py", line 121, in synloop
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 274, in _ewait
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     connection.drain_events(timeout=2.0)
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 301, in drain_events
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     self.check_error()
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     return self.transport.drain_events(self.connection, **kwargs)
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/kombu/transport/qpid.py", line 1693, in drain_events
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 267, in check_error
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     receiver = self.session.next_receiver(timeout=timeout)
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "<string>", line 6, in next_receiver
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256)     raise e
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 777, in next_receiver
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:WARNING: (31580-24256) ConnectionError: connection aborted
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     if self._ecwait(lambda: self.incoming, timeout):
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 57, in _ecwait
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     result = self._ewait(lambda: self.closed or predicate(), timeout)
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 637, in _ewait
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     result = self.connection._ewait(lambda: self.error or predicate(), timeout)
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 274, in _ewait
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     self.check_error()
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 267, in check_error
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104)     raise e
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:WARNING: (31593-63104) ConnectionError: connection aborted
       Jan 29 16:51:37 localhost.localdomain pulp[31598]: gofer.messaging.adapter.qpid.reliability:WARNING: connection aborted
       Jan 29 16:51:37 localhost.localdomain pulp[31597]: gofer.messaging.adapter.qpid.reliability:WARNING: connection aborted
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576) consumer: Connection to broker lost. Trying to re-establish the connection...
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576) Traceback (most recent call last):
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 317, in start
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     blueprint.start(self)
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     step.start(parent)
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 593, in start
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     c.loop(*c.loop_args())
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/celery/worker/loops.py", line 121, in synloop
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     connection.drain_events(timeout=2.0)
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 301, in drain_events
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     return self.transport.drain_events(self.connection, **kwargs)
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/kombu/transport/qpid.py", line 1693, in drain_events
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     receiver = self.session.next_receiver(timeout=timeout)
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "<string>", line 6, in next_receiver
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 777, in next_receiver
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     if self._ecwait(lambda: self.incoming, timeout):
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 57, in _ecwait
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     result = self._ewait(lambda: self.closed or predicate(), timeout)
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 637, in _ewait
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     result = self.connection._ewait(lambda: self.error or predicate(), timeout)
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 274, in _ewait
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     self.check_error()
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 267, in check_error
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576)     raise e
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:WARNING: (31588-48576) ConnectionError: connection aborted
       Jan 29 16:51:37 localhost.localdomain qpidd[30189]: 2019-01-29 16:51:37 [Broker] notice Broker (pid=30189) shut-down
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 104] Connection reset by peer.
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104) Trying again in 2.00 seconds...
       Jan 29 16:51:37 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104)
       Jan 29 16:51:37 localhost.localdomain systemd[1]: Stopped An AMQP message broker daemon..
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 104] Connection reset by peer.
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576) Trying again in 2.00 seconds...
       Jan 29 16:51:37 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576)
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 104] Connection reset by peer.
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256) Trying again in 2.00 seconds...
       Jan 29 16:51:37 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256)
       Jan 29 16:51:39 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:39 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104) Trying again in 4.00 seconds...
       Jan 29 16:51:39 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104)
       Jan 29 16:51:39 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:39 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576) Trying again in 4.00 seconds...
       Jan 29 16:51:39 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576)
       Jan 29 16:51:39 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:39 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256) Trying again in 4.00 seconds...
       Jan 29 16:51:39 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256)
       Jan 29 16:51:43 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:43 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104) Trying again in 6.00 seconds...
       Jan 29 16:51:43 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104)
       Jan 29 16:51:43 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:43 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576) Trying again in 6.00 seconds...
       Jan 29 16:51:43 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576)
       Jan 29 16:51:43 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:43 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256) Trying again in 6.00 seconds...
       Jan 29 16:51:43 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256)
       Jan 29 16:51:47 localhost.localdomain pulp[31599]: gofer.messaging.adapter.connect:INFO: connecting: qpid+ssl://192.168.122.46:5671
       Jan 29 16:51:47 localhost.localdomain pulp[31599]: gofer.messaging.adapter.qpid.connection:INFO: open: URL: ssl://192.168.122.46|SSL: ca: /etc/pki/pulp/ca.crt|key: None|certificate: /etc/pki/qpid/client.crt|host-validation: False
       Jan 29 16:51:47 localhost.localdomain pulp[31599]: gofer.messaging.adapter.connect:ERROR: connect: qpid+ssl://192.168.122.46:5671, failed: [Errno 111] Connection refused
       Jan 29 16:51:47 localhost.localdomain pulp[31599]: gofer.messaging.adapter.connect:INFO: retry in 10 seconds
       Jan 29 16:51:47 localhost.localdomain pulp[31598]: gofer.messaging.adapter.connect:INFO: connecting: qpid+ssl://192.168.122.46:5671
       Jan 29 16:51:47 localhost.localdomain pulp[31598]: gofer.messaging.adapter.qpid.connection:INFO: open: URL: ssl://192.168.122.46|SSL: ca: /etc/pki/pulp/ca.crt|key: None|certificate: /etc/pki/qpid/client.crt|host-validation: False
       Jan 29 16:51:47 localhost.localdomain pulp[31598]: gofer.messaging.adapter.connect:ERROR: connect: qpid+ssl://192.168.122.46:5671, failed: [Errno 111] Connection refused
       Jan 29 16:51:47 localhost.localdomain pulp[31597]: gofer.messaging.adapter.connect:INFO: connecting: qpid+ssl://192.168.122.46:5671
       Jan 29 16:51:47 localhost.localdomain pulp[31597]: gofer.messaging.adapter.qpid.connection:INFO: open: URL: ssl://192.168.122.46|SSL: ca: /etc/pki/pulp/ca.crt|key: None|certificate: /etc/pki/qpid/client.crt|host-validation: False
       Jan 29 16:51:47 localhost.localdomain pulp[31597]: gofer.messaging.adapter.connect:ERROR: connect: qpid+ssl://192.168.122.46:5671, failed: [Errno 111] Connection refused
       Jan 29 16:51:47 localhost.localdomain pulp[31597]: gofer.messaging.adapter.connect:INFO: retry in 10 seconds
       Jan 29 16:51:47 localhost.localdomain pulp[31598]: gofer.messaging.adapter.connect:INFO: retry in 10 seconds
       Jan 29 16:51:49 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:49 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104) Trying again in 8.00 seconds...
       Jan 29 16:51:49 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104)
       Jan 29 16:51:49 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:49 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576) Trying again in 8.00 seconds...
       Jan 29 16:51:49 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576)
       Jan 29 16:51:49 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:49 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256) Trying again in 8.00 seconds...
       Jan 29 16:51:49 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256)
       Jan 29 16:51:57 localhost.localdomain pulp[31599]: gofer.messaging.adapter.connect:INFO: connecting: qpid+ssl://192.168.122.46:5671
       Jan 29 16:51:57 localhost.localdomain pulp[31599]: gofer.messaging.adapter.qpid.connection:INFO: open: URL: ssl://192.168.122.46|SSL: ca: /etc/pki/pulp/ca.crt|key: None|certificate: /etc/pki/qpid/client.crt|host-validation: False
       Jan 29 16:51:57 localhost.localdomain pulp[31599]: gofer.messaging.adapter.connect:ERROR: connect: qpid+ssl://192.168.122.46:5671, failed: [Errno 111] Connection refused
       Jan 29 16:51:57 localhost.localdomain pulp[31599]: gofer.messaging.adapter.connect:INFO: retry in 12 seconds
       Jan 29 16:51:57 localhost.localdomain pulp[31597]: gofer.messaging.adapter.connect:INFO: connecting: qpid+ssl://192.168.122.46:5671
       Jan 29 16:51:57 localhost.localdomain pulp[31597]: gofer.messaging.adapter.qpid.connection:INFO: open: URL: ssl://192.168.122.46|SSL: ca: /etc/pki/pulp/ca.crt|key: None|certificate: /etc/pki/qpid/client.crt|host-validation: False
       Jan 29 16:51:57 localhost.localdomain pulp[31598]: gofer.messaging.adapter.connect:INFO: connecting: qpid+ssl://192.168.122.46:5671
       Jan 29 16:51:57 localhost.localdomain pulp[31598]: gofer.messaging.adapter.qpid.connection:INFO: open: URL: ssl://192.168.122.46|SSL: ca: /etc/pki/pulp/ca.crt|key: None|certificate: /etc/pki/qpid/client.crt|host-validation: False
       Jan 29 16:51:57 localhost.localdomain pulp[31597]: gofer.messaging.adapter.connect:ERROR: connect: qpid+ssl://192.168.122.46:5671, failed: [Errno 111] Connection refused
       Jan 29 16:51:57 localhost.localdomain pulp[31597]: gofer.messaging.adapter.connect:INFO: retry in 12 seconds
       Jan 29 16:51:57 localhost.localdomain pulp[31598]: gofer.messaging.adapter.connect:ERROR: connect: qpid+ssl://192.168.122.46:5671, failed: [Errno 111] Connection refused
       Jan 29 16:51:57 localhost.localdomain pulp[31598]: gofer.messaging.adapter.connect:INFO: retry in 12 seconds
       Jan 29 16:51:57 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:57 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104) Trying again in 10.00 seconds...
       Jan 29 16:51:57 localhost.localdomain pulp[31593]: celery.worker.consumer.consumer:ERROR: (31593-63104)
       Jan 29 16:51:57 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:57 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256) Trying again in 10.00 seconds...
       Jan 29 16:51:57 localhost.localdomain pulp[31580]: celery.worker.consumer.consumer:ERROR: (31580-24256)
       Jan 29 16:51:57 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576) consumer: Cannot connect to qpid://192.168.122.46:5671//: [Errno 111] Connection refused.
       Jan 29 16:51:57 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576) Trying again in 10.00 seconds...
       Jan 29 16:51:57 localhost.localdomain pulp[31588]: celery.worker.consumer.consumer:ERROR: (31588-48576)
       Jan 29 16:52:01 localhost.localdomain pulp[31577]: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@localhost.localdomain' has gone missing, removing from list of workers
       Jan 29 16:52:01 localhost.localdomain pulp[31577]: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@localhost.localdomain is missing. Canceling the tasks in its queue.
       Jan 29 16:52:01 localhost.localdomain pulp[31577]: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@localhost.localdomain' has gone missing, removing from list of workers
       Jan 29 16:52:01 localhost.localdomain pulp[31577]: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@localhost.localdomain is missing. Canceling the tasks in its queue

« Previous
1
2
3
…
5
Next »

(1-1/5)

Project

Profile

Help

Pulp

Issue #4387 » amqp-failure.log