Project

Profile

Help

Story #87

closed

As a user, I can subscribe to an amqp topic to monitor what pulp tasks are doing

Added by mhrivnak over 7 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.7.0
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
February 2015
Quarter:

Description

Deliverables:

  • make pulp always publish messages to a topic exchange about task state changes
  • make pulp always publish messages to a topic exchange about progress reports
  • mmccune says jsherril should be the katello contact for this. Get his feedback during the design phase.
  • Documentation of this feature
  • Add a note to the release notes

Related issues

Related to Pulp - Task #1160: Remove http, amqp, and e-mail events that were deprecated in Pulp 2.7.0CLOSED - WONTFIX

Actions
Actions #1

Updated by bmbouter over 7 years ago

  • Description updated (diff)
Actions #2

Updated by jortel@redhat.com over 7 years ago

  • Description updated (diff)

We currently have 2 sections in server.conf to specify message broker properties and credentials. One for celery and one for agent messaging. Which broker should we be using for events? If we decide use the agent broker specified under [messaging], I can point the developer of this feature to code that can be used as an example of using the [messaging] information to create a kombu connection in 2.6.

Actions #3

Updated by mhrivnak over 7 years ago

I think we should use the workers' broker settings. Especially in light of recent discussions, it seems like all kinds of changes could potentially happen with agent-related messaging. A monitoring solution may also want to listen in on other internal messaging traffic anyway.

Actions #4

Updated by cduryee over 7 years ago

Once this is implemented, can we add a task to remove the http and email notifiers in 3.0?

Actions #6

Updated by cduryee over 7 years ago

  • Sprint/Milestone set to 9
Actions #7

Updated by cduryee over 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to cduryee
Actions #8

Updated by cduryee over 7 years ago

  • % Done changed from 0 to 10
Actions #9

Updated by cduryee over 7 years ago

  • Description updated (diff)

"remove existing notifier" removed per mhrivnak, needs to be done in 3.0

Actions #10

Updated by cduryee over 7 years ago

Justin,

For the task state change message, do you need anything besides the task's UUID and new state?

Also, can you paste an example of how Katello gets sync progress reports currently?

Actions #11

Updated by cduryee over 7 years ago

  • % Done changed from 10 to 30
Actions #12

Updated by rbarlow over 7 years ago

  • Tags Sprint Candidate added
Actions #13

Updated by mhrivnak over 7 years ago

  • Sprint/Milestone changed from 9 to 12
Actions #14

Updated by cduryee over 7 years ago

  • % Done changed from 30 to 50
Actions #15

Updated by cduryee over 7 years ago

  • Status changed from ASSIGNED to POST
  • % Done changed from 50 to 100

Added by cduryee over 7 years ago

Revision 68b41868

Emit event notifications when TaskStatus is saved or updated.

This patch adds new functionality to emit event notifications when TaskStatus objects are changed. This relies on mongoengine's signals library; when a TaskStatus is updated we publish a message. We override CriteriaQuerySet's update() method as well to send a message upon updating a collection of TaskStatuses.

re #87

Added by cduryee over 7 years ago

Revision 68b41868

Emit event notifications when TaskStatus is saved or updated.

This patch adds new functionality to emit event notifications when TaskStatus objects are changed. This relies on mongoengine's signals library; when a TaskStatus is updated we publish a message. We override CriteriaQuerySet's update() method as well to send a message upon updating a collection of TaskStatuses.

re #87

Actions #17

Updated by cduryee over 7 years ago

  • Status changed from POST to MODIFIED
Actions #18

Updated by bmbouter over 7 years ago

  • Tags deleted (Sprint Candidate)

Added by cduryee over 7 years ago

Revision abfc8035

Update docs for task notification messages

The previous docs were not suitable for someone who wanted to integrate with Pulp but was not already familiar with TaskStatus objects.

This patch adds some examples and further explanation on task notifications and how to use them. It also adds some comments to server.conf about the new options.

re #87

Added by cduryee over 7 years ago

Revision abfc8035

Update docs for task notification messages

The previous docs were not suitable for someone who wanted to integrate with Pulp but was not already familiar with TaskStatus objects.

This patch adds some examples and further explanation on task notifications and how to use them. It also adds some comments to server.conf about the new options.

re #87

Actions #19

Updated by dkliban@redhat.com over 7 years ago

  • Platform Release set to 2.7.0
  • Groomed set to No
  • Sprint Candidate set to No
Actions #20

Updated by dkliban@redhat.com over 7 years ago

  • Status changed from MODIFIED to 5
Actions #21

Updated by dkliban@redhat.com about 7 years ago

This story can be verified by running the script found here[0]. You should start the script and then do a sync of a repo. If you see that messages are being printed by your script, then the notifications are working.

[0] - https://github.com/pulp/pulp/blob/master/docs/dev-guide/integration/events/index.rst

Actions #22

Updated by bmbouter about 7 years ago

  • Related to Task #1160: Remove http, amqp, and e-mail events that were deprecated in Pulp 2.7.0 added
Actions #23

Updated by rbarlow over 6 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #24

Updated by bmbouter over 4 years ago

  • Sprint set to February 2015
Actions #25

Updated by bmbouter over 4 years ago

  • Sprint/Milestone deleted (12)
Actions #26

Updated by bmbouter over 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF