Project

Profile

Help

Story #87

As a user, I can subscribe to an amqp topic to monitor what pulp tasks are doing

Added by mhrivnak almost 7 years ago. Updated 4 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.7.0
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
February 2015
Quarter:

Description

Deliverables:

  • make pulp always publish messages to a topic exchange about task state changes
  • make pulp always publish messages to a topic exchange about progress reports
  • mmccune says jsherril should be the katello contact for this. Get his feedback during the design phase.
  • Documentation of this feature
  • Add a note to the release notes

Related issues

Related to Pulp - Task #1160: Remove http, amqp, and e-mail events that were deprecated in Pulp 2.7.0CLOSED - WONTFIX

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 68b41868 View on GitHub
Added by cduryee over 6 years ago

Emit event notifications when TaskStatus is saved or updated.

This patch adds new functionality to emit event notifications when TaskStatus objects are changed. This relies on mongoengine's signals library; when a TaskStatus is updated we publish a message. We override CriteriaQuerySet's update() method as well to send a message upon updating a collection of TaskStatuses.

re #87

Revision 68b41868 View on GitHub
Added by cduryee over 6 years ago

Emit event notifications when TaskStatus is saved or updated.

This patch adds new functionality to emit event notifications when TaskStatus objects are changed. This relies on mongoengine's signals library; when a TaskStatus is updated we publish a message. We override CriteriaQuerySet's update() method as well to send a message upon updating a collection of TaskStatuses.

re #87

Revision abfc8035 View on GitHub
Added by cduryee over 6 years ago

Update docs for task notification messages

The previous docs were not suitable for someone who wanted to integrate with Pulp but was not already familiar with TaskStatus objects.

This patch adds some examples and further explanation on task notifications and how to use them. It also adds some comments to server.conf about the new options.

re #87

Revision abfc8035 View on GitHub
Added by cduryee over 6 years ago

Update docs for task notification messages

The previous docs were not suitable for someone who wanted to integrate with Pulp but was not already familiar with TaskStatus objects.

This patch adds some examples and further explanation on task notifications and how to use them. It also adds some comments to server.conf about the new options.

re #87

History

#1 Updated by bmbouter almost 7 years ago

  • Description updated (diff)

#2 Updated by jortel@redhat.com almost 7 years ago

  • Description updated (diff)

We currently have 2 sections in server.conf to specify message broker properties and credentials. One for celery and one for agent messaging. Which broker should we be using for events? If we decide use the agent broker specified under [messaging], I can point the developer of this feature to code that can be used as an example of using the [messaging] information to create a kombu connection in 2.6.

#3 Updated by mhrivnak almost 7 years ago

I think we should use the workers' broker settings. Especially in light of recent discussions, it seems like all kinds of changes could potentially happen with agent-related messaging. A monitoring solution may also want to listen in on other internal messaging traffic anyway.

#4 Updated by cduryee almost 7 years ago

Once this is implemented, can we add a task to remove the http and email notifiers in 3.0?

#6 Updated by cduryee almost 7 years ago

  • Sprint/Milestone set to 9

#7 Updated by cduryee over 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to cduryee

#8 Updated by cduryee over 6 years ago

  • % Done changed from 0 to 10

#9 Updated by cduryee over 6 years ago

  • Description updated (diff)

"remove existing notifier" removed per mhrivnak, needs to be done in 3.0

#10 Updated by cduryee over 6 years ago

Justin,

For the task state change message, do you need anything besides the task's UUID and new state?

Also, can you paste an example of how Katello gets sync progress reports currently?

#11 Updated by cduryee over 6 years ago

  • % Done changed from 10 to 30

#12 Updated by rbarlow over 6 years ago

  • Tags Sprint Candidate added

#13 Updated by mhrivnak over 6 years ago

  • Sprint/Milestone changed from 9 to 12

#14 Updated by cduryee over 6 years ago

  • % Done changed from 30 to 50

#15 Updated by cduryee over 6 years ago

  • Status changed from ASSIGNED to POST
  • % Done changed from 50 to 100

#17 Updated by cduryee over 6 years ago

  • Status changed from POST to MODIFIED

#18 Updated by bmbouter over 6 years ago

  • Tags deleted (Sprint Candidate)

#19 Updated by dkliban@redhat.com over 6 years ago

  • Platform Release set to 2.7.0
  • Groomed set to No
  • Sprint Candidate set to No

#20 Updated by dkliban@redhat.com over 6 years ago

  • Status changed from MODIFIED to 5

#21 Updated by dkliban@redhat.com over 6 years ago

This story can be verified by running the script found here[0]. You should start the script and then do a sync of a repo. If you see that messages are being printed by your script, then the notifications are working.

[0] - https://github.com/pulp/pulp/blob/master/docs/dev-guide/integration/events/index.rst

#22 Updated by bmbouter about 6 years ago

  • Related to Task #1160: Remove http, amqp, and e-mail events that were deprecated in Pulp 2.7.0 added

#23 Updated by rbarlow over 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#24 Updated by bmbouter over 3 years ago

  • Sprint set to February 2015

#25 Updated by bmbouter over 3 years ago

  • Sprint/Milestone deleted (12)

#26 Updated by bmbouter over 2 years ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF