Project

Profile

Help

Story #5522

Story #5517: [EPIC] Automation Hub Release Blockers

As a user, I can filter ImportCollection reposnse messages by date to receive limit the set of messages returned

Added by bmbouter 2 months ago. Updated about 2 months ago.

Status:
MODIFIED
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
Yes
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 60

Description

Problem

Each time the user fetches logging output from GALAXY_API_ROOT/v3/imports/collections/<uuid:pk>/ it comes from this viewset

The issue is that the user wants to receive new log messages, but they get all logs every time.

Solution

Rename the CollectionImport Model by:

1. removing the "messages" attribute
2. creating three new attributes
  • message <--- long string, required.
  • level. <--- this should be a choice field w/ the standard Python logging levels enumerated
  • time. This one can be configured to auto-record at save using the django ORM syntax for that.

3. The "add_log_record" method should be removed
4. Port the logutils.CollectionImportHandler to create a CollectionImport for each call to emit()
5. Rename CollectionImport to CollectionImportMessage, and update migrations accordingly.

Associated revisions

Revision 395ba61e View on GitHub
Added by daviddavis about 2 months ago

Add since filter for collection import messages

fixes #5522
https://pulp.plan.io/issues/5522

History

#1 Updated by bmbouter 2 months ago

  • Parent task set to #5517

#2 Updated by bmbouter 2 months ago

  • Subject changed from As a user, I can provide a 'since' attribute to filter response data for ImportCollection to As a user, I can filter ImportCollection reposnse messages by date to receive limit the set of messages returned
  • Description updated (diff)

#3 Updated by bmbouter 2 months ago

  • Description updated (diff)

#4 Updated by daviddavis 2 months ago

  • Groomed changed from No to Yes
  • Sprint set to Sprint 60

#5 Updated by daviddavis 2 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to daviddavis

#6 Updated by daviddavis 2 months ago

For the time field, it looks like we currently use the log record's created field0. Will using the db record creation timestamp cause different values/ordering?

[0] https://github.com/pulp/pulp_ansible/blob/055305d7b928f9cef84bacb6b086ea9a41b23a89/pulp_ansible/app/models.py#L70

#7 Updated by bmbouter 2 months ago

daviddavis wrote:

For the time field, it looks like we currently use the log record's created field0. Will using the db record creation timestamp cause different values/ordering?

[0] https://github.com/pulp/pulp_ansible/blob/055305d7b928f9cef84bacb6b086ea9a41b23a89/pulp_ansible/app/models.py#L70

It likely will change the values but not the ordering (I think). Messages created one after the other would still be time-sorted correctly if they are sorted on that field by default.

The values changing I think is ok because two representations of the same moment in time could be translated. I'm making this up, but say the logger uses epoch, but the DB uses a strfmt() based timestamp, I think the user would submit the type required from the API and the internals would use it's datetime equivalent as it queries the db, so it should "just work".

Maybe I'm missing something though, not working on the code directly.

#8 Updated by daviddavis about 2 months ago

Follow up question: do we need a time field since it duplicates the pulp_created field?

#9 Updated by bmbouter about 2 months ago

daviddavis wrote:

Follow up question: do we need a time field since it duplicates the pulp_created field?

I don't think we need it, it would be redundant.

#10 Updated by daviddavis about 2 months ago

This is a bit more complex than I thought. Currently, there is a collection imports endpoint0. I thought I could "fake" a CollectionImport in the API by using the collection import Task instead. However, the design is complex and not very straightforward. Also, it would mean that if galaxy ever wanted to store collection import data, they would have no place to store it. I think it might be better to keep the CollectionImport model instead and create a new CollectionImportMessage model.

[0] https://github.com/pulp/pulp_ansible/blob/078942a1e695135af09fcd0f4759824d8a3df498/pulp_ansible/app/urls.py#L67-L71

#11 Updated by daviddavis about 2 months ago

I added a since filter to just filter the messages as they are now. I think this is probably the easiest/simplest solution:

https://github.com/pulp/pulp_ansible/pull/229

#12 Updated by daviddavis about 2 months ago

  • Status changed from ASSIGNED to MODIFIED
  • % Done changed from 0 to 100

Please register to edit this issue

Also available in: Atom PDF