Project

Profile

Help

Issue #2969

progress reports omitted by task API

Added by dkliban@redhat.com about 2 years ago. Updated 6 months ago.

Status:
MODIFIED
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Severity:
3. High
Version:
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 24

Description

I discovered this bug while writing the pulp_example plugin. The sync code that could demonstrate this problem has not been merged (yet). HOwever, the issue can be demonstrated by adding the following diff to Pulp.

diff --git a/platform/pulpcore/app/tasks/importer.py b/platform/pulpcore/app/tasks/importer.py
index 0ef18a8..24d69b5 100644
--- a/platform/pulpcore/app/tasks/importer.py
+++ b/platform/pulpcore/app/tasks/importer.py
@@ -29,13 +29,18 @@ def update(importer_id, app_label, serializer_name, data=None, partial=False):
             due to validation error. This theoretically should never occur since validation is
             performed before the task is dispatched.
     """ 
-    instance = models.Importer.objects.get(id=importer_id).cast()
-    data_querydict = QueryDict('', mutable=True)
-    data_querydict.update(data)
-    serializer_class = get_plugin_config(app_label).named_serializers[serializer_name]
-    serializer = serializer_class(instance, data=data_querydict, partial=partial)
-    serializer.is_valid(raise_exception=True)
-    serializer.save()
+    from pulpcore.plugin.models import ProgressBar
+    progress_bar = ProgressBar(message="Updating importer", total=1)
+    progress_bar.save()
+    with progress_bar:
+        instance = models.Importer.objects.get(id=importer_id).cast()
+        data_querydict = QueryDict('', mutable=True)
+        data_querydict.update(data)
+        serializer_class = get_plugin_config(app_label).named_serializers[serializer_name]
+        serializer = serializer_class(instance, data=data_querydict, partial=partial)
+        serializer.is_valid(raise_exception=True)
+        serializer.save()
+        progress_bar.increment()

Assuming that pulp 3 is running on localhost:1234, you can create a repository:

http POST http://localhost:1234/api/v3/repositories/ name=mycoolrepo notes:={} scratchpad:={}

Add a file importer:

http POST http://localhost:1234/api/v3/repositories/mycoolrepo/importers/file/ name=mycoolimporter download_policy='immediate' repository='http://localhost:1234/api/v3/repositories/mycoolrepo/'

Modify the importer:

[vagrant@pulp3 pulp]$ http PATCH http://localhost:1234/api/v3/repositories/mycoolrepo/importers/file/mycoolimporter/ download_policy='on_demand'
HTTP/1.0 202 Accepted
Allow: GET, PUT, PATCH, DELETE, HEAD, OPTIONS
Content-Length: 135
Content-Type: application/json
Date: Thu, 10 Aug 2017 15:37:42 GMT
Server: WSGIServer/0.2 CPython/3.5.3
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN

[
    {
        "_href": "http://localhost:1234/api/v3/tasks/36ee48dc-0a41-42b3-bd99-b4f222cbd5b9/",
        "task_id": "36ee48dc-0a41-42b3-bd99-b4f222cbd5b9" 
    }
]

The last command returned a task id and a URL for that task. A GET request of that URL returns the task.

[vagrant@pulp3 pulp]$ http http://localhost:1234/api/v3/tasks/36ee48dc-0a41-42b3-bd99-b4f222cbd5b9/
HTTP/1.0 200 OK
Allow: GET, OPTIONS
Content-Length: 349
Content-Type: application/json
Date: Thu, 10 Aug 2017 15:39:53 GMT
Server: WSGIServer/0.2 CPython/3.5.3
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN

{
    "_href": "http://localhost:1234/api/v3/tasks/36ee48dc-0a41-42b3-bd99-b4f222cbd5b9/",
    "error": null,
    "finished_at": "2017-08-10T15:37:43.044201Z",
    "group": null,
    "non_fatal_errors": [],
    "parent": null,
    "started_at": "2017-08-10T15:37:42.992822Z",
    "state": "completed",
    "tags": [],
    "worker": "http://localhost:1234/api/v3/workers/reserved_resource_worker_1@pulp3.dev/" 
}

The task should include the progress report that was generated. However, it does not. The progress report does exist in the database though:

[vagrant@pulp3 pulp]$ psql --u pulp
psql (9.5.7)
Type "help" for help.

pulp=# select * from pulp_app_progressreport ;
                  id                  |                          message                           |   state   | total | done | suffix |               task_id                
--------------------------------------+------------------------------------------------------------+-----------+-------+------+--------+--------------------------------------
 5fac01d4-f07e-4fda-b635-9ba468e04183 | Updating importer                                          | completed |     1 |    1 |        | 36ee48dc-0a41-42b3-bd99-b4f222cbd5b9

Associated revisions

Revision 5b66ba51 View on GitHub
Added by werwty about 2 years ago

Update task to serialize progress_report

closes #2969
https://pulp.plan.io/issues/2969

Revision 5b66ba51 View on GitHub
Added by werwty about 2 years ago

Update task to serialize progress_report

closes #2969
https://pulp.plan.io/issues/2969

Revision 5b66ba51 View on GitHub
Added by werwty about 2 years ago

Update task to serialize progress_report

closes #2969
https://pulp.plan.io/issues/2969

History

#1 Updated by bmbouter about 2 years ago

Having the field refer to a list of progress reports (or a set since they are truly unordered) would be good. The field name on the Task serializer could be 'progress_reports' or 'progress'.

#2 Updated by ttereshc about 2 years ago

  • Priority changed from Normal to High
  • Sprint/Milestone set to 43
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes

#3 Updated by mhrivnak about 2 years ago

https://github.com/pulp/pulp/blob/253ae59c/platform/pulpcore/app/serializers/progress.py#L46-L48

That serializer is already meant for embedding the progress report in the task detail view.

#4 Updated by bizhang about 2 years ago

  • Status changed from NEW to POST
  • Assignee set to bizhang

#5 Updated by werwty about 2 years ago

  • Status changed from POST to MODIFIED

#6 Updated by bmbouter almost 2 years ago

  • Tags deleted (Pulp 3 Plugin Writer Alpha)

Cleaning up Redmine tags

#7 Updated by bmbouter over 1 year ago

  • Sprint set to Sprint 24

#8 Updated by bmbouter over 1 year ago

  • Sprint/Milestone deleted (43)

#9 Updated by daviddavis 6 months ago

  • Sprint/Milestone set to 3.0

#10 Updated by bmbouter 6 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF