Project

Profile

Help

Maintenance: Planio will be observing a scheduled maintenance window this Sunday, November 10, 2024 from 20:00 UTC until 21:00 UTC to perform important network maintenance in our primary data center. Your Planio account will be unavailable for a few minutes during this maintenance window.

Issue #5164

closed

Progress Reports incorrectly update counts in error or canceled situations

Added by bmbouter over 5 years ago. Updated about 5 years ago.

Status:
CLOSED - NOTABUG
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 59
Quarter:

Description

A user contacted me with a task status for a failed task (see below) and I noticed its progress reports looked incorrect. The "canceled" and "errored" progress reports show done counts equal to total counts which is incorrect for a state other than "completed".

In terms of reproducing, this was with pulp_file doing a typical sync, and it failed to find the PULP_MANIFEST. I believe the issue is in core though here

   {
      "_href": "/pulp/api/v3/tasks/01f034b0-a419-403a-88e8-b337a54c533a/",
      "_created": "2019-07-18T10:40:18.520527Z",
      "state": "failed",
      "name": "pulp_file.app.tasks.synchronizing.synchronize",
      "started_at": "2019-07-18T10:40:18.665278Z",
      "finished_at": "2019-07-18T10:52:12.468158Z",
      "non_fatal_errors": [],
      "error": {
        "code": null,
        "description": "404, message='Not Found'",
        "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 812, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 588, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 594, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_file/app/tasks/synchronizing.py\", line 45, in synchronize\n    dv.create()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 169, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/lib64/python3.6/asyncio/base_events.py\", line 484, in run_until_complete\n    return future.result()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 209, in create_pipeline\n    await asyncio.gather(*futures)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 43, in __call__\n    await self.run()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py\", line 132, in run\n    pb.done += task.result()  # download_count\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py\", line 155, in _handle_content_unit\n    await asyncio.gather(*downloaders_for_content)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/models.py\", line 78, in download\n    download_result = await downloader.run(extra_data=self.extra_data)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/download/base.py\", line 212, in run\n    return await self._run(extra_data=extra_data)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/backoff/_async.py\", line 131, in retry\n    ret = await target(*args, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/download/http.py\", line 183, in _run\n    response.raise_for_status()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/aiohttp/client_reqrep.py\", line 942, in raise_for_status\n    headers=self.headers)\n"
      },
      "worker": "/pulp/api/v3/workers/e256600c-c8bb-4dec-affe-f047e617ad50/",
      "parent": null,
      "spawned_tasks": [],
      "progress_reports": [
        {
          "message": "Downloading Metadata",
          "state": "completed",
          "total": 1,
          "done": 1,
          "suffix": null
        },
        {
          "message": "Downloading Artifacts",
          "state": "failed",
          "total": 35195,
          "done": 35195,
          "suffix": null
        },
        {
          "message": "Parsing Metadata Lines",
          "state": "canceled",
          "total": 100000,
          "done": 35501,
          "suffix": null
        },
        {
          "message": "Associating Content",
          "state": "canceled",
          "total": 35191,
          "done": 35191,
          "suffix": null
        }
      ],
      "created_resources": []
    }

Related issues

Related to Pulp - Story #5444: [Epic] Consolidate ProgressReporting into a single model so it's simpler for plugin writersCLOSED - CURRENTRELEASEfao89

Actions
Actions #1

Updated by jhutar@redhat.com over 5 years ago

Just for record, please note it failed for 404 on some files, not on the manifest. But I assume that is not important in this specific issue report. Thank you!

Actions #2

Updated by amacdona@redhat.com over 5 years ago

  • Sprint/Milestone set to 3.0.0
  • Triaged changed from No to Yes
  • Sprint set to Sprint 56
Actions #3

Updated by rchan over 5 years ago

  • Sprint changed from Sprint 56 to Sprint 57
Actions #4

Updated by rchan about 5 years ago

  • Sprint changed from Sprint 57 to Sprint 58
Actions #5

Updated by fao89 about 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fao89
Actions #6

Updated by fao89 about 5 years ago

  • Status changed from ASSIGNED to NEW
  • Assignee deleted (fao89)
Actions #7

Updated by fao89 about 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fao89
Actions #9

Updated by rchan about 5 years ago

  • Sprint changed from Sprint 58 to Sprint 59
Actions #10

Updated by fao89 about 5 years ago

  • Status changed from POST to NEW
  • Assignee deleted (fao89)
Actions #11

Updated by fao89 about 5 years ago

  • Related to Story #5444: [Epic] Consolidate ProgressReporting into a single model so it's simpler for plugin writers added
Actions #12

Updated by fao89 about 5 years ago

  • Status changed from NEW to CLOSED - NOTABUG

it's likely been fixed by #5444

Also available in: Atom PDF