Project

Profile

Help

Issue #2458

Lazy download repo content task fails silently.

Added by jortel@redhat.com almost 4 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.10.3
Platform Release:
2.12.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 14
Quarter:

Description

 pulp-admin repo download --repo-id lazy
+----------------------------------------------------------------------+
                     Downloading Repository [lazy]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Download Repository Content
[==========================                        ] 53%
17 of 32 items
... failed

Task Succeeded

This needs to be logged at INFO. Otherwise, this fails with nothing in the log. I spend 4 hours troubleshooting this.

        except (InvalidChecksumType, VerificationException, IOError), e:
            _logger.debug(_('Download of {path} failed: {reason}.').format(
                path=catalog_entry.path, reason=str(e)))
            path_entry[PATH_DOWNLOADED] = False
            self.progress_failures += 1
self.report()

[1] https://github.com/pulp/pulp/blob/master/server/pulp/server/controllers/repository.py#L1710

Associated revisions

Revision bb8b02cd View on GitHub
Added by jortel@redhat.com over 3 years ago

Log VerificationException at INFO. closes #2458

Revision bb8b02cd View on GitHub
Added by jortel@redhat.com over 3 years ago

Log VerificationException at INFO. closes #2458

History

#1 Updated by jortel@redhat.com almost 4 years ago

  • Description updated (diff)
  • Version set to 2.10.3

#2 Updated by jortel@redhat.com almost 4 years ago

The task reports as succeeded. Does this make sense?

#3 Updated by mhrivnak almost 4 years ago

wrote:

The task reports as succeeded. Does this make sense?

I think yes, as long as its progress report or result communicates the one or more files failed to download. This is similar to how syncs work.

#4 Updated by bizhang almost 4 years ago

  • Triaged changed from No to Yes

#5 Updated by bmbouter almost 4 years ago

These sounds like non-fatal exceptions so I expect the final task status will be succeeded, but errors should be reported with a progress report.

#6 Updated by jortel@redhat.com almost 4 years ago

bmbouter wrote:

These sounds like non-fatal exceptions so I expect the final task status will be succeeded, but errors should be reported with a progress report.

By progress report I think we mean the result report that summarizes failure statistics like "400 downloads failed" and not a listing of all 400 URLs etc.

#7 Updated by jortel@redhat.com almost 4 years ago

wrote:

bmbouter wrote:

These sounds like non-fatal exceptions so I expect the final task status will be succeeded, but errors should be reported with a progress report.

By progress report I think we mean the result report that summarizes failure statistics like "400 downloads failed" and not a listing of all 400 URLs etc. The main thing is that the task state is not FAILED.

#8 Updated by mhrivnak almost 4 years ago

  • Sprint/Milestone set to 32

#9 Updated by bmbouter almost 4 years ago

Yes @jortel you are right. It should be on the result report not a "progress report". Also +1 to summarizing the failure.

#10 Updated by jortel@redhat.com over 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jortel@redhat.com

#11 Updated by jortel@redhat.com over 3 years ago

Looks like this task is running a LazyUnitDownloadStep which followed the flawed practice of reporting both progress and result (summary) using a progress report. And, although, the LazyUnitDownloadStep does not subclass a PublishStep, it tries to act like one and submit a progress report that can be rendered by a PublishStepStatusRenderer. This is exactly how things are set up. The DownloadRepositoryCommand IsA SyncPublishCommand and is created with the PublishStepStatusRenderer.

Given how much of this will be rewritten in pulp3 (not using PublishStep), I don't think it's worth the effort to fix this in pulp2.

[jortel@f23d pulp]$ pulp-admin tasks details --task-id=f5ad1e36-ee73-422a-9cb1-f3820f9c0e0a
+----------------------------------------------------------------------+
                              Task Details
+----------------------------------------------------------------------+

Operations:       download
Resources:        zoo (repository)
State:            Successful
Start Time:       2017-01-23T19:49:25Z
Finish Time:      2017-01-23T19:49:26Z
Result:           N/A
Task Id:          f5ad1e36-ee73-422a-9cb1-f3820f9c0e0a
Progress Report:  
  Background Download: 
    Description:   Download Repository Content
    Details:       
    Error Details: 
    Items Total:   32
    Num Failures:  1
    Num Processed: 1
    Num Success:   0
    State:         FAILED
    Step Id:       6f64950c-af41-4d20-b219-d8216c9dbaba
    Step Type:     background_download

#12 Updated by jortel@redhat.com over 3 years ago

  • Status changed from ASSIGNED to POST

#13 Updated by Ichimonji10 over 3 years ago

Is there a procedure for reproducing the failure described by this issue?

#14 Updated by jortel@redhat.com over 3 years ago

  • Status changed from POST to MODIFIED

#15 Updated by bizhang over 3 years ago

  • Platform Release set to 2.12.1

#16 Updated by bizhang over 3 years ago

  • Status changed from MODIFIED to 5

#17 Updated by bizhang over 3 years ago

  • Version changed from 2.10.3 to 2.12.1

#18 Updated by bizhang over 3 years ago

  • Version changed from 2.12.1 to 2.10.3

#19 Updated by bizhang over 3 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#20 Updated by jortel@redhat.com about 3 years ago

To test this, I suggest:

  1. Creating a repository with a on_demand download policy.
  2. Synchronize the repository.
  3. Using mongo, update one of the URLs in the catalog entries for that importer to be incorrect.
  4. As described in the description, kick off the repository download task.

#21 Updated by kersom about 3 years ago

Related to step 3. Is there another way to update the URL in catalog without use mongo?

#23 Updated by Ichimonji10 about 3 years ago

If there's a way to reproduce this issue by accessing Pulp's public interface, or by performing actions like starting or stopping services, we can definitely test this. As it is, the procedure for reproducing this issue requires an uncomfortable amount of knowledge about how Pulp works internally, and I think QE should avoid creating an automated test for this issue.

#24 Updated by bmbouter over 2 years ago

  • Sprint set to Sprint 16

#25 Updated by bmbouter over 2 years ago

  • Sprint changed from Sprint 16 to Sprint 14

#26 Updated by bmbouter over 2 years ago

  • Sprint/Milestone deleted (32)

#27 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF