Issue #2458
closed
Lazy download repo content task fails silently.
Status:
CLOSED - CURRENTRELEASE
Description
pulp-admin repo download --repo-id lazy
+----------------------------------------------------------------------+
Downloading Repository [lazy]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Download Repository Content
[========================== ] 53%
17 of 32 items
... failed
Task Succeeded
This needs to be logged at INFO. Otherwise, this fails with nothing in the log. I spend 4 hours troubleshooting this.
except (InvalidChecksumType, VerificationException, IOError), e:
_logger.debug(_('Download of {path} failed: {reason}.').format(
path=catalog_entry.path, reason=str(e)))
path_entry[PATH_DOWNLOADED] = False
self.progress_failures += 1
self.report()
[1] https://github.com/pulp/pulp/blob/master/server/pulp/server/controllers/repository.py#L1710
- Description updated (diff)
- Version set to 2.10.3
The task reports as succeeded. Does this make sense?
jortel@redhat.com wrote:
The task reports as succeeded. Does this make sense?
I think yes, as long as its progress report or result communicates the one or more files failed to download. This is similar to how syncs work.
- Triaged changed from No to Yes
These sounds like non-fatal exceptions so I expect the final task status will be succeeded, but errors should be reported with a progress report.
bmbouter wrote:
These sounds like non-fatal exceptions so I expect the final task status will be succeeded, but errors should be reported with a progress report.
By progress report I think we mean the result report that summarizes failure statistics like "400 downloads failed" and not a listing of all 400 URLs etc.
jortel@redhat.com wrote:
bmbouter wrote:
These sounds like non-fatal exceptions so I expect the final task status will be succeeded, but errors should be reported with a progress report.
By progress report I think we mean the result report that summarizes failure statistics like "400 downloads failed" and not a listing of all 400 URLs etc. The main thing is that the task state is not FAILED.
- Sprint/Milestone set to 32
Yes @jortel you are right. It should be on the result report not a "progress report". Also +1 to summarizing the failure.
- Status changed from NEW to ASSIGNED
- Assignee set to jortel@redhat.com
Looks like this task is running a LazyUnitDownloadStep which followed the flawed practice of reporting both progress and result (summary) using a progress report. And, although, the LazyUnitDownloadStep does not subclass a PublishStep, it tries to act like one and submit a progress report that can be rendered by a PublishStepStatusRenderer. This is exactly how things are set up. The DownloadRepositoryCommand IsA SyncPublishCommand and is created with the PublishStepStatusRenderer.
Given how much of this will be rewritten in pulp3 (not using PublishStep), I don't think it's worth the effort to fix this in pulp2.
[jortel@f23d pulp]$ pulp-admin tasks details --task-id=f5ad1e36-ee73-422a-9cb1-f3820f9c0e0a
+----------------------------------------------------------------------+
Task Details
+----------------------------------------------------------------------+
Operations: download
Resources: zoo (repository)
State: Successful
Start Time: 2017-01-23T19:49:25Z
Finish Time: 2017-01-23T19:49:26Z
Result: N/A
Task Id: f5ad1e36-ee73-422a-9cb1-f3820f9c0e0a
Progress Report:
Background Download:
Description: Download Repository Content
Details:
Error Details:
Items Total: 32
Num Failures: 1
Num Processed: 1
Num Success: 0
State: FAILED
Step Id: 6f64950c-af41-4d20-b219-d8216c9dbaba
Step Type: background_download
- Status changed from ASSIGNED to POST
Is there a procedure for reproducing the failure described by this issue?
- Status changed from POST to MODIFIED
- Platform Release set to 2.12.1
- Status changed from MODIFIED to 5
- Version changed from 2.10.3 to 2.12.1
- Version changed from 2.12.1 to 2.10.3
- Status changed from 5 to CLOSED - CURRENTRELEASE
To test this, I suggest:
- Creating a repository with a on_demand download policy.
- Synchronize the repository.
- Using mongo, update one of the URLs in the catalog entries for that importer to be incorrect.
- As described in the description, kick off the repository download task.
Related to step 3. Is there another way to update the URL in catalog without use mongo?
If there's a way to reproduce this issue by accessing Pulp's public interface, or by performing actions like starting or stopping services, we can definitely test this. As it is, the procedure for reproducing this issue requires an uncomfortable amount of knowledge about how Pulp works internally, and I think QE should avoid creating an automated test for this issue.
- Sprint changed from Sprint 16 to Sprint 14
- Sprint/Milestone deleted (
32)
Also available in: Atom
PDF
.
Log VerificationException at INFO. closes #2458