Project

Profile

Help

Story #8881

closed

As a user, Pulp will retry downloads by default 3 times and I can configure this value on my Remote

Added by bmbouter over 3 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 99
Quarter:

Description

Background

Currently Pulp will only retry on specific http error codes:

429 - Too Many Requests 502 - Bad Gateway 503 - Service Unavailable 504 - Gateway Timeout

That happens here: https://github.com/pulp/pulpcore/blob/master/pulpcore/download/http.py#L15-L32

The need to retry

  1. It's expected by users. Almost all (if not all) download tools, e.g. curl, wget, etc have retry behaviors in them
  2. With sync having so many downloads occur and a single error halting the entire process, as it is now, Pulp is not very reliable

What to retry on

With the implementation of this feature Pulp will retry in the following situations:

  • All HTTP 5xx response codes
  • HTTP 429
  • Socket timeouts and TCP disconnects

This is a simplified set of cases that mimics the retry behaviors outlined by AWS and Google Cloud.

Exponential Backoff and Jitter

Retries will continue to use the backoff and jitter already used today.

Number of Retries

The default will be 5, which was the Pulp2 default.

User configuration

The Remote will get a new field:

retry_count = models.PositiveIntegerField(default=5)


Related issues

Related to RPM Support - Issue #6589: Issues synching large Redhat reposCLOSED - DUPLICATEActions
Has duplicate Pulp - Issue #8878: default Download Concurrency is too high for many centos mirrorsCLOSED - DUPLICATEdalleyActions
Blocked by Pulp - Issue #8899: IntegerFields cannot be set null after being set to a value even if allow_null=TrueCLOSED - NOTABUGActions

Also available in: Atom PDF