Project

Profile

Help

Task #1333

Document that syncs give up if network fails

Added by pthomas@redhat.com about 5 years ago. Updated over 1 year ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Category:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Documentation, Easy Fix, Pulp 2
Sprint:
Quarter:

Description

As shown below, if a network error occurs during a sync and several requests fail, the sync will skip the rest of its downloads. This is good, because trying to make thousands more requests to a struggling or missing remote service could take a lot of time, could make that remove service's problems worse, and is unlikely to succeed.

It would be helpful to document that behavior. I think this logic is in nectar, so most importers will behave this way. Here is an example of that behavior "in the wild":

[root@ibm-x3550m3-06 ~]# rpm -qa pulp-server
pulp-server-2.7.0-0.7.beta.el7.noarch
[root@ibm-x3550m3-06 ~]# 
[root@ibm-x3550m3-06 ~]# pulp-admin node  sync run --node-id node1
This command may be exited via ctrl+c without affecting the request.

[-]
Accepted...

(1/1) Repository: centos
[====                                              ] 9%

Error occurred during synchronization, check the child node logs for details

+----------------------------------------------------------------------+
                       Child Node Synchronization
+----------------------------------------------------------------------+

Repository: 
  Action:          Merged
  Content Sources: 
    Downloads:     
      Source Id:       Parent Node
      Total Failed:    4611
      Total Succeeded: 490
    Total Sources: 0
  Id:              centos
  Units:           
    Added:   490
    Removed: 0
    Updated: 0

+----------------------------------------------------------------------+
               The following [4611] errors were reported
+----------------------------------------------------------------------+

- 01: Received error [A connection error occurred] while downloading a unit file
at URL
[https://ibm-x3550m3-06.lab.eng.brq.redhat.com/pulp/nodes/https/repos/centos/con
tent/rpm/evolution-mapi-devel/3.8.5/6.el7/i686/ec7af664520b926516f473cf086388373
14ad9d5bc2a03af79d945228d49f7ca/evolution-mapi-devel-3.8.5-6.el7.i686.rpm] for
repository [centos]. The cause could be that the repository has not been
published.

- 02: Received error [Download skipped] while downloading a unit file at URL
[https://ibm-x3550m3-06.lab.eng.brq.redhat.com/pulp/nodes/https/repos/centos/con
tent/rpm/OpenEXR/1.7.1/7.el7/x86_64/4abca5307be39571089dd481eeb307bcf29fbb473629
005cae5721079108546a/OpenEXR-1.7.1-7.el7.x86_64.rpm] for repository [centos].
The cause could be that the repository has not been published.

- 03: Received error [Download skipped] while downloading a unit file at URL
[https://ibm-x3550m3-06.lab.eng.brq.redhat.com/pulp/nodes/https/repos/centos/con
tent/rpm/texlive-charter-doc/svn15878.0/32.el7/noarch/e068aee3536f1e724610ebe09b
d9c077fa82275d147df9a45e55abead8e18ddd/texlive-charter-doc-svn15878.0-32.el7.noa
rch.rpm] for repository [centos]. The cause could be that the repository has not
been published.

- 04: Received error [Download skipped] while downloading a unit file at URL
[https://ibm-x3550m3-06.lab.eng.brq.redhat.com/pulp/nodes/https/repos/centos/con
tent/rpm/mythes-uk/1.6.5/6.el7/noarch/753a0d54ce20196ac1cc61914258f38f8f3cd13f9d
a280fc562f59c9b503062a/mythes-uk-1.6.5-6.el7.noarch.rpm] for repository
[centos]. The cause could be that the repository has not been published.

- 05: Received error [Download skipped] while downloading a unit file at URL
[https://ibm-x3550m3-06.lab.eng.brq.redhat.com/pulp/nodes/https/repos/centos/con
tent/rpm/libbluray-devel/0.2.3/5.el7/x86_64/424fbab8446ab76bab7454ddac4b98daa6aa
b71f461ec672cdaf0de84c8c99a5/libbluray-devel-0.2.3-5.el7.x86_64.rpm] for
repository [centos]. The cause could be that the repository has not been
published.

- 06: Received error [Download skipped] while downloading a unit file at URL
[https://ibm-x3550m3-06.lab.eng.brq.redhat.com/pulp/nodes/https/repos/centos/con
tent/rpm/libgnome/2.32.1/9.el7/i686/59c0d5006cc901129acecaee16f7cfdda57873d8c520
ace99c63ac6036124203/libgnome-2.32.1-9.el7.i686.rpm] for repository [centos].
The cause could be that the repository has not been published.

- 07: Received error [Download skipped] while downloading a unit file at URL
[https://ibm-x3550m3-06.lab.eng.brq.redhat.com/pulp/nodes/https/repos/centos/con
tent/rpm/alsa-plugins-vdownmix/1.0.27/3.el7/i686/ffd0f28a6540ec51b9abc6843caa1f3
a461554097b5bf49044f0f55bb8fe18ca/alsa-plugins-vdownmix-1.0.27-3.el7.i686.rpm]
for repository [centos]. The cause could be that the repository has not been
published.

History

#1 Updated by pthomas@redhat.com about 5 years ago

Steps to reproduce.

1. Create & Sync a centos repo

  pulp-admin rpm repo create --repo-id centos --feed  http://mirror.mojohost.com/centos/7/os/x86_64/

2. Install & configure node parent & node child

3. Run node sync

#2 Updated by dkliban@redhat.com about 5 years ago

  • Tags Documentation added

I logged in to this machine and determined that the problem seems to be caused by network problem. We should document in the troubleshooting guide that a node sync will stop syncing at the first sign of network connectivity problems. However, when the node sync is run again, the sync resumes where it left off.

#3 Updated by mhrivnak almost 5 years ago

  • Tracker changed from Issue to Task
  • Subject changed from node sync errors when syncing big repos (centos) to Document that syncs give up if network fails
  • Description updated (diff)
  • Groomed set to No
  • Sprint Candidate set to Yes
  • Tags Easy Fix added

#4 Updated by mhrivnak over 4 years ago

  • Sprint Candidate changed from Yes to No

#5 Updated by bmbouter over 1 year ago

  • Status changed from NEW to CLOSED - WONTFIX

#6 Updated by bmbouter over 1 year ago

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.

#7 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF