Project

Profile

Help

Issue #8512

closed

deb repo - apt install - File has unexpected size - http pipeline

Added by jsherril@redhat.com over 3 years ago. Updated over 3 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Debian:
Platform Release:
Target Release - Debian:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Quarter:

Description

A katello user opened this: https://projects.theforeman.org/issues/32178

I'm getting unexpected size error on ubuntu 18.04 when trying to install from a deb repository. The problem seems to be caused by apt pipelining requests. Only happens on katello deb repos, and when there are many packages to download as seen below. I have ubuntu mirror in the pub directory, not through katello.

My guess is this is issue with pulp, as httpd proxies the requests to it, right?

# apt install -y ...
<snip>
Get:29 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic/main amd64 libxi6 amd64 2:1.7.9-1 [29.2 kB]
Get:30 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic/main amd64 libxrender1 amd64 1:0.9.10-1 [18.7 kB]
Get:31 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic-updates/main amd64 x11-common all 1:7.7+19ubuntu7.1 [22.5 kB]
Get:32 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic/main amd64 libxtst6 amd64 2:1.2.3-1 [12.8 kB]
Get:33 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic-updates/universe amd64 openjdk-8-jre-headless amd64 8u282-b08-0ubuntu1~18.04 [28.2 MB]
Get:34 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic-updates/main amd64 ca-certificates-java all 20180516ubuntu1~18.04.1 [12.2 kB]
Get:35 http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic default/all amd64 scylla-enterprise-conf amd64 2020.1.7-0.20210304.e42189e09-1 [11.8 kB]
Get:36 http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic default/all amd64 scylla-enterprise-python3 amd64 3.7.7-0.20210304.e42189e09-1 [20.0 MB]
Err:36 http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic default/all amd64 scylla-enterprise-python3 amd64 3.7.7-0.20210304.e42189e09-1
  File has unexpected size (20004486 != 86464). Mirror sync in progress? [IP: 10.1.1.8 80]
  Hashes of expected file:
   - SHA256:51a6c7c50ebd72be856b02a9fbe8ac39739e3f0c36bcf6a7c5f1d2d83092286c
   - SHA1:02cf7a581beeaf1b067600c72c268dc0b9e713bd [weak]
   - MD5Sum:aa1e11ed2ba92ca3da8a9785cdfae2ed [weak]
Get:37 http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic default/all amd64 scylla-enterprise-server amd64 2020.1.7-0.20210304.e42189e09-1 [35.7 MB]
Err:37 http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic default/all amd64 scylla-enterprise-server amd64 2020.1.7-0.20210304.e42189e09-1
  File has unexpected size (35725644 != 86464). Mirror sync in progress? [IP: 10.1.1.8 80]
  Hashes of expected file:
   - SHA256:74cbe6f7ba359869f84bf671941e9e7d60b4c302de836f35e73f12b37fe57c46
   - SHA1:ba106ac7ae29c5821a9d3c01d6f601352081d6e3 [weak]
   - MD5Sum:83f1ad8d23dcbf950096c28b88059442 [weak]
Get:38 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic-updates/universe amd64 sntp amd64 1:4.2.8p10+dfsg-5ubuntu7.3 [86.5 kB]
Fetched 38.1 MB in 1s (53.8 MB/s)
E: Failed to fetch http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic/pool/all/s/scylla-enterprise-python3/scylla-enterprise-python3_3.7.7-0.20210304.e42189e09-1_amd64.deb  File has unexpected size (20004486 != 86464). Mirror sync in progress? [IP: 10.1.1.8 80]
   Hashes of expected file:
    - SHA256:51a6c7c50ebd72be856b02a9fbe8ac39739e3f0c36bcf6a7c5f1d2d83092286c
    - SHA1:02cf7a581beeaf1b067600c72c268dc0b9e713bd [weak]
    - MD5Sum:aa1e11ed2ba92ca3da8a9785cdfae2ed [weak]
E: Failed to fetch http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic/pool/all/s/scylla-enterprise-server/scylla-enterprise-server_2020.1.7-0.20210304.e42189e09-1_amd64.deb  File has unexpected size (35725644 != 86464). Mirror sync in progress? [IP: 10.1.1.8 80]
   Hashes of expected file:
    - SHA256:74cbe6f7ba359869f84bf671941e9e7d60b4c302de836f35e73f12b37fe57c46
    - SHA1:ba106ac7ae29c5821a9d3c01d6f601352081d6e3 [weak]
    - MD5Sum:83f1ad8d23dcbf950096c28b88059442 [weak]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

# apt -oAcquire::http::Pipeline-Depth=0 install ...
# install completes with issue

Related issues

Is duplicate of Debian Support - Issue #8506: apt install - File has unexpected size - http pipelineCLOSED - CURRENTRELEASEActions
Actions #1

Updated by jsherril@redhat.com over 3 years ago

From the user:

Opened a bug for apt for Ubuntu: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1921626

Managed to reproduce the issue with bionic, focal, and hirsute.

Turns out apt relies on "Size" field from "Packages" file to match pipeline responses correctly. The file produced by pulp doesn't have "Size" field for packages.

root@aptbug:~# curl -sS https://repo.saltproject.io/py3/ubuntu/18.04/amd64/latest/dists/bionic/main/binary-amd64/Packages | grep -c ^Size
14
root@aptbug:~# curl -sS http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Saltstack/ubuntu_18_04_salt_latest/dists/default/all/binary-amd64/Packages | grep -c ^Size
0

https://salsa.debian.org/apt-team/apt/-/blob/0d25ce3d466ecddea02d171981f011f7dbf95e08/methods/basehttp.cc#L711-731

Actions #2

Updated by quba42 over 3 years ago

  • Is duplicate of Issue #8506: apt install - File has unexpected size - http pipeline added
Actions #3

Updated by jsherril@redhat.com over 3 years ago

  • Status changed from NEW to CLOSED - DUPLICATE

Also available in: Atom PDF