Issue #8512
closeddeb repo - apt install - File has unexpected size - http pipeline
Description
A katello user opened this: https://projects.theforeman.org/issues/32178
I'm getting unexpected size error on ubuntu 18.04 when trying to install from a deb repository. The problem seems to be caused by apt pipelining requests. Only happens on katello deb repos, and when there are many packages to download as seen below. I have ubuntu mirror in the pub directory, not through katello.
My guess is this is issue with pulp, as httpd proxies the requests to it, right?
# apt install -y ...
<snip>
Get:29 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic/main amd64 libxi6 amd64 2:1.7.9-1 [29.2 kB]
Get:30 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic/main amd64 libxrender1 amd64 1:0.9.10-1 [18.7 kB]
Get:31 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic-updates/main amd64 x11-common all 1:7.7+19ubuntu7.1 [22.5 kB]
Get:32 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic/main amd64 libxtst6 amd64 2:1.2.3-1 [12.8 kB]
Get:33 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic-updates/universe amd64 openjdk-8-jre-headless amd64 8u282-b08-0ubuntu1~18.04 [28.2 MB]
Get:34 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic-updates/main amd64 ca-certificates-java all 20180516ubuntu1~18.04.1 [12.2 kB]
Get:35 http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic default/all amd64 scylla-enterprise-conf amd64 2020.1.7-0.20210304.e42189e09-1 [11.8 kB]
Get:36 http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic default/all amd64 scylla-enterprise-python3 amd64 3.7.7-0.20210304.e42189e09-1 [20.0 MB]
Err:36 http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic default/all amd64 scylla-enterprise-python3 amd64 3.7.7-0.20210304.e42189e09-1
File has unexpected size (20004486 != 86464). Mirror sync in progress? [IP: 10.1.1.8 80]
Hashes of expected file:
- SHA256:51a6c7c50ebd72be856b02a9fbe8ac39739e3f0c36bcf6a7c5f1d2d83092286c
- SHA1:02cf7a581beeaf1b067600c72c268dc0b9e713bd [weak]
- MD5Sum:aa1e11ed2ba92ca3da8a9785cdfae2ed [weak]
Get:37 http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic default/all amd64 scylla-enterprise-server amd64 2020.1.7-0.20210304.e42189e09-1 [35.7 MB]
Err:37 http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic default/all amd64 scylla-enterprise-server amd64 2020.1.7-0.20210304.e42189e09-1
File has unexpected size (35725644 != 86464). Mirror sync in progress? [IP: 10.1.1.8 80]
Hashes of expected file:
- SHA256:74cbe6f7ba359869f84bf671941e9e7d60b4c302de836f35e73f12b37fe57c46
- SHA1:ba106ac7ae29c5821a9d3c01d6f601352081d6e3 [weak]
- MD5Sum:83f1ad8d23dcbf950096c28b88059442 [weak]
Get:38 http://foreman.example.com/pub/mirror/archive.ubuntu.com/ubuntu bionic-updates/universe amd64 sntp amd64 1:4.2.8p10+dfsg-5ubuntu7.3 [86.5 kB]
Fetched 38.1 MB in 1s (53.8 MB/s)
E: Failed to fetch http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic/pool/all/s/scylla-enterprise-python3/scylla-enterprise-python3_3.7.7-0.20210304.e42189e09-1_amd64.deb File has unexpected size (20004486 != 86464). Mirror sync in progress? [IP: 10.1.1.8 80]
Hashes of expected file:
- SHA256:51a6c7c50ebd72be856b02a9fbe8ac39739e3f0c36bcf6a7c5f1d2d83092286c
- SHA1:02cf7a581beeaf1b067600c72c268dc0b9e713bd [weak]
- MD5Sum:aa1e11ed2ba92ca3da8a9785cdfae2ed [weak]
E: Failed to fetch http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Scylla/scylladb-enterprise-2020_1_bionic/pool/all/s/scylla-enterprise-server/scylla-enterprise-server_2020.1.7-0.20210304.e42189e09-1_amd64.deb File has unexpected size (35725644 != 86464). Mirror sync in progress? [IP: 10.1.1.8 80]
Hashes of expected file:
- SHA256:74cbe6f7ba359869f84bf671941e9e7d60b4c302de836f35e73f12b37fe57c46
- SHA1:ba106ac7ae29c5821a9d3c01d6f601352081d6e3 [weak]
- MD5Sum:83f1ad8d23dcbf950096c28b88059442 [weak]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
# apt -oAcquire::http::Pipeline-Depth=0 install ...
# install completes with issue
Related issues
Updated by jsherril@redhat.com over 3 years ago
From the user:
Opened a bug for apt for Ubuntu: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1921626
Managed to reproduce the issue with bionic, focal, and hirsute.
Turns out apt relies on "Size" field from "Packages" file to match pipeline responses correctly. The file produced by pulp doesn't have "Size" field for packages.
root@aptbug:~# curl -sS https://repo.saltproject.io/py3/ubuntu/18.04/amd64/latest/dists/bionic/main/binary-amd64/Packages | grep -c ^Size
14
root@aptbug:~# curl -sS http://foreman.example.com/pulp/deb/MyOrg/Library/custom/Saltstack/ubuntu_18_04_salt_latest/dists/default/all/binary-amd64/Packages | grep -c ^Size
0
Updated by quba42 over 3 years ago
- Is duplicate of Issue #8506: apt install - File has unexpected size - http pipeline added
Updated by jsherril@redhat.com over 3 years ago
- Status changed from NEW to CLOSED - DUPLICATE