Project

Profile

Help

Issue #8232

APT publications do not retain all fields from upstream package indecies

Added by mathiasdasilva 9 months ago. Updated 26 days ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Debian:
master
Platform Release:
Target Release - Debian:
OS:
CentOS 7
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Dear support,

When we plug our servers to our apt pulp repository, same packages are installed again an again. I checked and there is a difference between source package manifest and pulp deb manifest. There is an example for wget :

Ubuntu repository :

Package: wget Architecture: amd64 Version: 1.20.3-1ubuntu1 Multi-Arch: foreign Priority: standard Section: web Origin: Ubuntu Maintainer: Ubuntu Developers Original-Maintainer: Noël Köthe Bugs: https://bugs.launchpad.net/ubuntu/+filebug Installed-Size: 992 Depends: libc6 (>= 2.17), libidn2-0 (>= 0.6), libpcre2-8-0 (>= 10.32), libpsl5 (>= 0.16.0), libssl1.1 (>= 1.1.0), libuuid1 (>= 2.16), zlib1g (>= 1:1.1.4) Recommends: ca-certificates Conflicts: wget-ssl Filename: pool/main/w/wget/wget_1.20.3-1ubuntu1_amd64.deb Size: 348824 MD5sum: ae9eb859432505828dab97c80edcb5b5 SHA1: 45504b6e948d4f1a7e2ba0233bb57a67b392ceea SHA256: 68144dde6d45e1a54fa983d6d10be7043e6695e259c8d2b12fd03ea5e475d56a Homepage: https://www.gnu.org/software/wget/ Description: retrieves files from the web Task: standard Description-md5: 63a4a740bcd9e8e94bf661e4f1806e02

Pulp repository : Package: wget Version: 1.20.3-1ubuntu1 Architecture: amd64 Section: web Priority: standard Origin: Ubuntu Bugs: https://bugs.launchpad.net/ubuntu/+filebug Installed-Size: 992 Maintainer: Ubuntu Developers Description: retrieves files from the web Homepage: https://www.gnu.org/software/wget/ Conflicts: wget-ssl Depends: libc6 (>= 2.17), libidn2-0 (>= 0.6), libpcre2-8-0 (>= 10.32), libpsl5 (>= 0.16.0), libssl1.1 (>= 1.1.0), libuuid1 (>= 2.16), zlib1g (>= 1:1.1.4) Recommends: ca-certificates MD5sum: ae9eb859432505828dab97c80edcb5b5 SHA1: 45504b6e948d4f1a7e2ba0233bb57a67b392ceea SHA256: 68144dde6d45e1a54fa983d6d10be7043e6695e259c8d2b12fd03ea5e475d56a Filename: pool/main/w/wget/wget_1.20.3-1ubuntu1_amd64.deb

This behavior seems to be identical to this ticket : https://pulp.plan.io/issues/7190

Pulp version:

Pulp core: 3.9.0 Pulp deb: 2.9.0

Best regards,


Related issues

Related to Debian Support - Issue #8506: apt install - File has unexpected size - http pipelineCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 0cad4469 View on GitHub
Added by mbucher 3 months ago

Add store additional package-index fields

Closes #8232 https://pulp.plan.io/issues/8232

History

#1 Updated by quba42 9 months ago

This suggests pulp_deb is losing the following fields: Multi-Arch, Original-Maintainer, Size, Task, Description-md5

I will need to investigate.

#2 Updated by mathiasdasilva 9 months ago

If you need it, I can send our repository / remote / publication / distribution configuration.

#3 Updated by quba42 9 months ago

The options for the remote are generally useful.

The only other thing beyond that that might be of interest is what combination of "structured" and/or "simple" mode you used for the APT publisher. (This is probably irrelevant to this particular issue but can't hurt either.)

#4 Updated by mathiasdasilva 9 months ago

There is the remote option :

   {
        "architectures": "amd64",
        "ca_cert": null,
        "client_cert": null,
        "client_key": null,
        "components": "main restricted",
        "connect_timeout": null,
        "distributions": "bionic bionic-security xenial xenial-security focal focal-security",
        "download_concurrency": 20,
        "gpgkey": null,
        "ignore_missing_package_indices": false,
        "name": "ubuntu",
        "password": null,
        "policy": "on_demand",
        "proxy_url": null,
        "pulp_created": "2021-02-09T11:21:22.167056Z",
        "pulp_href": "/pulp/api/v3/remotes/deb/apt/b6c2576e-c050-4ac1-8250-74178bacfd2c/",
        "pulp_last_updated": "2021-02-10T07:29:19.060281Z",
        "sock_connect_timeout": null,
        "sock_read_timeout": null,
        "sync_installer": false,
        "sync_sources": false,
        "sync_udebs": false,
        "tls_validation": true,
        "total_timeout": null,
        "url": "http://archive.ubuntu.com/ubuntu/",
        "username": null
    },

The publication is in structured mode. I also tried with simple and we've got same behavior.

#5 Updated by quba42 7 months ago

  • Related to Issue #8506: apt install - File has unexpected size - http pipeline added

#6 Updated by quba42 6 months ago

  • Sprint/Milestone set to Katello

#7 Updated by quba42 6 months ago

  • Priority changed from Normal to High

#8 Updated by pulpbot 5 months ago

  • Status changed from NEW to POST

#9 Updated by quba42 3 months ago

  • Subject changed from Pulp 3 - pulp-deb : APT client installing same patches again and again to APT publications do not retain all fields from upstream package indecies

Note the Size field has now been fixed under a separate issue: https://pulp.plan.io/issues/8506

I expect this required field was responsible for the symptoms experienced here.

We will be adding functionality to retain arbitrary Package index fields not handled elsewhere under this ticket.

#10 Updated by mbucher 3 months ago

  • Status changed from POST to MODIFIED

#11 Updated by quba42 about 2 months ago

  • Sprint/Milestone changed from Katello to 2.15.0

#12 Updated by pulpbot about 2 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

#13 Updated by mathiasdasilva about 1 month ago

Hello,

Thank's for the fix on Size field. I updated to pulp_deb 15 but i still got the issue.

I checked and the Description-md5 filed is still missing from Release file.

#14 Updated by quba42 26 days ago

mathiasdasilva wrote:

Hello,

Thank's for the fix on Size field. I updated to pulp_deb 15 but i still got the issue.

I checked and the Description-md5 filed is still missing from Release file.

Description-md5 is a field within Packages files not Release files, but I am guessing that was just a slip of the tongue.

I have re-tested, and for upstream repositories containing Description-md5 fields in their package indices (Packages files), I am getting the Description-md5 field in the indices published by pulp_deb. Is it possible the repo you are syncing, does not contain the field? It all seems to be working as expected for me.

Please register to edit this issue

Also available in: Atom PDF