Project

Profile

Help

Issue #8232

closed

APT publications do not retain all fields from upstream package indecies

Added by mathiasdasilva about 3 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Debian:
master
Platform Release:
Target Release - Debian:
OS:
CentOS 7
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Dear support,

When we plug our servers to our apt pulp repository, same packages are installed again an again. I checked and there is a difference between source package manifest and pulp deb manifest. There is an example for wget :

Ubuntu repository :

Package: wget Architecture: amd64 Version: 1.20.3-1ubuntu1 Multi-Arch: foreign Priority: standard Section: web Origin: Ubuntu Maintainer: Ubuntu Developers Original-Maintainer: Noël Köthe Bugs: https://bugs.launchpad.net/ubuntu/+filebug Installed-Size: 992 Depends: libc6 (>= 2.17), libidn2-0 (>= 0.6), libpcre2-8-0 (>= 10.32), libpsl5 (>= 0.16.0), libssl1.1 (>= 1.1.0), libuuid1 (>= 2.16), zlib1g (>= 1:1.1.4) Recommends: ca-certificates Conflicts: wget-ssl Filename: pool/main/w/wget/wget_1.20.3-1ubuntu1_amd64.deb Size: 348824 MD5sum: ae9eb859432505828dab97c80edcb5b5 SHA1: 45504b6e948d4f1a7e2ba0233bb57a67b392ceea SHA256: 68144dde6d45e1a54fa983d6d10be7043e6695e259c8d2b12fd03ea5e475d56a Homepage: https://www.gnu.org/software/wget/ Description: retrieves files from the web Task: standard Description-md5: 63a4a740bcd9e8e94bf661e4f1806e02

Pulp repository : Package: wget Version: 1.20.3-1ubuntu1 Architecture: amd64 Section: web Priority: standard Origin: Ubuntu Bugs: https://bugs.launchpad.net/ubuntu/+filebug Installed-Size: 992 Maintainer: Ubuntu Developers Description: retrieves files from the web Homepage: https://www.gnu.org/software/wget/ Conflicts: wget-ssl Depends: libc6 (>= 2.17), libidn2-0 (>= 0.6), libpcre2-8-0 (>= 10.32), libpsl5 (>= 0.16.0), libssl1.1 (>= 1.1.0), libuuid1 (>= 2.16), zlib1g (>= 1:1.1.4) Recommends: ca-certificates MD5sum: ae9eb859432505828dab97c80edcb5b5 SHA1: 45504b6e948d4f1a7e2ba0233bb57a67b392ceea SHA256: 68144dde6d45e1a54fa983d6d10be7043e6695e259c8d2b12fd03ea5e475d56a Filename: pool/main/w/wget/wget_1.20.3-1ubuntu1_amd64.deb

This behavior seems to be identical to this ticket : https://pulp.plan.io/issues/7190

Pulp version:

Pulp core: 3.9.0 Pulp deb: 2.9.0

Best regards,


Related issues

Related to Debian Support - Issue #8506: apt install - File has unexpected size - http pipelineCLOSED - CURRENTRELEASEActions
Actions #1

Updated by quba42 about 3 years ago

This suggests pulp_deb is losing the following fields: Multi-Arch, Original-Maintainer, Size, Task, Description-md5

I will need to investigate.

Actions #2

Updated by mathiasdasilva about 3 years ago

If you need it, I can send our repository / remote / publication / distribution configuration.

Actions #3

Updated by quba42 about 3 years ago

The options for the remote are generally useful.

The only other thing beyond that that might be of interest is what combination of "structured" and/or "simple" mode you used for the APT publisher. (This is probably irrelevant to this particular issue but can't hurt either.)

Actions #4

Updated by mathiasdasilva about 3 years ago

There is the remote option :

   {
        "architectures": "amd64",
        "ca_cert": null,
        "client_cert": null,
        "client_key": null,
        "components": "main restricted",
        "connect_timeout": null,
        "distributions": "bionic bionic-security xenial xenial-security focal focal-security",
        "download_concurrency": 20,
        "gpgkey": null,
        "ignore_missing_package_indices": false,
        "name": "ubuntu",
        "password": null,
        "policy": "on_demand",
        "proxy_url": null,
        "pulp_created": "2021-02-09T11:21:22.167056Z",
        "pulp_href": "/pulp/api/v3/remotes/deb/apt/b6c2576e-c050-4ac1-8250-74178bacfd2c/",
        "pulp_last_updated": "2021-02-10T07:29:19.060281Z",
        "sock_connect_timeout": null,
        "sock_read_timeout": null,
        "sync_installer": false,
        "sync_sources": false,
        "sync_udebs": false,
        "tls_validation": true,
        "total_timeout": null,
        "url": "http://archive.ubuntu.com/ubuntu/",
        "username": null
    },

The publication is in structured mode. I also tried with simple and we've got same behavior.

Actions #5

Updated by quba42 almost 3 years ago

  • Related to Issue #8506: apt install - File has unexpected size - http pipeline added
Actions #6

Updated by quba42 almost 3 years ago

  • Sprint/Milestone set to Katello
Actions #7

Updated by quba42 almost 3 years ago

  • Priority changed from Normal to High
Actions #8

Updated by pulpbot almost 3 years ago

  • Status changed from NEW to POST
Actions #9

Updated by quba42 over 2 years ago

  • Subject changed from Pulp 3 - pulp-deb : APT client installing same patches again and again to APT publications do not retain all fields from upstream package indecies

Note the Size field has now been fixed under a separate issue: https://pulp.plan.io/issues/8506

I expect this required field was responsible for the symptoms experienced here.

We will be adding functionality to retain arbitrary Package index fields not handled elsewhere under this ticket.

Added by mbucher over 2 years ago

Revision 0cad4469 | View on GitHub

Add store additional package-index fields

Closes #8232 https://pulp.plan.io/issues/8232

Actions #10

Updated by mbucher over 2 years ago

  • Status changed from POST to MODIFIED
Actions #11

Updated by quba42 over 2 years ago

  • Sprint/Milestone changed from Katello to 2.15.0
Actions #12

Updated by pulpbot over 2 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #13

Updated by mathiasdasilva over 2 years ago

Hello,

Thank's for the fix on Size field. I updated to pulp_deb 15 but i still got the issue.

I checked and the Description-md5 filed is still missing from Release file.

Actions #14

Updated by quba42 over 2 years ago

mathiasdasilva wrote:

Hello,

Thank's for the fix on Size field. I updated to pulp_deb 15 but i still got the issue.

I checked and the Description-md5 filed is still missing from Release file.

Description-md5 is a field within Packages files not Release files, but I am guessing that was just a slip of the tongue.

I have re-tested, and for upstream repositories containing Description-md5 fields in their package indices (Packages files), I am getting the Description-md5 field in the indices published by pulp_deb. Is it possible the repo you are syncing, does not contain the field? It all seems to be working as expected for me.

Also available in: Atom PDF