Issue #8232
closedAPT publications do not retain all fields from upstream package indecies
Description
Dear support,
When we plug our servers to our apt pulp repository, same packages are installed again an again. I checked and there is a difference between source package manifest and pulp deb manifest. There is an example for wget :
Ubuntu repository :
Package: wget Architecture: amd64 Version: 1.20.3-1ubuntu1 Multi-Arch: foreign Priority: standard Section: web Origin: Ubuntu Maintainer: Ubuntu Developers ubuntu-devel-discuss@lists.ubuntu.com Original-Maintainer: Noël Köthe noel@debian.org Bugs: https://bugs.launchpad.net/ubuntu/+filebug Installed-Size: 992 Depends: libc6 (>= 2.17), libidn2-0 (>= 0.6), libpcre2-8-0 (>= 10.32), libpsl5 (>= 0.16.0), libssl1.1 (>= 1.1.0), libuuid1 (>= 2.16), zlib1g (>= 1:1.1.4) Recommends: ca-certificates Conflicts: wget-ssl Filename: pool/main/w/wget/wget_1.20.3-1ubuntu1_amd64.deb Size: 348824 MD5sum: ae9eb859432505828dab97c80edcb5b5 SHA1: 45504b6e948d4f1a7e2ba0233bb57a67b392ceea SHA256: 68144dde6d45e1a54fa983d6d10be7043e6695e259c8d2b12fd03ea5e475d56a Homepage: https://www.gnu.org/software/wget/ Description: retrieves files from the web Task: standard Description-md5: 63a4a740bcd9e8e94bf661e4f1806e02
Pulp repository : Package: wget Version: 1.20.3-1ubuntu1 Architecture: amd64 Section: web Priority: standard Origin: Ubuntu Bugs: https://bugs.launchpad.net/ubuntu/+filebug Installed-Size: 992 Maintainer: Ubuntu Developers ubuntu-devel-discuss@lists.ubuntu.com Description: retrieves files from the web Homepage: https://www.gnu.org/software/wget/ Conflicts: wget-ssl Depends: libc6 (>= 2.17), libidn2-0 (>= 0.6), libpcre2-8-0 (>= 10.32), libpsl5 (>= 0.16.0), libssl1.1 (>= 1.1.0), libuuid1 (>= 2.16), zlib1g (>= 1:1.1.4) Recommends: ca-certificates MD5sum: ae9eb859432505828dab97c80edcb5b5 SHA1: 45504b6e948d4f1a7e2ba0233bb57a67b392ceea SHA256: 68144dde6d45e1a54fa983d6d10be7043e6695e259c8d2b12fd03ea5e475d56a Filename: pool/main/w/wget/wget_1.20.3-1ubuntu1_amd64.deb
This behavior seems to be identical to this ticket : https://pulp.plan.io/issues/7190
Pulp version:
Pulp core: 3.9.0 Pulp deb: 2.9.0
Best regards,
Related issues
Updated by quba42 almost 4 years ago
This suggests pulp_deb is losing the following fields: Multi-Arch, Original-Maintainer, Size, Task, Description-md5
I will need to investigate.
Updated by mathiasdasilva almost 4 years ago
If you need it, I can send our repository / remote / publication / distribution configuration.
Updated by quba42 almost 4 years ago
The options for the remote are generally useful.
The only other thing beyond that that might be of interest is what combination of "structured" and/or "simple" mode you used for the APT publisher. (This is probably irrelevant to this particular issue but can't hurt either.)
Updated by mathiasdasilva almost 4 years ago
There is the remote option :
{
"architectures": "amd64",
"ca_cert": null,
"client_cert": null,
"client_key": null,
"components": "main restricted",
"connect_timeout": null,
"distributions": "bionic bionic-security xenial xenial-security focal focal-security",
"download_concurrency": 20,
"gpgkey": null,
"ignore_missing_package_indices": false,
"name": "ubuntu",
"password": null,
"policy": "on_demand",
"proxy_url": null,
"pulp_created": "2021-02-09T11:21:22.167056Z",
"pulp_href": "/pulp/api/v3/remotes/deb/apt/b6c2576e-c050-4ac1-8250-74178bacfd2c/",
"pulp_last_updated": "2021-02-10T07:29:19.060281Z",
"sock_connect_timeout": null,
"sock_read_timeout": null,
"sync_installer": false,
"sync_sources": false,
"sync_udebs": false,
"tls_validation": true,
"total_timeout": null,
"url": "http://archive.ubuntu.com/ubuntu/",
"username": null
},
The publication is in structured mode. I also tried with simple and we've got same behavior.
Updated by quba42 over 3 years ago
- Related to Issue #8506: apt install - File has unexpected size - http pipeline added
Updated by pulpbot over 3 years ago
- Status changed from NEW to POST
Updated by quba42 over 3 years ago
- Subject changed from Pulp 3 - pulp-deb : APT client installing same patches again and again to APT publications do not retain all fields from upstream package indecies
Note the Size field has now been fixed under a separate issue: https://pulp.plan.io/issues/8506
I expect this required field was responsible for the symptoms experienced here.
We will be adding functionality to retain arbitrary Package index fields not handled elsewhere under this ticket.
Added by mbucher over 3 years ago
Updated by mbucher over 3 years ago
- Status changed from POST to MODIFIED
Applied in changeset 0cad44697aa8f2db88fdfc4000fcfc408f9e81b4.
Updated by quba42 over 3 years ago
- Sprint/Milestone changed from Katello to 2.15.0
Updated by pulpbot over 3 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Updated by mathiasdasilva about 3 years ago
Hello,
Thank's for the fix on Size field. I updated to pulp_deb 15 but i still got the issue.
I checked and the Description-md5 filed is still missing from Release file.
Updated by quba42 about 3 years ago
mathiasdasilva wrote:
Hello,
Thank's for the fix on Size field. I updated to pulp_deb 15 but i still got the issue.
I checked and the Description-md5 filed is still missing from Release file.
Description-md5 is a field within Packages files not Release files, but I am guessing that was just a slip of the tongue.
I have re-tested, and for upstream repositories containing Description-md5 fields in their package indices (Packages files), I am getting the Description-md5 field in the indices published by pulp_deb. Is it possible the repo you are syncing, does not contain the field? It all seems to be working as expected for me.
Add store additional package-index fields
Closes #8232 https://pulp.plan.io/issues/8232