Project

Profile

Help

Test #4542

closed

[pulp_rpm] - Repository publishing duplicates RPM files under Packages and Packages/<LETTER> structure

Added by bherring over 5 years ago. Updated almost 4 years ago.

Status:
CLOSED - COMPLETE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Version:
Platform Release:
Tags:
Pulp 2
Sprint:
Quarter:

Description

Some repositories are getting content duplicated on its repodata. For example, the CentoOS 7 Base repository, gets duplicated symlinks as follows:

[root@pulp ~]# pulp-admin  rpm repo list --details --repo-id centos-7-base
+----------------------------------------------------------------------+
                            RPM Repositories
+----------------------------------------------------------------------+

Id:                   centos-7-base
Display Name:         None
Description:          None
Content Unit Counts:  
  Distribution:        1
  Package Category:    11
  Package Environment: 10
  Package Group:       88
  Package Langpacks:   1
  Rpm:                 10019
Notes:                
Scratchpad:           
  Checksum Type: sha256
Importers:            
  Config:               
    Feed: http://centos.mirror.constant.com/7/os/x86_64/
  Id:                   yum_importer
  Importer Type Id:     yum_importer
  Last Override Config: 
  Last Sync:            2019-03-13T19:58:14Z
  Last Updated:         2019-03-13T19:54:40Z
  Repo Id:              centos-7-base
  Scratchpad:           
    Repomd Revision: 1543161601
Distributors:         
  Auto Publish:         True
  Config:               
    Http:         True
    Https:        True
    Relative URL: 7/os/x86_64/
  Distributor Type Id:  yum_distributor
  Id:                   yum_distributor
  Last Override Config: 
  Last Publish:         2019-03-14T04:17:40Z
  Last Updated:         2019-03-13T19:54:40Z
  Repo Id:              centos-7-base
  Scratchpad:           
  Auto Publish:         False
  Config:               
    Http:         True
    Https:        True
    Relative URL: 7/os/x86_64/
  Distributor Type Id:  export_distributor
  Id:                   export_distributor
  Last Override Config: 
  Last Publish:         None
  Last Updated:         2019-03-13T19:54:40Z
  Repo Id:              centos-7-base
  Scratchpad:           

Once the repository is published, it ended up having duplicated RPMs

pulp-admin  rpm repo  publish run  --repo-id df05f52b-431e-483d-9878-9ffef41d70c8 --force-full

○ → tree /var/lib/pulp/published/yum/https/repos/7/os/x86_64/Packages/  | grep 389-ds-base
│   ├── 389-ds-base-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/07/cb4bf5d6199cdc7192cbdbbf283cccdef38f18751c53660107b14232d2d349/389-ds-base-1.3.8.4-15.el7.x86_64.rpm
│   ├── 389-ds-base-devel-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/ab/e6f96db9ad2c0f7f097f71b506b25a0d83a4a9926e904100f006361ef66d1f/389-ds-base-devel-1.3.8.4-15.el7.x86_64.rpm
│   ├── 389-ds-base-libs-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/94/5011e802dbf9d470522eac943442a8718e08ba321dc0649a55a0d7f130e363/389-ds-base-libs-1.3.8.4-15.el7.x86_64.rpm
│   └── 389-ds-base-snmp-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/e9/b6b32a0421e87ab83a6ea7346f51e3929746e10eeb2c9b5f05e8278c9e7b61/389-ds-base-snmp-1.3.8.4-15.el7.x86_64.rpm
├── 389-ds-base-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/07/cb4bf5d6199cdc7192cbdbbf283cccdef38f18751c53660107b14232d2d349/389-ds-base-1.3.8.4-15.el7.x86_64.rpm
├── 389-ds-base-devel-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/ab/e6f96db9ad2c0f7f097f71b506b25a0d83a4a9926e904100f006361ef66d1f/389-ds-base-devel-1.3.8.4-15.el7.x86_64.rpm
├── 389-ds-base-libs-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/94/5011e802dbf9d470522eac943442a8718e08ba321dc0649a55a0d7f130e363/389-ds-base-libs-1.3.8.4-15.el7.x86_64.rpm
├── 389-ds-base-snmp-1.3.8.4-15.el7.x86_64.rpm -> /var/lib/pulp/content/units/rpm/e9/b6b32a0421e87ab83a6ea7346f51e3929746e10eeb2c9b5f05e8278c9e7b61/389-ds-base-snmp-1.3.8.4-15.el7.x86_64.rpm

That creates a big problem whenever exporting the content via ISO , as the size of the repo will be twice big.

The repodata is being created correctly as it only references the files under the new strucuture Packages/<first_letter>/

○ → zcat  /var/lib/pulp/published/yum/https/repos/7/os/x86_64/Packages/repodata/a3f98f3b850725e0ac52472653efd309be6ba436296e66b1b997ca92c95fcdf1-primary.xml.gz  | grep 'href' | grep 389-ds-base
  <location href="Packages/3/389-ds-base-libs-1.3.8.4-15.el7.x86_64.rpm"/>
  <location href="Packages/3/389-ds-base-1.3.8.4-15.el7.x86_64.rpm"/>
  <location href="Packages/3/389-ds-base-snmp-1.3.8.4-15.el7.x86_64.rpm"/>
  <location href="Packages/3/389-ds-base-devel-1.3.8.4-15.el7.x86_64.rpm"/>

Related issues

Copied from RPM Support - Issue #4541: Repository publishing duplicates RPM files under Packages and Packages/<LETTER> structureCLOSED - CURRENTRELEASEdkliban@redhat.comActions

Also available in: Atom PDF