Project

Profile

Help

Story #3377

Add support for SUSE Errata format

Added by jomitsch@redhat.com about 1 year ago. Updated 5 days ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
% Done:

100%

Platform Release:
2.16.0
Blocks Release:
Backwards Incompatible:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
Yes
Verification Required:
No
Sprint:
Sprint 33

Description

### SLES repo syncs fine, but no errata showing up in counts

Id:                  SLES12-SP3-Updates
Display Name:        None
Description:         None
Content Unit Counts: 
  Drpm:                   412
  Rpm:                    1431
  Srpm:                   369
  Yum Repo Metadata File: 2

### Repo details

Id:                   SLES12-SP3-Updates
Display Name:         None
Description:          None
Content Unit Counts:
  Drpm:                   412
  Rpm:                    1431
  Srpm:                   369
  Yum Repo Metadata File: 2
Notes:
Scratchpad:
  Checksum Type: sha256
Importers:            
  Config:               
    Download Policy:  on_demand
    Feed:             https://updates.suse.com/SUSE/Updates/SLE-SERVER/12-SP3/x8
                      6_64/update/
    Query Auth Token: u8cGUGITIGZ5mVjDaJYNhTvsMlJbJYXNtYGVNvdaUPiGXbmVKoGE2n0-dr
                      LXU0Ac1RSIVZc_CprERYEXsRhhme0Xc3Qpb1PX13iSCqyDwuAnJLrIXpsi
                      cDB_Yv8R32YfglLx8oaoJV3rt0Q
  Id:                   yum_importer
  Importer Type Id:     yum_importer
  Last Override Config: 
    Type Skip List: iso, erratum, distribution
  Last Sync:            2018-02-15T22:41:07Z
  Last Updated:         2018-02-15T22:38:38Z
  Repo Id:              SLES12-SP3-Updates
  Scratchpad:           
    Repomd Revision: 1518707915
Distributors:         
  Auto Publish:         True
  Config:               
    Checksum Type: sha256
    Http:          True
    Https:         True
    Relative URL:  SLES12-SP3-Updates
  Distributor Type Id:  yum_distributor
  Id:                   yum_distributor
  Last Override Config: 
  Last Publish:         2018-02-15T22:41:58Z
  Last Updated:         2018-02-15T22:41:07Z
  Repo Id:              SLES12-SP3-Updates
  Scratchpad:           
  Auto Publish:         False
  Config:               
    Http:         True
    Https:        True
    Relative URL: SLES12-SP3-Updates
  Distributor Type Id:  export_distributor
  Id:                   export_distributor
  Last Override Config: 
  Last Publish:         None
  Last Updated:         2018-02-15T22:33:33Z
  Repo Id:              SLES12-SP3-Updates
  Scratchpad: 

Sync completed without errors, I've tried with the following repos without seeing any errata counts: SLES12-SP2-Pool, SLES12-SP3-Pool, SLES12-SP2-Updates, SLES12-SP3-Updates

Judging from the updateinfo.xml files in the CDN, errata should be present.

To reproduce:

- Get a developer subscription at https://www.suse.com/subscriptions/sles/developer/
- List repo info using the developer subscrption token, I use this script to do so: https://gist.github.com/johnpmitsch/0297e7b0c6ac7e261dbfd559d4903133
- Create any SUSE repo in pulp using info from that list, I use this script: https://gist.github.com/johnpmitsch/b08415e906dcb000b244bc771838fa01
- Sync SLES12-SP3-Updates or another repo with errata

Associated revisions

Revision 1bbf8cc4 View on GitHub
Added by werwty about 1 year ago

Allow errata to be synced with query_auth_token.

Metadata sync currently works with query_auth_token. Removing
TYPE_ID_ERRATA from QUERY_AUTH_TOKEN_UNSUPPORTED allows pulp to sync
updateinfo.xml from SLES, and generate errata content.

Add relogin_suggested, restart_suggested to errata model since these
fields are available on SUSE metadata.

Add migrations to create relogin_suggested and restart_suggested fields
to pulp's errata models in database

fixes #3377
https://pulp.plan.io/issues/3377

History

#1 Updated by jomitsch@redhat.com about 1 year ago

pulp versions:

[root@bacon ~]# rpm -qa | grep pulp
pulp-docker-plugins-3.1.1-1.el7.noarch
python-pulp-ostree-common-1.3.0-1.el7.noarch
pulp-ostree-plugins-1.3.0-1.el7.noarch
rubygem-smart_proxy_pulp-1.3.0-1.el7.noarch
python-pulp-bindings-2.15.1-1.el7.noarch
python-pulp-agent-lib-2.15.1-1.el7.noarch
pulp-client-1.0-1.noarch
python-pulp-common-2.15.1-1.el7.noarch
python-kombu-3.0.33-8.pulp.el7.noarch
python-pulp-repoauth-2.15.1-1.el7.noarch
pulp-selinux-2.15.1-1.el7.noarch
pulp-server-2.15.1-1.el7.noarch
python-pulp-streamer-2.15.1-1.el7.noarch
python-pulp-rpm-common-2.15.1-1.el7.noarch
python-pulp-oid_validation-2.15.1-1.el7.noarch
python-pulp-docker-common-3.1.1-1.el7.noarch
python-pulp-puppet-common-2.15.1-1.el7.noarch
pulp-admin-client-2.15.1-1.el7.noarch
pulp-rpm-handlers-2.15.1-1.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
pulp-katello-1.0.2-1.el7.noarch
pulp-rpm-plugins-2.15.1-1.el7.noarch
pulp-puppet-plugins-2.15.1-1.el7.noarch
python-pulp-client-lib-2.15.1-1.el7.noarch
pulp-rpm-admin-extensions-2.15.1-1.el7.noarch

#3 Updated by dalley about 1 year ago

  • Triaged changed from No to Yes

#4 Updated by bizhang about 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bizhang

#5 Updated by bizhang about 1 year ago

  • Sprint/Milestone set to 56

#6 Updated by Ichimonji10 about 1 year ago

The updateinfo.xml files are different for Red Hat and SUSE repositories. To quote from Pulp Smash :

The purpose of an updateinfo.xml file is to document the errata (i.e. maintenance patches) provided by a repository. For an overview of the structure and contents of such a file, see openSUSE:Standards Rpm Metadata UpdateInfo. Beware that yum and dnf do not adhere to the openSUSE standards. We link to their standard anyway because it’s easier to understand than yum or dnf’s source code. (!)

One discrepancy is in the schema for the <pkglist> element. According to the yum source code, it has the following structure:

<!ELEMENT pkglist (collection+)>
<!ELEMENT collection (name?, package+)>
<!ATTLIST collection short CDATA #IMPLIED>
<!ATTLIST collection name CDATA #IMPLIED>
<!ELEMENT name (#PCDATA)>

Here’s a concrete example of what that might look like:

<pkglist>
<collection name="…" short="…">
<name>…</name>
<package>…</package>
<package>…</package>
</collection>
</pkglist>

yum (and, therefore, dnf) allows a <collection> element to have “name” and “short” attributes. openSUSE does not.

I don't know if this specific discrepancy is at fault. But it's worth keeping in mind.

#7 Updated by bizhang about 1 year ago

Errata sync currently does not support the `query_auth_token` option used to sync other SUSE contents0.

For SLES>12 basic auth isn't supported, so query_auth_token has to be used.

Removing TYPE_ID_ERRATA from QUERY_AUTH_TOKEN_UNSUPPORTED1 seems to work out of the box [2]
Looking at the original implementation of query_auth_token [3] it appears that Errata is unsupported was because it wasn't a part of the scope

"Summing up
Assuming the Distribution and Errata content types can be ignored, the bulk of the work for munging the urls would need to be done in MetadataFiles"

[0] https://docs.pulpproject.org/plugins/pulp_rpm/tech-reference/yum-plugins.html?highlight=query_auth_token#configuration-parameters
[1] https://github.com/pulp/pulp_rpm/blob/master/common/pulp_rpm/common/ids.py#L66
[2] https://paste.fedoraproject.org/paste/bOs-E1ewq8C0OMwgYnbu7g
[3] https://pulp.plan.io/issues/1358

#8 Updated by bizhang about 1 year ago

  • Status changed from ASSIGNED to POST

#9 Updated by bizhang about 1 year ago

  • Tracker changed from Issue to Story
  • Subject changed from Errata not showing for SUSE repo to Add support for SUSE Errata format
  • % Done set to 0

#11 Updated by bmbouter about 1 year ago

  • Sprint set to Sprint 33

#12 Updated by bmbouter about 1 year ago

  • Sprint/Milestone deleted (56)

#13 Updated by werwty about 1 year ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#14 Updated by bizhang about 1 year ago

  • Smash Test set to 904

#15 Updated by ttereshc about 1 year ago

  • Platform Release set to 2.16.0

#16 Updated by bmbouter about 1 year ago

  • Status changed from MODIFIED to ON_QA

#17 Updated by Ichimonji10 about 1 year ago

  • Verified changed from No to Yes

#18 Updated by bmbouter about 1 year ago

  • Status changed from ON_QA to CLOSED - CURRENTRELEASE

#19 Updated by bmbouter 5 days ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF