Story #3377: Add support for SUSE Errata format - RPM Support - Pulp

Actions

Send by e-mail Copy link

Story #3377

closed

Add support for SUSE Errata format

Added by jomitsch@redhat.com almost 7 years ago. Updated over 5 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

bizhang

Sprint/Milestone:

Start date:

Due date:

% Done:

100%

Estimated time:

Platform Release:

2.16.0

Groomed:

Sprint Candidate:

Tags:

Pulp 2

Sprint:

Sprint 33

Quarter:

Description

### SLES repo syncs fine, but no errata showing up in counts

Id:                  SLES12-SP3-Updates
Display Name:        None
Description:         None
Content Unit Counts: 
  Drpm:                   412
  Rpm:                    1431
  Srpm:                   369
  Yum Repo Metadata File: 2

### Repo details

Id:                   SLES12-SP3-Updates
Display Name:         None
Description:          None
Content Unit Counts:
  Drpm:                   412
  Rpm:                    1431
  Srpm:                   369
  Yum Repo Metadata File: 2
Notes:
Scratchpad:
  Checksum Type: sha256
Importers:            
  Config:               
    Download Policy:  on_demand
    Feed:             https://updates.suse.com/SUSE/Updates/SLE-SERVER/12-SP3/x8
                      6_64/update/
    Query Auth Token: u8cGUGITIGZ5mVjDaJYNhTvsMlJbJYXNtYGVNvdaUPiGXbmVKoGE2n0-dr
                      LXU0Ac1RSIVZc_CprERYEXsRhhme0Xc3Qpb1PX13iSCqyDwuAnJLrIXpsi
                      cDB_Yv8R32YfglLx8oaoJV3rt0Q
  Id:                   yum_importer
  Importer Type Id:     yum_importer
  Last Override Config: 
    Type Skip List: iso, erratum, distribution
  Last Sync:            2018-02-15T22:41:07Z
  Last Updated:         2018-02-15T22:38:38Z
  Repo Id:              SLES12-SP3-Updates
  Scratchpad:           
    Repomd Revision: 1518707915
Distributors:         
  Auto Publish:         True
  Config:               
    Checksum Type: sha256
    Http:          True
    Https:         True
    Relative URL:  SLES12-SP3-Updates
  Distributor Type Id:  yum_distributor
  Id:                   yum_distributor
  Last Override Config: 
  Last Publish:         2018-02-15T22:41:58Z
  Last Updated:         2018-02-15T22:41:07Z
  Repo Id:              SLES12-SP3-Updates
  Scratchpad:           
  Auto Publish:         False
  Config:               
    Http:         True
    Https:        True
    Relative URL: SLES12-SP3-Updates
  Distributor Type Id:  export_distributor
  Id:                   export_distributor
  Last Override Config: 
  Last Publish:         None
  Last Updated:         2018-02-15T22:33:33Z
  Repo Id:              SLES12-SP3-Updates
  Scratchpad:

Sync completed without errors, I've tried with the following repos without seeing any errata counts: SLES12-SP2-Pool, SLES12-SP3-Pool, SLES12-SP2-Updates, SLES12-SP3-Updates

Judging from the updateinfo.xml files in the CDN, errata should be present.

To reproduce:

- Get a developer subscription at https://www.suse.com/subscriptions/sles/developer/
- List repo info using the developer subscrption token, I use this script to do so: https://gist.github.com/johnpmitsch/0297e7b0c6ac7e261dbfd559d4903133
- Create any SUSE repo in pulp using info from that list, I use this script: https://gist.github.com/johnpmitsch/b08415e906dcb000b244bc771838fa01
- Sync SLES12-SP3-Updates or another repo with errata

Actions

Copy link

Updated by jomitsch@redhat.com almost 7 years ago

pulp versions:

[root@bacon ~]# rpm -qa | grep pulp
pulp-docker-plugins-3.1.1-1.el7.noarch
python-pulp-ostree-common-1.3.0-1.el7.noarch
pulp-ostree-plugins-1.3.0-1.el7.noarch
rubygem-smart_proxy_pulp-1.3.0-1.el7.noarch
python-pulp-bindings-2.15.1-1.el7.noarch
python-pulp-agent-lib-2.15.1-1.el7.noarch
pulp-client-1.0-1.noarch
python-pulp-common-2.15.1-1.el7.noarch
python-kombu-3.0.33-8.pulp.el7.noarch
python-pulp-repoauth-2.15.1-1.el7.noarch
pulp-selinux-2.15.1-1.el7.noarch
pulp-server-2.15.1-1.el7.noarch
python-pulp-streamer-2.15.1-1.el7.noarch
python-pulp-rpm-common-2.15.1-1.el7.noarch
python-pulp-oid_validation-2.15.1-1.el7.noarch
python-pulp-docker-common-3.1.1-1.el7.noarch
python-pulp-puppet-common-2.15.1-1.el7.noarch
pulp-admin-client-2.15.1-1.el7.noarch
pulp-rpm-handlers-2.15.1-1.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
pulp-katello-1.0.2-1.el7.noarch
pulp-rpm-plugins-2.15.1-1.el7.noarch
pulp-puppet-plugins-2.15.1-1.el7.noarch
python-pulp-client-lib-2.15.1-1.el7.noarch
pulp-rpm-admin-extensions-2.15.1-1.el7.noarch

Actions

Copy link

Updated by dalley almost 7 years ago

Triaged changed from No to Yes

Actions

Copy link

Updated by bizhang almost 7 years ago

Status changed from NEW to ASSIGNED
Assignee set to bizhang

Actions

Copy link

Updated by bizhang almost 7 years ago

Sprint/Milestone set to 56

Actions

Copy link

Updated by Ichimonji10 almost 7 years ago

The updateinfo.xml files are different for Red Hat and SUSE repositories. To quote from Pulp Smash :

The purpose of an updateinfo.xml file is to document the errata (i.e. maintenance patches) provided by a repository. For an overview of the structure and contents of such a file, see openSUSE:Standards Rpm Metadata UpdateInfo. Beware that yum and dnf do not adhere to the openSUSE standards. We link to their standard anyway because it’s easier to understand than yum or dnf’s source code. (!)

One discrepancy is in the schema for the <pkglist> element. According to the yum source code, it has the following structure:
<!ELEMENT pkglist (collection+)>
<!ELEMENT collection (name?, package+)>
<!ATTLIST collection short CDATA #IMPLIED>
<!ATTLIST collection name CDATA #IMPLIED>
<!ELEMENT name (#PCDATA)>
Here’s a concrete example of what that might look like:
<pkglist>
<collection name="…" short="…">
<name>…</name>
<package>…</package>
<package>…</package>
</collection>
</pkglist>
yum (and, therefore, dnf) allows a <collection> element to have “name” and “short” attributes. openSUSE does not.

I don't know if this specific discrepancy is at fault. But it's worth keeping in mind.

Actions

Copy link

Updated by bizhang almost 7 years ago

Errata sync currently does not support the `query_auth_token` option used to sync other SUSE contents[0].

For SLES>12 basic auth isn't supported, so query_auth_token has to be used.

Removing TYPE_ID_ERRATA from QUERY_AUTH_TOKEN_UNSUPPORTED[1] seems to work out of the box [2]
Looking at the original implementation of query_auth_token [3] it appears that Errata is unsupported was because it wasn't a part of the scope

"Summing up
Assuming the Distribution and Errata content types can be ignored, the bulk of the work for munging the urls would need to be done in MetadataFiles"

[0] https://docs.pulpproject.org/plugins/pulp_rpm/tech-reference/yum-plugins.html?highlight=query_auth_token#configuration-parameters
[1] https://github.com/pulp/pulp_rpm/blob/master/common/pulp_rpm/common/ids.py#L66
[2] https://paste.fedoraproject.org/paste/bOs-E1ewq8C0OMwgYnbu7g
[3] https://pulp.plan.io/issues/1358

Actions

Copy link

Updated by bizhang almost 7 years ago

Status changed from ASSIGNED to POST

PR: https://github.com/pulp/pulp_rpm/pull/1088

Actions

Copy link

Updated by bizhang almost 7 years ago

Tracker changed from Issue to Story
Subject changed from Errata not showing for SUSE repo to Add support for SUSE Errata format
% Done set to 0

Actions

Copy link

#11

Updated by bmbouter almost 7 years ago

Sprint set to Sprint 33

Actions

Copy link

#12

Updated by bmbouter almost 7 years ago

Sprint/Milestone deleted (56)

Added by werwty almost 7 years ago

Revision 1bbf8cc4 | View on GitHub

Allow errata to be synced with query_auth_token.

Metadata sync currently works with query_auth_token. Removing TYPE_ID_ERRATA from QUERY_AUTH_TOKEN_UNSUPPORTED allows pulp to sync updateinfo.xml from SLES, and generate errata content.

Add relogin_suggested, restart_suggested to errata model since these fields are available on SUSE metadata.

Add migrations to create relogin_suggested and restart_suggested fields to pulp's errata models in database

fixes #3377 https://pulp.plan.io/issues/3377

Actions

Copy link

#13