Task #927
closed
Test with openSUSE
Added by mhrivnak over 9 years ago. Updated over 5 years ago.
0%
Description
Test basic yum repo management for opensuse.
Deliverables
- sync and publish opensuse repos
- if possible, install and provision from a pulp-published repo
- install software and updates from a pulp-published repo
- file issues for any problems or gaps that are discovered
Files
| create_package_descr (21.2 KB) create_package_descr | semyers, 10/26/2015 04:45 PM |
Related issues
Updated by bmbouter over 9 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to bmbouter
- Sprint/Milestone set to 15
Updated by mhrivnak over 9 years ago
- Related to Story #240: YUM repositories using deltainfo don't get delta RPMs published added
Updated by bmbouter over 9 years ago
- Sprint/Milestone changed from 15 to 16
- Groomed set to No
- Sprint Candidate set to No
Updated by bmbouter over 9 years ago
- Groomed changed from No to Yes
- Tags deleted (
Groomed)
Updated by bmbouter over 9 years ago
- Sprint Candidate changed from No to Yes
- Tags deleted (
Sprint Candidate)
Updated by dkliban@redhat.com over 9 years ago
- Assignee changed from bmbouter to dkliban@redhat.com
Updated by dkliban@redhat.com over 9 years ago
- Status changed from ASSIGNED to NEW
- Assignee deleted (
dkliban@redhat.com)
Updated by dkliban@redhat.com over 9 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to dkliban@redhat.com
Updated by dkliban@redhat.com over 9 years ago
- Status changed from ASSIGNED to NEW
Updated by semyers about 9 years ago
- Status changed from NEW to ASSIGNED
I'm seeing this on the consumer side when pulling from the OpenSUSE 13.2 main repo: https://pulp.plan.io/issues/1322
I'm going to get off of master for the time being, and test this on 2.6.
Updated by semyers about 9 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to semyers
Updated by semyers about 9 years ago
Syncing an openSUSE repo with the RPM plugin worked well enough, but attempting to do an openSUSE netinstall aimed at the published repo made the installer sad.
For a quick summary of what's wrong, here's what the "main" openSUSE yast repo looks like:
http://download.opensuse.org/distribution/13.2/repo/oss/
And here's what a normal yum repo looks like (fedora 22):
http://ftp.linux.ncsu.edu/pub/fedora/linux/releases/22/Everything/x86_64/os/
Inside the yast repo, you can peek into the "suse" directory and see the expected "repodata" folder, and this is enough for pulp to sync with the rpm plugin. However, watching pulp's httpd logs when trying to use the published repo from the openSUSE network install reveals that openSUSE is very interested in files in the repo's base dir, /oss, specifically /oss/content. To me, this means that for pulp to properly sync and publish a yast repo, we'll need to either pull more metadata than we currently are and/or present the metadata we have in a way that makes the published repo look like a proper yast repo.
Given the difference in the location of the "repodata" dir from the root of the repository, and the fact that its location is configurable in a YaST repo[0], my thinking at this point is that pulp_rpm probably isn't the tool to be used when syncing yast repos. That said, I don't know enough about pulp to have any good reason to think that, so I'll be asking the team about this tomorrow.
But hey, at least they publish their repo standards! :)
[0]: https://en.opensuse.org/openSUSE:Standards_YaST2_Repository_Metadata
Updated by semyers about 9 years ago
- File create_package_descr create_package_descr added
What is the minimum amount of work required to distribute a yum repo as a YaST repo?
To answer that question, I've found this rather helpful document: https://en.opensuse.org/SDB:Creating_YaST_installation_sources
As far as I can tell, these are the major differences between a yum repo and a YaST repo:
- yum repos are organized somewhat arbitrarily, as a directory containing rpm packages and a repodata dir that lets clients make sense of it all
- YaST repos are more structured, with a top-level directory containing a few metadata files, including one that says which directory contains all the packages (organized by architecture) and the repodata dir
metadata files¶
These files are based at the root of the repository, and are both required.
The media descriptor file¶
media.1/media can be created at distribution time. The content of this file is as simple as
Pulp
YYYYMMDDhhmmss
1
Regarding the first field, "Pulp", the docs for this file state that this field "...should contain an identifier to quickly verify the media set the media belongs to. This identifier is not shown to the user but should contain human-readable data for debugging purposes".
The content descriptor file¶
content can also be created a distribution time. It has several fields available for use, documented here, but I'll outline what I believe are the fields we should (or must?) include, based on what I've seen in "real" openSUSE repos:
CONTENTSTYLE, DATADIR, DESCRDIR, DISTRO, LINGUAS, VENDOR
CONTENTSTYLE must be first in the file, and must be set to 11 (identifies this as a Code11 YaST repo, which we're impersonating)
DATADIR is the directory (specifically: a subdirectory in the same directory as the content file) where the packages are contained. The DATADIRs that I've observed contain packages sorted into dirs by arch, as well as #repodata@, which we already know how to make
DESCRDIR is a little bit tricker. It's created by an awesome perl script (redundant, I know) that you can find linked on the "Creating_YaST_installation_sources" page in the openSUSE repo, or you can look at the one that I've conveniently attached to this ticket. :) I expect we'd reimplement this in python, if this hasn't already been done (I haven't looked).
DISTRO appears to be an undocumented abbreviation of DISTRIBUTION, which (acording to the docs) is "Some string denoting the distribution. The same string is most probably used in the .rpms to denote the distribution. Usually a composition of name version and flavor." Based on that, I expect we can (probably?) snag it from the packages. Given the requirement that it's "some string", I'm not sure how important it is that this contains a useful value.
LINGUAS (not LANGUAGES, apparently) is a space-delimited list of languages supported by this repo. I'm not quite sure how to populate this, but it's related to what we generate at in the DESCRDIR step with the create_package_descr script, and is possibly derivable from package metadata.
VENDOR is, predictably, the name of the repository vendor. In our case, likely "Pulp".
LABEL is the nicely-formatted human-readable string that will be shown by packaging tools using this repo. It is not required, but seems like a nice thing to do. :)
In addition, there are META, HASH, and KEY tags, which all contain the expected hashes of non-package files that may be downloaded from this repo (e.g. the packages files found in the DESCRDIR)
Conflicting information¶
The wiki page for the content file include a section (at the time I write this, at least) with a little bit of Q&A, specifically someone asking what fields are required:
https://en.opensuse.org/openSUSE:Standards_YaST2_Repository_Metadata_content#Not_documented.2Funclear
It says that NAME, VERSION, BASEARCHS, and VENDOR are required. But, based on the content file for the main opensuse repo, that can't be true. Only VENDOR can be found in this file, despite it clearly stating CONTENTSTYLE 11:
http://download.opensuse.org/distribution/13.2/repo/oss/content
When conflicts like this appear, it seems preferable to defer to the actual repo that openSUSE users connect to, rather than relying on potentially outdated wiki docs. It would probably also be good to correct their wiki where possible.
Cryptography¶
gpg¶
The content file may be signed, by including content.asc alongside the content file. Additionally, the public part of the key used to generate that signature can be included in content.key.
https¶
Nope!
In the iso installer, I wasn't able to get openSUSE to even do https without booting into the advanced install env. Even when I did, my pulp install was using a self-signed cert so I was never able to establish a connection from the installer due to a lack of trust. The repositories published by openSUSE itself are not secured by HTTPS, instead appearing to completely rely on the GPG-signed content file.
Updated by semyers about 9 years ago
Story so far:
- sync and publish opensuse repos
This works. The layout of the openSUSE repository is mangled when republished by pulp, but it doesn't seem to matter. openSUSE indexes the published repo and installs packages just fine.
- if possible, install and provision from a pulp-published repo
This might be possible, if pulp were to publish a YaST repo as described in earlier comments. Satellite 5 currently does this by exposing the openSUSE install iso via HTTP with some manual work, and using SUSE's version of kickstarting (autoyast) to install from the exposed iso. You can, incidentally, add a pulp repo as a content source in the autoyast file at this point if desired, but it's not required for the install.
There are community plugins to help foreman out with making the autoyast kickstart XML file. What remains, it seems, is talking with Sat 6 folks about what I've learned here, and replicating what I've found so far in SLES.
- install software and updates from a pulp-published repo
This completely works. After publishing a normal yum repo via pulp, you can add the repo to SUSE with zypper:
zypper addrepository -f http://hostname/pulp/path/to/repo reponame
zypper search -s <package_name> is an easy way to verify what repo a package can come from.
Updated by semyers about 9 years ago
Just quick update, I'm still working with the sat6 team to identify gaps between what sat5 provides and what sat6 provides based on my conclusions in the previous comment. Once they're identified and any outstanding RFEs are created, this should be done.
Updated by semyers almost 9 years ago
- Related to Story #1391: As a user, I can distribute the *contents* of an iso added
Updated by semyers almost 9 years ago
- Status changed from ASSIGNED to CLOSED - NOTABUG
openSUSE provisioning is something that is out of scope for pulp, and should be handled by foreman/satellite. The satellite team is still identifying the steps that would need to be taken to get feature parity in satellite 6 (via foreman) to what we have in satellite 5.
Pulp can help foreman out a little bit by exposing the contents of isos, which I've opened as story #1391.
closing with NOTABUG, since this wasn't a bug. :)
.