Issue #982
closed
"rpm repo export --start-date" does not generate repodata
Description
1st, generate a good test with valid repodata
pulp-admin rpm repo create --repo-id=fakepulp \
--feed=https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.6/fedora-21/x86_64/
pulp-admin rpm repo sync run --repo-id=fakepulp
pulp-admin rpm repo export run --repo-id=fakepulp
mkdir -p /media/foo
mount -o loop /var/lib/pulp/published/yum/https/exports/repos/fakepulp/fakepulp-2015-05-13T13.01-01.iso /media/foo/
find /media/foo |grep repodata
/media/foo/fakepulp/repodata
/media/foo/fakepulp/repodata/3aeed7b98df3f72c98cfb616e2aab652a4d4881ef16db9e570d0af3f7a9a868e-other.xml.gz
/media/foo/fakepulp/repodata/685a9837d9f328ed161556a85f6f22dc404a0681d73621f31e6b3929d48682d1-updateinfo.xml.gz
/media/foo/fakepulp/repodata/85302be28e0c299568d817859421084d167b1dff01a1dcce9a31861024fc54aa-primary.xml.gz
/media/foo/fakepulp/repodata/c57e62917f1db07c23e89f5cd2ad404d80ed9119cffa9cdac2e72f67f58c5b91-filelists.xml.gz
/media/foo/fakepulp/repodata/dfd6924bc302c3dd52a1cfa2cc36179308e2f03c529828b856c35cae635d90c7-comps.xml
/media/foo/fakepulp/repodata/repomd.xml
Now try and limit the content with --start-date
No repodata will be created for default iso or if specify --export-dir content location
pulp-admin rpm repo export run --repo-id=fakepulp --start-date "2015-03-15T00:00:00"
mount -o loop /var/lib/pulp/published/yum/https/exports/repos/fakepulp/fakepulp-2015-05-13T13.23-01.iso /media/foo/
find /media/foo |grep repodata
no results...
find /media/foo
/media/foo
/media/foo/fakepulp
/media/foo/fakepulp/createrepo-0.9.9-21.2.pulp.fc21.noarch.json
/media/foo/fakepulp/createrepo-0.9.9-21.2.pulp.fc21.noarch.rpm
/media/foo/fakepulp/gofer-2.6.1-1.fc21.noarch.json
/media/foo/fakepulp/gofer-2.6.1-1.fc21.noarch.rpm
/media/foo/fakepulp/gofer-package-2.6.1-1.fc21.noarch.json
/media/foo/fakepulp/gofer-package-2.6.1-1.fc21.noarch.rpm
/media/foo/fakepulp/gofer-system-2.6.1-1.fc21.noarch.json
.
.
No repodata folder or xml data is created which makes it impossible to import this content.
Updated by mhrivnak over 8 years ago
- Status changed from NEW to CLOSED - NOTABUG
This is working as designed. The documentation here should explain it: https://pulp-rpm.readthedocs.org/en/2.6-release/user-guide/recipes.html#export-repositories-and-repository-groups
The basic idea is that you'll do a full export that can be sync'd to a disconnected pulp, and then incremental changes after that date will be uploaded to the corresponding repo on the other pulp.
Updated by patrick.wirth over 8 years ago
- Status changed from CLOSED - NOTABUG to MODIFIED
With your feedback, I did some additional tests.
Additional details :
- This time I did a full export of a repo
- Desired repodata was created this time
- Moved the generated iso file to the isolated network.
- Mounted the iso on a loopback
attempted to import from the loopback mount
<snip>
Downloading metadata...
[-]
... failed
[Errno 13] Permission denied:
u'///pulp-imports/pulp-2-updates-live/repodata/repomd.xml'
Task Failed
<snip>
THIS import DID work if I copied the iso files to a local filesystem
via `tar -cf - * | ( cd /pulp-imports; tar -xf -)
So I re-tried the repo import of a "start-date" export from a local filesytem instead of a loopback device.
Still, no good.
<snip>
Downloading metadata...
[-]
... failed
[Errno 2] No such file or directory:
u'///pulp-imports/rhel-server-6.5-x86_64-updates-live/repodata/repomd.xml'
<snip>
Updated by patrick.wirth over 8 years ago
So it appears that the pulp-admin wants to see if it can update the 'repodata/repomd.xml' during an import. But as far as I can tell, repomd.xml was not modified in any way. Same md5, same permissions.
When the file is missing, the import fails. Which was my original problem and remains the same if the filesystem is read-writable or a read-only mount.
Updated by rbarlow over 8 years ago
- Status changed from MODIFIED to NEW
- Triaged changed from Yes to No
Modified means that the fix has been merged in our git repository, so I'm moving it back to new.
Updated by mhrivnak over 8 years ago
When you do a sync from the local filesystem, the apache user needs the ability to read the corresponding location on disk. This sounds like a simple case of reconciling filesystem permissions. Perhaps selinux is involved?
pulp-admin does not need access to the repository and will not interact with repomd.xml in any way.
Updated by patrick.wirth over 8 years ago
selinux is already in permissive.
I had already bumped into the apache permissions when setting up exports, that was an early fix.
I agree, pulp-admin should not need to modify that file for an import.
Just to be clear. I did the following
pulp-admin rpm repo sync run --repo-id "pulp-2-updates-live" \
--feed=file:///pulp-imports/pulp-2-updates-live
pulp-admin rpm repo sync run --repo-id "pulp-2-updates-live"
Updated by patrick.wirth over 8 years ago
sorry, it was
pulp-admin rpm repo update run --repo-id "pulp-2-updates-live" \
--feed=file:///pulp-imports/pulp-2-updates-live
Updated by mhrivnak over 8 years ago
- Status changed from NEW to CLOSED - NOTABUG
This looks solidly like the OS is telling you that the apache user can't read the repomd.xml file. One way you could confirm this and try to debug is to become the apache user in a shell session and try to read that file.
This does not look like a pulp bug, and this filesystem permission error is definitely not related to the initial bug you reported. If you find that there is something pulp could or should improve related to the error you see now, please open a new bug.
.