Project

Profile

Help

Issue #982

"rpm repo export --start-date" does not generate repodata

Added by patrick.wirth over 5 years ago. Updated over 1 year ago.

Status:
CLOSED - NOTABUG
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.6.0
Platform Release:
OS:
RHEL 6
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

1st, generate a good test with valid repodata

pulp-admin rpm repo create --repo-id=fakepulp \
--feed=https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.6/fedora-21/x86_64/
pulp-admin rpm repo sync run --repo-id=fakepulp
pulp-admin rpm repo export run --repo-id=fakepulp
mkdir -p /media/foo

mount -o loop /var/lib/pulp/published/yum/https/exports/repos/fakepulp/fakepulp-2015-05-13T13.01-01.iso /media/foo/

find /media/foo |grep repodata
/media/foo/fakepulp/repodata
/media/foo/fakepulp/repodata/3aeed7b98df3f72c98cfb616e2aab652a4d4881ef16db9e570d0af3f7a9a868e-other.xml.gz
/media/foo/fakepulp/repodata/685a9837d9f328ed161556a85f6f22dc404a0681d73621f31e6b3929d48682d1-updateinfo.xml.gz
/media/foo/fakepulp/repodata/85302be28e0c299568d817859421084d167b1dff01a1dcce9a31861024fc54aa-primary.xml.gz
/media/foo/fakepulp/repodata/c57e62917f1db07c23e89f5cd2ad404d80ed9119cffa9cdac2e72f67f58c5b91-filelists.xml.gz
/media/foo/fakepulp/repodata/dfd6924bc302c3dd52a1cfa2cc36179308e2f03c529828b856c35cae635d90c7-comps.xml
/media/foo/fakepulp/repodata/repomd.xml

Now try and limit the content with --start-date
No repodata will be created for default iso or if specify --export-dir content location

pulp-admin rpm repo export run --repo-id=fakepulp --start-date "2015-03-15T00:00:00"

mount -o loop /var/lib/pulp/published/yum/https/exports/repos/fakepulp/fakepulp-2015-05-13T13.23-01.iso /media/foo/

find /media/foo |grep repodata
no results...

find /media/foo
/media/foo
/media/foo/fakepulp
/media/foo/fakepulp/createrepo-0.9.9-21.2.pulp.fc21.noarch.json
/media/foo/fakepulp/createrepo-0.9.9-21.2.pulp.fc21.noarch.rpm
/media/foo/fakepulp/gofer-2.6.1-1.fc21.noarch.json
/media/foo/fakepulp/gofer-2.6.1-1.fc21.noarch.rpm
/media/foo/fakepulp/gofer-package-2.6.1-1.fc21.noarch.json
/media/foo/fakepulp/gofer-package-2.6.1-1.fc21.noarch.rpm
/media/foo/fakepulp/gofer-system-2.6.1-1.fc21.noarch.json
.
.

No repodata folder or xml data is created which makes it impossible to import this content.

History

#1 Updated by mhrivnak over 5 years ago

  • Status changed from NEW to CLOSED - NOTABUG

This is working as designed. The documentation here should explain it: https://pulp-rpm.readthedocs.org/en/2.6-release/user-guide/recipes.html#export-repositories-and-repository-groups

The basic idea is that you'll do a full export that can be sync'd to a disconnected pulp, and then incremental changes after that date will be uploaded to the corresponding repo on the other pulp.

#2 Updated by jortel@redhat.com over 5 years ago

  • Triaged changed from No to Yes

#3 Updated by patrick.wirth over 5 years ago

  • Status changed from CLOSED - NOTABUG to MODIFIED

With your feedback, I did some additional tests.

Additional details :
- This time I did a full export of a repo
- Desired repodata was created this time
- Moved the generated iso file to the isolated network.
- Mounted the iso on a loopback

attempted to import from the loopback mount

   <snip>
   Downloading metadata...
   [-]
   ... failed

   [Errno 13] Permission denied:
   u'///pulp-imports/pulp-2-updates-live/repodata/repomd.xml'

   Task Failed
   <snip>

THIS import DID work if I copied the iso files to a local filesystem
via `tar -cf - * | ( cd /pulp-imports; tar -xf -)

So I re-tried the repo import of a "start-date" export from a local filesytem instead of a loopback device.

Still, no good.

   <snip>

   Downloading metadata...
   [-]
   ... failed

   [Errno 2] No such file or directory:
   u'///pulp-imports/rhel-server-6.5-x86_64-updates-live/repodata/repomd.xml'
   <snip>

#4 Updated by patrick.wirth over 5 years ago

So it appears that the pulp-admin wants to see if it can update the 'repodata/repomd.xml' during an import. But as far as I can tell, repomd.xml was not modified in any way. Same md5, same permissions.

When the file is missing, the import fails. Which was my original problem and remains the same if the filesystem is read-writable or a read-only mount.

#5 Updated by rbarlow over 5 years ago

  • Status changed from MODIFIED to NEW
  • Triaged changed from Yes to No

Modified means that the fix has been merged in our git repository, so I'm moving it back to new.

#6 Updated by mhrivnak over 5 years ago

When you do a sync from the local filesystem, the apache user needs the ability to read the corresponding location on disk. This sounds like a simple case of reconciling filesystem permissions. Perhaps selinux is involved?

pulp-admin does not need access to the repository and will not interact with repomd.xml in any way.

#7 Updated by patrick.wirth over 5 years ago

selinux is already in permissive.

I had already bumped into the apache permissions when setting up exports, that was an early fix.

I agree, pulp-admin should not need to modify that file for an import.

Just to be clear. I did the following

pulp-admin rpm repo sync run --repo-id "pulp-2-updates-live" \
--feed=file:///pulp-imports/pulp-2-updates-live

pulp-admin rpm repo sync run --repo-id "pulp-2-updates-live"

#8 Updated by patrick.wirth over 5 years ago

sorry, it was

pulp-admin rpm repo update run --repo-id "pulp-2-updates-live" \
--feed=file:///pulp-imports/pulp-2-updates-live

#9 Updated by mhrivnak over 5 years ago

  • Status changed from NEW to CLOSED - NOTABUG

This looks solidly like the OS is telling you that the apache user can't read the repomd.xml file. One way you could confirm this and try to debug is to become the apache user in a shell session and try to read that file.

This does not look like a pulp bug, and this filesystem permission error is definitely not related to the initial bug you reported. If you find that there is something pulp could or should improve related to the error you see now, please open a new bug.

#10 Updated by mhrivnak over 5 years ago

  • Triaged changed from No to Yes

#11 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF