Issue #7244
closedHard links not being used during RPM content type migration
Description
I did a migration test with ~85 GB of content in my Pulp data directory. After the migration, the disk utilization essentially doubled. I checked, and hard links weren't being used for the Pulp 2 RPMs, which probably explains the high disk usage.
My Pulp data directory is on a separate hard drive under /data/pulp
, with a soft link to /var/lib/pulp
Related issues
Updated by iballou over 4 years ago
Migration plugin version: pulp-2to3-migration (0.2.0b7.dev0)
Updated by iballou over 4 years ago
I did not see this warning in journalctl: https://github.com/pulp/pulp-2to3-migration/blob/0.2/pulp_2to3_migration/app/plugin/content.py#L133
I also tested this and it worked: os.link("/var/lib/pulp/foo.txt", "/var/lib/pulp/baz.txt")
Updated by ttereshc over 4 years ago
- Priority changed from Normal to High
- Triaged changed from No to Yes
- Sprint set to Sprint 78
Updated by ipanova@redhat.com over 4 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ipanova@redhat.com
Updated by iballou over 4 years ago
I was able to capture the error. This is one of many:
Sep 1 14:40:29 centos7-katello-devel pulpcore-worker-1: pulp: pulp_2to3_migration.app.plugin.content:INFO: Hard link cannot be created, file will be copied Sep 1 14:40:29 centos7-katello-devel pulpcore-worker-1: pulp: pulp_2to3_migration.app.plugin.content:INFO: [Errno 1] Operation not permitted: '/var/lib/pulp/content/units/rpm/77/17fd26f4f352d19bdeaa21b1fa16a3053fa9a7d0b5f69f88577108f8044120/camel-0.1-1.noarch.rpm' -> '/var/lib/pulp/docroot/artifact/82/e497ca3e7affb5682362b50537af290d0a202e4ad00b6ef62357a2cc9819ba'
Updated by ipanova@redhat.com over 4 years ago
iballou wrote:
I was able to capture the error. This is one of many:
[...]
yeah, thanks for confirmation. So this error points to the permissions and ownership which are not set correctly. In order to create the hardlink from /var/lib/pulp, apache should be part of pulp group, and pulp group should have write permissions. I will check more, i think even pullp installer has wrong permissions.
Updated by iballou over 4 years ago
The apache user can make hard links from the content dir to the artifact dir, but the pulp user cannot. Is it the pulp user who is trying to make the link instead of the apache user?
Updated by ipanova@redhat.com over 4 years ago
I filed 2 issues https://pulp.plan.io/issues/7445 and https://pulp.plan.io/issues/7443
These are expected permissions and ownership https://github.com/pulp/pulp_installer/blob/master/roles/pulp_common/tasks/install.yml#L107-L133
Updated by ipanova@redhat.com over 4 years ago
Based on the comments I suggest to close this issue and fix the 2filed above + make sure that foreman installer does set properly permissions/ownership
Updated by ipanova@redhat.com over 4 years ago
- Status changed from ASSIGNED to CLOSED - DUPLICATE
Updated by ipanova@redhat.com over 4 years ago
- Has duplicate Issue #7445: Pulp2 overrides permissions set by pulp3 added