Project

Profile

Help

Issue #7244

Hard links not being used during RPM content type migration

Added by iballou over 1 year ago. Updated about 1 year ago.

Status:
CLOSED - DUPLICATE
Priority:
High
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 80
Quarter:

Description

I did a migration test with ~85 GB of content in my Pulp data directory. After the migration, the disk utilization essentially doubled. I checked, and hard links weren't being used for the Pulp 2 RPMs, which probably explains the high disk usage.

My Pulp data directory is on a separate hard drive under /data/pulp, with a soft link to /var/lib/pulp


Related issues

Has duplicate Pulp - Issue #7445: Pulp2 overrides permissions set by pulp3CLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

History

#1 Updated by iballou over 1 year ago

Migration plugin version: pulp-2to3-migration (0.2.0b7.dev0)

#2 Updated by iballou over 1 year ago

I did not see this warning in journalctl: https://github.com/pulp/pulp-2to3-migration/blob/0.2/pulp_2to3_migration/app/plugin/content.py#L133

I also tested this and it worked: os.link("/var/lib/pulp/foo.txt", "/var/lib/pulp/baz.txt")

#3 Updated by ttereshc over 1 year ago

  • Priority changed from Normal to High
  • Triaged changed from No to Yes
  • Sprint set to Sprint 78

#4 Updated by rchan over 1 year ago

  • Sprint changed from Sprint 78 to Sprint 79

#5 Updated by iballou over 1 year ago

The filesystem type is EXT4.

#6 Updated by rchan over 1 year ago

  • Sprint changed from Sprint 79 to Sprint 80

#7 Updated by ipanova@redhat.com over 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ipanova@redhat.com

#8 Updated by iballou about 1 year ago

I was able to capture the error. This is one of many:

Sep  1 14:40:29 centos7-katello-devel pulpcore-worker-1: pulp: pulp_2to3_migration.app.plugin.content:INFO: Hard link cannot be created, file will be copied
Sep  1 14:40:29 centos7-katello-devel pulpcore-worker-1: pulp: pulp_2to3_migration.app.plugin.content:INFO: [Errno 1] Operation not permitted: '/var/lib/pulp/content/units/rpm/77/17fd26f4f352d19bdeaa21b1fa16a3053fa9a7d0b5f69f88577108f8044120/camel-0.1-1.noarch.rpm' -> '/var/lib/pulp/docroot/artifact/82/e497ca3e7affb5682362b50537af290d0a202e4ad00b6ef62357a2cc9819ba'

#9 Updated by ipanova@redhat.com about 1 year ago

iballou wrote:

I was able to capture the error. This is one of many:

[...]

yeah, thanks for confirmation. So this error points to the permissions and ownership which are not set correctly. In order to create the hardlink from /var/lib/pulp, apache should be part of pulp group, and pulp group should have write permissions. I will check more, i think even pullp installer has wrong permissions.

#10 Updated by iballou about 1 year ago

The apache user can make hard links from the content dir to the artifact dir, but the pulp user cannot. Is it the pulp user who is trying to make the link instead of the apache user?

#12 Updated by ipanova@redhat.com about 1 year ago

Based on the comments I suggest to close this issue and fix the 2filed above + make sure that foreman installer does set properly permissions/ownership

#13 Updated by ipanova@redhat.com about 1 year ago

  • Status changed from ASSIGNED to CLOSED - DUPLICATE

#14 Updated by ipanova@redhat.com about 1 year ago

  • Has duplicate Issue #7445: Pulp2 overrides permissions set by pulp3 added

Please register to edit this issue

Also available in: Atom PDF