Project

Profile

Help

Issue #7244

closed

Hard links not being used during RPM content type migration

Added by iballou over 3 years ago. Updated over 3 years ago.

Status:
CLOSED - DUPLICATE
Priority:
High
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 80
Quarter:

Description

I did a migration test with ~85 GB of content in my Pulp data directory. After the migration, the disk utilization essentially doubled. I checked, and hard links weren't being used for the Pulp 2 RPMs, which probably explains the high disk usage.

My Pulp data directory is on a separate hard drive under /data/pulp, with a soft link to /var/lib/pulp


Related issues

Has duplicate Pulp - Issue #7445: Pulp2 overrides permissions set by pulp3CLOSED - CURRENTRELEASEggaineyActions
Actions #1

Updated by iballou over 3 years ago

Migration plugin version: pulp-2to3-migration (0.2.0b7.dev0)

Actions #2

Updated by iballou over 3 years ago

I did not see this warning in journalctl: https://github.com/pulp/pulp-2to3-migration/blob/0.2/pulp_2to3_migration/app/plugin/content.py#L133

I also tested this and it worked: os.link("/var/lib/pulp/foo.txt", "/var/lib/pulp/baz.txt")

Actions #3

Updated by ttereshc over 3 years ago

  • Priority changed from Normal to High
  • Triaged changed from No to Yes
  • Sprint set to Sprint 78
Actions #4

Updated by rchan over 3 years ago

  • Sprint changed from Sprint 78 to Sprint 79
Actions #5

Updated by iballou over 3 years ago

The filesystem type is EXT4.

Actions #6

Updated by rchan over 3 years ago

  • Sprint changed from Sprint 79 to Sprint 80
Actions #7

Updated by ipanova@redhat.com over 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ipanova@redhat.com
Actions #8

Updated by iballou over 3 years ago

I was able to capture the error. This is one of many:

Sep  1 14:40:29 centos7-katello-devel pulpcore-worker-1: pulp: pulp_2to3_migration.app.plugin.content:INFO: Hard link cannot be created, file will be copied
Sep  1 14:40:29 centos7-katello-devel pulpcore-worker-1: pulp: pulp_2to3_migration.app.plugin.content:INFO: [Errno 1] Operation not permitted: '/var/lib/pulp/content/units/rpm/77/17fd26f4f352d19bdeaa21b1fa16a3053fa9a7d0b5f69f88577108f8044120/camel-0.1-1.noarch.rpm' -> '/var/lib/pulp/docroot/artifact/82/e497ca3e7affb5682362b50537af290d0a202e4ad00b6ef62357a2cc9819ba'
Actions #9

Updated by ipanova@redhat.com over 3 years ago

iballou wrote:

I was able to capture the error. This is one of many:

[...]

yeah, thanks for confirmation. So this error points to the permissions and ownership which are not set correctly. In order to create the hardlink from /var/lib/pulp, apache should be part of pulp group, and pulp group should have write permissions. I will check more, i think even pullp installer has wrong permissions.

Actions #10

Updated by iballou over 3 years ago

The apache user can make hard links from the content dir to the artifact dir, but the pulp user cannot. Is it the pulp user who is trying to make the link instead of the apache user?

Actions #12

Updated by ipanova@redhat.com over 3 years ago

Based on the comments I suggest to close this issue and fix the 2filed above + make sure that foreman installer does set properly permissions/ownership

Actions #13

Updated by ipanova@redhat.com over 3 years ago

  • Status changed from ASSIGNED to CLOSED - DUPLICATE
Actions #14

Updated by ipanova@redhat.com over 3 years ago

  • Has duplicate Issue #7445: Pulp2 overrides permissions set by pulp3 added

Also available in: Atom PDF