Project

Profile

Help

Issue #2387

Publishes fail on RHEL 6

Added by Ichimonji10 over 2 years ago. Updated 5 days ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Urgent
Category:
-
Sprint/Milestone:
-
Severity:
3. High
Version:
2.10.2
Platform Release:
2.10.2
Blocks Release:
2.10.z
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
Yes
Verification Required:
No
Sprint:
Sprint 11

Description

To reproduce this issue, provision a RHEL 6 system, install Pulp on it, and execute the following script:

pulp-admin rpm repo create --repo-id foo --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned/
pulp-admin rpm repo sync run --repo-id foo
wget --no-check-certificate https://localhost/pulp/repos/pulp/pulp/fixtures/rpm-unsigned/bear-4.1-1.noarch.rpm
pulp-admin rpm repo delete --repo-id foo

The pulp-admin rpm repo sync run --repo-id foo command generates an error:

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
32 of 32 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[                                                  ] 0%
0 of 4 items

Task Failed

[Errno 13] Permission denied

Disabling SELinux fixes this error. To diagnose this error, I wrote up this script:

#!/usr/bin/env bash
#
# Demonstrate the issue with Pulp and get information about it. All statements
# with `|| true` prepended are known to return non-zero exit codes.
#
set -euo pipefail

# Configure SELinux
setenforce 0
echo > /var/log/audit/audit.log
semodule -R

# Execute problematic actions
pulp-admin rpm repo create --repo-id foo --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned/
pulp-admin rpm repo sync run --repo-id foo
wget --no-check-certificate https://localhost/pulp/repos/pulp/pulp/fixtures/rpm-unsigned/bear-4.1-1.noarch.rpm
pulp-admin rpm repo delete --repo-id foo

# Get info
set -x
audit2allow -al
audit2allow -Ral || true
cat /var/log/audit/audit.log

# Reset SELinux
set +x
setenforce 1

The following is printed:

+ audit2allow -al

#============= celery_t ==============

#!!!! This avc is a constraint violation.  You will need to add an attribute to either the source or target type to make it work.
#Contraint rule:
allow celery_t httpd_sys_rw_content_t:dir relabelto;

#!!!! This avc is a constraint violation.  You will need to add an attribute to either the source or target type to make it work.
#Contraint rule:
allow celery_t httpd_sys_rw_content_t:file relabelto;

#!!!! This avc is a constraint violation.  You will need to add an attribute to either the source or target type to make it work.
#Contraint rule:
allow celery_t httpd_sys_rw_content_t:lnk_file relabelto;
+ audit2allow -Ral

require {
        type celery_t;
        type httpd_sys_rw_content_t;
        class lnk_file relabelto;
        class file relabelto;
        class dir relabelto;
}

#============= celery_t ==============

#!!!! This avc is a constraint violation.  You will need to add an attribute to either the source or target type to make it work.
#Contraint rule:
allow celery_t httpd_sys_rw_content_t:dir relabelto;

#!!!! This avc is a constraint violation.  You will need to add an attribute to either the source or target type to make it work.
#Contraint rule:
allow celery_t httpd_sys_rw_content_t:file relabelto;

#!!!! This avc is a constraint violation.  You will need to add an attribute to either the source or target type to make it work.
#Contraint rule:
allow celery_t httpd_sys_rw_content_t:lnk_file relabelto;
+ cat /var/log/audit/audit.log

type=MAC_POLICY_LOAD msg=audit(1478278679.827:6541): policy loaded auid=0 ses=8
type=SYSCALL msg=audit(1478278679.827:6541): arch=c000003e syscall=1 success=yes exit=8447991 a0=4 a1=7f834d753000 a2=80e7f7 a3=7fff5d33a0f0 items=0 ppid=17451 pid=17452 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=8 comm="load_policy" exe="/sbin/load_policy" subj=unconfined_u:unconfined_r:load_policy_t:s0-s0:c0.c1023 key=(null)
type=AVC msg=audit(1478278693.893:6542): avc:  denied  { relabelto } for  pid=17425 comm="python" name="1478278692.65" dev=dm-0 ino=788189 scontext=unconfined_u:system_r:celery_t:s0 tcontext=system_u:object_r:httpd_sys_rw_content_t:s0 tclass=dir
type=SYSCALL msg=audit(1478278693.893:6542): arch=c000003e syscall=189 success=yes exit=0 a0=3b87de4 a1=7f04d509147d a2=4759460 a3=2c items=0 ppid=17212 pid=17425 auid=0 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=8 comm="python" exe="/usr/bin/python" subj=unconfined_u:system_r:celery_t:s0 key=(null)
type=AVC msg=audit(1478278693.895:6543): avc:  denied  { relabelto } for  pid=17425 comm="python" name="shark-0.1-1.noarch.rpm" dev=dm-0 ino=788213 scontext=unconfined_u:system_r:celery_t:s0 tcontext=system_u:object_r:httpd_sys_rw_content_t:s0 tclass=lnk_file
type=SYSCALL msg=audit(1478278693.895:6543): arch=c000003e syscall=189 success=yes exit=0 a0=3ba8e24 a1=7f04d509147d a2=4758120 a3=2c items=0 ppid=17212 pid=17425 auid=0 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=8 comm="python" exe="/usr/bin/python" subj=unconfined_u:system_r:celery_t:s0 key=(null)
type=AVC msg=audit(1478278693.931:6544): avc:  denied  { relabelto } for  pid=17425 comm="python" name="11642ea5192aeb4b050b08c7619b365d8982475b081388a196f0346bd4438fc9-comps.xml" dev=dm-0 ino=788234 scontext=unconfined_u:system_r:celery_t:s0 tcontext=system_u:object_r:httpd_sys_rw_content_t:s0 tclass=file
type=SYSCALL msg=audit(1478278693.931:6544): arch=c000003e syscall=189 success=yes exit=0 a0=3bbed14 a1=7f04d509147d a2=4babf30 a3=2c items=0 ppid=17212 pid=17425 auid=0 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=8 comm="python" exe="/usr/bin/python" subj=unconfined_u:system_r:celery_t:s0 key=(null)

This error can be produced at least with Pulp 2.10 installed. It may also occur with earlier and later versions of Pulp installed. I'll do some research and update this issue when I figure out which versions of Pulp suffer from this issue.

This issue is somewhat similar to https://pulp.plan.io/issues/2277.

Associated revisions

Revision 6bc0e1e9 View on GitHub
Added by dkliban@redhat.com over 2 years ago

SELinux permission for celery to change domain object id of files

On EL6, calling selinux.restorecon() requires changing the domain object id for the file
whose security context is being restored. This patch provides that permission for celery
on EL6.

closes #2387
https://pulp.plan.io/issues/2387

Revision 6bc0e1e9 View on GitHub
Added by dkliban@redhat.com over 2 years ago

SELinux permission for celery to change domain object id of files

On EL6, calling selinux.restorecon() requires changing the domain object id for the file
whose security context is being restored. This patch provides that permission for celery
on EL6.

closes #2387
https://pulp.plan.io/issues/2387

Revision 6bc0e1e9 View on GitHub
Added by dkliban@redhat.com over 2 years ago

SELinux permission for celery to change domain object id of files

On EL6, calling selinux.restorecon() requires changing the domain object id for the file
whose security context is being restored. This patch provides that permission for celery
on EL6.

closes #2387
https://pulp.plan.io/issues/2387

Revision 9e767456 View on GitHub
Added by dkliban@redhat.com over 2 years ago

Handles exception when restorecon is called with SELinux disabled.

closes #2387
https://pulp.plan.io/issues/2387

Revision 9e767456 View on GitHub
Added by dkliban@redhat.com over 2 years ago

Handles exception when restorecon is called with SELinux disabled.

closes #2387
https://pulp.plan.io/issues/2387

Revision 9e767456 View on GitHub
Added by dkliban@redhat.com over 2 years ago

Handles exception when restorecon is called with SELinux disabled.

closes #2387
https://pulp.plan.io/issues/2387

Revision 030efd45 View on GitHub
Added by dkliban@redhat.com over 2 years ago

Adds more SELinux for EL6

EL6 requires very granular permissions for running restorecon. This patch grants celery_t those
permissions.

re #2387
https://pulp.plan.io/issues/2387

Revision 030efd45 View on GitHub
Added by dkliban@redhat.com over 2 years ago

Adds more SELinux for EL6

EL6 requires very granular permissions for running restorecon. This patch grants celery_t those
permissions.

re #2387
https://pulp.plan.io/issues/2387

Revision 030efd45 View on GitHub
Added by dkliban@redhat.com over 2 years ago

Adds more SELinux for EL6

EL6 requires very granular permissions for running restorecon. This patch grants celery_t those
permissions.

re #2387
https://pulp.plan.io/issues/2387

History

#1 Updated by Ichimonji10 over 2 years ago

Here's a traceback from the API:

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 488, in __call__
    return super(Task, self).__call__(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 103, in __call__
    return super(PulpTask, self).__call__(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 971, in publish
    result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
  File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1063, in check_publish
    result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
  File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1115, in _do_publish
    publish_report = publish_repo(transfer_repo, conduit, call_config)
  File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 673, in wrap_f
    return f(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
    return self._publisher.process_lifecycle()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 566, in process_lifecycle
    super(PluginStep, self).process_lifecycle()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 163, in process_lifecycle
    step.process()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 253, in process
    self._process_block()
  File"/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 297, in _process_block
    self.process_main()
  File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 905, in process_main
    selinux.restorecon(timestamp_master_dir.encode('utf-8'), recursive=True)
  File "/usr/lib64/python2.6/site-packages/selinux/__init__.py", line 80, in restorecon
    status, context = matchpathcon(path, mode)
OSError: [Errno 13] Permission denied

#2 Updated by Ichimonji10 over 2 years ago

  • Version set to 2.10.2

Tested against Pulp 2.9, 2.10 and 2.11. This issue is present on 2.10 and 2.11.

Here's the packages on the Pulp 2.10 system:

# rpm -qa | sort | grep -i pulp
mod_wsgi-3.4-2.pulp.el6.x86_64
pulp-admin-client-2.10.2-0.1.alpha.git.23.3357589.el6.noarch
pulp-docker-admin-extensions-2.1.1-0.1.alpha.git.25.49d4e82.el6.noarch
pulp-docker-plugins-2.1.1-0.1.alpha.git.25.49d4e82.el6.noarch
pulp-puppet-admin-extensions-2.10.2-0.1.alpha.git.2.09f76df.el6.noarch
pulp-puppet-plugins-2.10.2-0.1.alpha.git.2.09f76df.el6.noarch
pulp-python-admin-extensions-1.1.4-0.1.alpha.git.28.71c18b3.el6.noarch
pulp-python-plugins-1.1.4-0.1.alpha.git.28.71c18b3.el6.noarch
pulp-rpm-admin-extensions-2.10.2-0.1.alpha.git.10.c83a0f9.el6.noarch
pulp-rpm-plugins-2.10.2-0.1.alpha.git.10.c83a0f9.el6.noarch
pulp-selinux-2.10.2-0.1.alpha.git.23.3357589.el6.noarch
pulp-server-2.10.2-0.1.alpha.git.23.3357589.el6.noarch
python-isodate-0.5.0-4.pulp.el6.noarch
python-kombu-3.0.33-6.pulp.el6.noarch
python-pulp-bindings-2.10.2-0.1.alpha.git.23.3357589.el6.noarch
python-pulp-client-lib-2.10.2-0.1.alpha.git.23.3357589.el6.noarch
python-pulp-common-2.10.2-0.1.alpha.git.23.3357589.el6.noarch
python-pulp-docker-common-2.1.1-0.1.alpha.git.25.49d4e82.el6.noarch
python-pulp-oid_validation-2.10.2-0.1.alpha.git.23.3357589.el6.noarch
python-pulp-puppet-common-2.10.2-0.1.alpha.git.2.09f76df.el6.noarch
python-pulp-python-common-1.1.4-0.1.alpha.git.28.71c18b3.el6.noarch
python-pulp-repoauth-2.10.2-0.1.alpha.git.23.3357589.el6.noarch
python-pulp-rpm-common-2.10.2-0.1.alpha.git.10.c83a0f9.el6.noarch
python-pulp-streamer-2.10.2-0.1.alpha.git.23.3357589.el6.noarch

Here's the packages on the 2.11 system:

# rpm -qa | sort | grep -i pulp
mod_wsgi-3.4-2.pulp.el6.x86_64
pulp-admin-client-2.11.1-0.1.alpha.git.29.66668b5.el6.noarch
pulp-docker-admin-extensions-2.2.1-0.1.alpha.git.3.e76d3d5.el6.noarch
pulp-docker-plugins-2.2.1-0.1.alpha.git.3.e76d3d5.el6.noarch
pulp-puppet-admin-extensions-2.11.1-0.1.alpha.git.6.507503a.el6.noarch
pulp-puppet-plugins-2.11.1-0.1.alpha.git.6.507503a.el6.noarch
pulp-python-admin-extensions-1.1.3-1.el6.noarch
pulp-python-plugins-1.1.3-1.el6.noarch
pulp-rpm-admin-extensions-2.11.1-0.1.alpha.git.6.b97fdce.el6.noarch
pulp-rpm-plugins-2.11.1-0.1.alpha.git.6.b97fdce.el6.noarch
pulp-selinux-2.11.1-0.1.alpha.git.29.66668b5.el6.noarch
pulp-server-2.11.1-0.1.alpha.git.29.66668b5.el6.noarch
python-isodate-0.5.0-4.pulp.el6.noarch
python-kombu-3.0.33-6.pulp.el6.noarch
python-pulp-bindings-2.11.1-0.1.alpha.git.29.66668b5.el6.noarch
python-pulp-client-lib-2.11.1-0.1.alpha.git.29.66668b5.el6.noarch
python-pulp-common-2.11.1-0.1.alpha.git.29.66668b5.el6.noarch
python-pulp-docker-common-2.2.1-0.1.alpha.git.3.e76d3d5.el6.noarch
python-pulp-oid_validation-2.11.1-0.1.alpha.git.29.66668b5.el6.noarch
python-pulp-puppet-common-2.11.1-0.1.alpha.git.6.507503a.el6.noarch
python-pulp-python-common-1.1.3-1.el6.noarch
python-pulp-repoauth-2.11.1-0.1.alpha.git.29.66668b5.el6.noarch
python-pulp-rpm-common-2.11.1-0.1.alpha.git.6.b97fdce.el6.noarch
python-pulp-streamer-2.11.1-0.1.alpha.git.29.66668b5.el6.noarch

#3 Updated by amacdona@redhat.com over 2 years ago

  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes
  • Blocks Release 2.10.z added

#4 Updated by mhrivnak over 2 years ago

  • Priority changed from Normal to Urgent
  • Sprint/Milestone set to 28

#5 Updated by dkliban@redhat.com over 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dkliban@redhat.com

#6 Updated by dkliban@redhat.com over 2 years ago

I was unable to reproduce. I installed using an ansible playbook and ended up with the following RPMs:

mod_wsgi.x86_64                     3.4-2.pulp.el6          @pulp
pulp-admin-client.noarch            2.10.2-0.1.alpha.git.23.42de850.el6                                                            @pulp
pulp-docker-admin-extensions.noarch 2.1.1-0.1.alpha.git.25.49d4e82.el6                                                            @pulp
pulp-docker-plugins.noarch          2.1.1-0.1.alpha.git.25.49d4e82.el6                                                            @pulp               
pulp-puppet-admin-extensions.noarch 2.10.2-0.1.alpha.git.2.09f76df.el6                                                            @pulp               
pulp-puppet-plugins.noarch          2.10.2-0.1.alpha.git.2.09f76df.el6                                                            @pulp               
pulp-python-admin-extensions.noarch 1.1.4-0.1.alpha.git.28.71c18b3.el6                                                            @pulp               
pulp-python-plugins.noarch          1.1.4-0.1.alpha.git.28.71c18b3.el6                                                            @pulp               
pulp-rpm-admin-extensions.noarch    2.10.2-0.1.alpha.git.15.d6a61d0.el6                                                            @pulp               
pulp-rpm-plugins.noarch             2.10.2-0.1.alpha.git.15.d6a61d0.el6                                                            @pulp               
pulp-selinux.noarch                 2.10.2-0.1.alpha.git.23.42de850.el6                                                            @pulp               
pulp-server.noarch                  2.10.2-0.1.alpha.git.23.42de850.el6                                                            @pulp               
python-amqp.noarch                  1.4.9-1.el6             @pulp               
python-billiard.x86_64              1:3.3.0.17-2.el6        @pulp               
python-bson.x86_64                  3.2-1.el6               @pulp               
python-celery.noarch                3.1.11-1.el6            @pulp               
python-gofer.noarch                 2.7.6-1.el6             @pulp               
python-gofer-qpid.noarch            2.7.6-1.el6             @pulp               
python-isodate.noarch               0.5.0-4.pulp.el6        @pulp               
python-kombu.noarch                 1:3.0.33-6.pulp.el6     @pulp               
python-mongoengine.noarch           0.10.5-1.el6            @pulp               
python-nectar.noarch                1.5.3-1.el6             @pulp               
python-pulp-bindings.noarch         2.10.2-0.1.alpha.git.23.42de850.el6                                                            @pulp               
python-pulp-client-lib.noarch       2.10.2-0.1.alpha.git.23.42de850.el6                                                            @pulp               
python-pulp-common.noarch           2.10.2-0.1.alpha.git.23.42de850.el6                                                            @pulp               
python-pulp-docker-common.noarch    2.1.1-0.1.alpha.git.25.49d4e82.el6                                                            @pulp               
python-pulp-oid_validation.noarch   2.10.2-0.1.alpha.git.23.42de850.el6                                                            @pulp               
python-pulp-puppet-common.noarch    2.10.2-0.1.alpha.git.2.09f76df.el6                                                            @pulp               
python-pulp-python-common.noarch    1.1.4-0.1.alpha.git.28.71c18b3.el6                                                            @pulp               
python-pulp-repoauth.noarch         2.10.2-0.1.alpha.git.23.42de850.el6                                                            @pulp               
python-pulp-rpm-common.noarch       2.10.2-0.1.alpha.git.15.d6a61d0.el6                                                            @pulp               
python-pulp-streamer.noarch         2.10.2-0.1.alpha.git.23.42de850.el6                                                            @pulp               
python-pymongo.x86_64               3.2-1.el6               @pulp               
python-pymongo-gridfs.x86_64        3.2-1.el6               @pulp               
python-semantic_version.noarch      2.2.0-6.el6             @pulp 

#7 Updated by dmcnabb over 2 years ago

wrote:

I was unable to reproduce. I installed using an ansible playbook and ended up with the following RPMs:

[...]

dkliban -

I have this issue as well. Could you try installing 2.10.0 first and then upgrading to 2.10.2 in order to reproduce this issue? Also, I have selinux disabled and I still have this issue. However, the error I see is a little different. The traceback is identical.

Nov 8 15:56:02 pulp-server pulp: pulp.server.async.tasks:INFO: Task failed : [17ecfada-b3a3-42cc-a2b1-74aec4fc9231]
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) Task pulp.server.managers.repo.publish.publish[id] raised unexpected: OSError(1, 'Operation not permitted')
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) Traceback (most recent call last):
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) R = retval = fun(*args, **kwargs)
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 488, in call
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return super(Task, self).__call__(*args, **kwargs)
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 103, in call
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return super(PulpTask, self).__call__(*args, **kwargs)
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in protected_call
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return self.run(*args, **kwargs)
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1095, in publish
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1187, in check_publish
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) result = do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1239, in _do_publish
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) publish_report = publish_repo(transfer_repo, conduit, call_config)
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 673, in wrap_f
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return f(*args, **kwargs)
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) return self._publisher.process_lifecycle()
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 566, in process_lifecycle
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) super(PluginStep, self).process_lifecycle()
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 163, in process_lifecycle
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) step.process()
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 253, in process
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) self._process_block()
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 297, in _process_block
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) self.process_main()
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 905, in process_main
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) selinux.restorecon(timestamp_master_dir.encode('utf-8'), recursive=True)
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) File "/usr/lib64/python2.6/site-packages/selinux/
_init__.py", line 83, in restorecon
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) lsetfilecon(path, context)
Nov 8 15:56:02 pulp-server pulp: celery.worker.job:ERROR: (6756-60832) OSError: [Errno 1] Operation not permitted

#8 Updated by Ichimonji10 over 2 years ago

Here's a reproducer.

# Reproduce https://pulp.plan.io/issues/2387

# Provision a RHEL 6.8 beaker system. Log in, and execute the following.
subscription-manager register
subscription-manager list --available | less  # pick an employee SKU
subscription-manager attach --pool="$poolid" 
yum -y update  # you may need to disable some unreachable repos
shutdown -r now
yum install -y libselinux-python

# Log out of the system. On the local control node, execute the following.
git clone https://github.com/pulp/pulp_packaging.git
cd pulp_packaging
echo $hostname > hosts
ansible-playbook ci/ansible/pulp_server.yaml -i hosts -e pulp_version='2.10'

# The system should have the nightly build of Pulp 2.10 installed. (2.10.2, as
# of this writing.) Log into the system and execute the following:
pulp-admin login -u admin
pulp-admin rpm repo create --repo-id foo --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned/
pulp-admin rpm repo sync run --repo-id foo
wget --no-check-certificate https://localhost/pulp/repos/pulp/pulp/fixtures/rpm-unsigned/bear-4.1-1.noarch.rpm
pulp-admin rpm repo delete --repo-id foo

#9 Updated by dkliban@redhat.com over 2 years ago

  • Status changed from ASSIGNED to POST

#10 Updated by dkliban@redhat.com over 2 years ago

  • Status changed from POST to MODIFIED

#11 Updated by dkliban@redhat.com over 2 years ago

  • Status changed from MODIFIED to ASSIGNED

Just discovered that we also get an exception if SELinux is fully disabled. I was able to reproduce this0.

[0] https://www.redhat.com/archives/pulp-list/2016-November/msg00008.html

#12 Updated by dkliban@redhat.com over 2 years ago

  • Status changed from ASSIGNED to POST

#13 Updated by bmbouter over 2 years ago

  • Status changed from POST to ASSIGNED

I went to review because it was in POST, but the PR has WIP so moving back to ASSIGNED

#14 Updated by semyers over 2 years ago

  • Platform Release set to 2.10.2

#15 Updated by dkliban@redhat.com over 2 years ago

  • Status changed from ASSIGNED to MODIFIED

#16 Updated by semyers over 2 years ago

  • Status changed from MODIFIED to ON_QA

#17 Updated by pthomas@redhat.com over 2 years ago

This seems to be failing for me. This is an upgrade setup where I went from 2.10 stable to 2.10.2 beta2

[root@ibm-x3250m4-02 ~]# pulp-admin rpm repo create --repo-id foo --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned/
Successfully created repository [foo]

[root@ibm-x3250m4-02 ~]# pulp-admin rpm repo sync run --repo-id foo
+----------------------------------------------------------------------+
                     Synchronizing Repository [foo]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[/]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       32/32 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[\]
... completed

Importing package groups/categories...
[-]
... completed

Cleaning duplicate packages...
[\]
... completed

Task Succeeded

Task Failed

[Errno 13] Permission denied

[root@ibm-x3250m4-02 ~]# rpm -qa |grep pulp
pulp-python-plugins-1.1.3-1.el6.noarch
python-isodate-0.5.0-4.pulp.el6.noarch
python-pulp-docker-common-2.1.0-1.el6.noarch
python-pulp-repoauth-2.10.2-0.2.beta.el6.noarch
python-pulp-bindings-2.10.2-0.2.beta.el6.noarch
mod_wsgi-3.4-2.pulp.el6.x86_64
python-pulp-common-2.10.2-0.2.beta.el6.noarch
python-pulp-puppet-common-2.10.2-0.1.beta.el6.noarch
pulp-admin-client-2.10.2-0.2.beta.el6.noarch
pulp-server-2.10.2-0.2.beta.el6.noarch
python-pulp-streamer-2.10.2-0.2.beta.el6.noarch
pulp-puppet-admin-extensions-2.10.2-0.1.beta.el6.noarch
python-pulp-python-common-1.1.3-1.el6.noarch
python-pulp-oid_validation-2.10.2-0.2.beta.el6.noarch
pulp-puppet-plugins-2.10.2-0.1.beta.el6.noarch
python-pulp-rpm-common-2.10.2-0.1.beta.el6.noarch
python-pulp-client-lib-2.10.2-0.2.beta.el6.noarch
pulp-rpm-plugins-2.10.2-0.1.beta.el6.noarch
pulp-python-admin-extensions-1.1.3-1.el6.noarch
pulp-selinux-2.10.2-0.2.beta.el6.noarch
pulp-rpm-admin-extensions-2.10.2-0.1.beta.el6.noarch
python-kombu-3.0.33-6.pulp.el6.noarch
pulp-docker-plugins-2.1.0-1.el6.noarch
pulp-docker-admin-extensions-2.1.0-1.el6.noarch
[root@ibm-x3250m4-02 ~]# hostname
ibm-x3250m4-02.lab.eng.rdu2.redhat.com
[root@ibm-x3250m4-02 ~]# 

#18 Updated by Ichimonji10 over 2 years ago

  • Status changed from ON_QA to ASSIGNED

Tail output from script:

+ audit2allow -al

+ audit2allow -Ral

+ cat /var/log/audit/audit.log

type=MAC_POLICY_LOAD msg=audit(1479154792.551:1998): policy loaded auid=0 ses=47
type=SYSCALL msg=audit(1479154792.551:1998): arch=c000003e syscall=1 success=yes exit=8450823 a0=4 a1=7fdee75e2000 a2=80f307 a3=7ffe77297af0 items=0 ppid=12936 pid=12937 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts2 ses=47 comm="load_policy" exe="/sbin/load_policy" subj=unconfined_u:unconfined_r:load_policy_t:s0-s0:c0.c1023 key=(null)
type=USER_ACCT msg=audit(1479154801.062:1999): user pid=12985 uid=0 auid=0 ses=15 subj=unconfined_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=CRED_ACQ msg=audit(1479154801.062:2000): user pid=12985 uid=0 auid=0 ses=15 subj=unconfined_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=LOGIN msg=audit(1479154801.071:2001): pid=12985 uid=0 subj=unconfined_u:system_r:crond_t:s0-s0:c0.c1023 old auid=0 new auid=0 old ses=15 new ses=48
type=USER_START msg=audit(1479154801.074:2002): user pid=12985 uid=0 auid=0 ses=48 subj=unconfined_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=CRED_DISP msg=audit(1479154801.155:2003): user pid=12985 uid=0 auid=0 ses=48 subj=unconfined_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=USER_END msg=audit(1479154801.155:2004): user pid=12985 uid=0 auid=0 ses=48 subj=unconfined_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
+ set +x

#19 Updated by mhrivnak over 2 years ago

  • Sprint/Milestone changed from 28 to 29

#20 Updated by dkliban@redhat.com over 2 years ago

  • Status changed from ASSIGNED to POST

#21 Updated by dkliban@redhat.com over 2 years ago

  • Status changed from POST to MODIFIED

#22 Updated by Ichimonji10 over 2 years ago

FYI, I just spun up a fresh VM and installed the beta RPMs. Publishes fail. Here's the tail end of the diagnostic script:

+ audit2allow -al

+ audit2allow -Ral

+ cat /var/log/audit/audit.log

type=MAC_POLICY_LOAD msg=audit(1479228980.563:1869): policy loaded auid=0 ses=2
type=SYSCALL msg=audit(1479228980.563:1869): arch=c000003e syscall=1 success=yes exit=8448003 a0=4 a1=7f965522f000 a2=80e803 a3=7ffd4f22a4c0 items=0 ppid=4869 pid=4870 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=2 comm="load_policy" exe="/sbin/load_policy" subj=unconfined_u:unconfined_r:load_policy_t:s0-s0:c0.c1023 key=(null)
+ set +x

Here's which Pulp packages are installed:

$ rpm -qa | sort | grep -i pulp
mod_wsgi-3.4-2.pulp.el6.x86_64
pulp-admin-client-2.10.2-0.2.beta.el6.noarch
pulp-docker-admin-extensions-2.1.0-1.el6.noarch
pulp-docker-plugins-2.1.0-1.el6.noarch
pulp-puppet-admin-extensions-2.10.2-0.1.beta.el6.noarch
pulp-puppet-plugins-2.10.2-0.1.beta.el6.noarch
pulp-python-admin-extensions-1.1.3-1.el6.noarch
pulp-python-plugins-1.1.3-1.el6.noarch
pulp-rpm-admin-extensions-2.10.2-0.1.beta.el6.noarch
pulp-rpm-plugins-2.10.2-0.1.beta.el6.noarch
pulp-selinux-2.10.2-0.2.beta.el6.noarch
pulp-server-2.10.2-0.2.beta.el6.noarch
python-isodate-0.5.0-4.pulp.el6.noarch
python-kombu-3.0.33-6.pulp.el6.noarch
python-pulp-bindings-2.10.2-0.2.beta.el6.noarch
python-pulp-client-lib-2.10.2-0.2.beta.el6.noarch
python-pulp-common-2.10.2-0.2.beta.el6.noarch
python-pulp-docker-common-2.1.0-1.el6.noarch
python-pulp-oid_validation-2.10.2-0.2.beta.el6.noarch
python-pulp-puppet-common-2.10.2-0.1.beta.el6.noarch
python-pulp-python-common-1.1.3-1.el6.noarch
python-pulp-repoauth-2.10.2-0.2.beta.el6.noarch
python-pulp-rpm-common-2.10.2-0.1.beta.el6.noarch
python-pulp-streamer-2.10.2-0.2.beta.el6.noarch

#23 Updated by semyers over 2 years ago

  • Status changed from MODIFIED to ON_QA

Beta 3 is available for testing.

#24 Updated by pthomas@redhat.com over 2 years ago

+ audit2allow -al

+ audit2allow -Ral

+ cat /var/log/audit/audit.log

type=MAC_POLICY_LOAD msg=audit(1479238942.691:1772): policy loaded auid=0 ses=20
type=SYSCALL msg=audit(1479238942.691:1772): arch=c000003e syscall=1 success=yes exit=8450871 a0=4 a1=7fba79f70000 a2=80f337 a3=7fff26402080 items=0 ppid=27407 pid=27408 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=20 comm="load_policy" exe="/sbin/load_policy" subj=unconfined_u:unconfined_r:load_policy_t:s0-s0:c0.c1023 key=(null)
+ set +x

rpm -qa |grep pulp |sort
mod_wsgi-3.4-2.pulp.el6.x86_64
pulp-admin-client-2.10.2-0.3.beta.el6.noarch
pulp-docker-admin-extensions-2.1.0-1.el6.noarch
pulp-docker-plugins-2.1.0-1.el6.noarch
pulp-puppet-admin-extensions-2.10.2-0.1.beta.el6.noarch
pulp-puppet-plugins-2.10.2-0.1.beta.el6.noarch
pulp-python-admin-extensions-1.1.3-1.el6.noarch
pulp-python-plugins-1.1.3-1.el6.noarch
pulp-rpm-admin-extensions-2.10.2-0.1.beta.el6.noarch
pulp-rpm-plugins-2.10.2-0.1.beta.el6.noarch
pulp-selinux-2.10.2-0.3.beta.el6.noarch
pulp-server-2.10.2-0.3.beta.el6.noarch
python-isodate-0.5.0-4.pulp.el6.noarch
python-kombu-3.0.33-6.pulp.el6.noarch
python-pulp-bindings-2.10.2-0.3.beta.el6.noarch
python-pulp-client-lib-2.10.2-0.3.beta.el6.noarch
python-pulp-common-2.10.2-0.3.beta.el6.noarch
python-pulp-docker-common-2.1.0-1.el6.noarch
python-pulp-oid_validation-2.10.2-0.3.beta.el6.noarch
python-pulp-puppet-common-2.10.2-0.1.beta.el6.noarch
python-pulp-python-common-1.1.3-1.el6.noarch
python-pulp-repoauth-2.10.2-0.3.beta.el6.noarch
python-pulp-rpm-common-2.10.2-0.1.beta.el6.noarch
python-pulp-streamer-2.10.2-0.3.beta.el6.noarch

#25 Updated by pthomas@redhat.com over 2 years ago

  • Status changed from ON_QA to VERIFIED

Verified on both el6 & el7


Successfully created repository [foo]

+----------------------------------------------------------------------+
                     Synchronizing Repository [foo]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[/]
... completed

Downloading repository content...
[-]
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Cleaning duplicate packages...
[-]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
32 of 32 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
4 of 4 items
... completed

Publishing Comps file
[==================================================] 100%
4 of 4 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Generating HTML files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

--2016-11-16 09:19:58--  https://localhost/pulp/repos/pulp/pulp/fixtures/rpm-unsigned/bear-4.1-1.noarch.rpm
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:443... connected.
WARNING: no certificate subject alternative name matches
    requested host name ‘localhost’.
HTTP request sent, awaiting response... 200 OK
Length: 1846 (1.8K) [application/x-rpm]
Saving to: ‘bear-4.1-1.noarch.rpm’

100%[===================================================>] 1,846       --.-K/s   in 0s      

2016-11-16 09:19:58 (109 MB/s) - ‘bear-4.1-1.noarch.rpm’ saved [1846/1846]

This command may be exited via ctrl+c without affecting the request.

[\]
Running...

Repository [foo] successfully deleted

+ audit2allow -al

+ audit2allow -Ral
could not open interface info [/var/lib/sepolgen/interface_info]
+ true
+ cat /var/log/audit/audit.log

type=MAC_STATUS msg=audit(1479305987.527:50539): enforcing=0 old_enforcing=1 auid=0 ses=13
type=SYSCALL msg=audit(1479305987.527:50539): arch=c000003e syscall=1 success=yes exit=1 a0=3 a1=7ffda0a6fd10 a2=1 a3=7ffda0a6fa90 items=0 ppid=13773 pid=13774 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=13 comm="setenforce" exe="/usr/sbin/setenforce" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)
type=MAC_POLICY_LOAD msg=audit(1479305987.684:50540): policy loaded auid=0 ses=13
type=SYSCALL msg=audit(1479305987.684:50540): arch=c000003e syscall=1 success=yes exit=3705290 a0=4 a1=7f5db3009010 a2=3889ca a3=7ffd91844e50 items=0 ppid=13775 pid=13776 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=13 comm="load_policy" exe="/usr/sbin/load_policy" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)
+ set +x

#26 Updated by semyers over 2 years ago

  • Status changed from VERIFIED to CLOSED - CURRENTRELEASE

#27 Updated by pulpbot about 2 years ago

  • Verified changed from No to Yes

#28 Updated by bmbouter about 1 year ago

  • Sprint set to Sprint 11

#29 Updated by bmbouter about 1 year ago

  • Sprint/Milestone deleted (29)

#30 Updated by bmbouter 5 days ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF