Actions
Issue #2961
closed
Pulp 2.14 broken on Fedora 26
Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.14.0
Platform Release:
2.14.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 25
Quarter:
Description
Pulp 2.14 beta 3 cannot successfully be installed and used on Fedora 26 using pulp_packaging. A couple changes have already been made to make Pulp 2.14 beta 3 installable on Fedora 26 using pulp_packaging:
With these changes in place, this error (and many more) are logged when Pulp starts up:
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: Unhandled Exception
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) error signing cert request: Signature ok
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) subject=CN = admin:admin:5988b93144e534662b1fc1a2
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) Getting CA Private Key
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) Can't open /etc/pki/pulp/ca.key for reading, Permission denied
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) 139724785673984:error:0200100D:system library:fopen:Permission denied:crypto/bio/bss_file.c:74:fopen('/etc/pki/pulp/ca.key','r')
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) 139724785673984:error:2006D002:BIO routines:BIO_new_file:system lib:crypto/bio/bss_file.c:83:
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) unable to load CA Private Key
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) unable to write 'random state'
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952)
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) Traceback (most recent call last):
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) response = wrapped_callback(request, *callback_args, **callback_kwargs)
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) return self.dispatch(request, *args, **kwargs)
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) return handler(request, *args, **kwargs)
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py", line 241, in _auth_decorator
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) return _verify_auth(self, operation, super_user_only, method, *args, **kwargs)
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/decorators.py", line 195, in _verify_auth
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) value = method(self, *args, **kwargs)
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) File "/usr/lib/python2.7/site-packages/pulp/server/webservices/views/root_actions.py", line 25, in post
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) key, certificate = factory.cert_generation_manager().make_admin_user_cert(user)
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) File "/usr/lib/python2.7/site-packages/pulp/server/managers/auth/cert/cert_generator.py", line 31, in make_admin_user_cert
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) return self.make_cert(self.encode_admin_user(user), expiration)
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) File "/usr/lib/python2.7/site-packages/pulp/server/managers/auth/cert/cert_generator.py", line 85, in make_cert
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) raise Exception("error signing cert request: %%s" %% output)
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) Exception: error signing cert request: Signature ok
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) subject=CN = admin:admin:5988b93144e534662b1fc1a2
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) Getting CA Private Key
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) Can't open /etc/pki/pulp/ca.key for reading, Permission denied
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) 139724785673984:error:0200100D:system library:fopen:Permission denied:crypto/bio/bss_file.c:74:fopen('/etc/pki/pulp/ca.key','r')
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) 139724785673984:error:2006D002:BIO routines:BIO_new_file:system lib:crypto/bio/bss_file.c:83:
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) unable to load CA Private Key
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952) unable to write 'random state'
Aug 07 15:05:38 fedora-26-pulp-2-14-beta pulp[31876]: pulp.server.webservices.middleware.exception:ERROR: (31876-77952)
It looks like /etc/pki/pulp/ca.key is unreadable:
[root@fedora-26-pulp-2-14-beta ~]# ls -laZ /etc/pki/pulp/
total 16
drwxr-xr-x. 3 root root system_u:object_r:pulp_cert_t:s0 83 Aug 7 15:02 .
drwxr-xr-x. 10 root root system_u:object_r:cert_t:s0 110 Aug 7 15:04 ..
-rw-r-----. 1 root apache unconfined_u:object_r:pulp_cert_t:s0 1753 Aug 7 15:02 ca.crt
-rw-------. 1 root apache unconfined_u:object_r:pulp_cert_t:s0 3247 Aug 7 15:02 ca.key
drwxr-xr-x. 2 apache apache system_u:object_r:pulp_cert_t:s0 6 Aug 2 12:44 content
-rw-r-----. 1 root apache unconfined_u:object_r:pulp_cert_t:s0 1679 Aug 7 15:02 rsa.key
-rw-r--r--. 1 root apache unconfined_u:object_r:pulp_cert_t:s0 451 Aug 7 15:02 rsa_pub.key
A work-around is to execute the following:
chmod g+r /etc/pki/pulp/ca.key
systemctl restart httpd pulp_{celerybeat,resource_manager,workers}
This done, Pulp will start, but different errors will start being logged. For example:
Aug 07 15:19:06 fedora-26-pulp-2-14-beta audit[954]: AVC avc: denied { read } for pid=954 comm="pulp_streamer" name="cpu" dev="sysfs" ino=33 scontext=system_u:system_r:streamer_t:s0 tcontext=system_u:object_r:sysfs_t:s0 tclass=dir permissive=0
Aug 07 15:19:07 fedora-26-pulp-2-14-beta audit[958]: AVC avc: denied { read } for pid=958 comm="celery" name="cpu" dev="sysfs" ino=33 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:sysfs_t:s0 tclass=dir permissive=0
Aug 07 15:19:07 fedora-26-pulp-2-14-beta audit[978]: AVC avc: denied { read } for pid=978 comm="celery" name="cpu" dev="sysfs" ino=33 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:sysfs_t:s0 tclass=dir permissive=0
Aug 07 15:19:07 fedora-26-pulp-2-14-beta audit[959]: AVC avc: denied { read } for pid=959 comm="celery" name="cpu" dev="sysfs" ino=33 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:sysfs_t:s0 tclass=dir permissive=0
And:
Aug 07 15:19:09 fedora-26-pulp-2-14-beta audit[978]: AVC avc: denied { getattr } for pid=978 comm="celery" name="/" dev="tmpfs" ino=10791 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=filesystem permissive=0
Aug 07 15:19:09 fedora-26-pulp-2-14-beta audit[978]: AVC avc: denied { getattr } for pid=978 comm="celery" name="/" dev="tmpfs" ino=10791 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=filesystem permissive=0
Aug 07 15:19:09 fedora-26-pulp-2-14-beta audit[978]: AVC avc: denied { getattr } for pid=978 comm="celery" name="/" dev="tmpfs" ino=10793 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=filesystem permissive=0
Aug 07 15:19:09 fedora-26-pulp-2-14-beta audit[978]: AVC avc: denied { getattr } for pid=978 comm="celery" name="/" dev="tmpfs" ino=10794 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=filesystem permissive=0
Aug 07 15:19:09 fedora-26-pulp-2-14-beta audit[978]: AVC avc: denied { getattr } for pid=978 comm="celery" name="/" dev="tmpfs" ino=18993 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=filesystem permissive=0
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) Unrecoverable error: OSError(38, 'Function not implemented')
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) Traceback (most recent call last):
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line 203, in start
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) self.blueprint.start(self)
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) step.start(parent)
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib/python2.7/site-packages/celery/bootsteps.py", line 370, in start
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) return self.obj.start()
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib/python2.7/site-packages/celery/concurrency/base.py", line 131, in start
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) self.on_start()
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib/python2.7/site-packages/celery/concurrency/prefork.py", line 112, in on_start
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) **self.options)
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib64/python2.7/site-packages/billiard/pool.py", line 952, in __init__
Aug 07 15:19:09 fedora-26-pulp-2-14-beta audit[958]: AVC avc: denied { getattr } for pid=958 comm="celery" name="/" dev="tmpfs" ino=10791 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=filesystem permissive=0
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) self._setup_queues()
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib64/python2.7/site-packages/billiard/pool.py", line 1321, in _setup_queues
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) self._inqueue = self._ctx.SimpleQueue()
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib64/python2.7/site-packages/billiard/context.py", line 150, in SimpleQueue
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) return SimpleQueue(ctx=self.get_context())
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib64/python2.7/site-packages/billiard/queues.py", line 377, in __init__
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) self._rlock = ctx.Lock()
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib64/python2.7/site-packages/billiard/context.py", line 105, in Lock
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) return Lock(ctx=self.get_context())
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib64/python2.7/site-packages/billiard/synchronize.py", line 182, in __init__
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
Aug 07 15:19:09 fedora-26-pulp-2-14-beta pulp[978]: celery.worker:CRITICAL: (978-64480) File "/usr/lib64/python2.7/site-packages/billiard/synchronize.py", line 72, in __init__
...and on it goes. I'm not sure which messages are important. The important bit is that Pulp is still screwed up after making /etc/pki/pulp/ca.key group-readable. One can try to work around this by disabling SELinux:
setenforce 0
echo > /var/log/audit/audit.log
semodule -R
systemctl restart httpd pulp_{celerybeat,resource_manager,workers}
This does produce a glorious amount of output:
[root@fedora-26-pulp-2-14-beta pulp]# audit2allow -al
#============= celery_t ==============
allow celery_t self:process execmem;
allow celery_t sysfs_t:dir read;
allow celery_t tmpfs_t:dir { add_name remove_name write };
allow celery_t tmpfs_t:file { create getattr link open read unlink write };
allow celery_t tmpfs_t:filesystem getattr;
[root@fedora-26-pulp-2-14-beta pulp]# audit2allow -Ral
could not open interface info [/var/lib/sepolgen/interface_info]
[root@fedora-26-pulp-2-14-beta pulp]# cat /var/log/audit/audit.log
type=USER_AVC msg=audit(1502136922.763:275): pid=671 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: received policyload notice (seqno=2) exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'
type=MAC_POLICY_LOAD msg=audit(1502136922.771:276): policy loaded auid=0 ses=1
type=USER_AVC msg=audit(1502136924.854:277): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='avc: received policyload notice (seqno=2) exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?'
type=SERVICE_STOP msg=audit(1502136925.402:278): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=pulp_celerybeat comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=SERVICE_START msg=audit(1502136925.404:279): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=pulp_celerybeat comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=AVC msg=audit(1502136925.816:280): avc: denied { read } for pid=2115 comm="celery" name="cpu" dev="sysfs" ino=33 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:sysfs_t:s0 tclass=dir permissive=1
type=SERVICE_STOP msg=audit(1502136926.651:281): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=httpd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=SERVICE_START msg=audit(1502136926.702:282): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=httpd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=SERVICE_STOP msg=audit(1502136927.385:283): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=pulp_resource_manager comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=SERVICE_START msg=audit(1502136927.386:284): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=pulp_resource_manager comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=AVC msg=audit(1502136932.568:285): avc: denied { getattr } for pid=2193 comm="celery" name="/" dev="tmpfs" ino=10791 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=filesystem permissive=1
type=AVC msg=audit(1502136932.568:286): avc: denied { write } for pid=2193 comm="celery" name="/" dev="tmpfs" ino=10791 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1502136932.568:287): avc: denied { add_name } for pid=2193 comm="celery" name="vcJczy" scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1502136932.568:288): avc: denied { create } for pid=2193 comm="celery" name="vcJczy" scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1502136932.568:289): avc: denied { read write open } for pid=2193 comm="celery" path="/dev/shm/vcJczy" dev="tmpfs" ino=55350 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1502136932.568:290): avc: denied { link } for pid=2193 comm="celery" name="vcJczy" dev="tmpfs" ino=55350 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1502136932.568:291): avc: denied { getattr } for pid=2193 comm="celery" path="/dev/shm/vcJczy" dev="tmpfs" ino=55350 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1502136932.569:292): avc: denied { remove_name } for pid=2193 comm="celery" name="vcJczy" dev="tmpfs" ino=55350 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=dir permissive=1
type=AVC msg=audit(1502136932.569:293): avc: denied { unlink } for pid=2193 comm="celery" name="vcJczy" dev="tmpfs" ino=55350 scontext=system_u:system_r:celery_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1
type=SERVICE_STOP msg=audit(1502137015.062:294): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=pulp_workers comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
type=SERVICE_STOP msg=audit(1502137015.094:295): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=pulp_worker-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
type=SERVICE_START msg=audit(1502137015.116:296): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=pulp_worker-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=SERVICE_START msg=audit(1502137015.124:297): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=pulp_workers comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=AVC msg=audit(1502137057.690:298): avc: denied { execmem } for pid=2447 comm="celery" scontext=system_u:system_r:celery_t:s0 tcontext=system_u:system_r:celery_t:s0 tclass=process permissive=1
type=USER_START msg=audit(1502137067.949:299): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1502137067.950:300): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:a1:7b:07:ac:67:e9:1f:90:0d:bf:ca:4c:71:e4:10:54:f1:78:be:2e:8a:5a:ed:9d:ab:dc:98:b2:85:6a:a6:30 direction=? spid=2481 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=USER_START msg=audit(1502137067.969:301): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1502137067.969:302): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:a1:7b:07:ac:67:e9:1f:90:0d:bf:ca:4c:71:e4:10:54:f1:78:be:2e:8a:5a:ed:9d:ab:dc:98:b2:85:6a:a6:30 direction=? spid=2496 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=USER_START msg=audit(1502137067.990:303): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1502137067.991:304): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:a1:7b:07:ac:67:e9:1f:90:0d:bf:ca:4c:71:e4:10:54:f1:78:be:2e:8a:5a:ed:9d:ab:dc:98:b2:85:6a:a6:30 direction=? spid=2516 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=USER_END msg=audit(1502137067.998:305): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=USER_START msg=audit(1502137068.006:306): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1502137068.006:307): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:a1:7b:07:ac:67:e9:1f:90:0d:bf:ca:4c:71:e4:10:54:f1:78:be:2e:8a:5a:ed:9d:ab:dc:98:b2:85:6a:a6:30 direction=? spid=2525 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=USER_START msg=audit(1502137068.026:308): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1502137068.026:309): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:a1:7b:07:ac:67:e9:1f:90:0d:bf:ca:4c:71:e4:10:54:f1:78:be:2e:8a:5a:ed:9d:ab:dc:98:b2:85:6a:a6:30 direction=? spid=2545 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=USER_END msg=audit(1502137068.033:310): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=USER_START msg=audit(1502137068.043:311): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1502137068.043:312): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:a1:7b:07:ac:67:e9:1f:90:0d:bf:ca:4c:71:e4:10:54:f1:78:be:2e:8a:5a:ed:9d:ab:dc:98:b2:85:6a:a6:30 direction=? spid=2559 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=SERVICE_STOP msg=audit(1502137069.821:313): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=httpd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=SERVICE_START msg=audit(1502137069.859:314): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=httpd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=USER_END msg=audit(1502137069.861:315): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=USER_END msg=audit(1502137076.379:316): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=USER_END msg=audit(1502137076.379:317): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=USER_END msg=audit(1502137076.379:318): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=USER_START msg=audit(1502137096.412:319): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1502137096.412:320): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:a1:7b:07:ac:67:e9:1f:90:0d:bf:ca:4c:71:e4:10:54:f1:78:be:2e:8a:5a:ed:9d:ab:dc:98:b2:85:6a:a6:30 direction=? spid=2869 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=USER_START msg=audit(1502137096.433:321): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1502137096.434:322): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:a1:7b:07:ac:67:e9:1f:90:0d:bf:ca:4c:71:e4:10:54:f1:78:be:2e:8a:5a:ed:9d:ab:dc:98:b2:85:6a:a6:30 direction=? spid=2884 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=USER_START msg=audit(1502137096.458:323): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1502137096.459:324): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:a1:7b:07:ac:67:e9:1f:90:0d:bf:ca:4c:71:e4:10:54:f1:78:be:2e:8a:5a:ed:9d:ab:dc:98:b2:85:6a:a6:30 direction=? spid=2904 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=USER_END msg=audit(1502137096.471:325): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=USER_START msg=audit(1502137096.483:326): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1502137096.487:327): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=SHA256:a1:7b:07:ac:67:e9:1f:90:0d:bf:ca:4c:71:e4:10:54:f1:78:be:2e:8a:5a:ed:9d:ab:dc:98:b2:85:6a:a6:30 direction=? spid=2918 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=SERVICE_STOP msg=audit(1502137098.267:328): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=httpd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=SERVICE_START msg=audit(1502137098.305:329): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=httpd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=USER_END msg=audit(1502137098.307:330): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=USER_END msg=audit(1502137106.448:331): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
type=USER_END msg=audit(1502137106.448:332): pid=1253 uid=0 auid=0 ses=1 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=192.168.100.1 addr=192.168.100.1 terminal=ssh res=success'
Unfortunately, many tests still fail:
python -m unittest pulp_smash.tests.platform.api_v2.test_login # success
python -m unittest pulp_smash.tests.docker.api_v2.test_sync_publish.V{1,2}RegistryTestCase # total failure
Updated by Ichimonji10 over 7 years ago
I've marked this issue as blocking 2.14 out of precaution. It may turn out that there's some issue with pulp_packaging that's messing up Pulp 2.14 on Fedora 26, and when installed through other methods, Pulp 2.14 is great on Fedora 26. But I don't know that right now, and as a result of this issue, we have no automated test results for Pulp 2.14 on Fedora 26. Until the validity of Pulp 2.14 on Fedora 26 is proven, I think we should assume that there's a blocking issue here.
Updated by ttereshc over 7 years ago
- Priority changed from Normal to High
- Sprint/Milestone set to 42
- Triaged changed from No to Yes
After more investigation it can be re-considered as a blocker.
Updated by Ichimonji10 over 7 years ago
For what it's worth, Fedora 26 ships with a new major version of Celery.
Fedora 24: python2-celery-3.1.20-2.fc24.noarch
Fedora 25: python2-celery-3.1.20-3.fc25.noarch
Fedora 26: python2-celery-4.0.2-2.fc26.noarch
RHEL 7: python-celery-3.1.17-1.el7.noarch
Updated by pcreech over 7 years ago
W/R to "It looks like /etc/pki/pulp/ca.key is unreadable:", it appears the default permissions for https://github.com/pulp/pulp/blob/master/server/bin/pulp-gen-ca-certificate#L35 result has changed.
On F25:
[root@f25host bob]# openssl genrsa -out test.key 4096 &> /dev/null
[root@f25host bob]# ls -l
total 4
-rw-r--r--. 1 root root 3243 Aug 8 11:16 test.key
On F26:
[root@f26host bob]# openssl genrsa -out test.key 4096 &> /dev/null
[root@f26host bob]# ls -l
total 4
-rw-------. 1 root root 3243 Aug 8 11:17 test.key
Updated by mhrivnak over 7 years ago
I think we should block releasing 2.14.0 on F26, but proceed with releasing on other platforms. pcreech said that would be relatively easy to do from a release engineering perspective.
Depending on what changes are required to get this fixed for F26, we could either make F26 packages available later once non-code issues are resolved (packaging, ansible, selinux, etc), or just wait and add F26 packages with 2.14.1.
Considering how few of our users deploy on Fedora (evidence suggest perhaps zero have tried Pulp on F26), it would be a shame to withhold 2.14 on the working platforms.
Updated by pcreech over 7 years ago
Release Blocker is for Fedora 26 builds only. All other platforms can be released.
Updated by bizhang about 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to bizhang
Updated by bizhang about 7 years ago
- Status changed from ASSIGNED to POST
Added by werwty about 7 years ago
Updated by jortel@redhat.com about 7 years ago
- Sprint/Milestone changed from 43 to 44
Updated by werwty about 7 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulp|1a13d6de52da8384ae74ba6dbfe65afa6d3b930d.
Updated by Ichimonji10 about 7 years ago
- Status changed from MODIFIED to ASSIGNED
Pulp 2.14 is still broken in major ways on F26. As a quick sanity check, I applied the following diff to Pulp Smash:
diff --git a/pulp_smash/tests/docker/api_v2/test_sync_publish.py b/pulp_smash/tests/docker/api_v2/test_sync_publish.py
index ef43fb2..037fcdf 100644
--- a/pulp_smash/tests/docker/api_v2/test_sync_publish.py
+++ b/pulp_smash/tests/docker/api_v2/test_sync_publish.py
@@ -219,9 +219,6 @@ class V1RegistryTestCase(SyncPublishMixin, unittest.TestCase):
super().setUpClass()
cls.cfg = config.get_config()
cls.repo = {}
- if (utils.os_is_f26(cls.cfg) and
- selectors.bug_is_untestable(3036, cls.cfg.version)):
- raise unittest.SkipTest('https://pulp.plan.io/issues/3036')
@classmethod
def tearDownClass(cls):
@@ -306,9 +303,6 @@ class V2RegistryTestCase(SyncPublishMixin, unittest.TestCase):
super().setUpClass()
cls.cfg = config.get_config()
cls.repo = {}
- if (utils.os_is_f26(cls.cfg) and
- selectors.bug_is_untestable(3036, cls.cfg.version)):
- raise unittest.SkipTest('https://pulp.plan.io/issues/3036')
for issue_id in (2287, 2384):
if selectors.bug_is_untestable(issue_id, cls.cfg.version):
raise unittest.SkipTest(
I then executed python -m unittest pulp_smash.tests.docker.api_v2.test_sync_publish.V{1,2}RegistryTestCase. It blew up. The journal didn't reference SELinux, but it did include entries like this:
Oct 03 16:30:54 fedora-26-pulp-2-14-beta pulp[1524]: crane.config:INFO: config loaded from /etc/crane.conf
Oct 03 16:30:54 fedora-26-pulp-2-14-beta pulp[1524]: crane.data:INFO: loading metadata from /var/lib/pulp/published/docker
Oct 03 16:30:54 fedora-26-pulp-2-14-beta pulp[1524]: crane.data:ERROR: aborting metadata load: metadata version 4 not supported
Oct 03 16:30:54 fedora-26-pulp-2-14-beta pulp[1524]: crane.search:INFO: no search backend configured
Oct 03 16:30:54 fedora-26-pulp-2-14-beta pulp[1524]: crane.app:INFO: application initialized
Oct 03 16:30:54 fedora-26-pulp-2-14-beta pulp[1524]: crane.data:INFO: loading metadata from /var/lib/pulp/published/docker
Oct 03 16:30:54 fedora-26-pulp-2-14-beta pulp[1524]: crane.data:ERROR: aborting metadata load: metadata version 4 not supported
Oct 03 16:30:55 fedora-26-pulp-2-14-beta pulp[1523]: crane.config:INFO: config loaded from /etc/crane.conf
Oct 03 16:30:55 fedora-26-pulp-2-14-beta pulp[1523]: crane.data:INFO: loading metadata from /var/lib/pulp/published/docker
Oct 03 16:30:55 fedora-26-pulp-2-14-beta pulp[1523]: crane.data:ERROR: aborting metadata load: metadata version 4 not supported
There may be other issues with Pulp 2.14 on Fedora 26. I can't say, though, because the relevant Jenkins runs consistently and catastrophically fail.
I don't think it's acceptable to release the 2.14.1 beta for F26 with Docker being broken and without the reassurance of Jenkins.
Updated by mhrivnak about 7 years ago
It looks like crane is missing from the F26 beta repo. As such, I assume your automation installed the older version of crane that comes with F26, and that is why you see the error ""aborting metadata load: metadata version 4 not supported "
Updated by Ichimonji10 about 7 years ago
Pulp is being installed from the pulp_server.yaml Ansible playbook in pulp_packaging.
Added by pcreech about 7 years ago
Updated by pcreech about 7 years ago
mhrivnak wrote:
It looks like crane is missing from the F26 beta repo. As such, I assume your automation installed the older version of crane that comes with F26, and that is why you see the error ""aborting metadata load: metadata version 4 not supported "
Crane is now showing up in fedora 26 nightly builds
Updated by rchan almost 7 years ago
Should this bug be dispositioned? The high priority along with no longer being in the current sprint along with not being fixed by now seems inconsistent.
Updated by dalley about 6 years ago
- Status changed from ASSIGNED to CLOSED - CURRENTRELEASE
Actions
.
Fix SELinux and cert key permissions
closes #2961 https://pulp.plan.io/issues/2961