Project

Profile

Help

Issue #5820

Updated by ttereshc over 2 years ago

Support for different plugins is added through the endpoints. 
 All endpoints are always loaded, which basically means that all the supported plugins have has to be installed. 

 If one of the supported plugins is not installed, workers won't be able to start: 
 <pre> 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com systemd[1]: pulpcore-worker@2.service: main process exited, code=exited, status=2/INVALIDARGUMENT 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com systemd[1]: Unit pulpcore-worker@2.service entered failed state. 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com systemd[1]: pulpcore-worker@2.service failed. 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: [2019-11-28 14:00:50 +0000] [6309] [ERROR] Exception in worker process 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: Traceback (most recent call last): 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: worker.init_process() 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py", line 119, in init_process 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.load_wsgi() 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.wsgi = self.app.wsgi() 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.callable = self.load() 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 49, in load 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return self.load_wsgiapp() 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return util.import_app(self.app_uri) 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/util.py", line 358, in import_app 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: mod = importlib.import_module(module) 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return _bootstrap._gcd_import(name[level:], package, level) 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 994, in _gcd_import 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 971, in _find_and_load 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 665, in _load_unlocked 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap_external>", line 678, in exec_module 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/wsgi.py", line 17, in <module> 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: application = get_wsgi_application() 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: django.setup(set_prefix=False) 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py", line 24, in setup 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: apps.populate(settings.INSTALLED_APPS) 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py", line 122, in populate 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: app_config.ready() 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/apps.py", line 75, in ready 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.import_viewsets() 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/apps.py", line 108, in import_viewsets 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.viewsets_module = import_module(viewsets_module_name) 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return _bootstrap._gcd_import(name[level:], package, level) 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 994, in _gcd_import 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 971, in _find_and_load 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 665, in _load_unlocked 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap_external>", line 678, in exec_module 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/viewsets.py", line 22, in <module> 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from .serializers import ( 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/serializers.py", line 9, in <module> 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from pulp_2to3_migration.app.plugin import PLUGIN_MIGRATORS 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/plugin/__init__.py", line 8, in <module> 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: PLUGIN_MIGRATORS[entry_point.name] = entry_point.load() 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py", line 2318, in load 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return self.resolve() 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in resolve 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: module = __import__(self.module_name, fromlist=['__name__'], level=0) 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/plugin/docker/migrator.py", line 7, in <module> 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from .pulp_2to3_models import ( 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/plugin/docker/pulp_2to3_models.py", line 6, in <module> 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from pulp_container.constants import MEDIA_TYPE 
 Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: ModuleNotFoundError: No module named 'pulp_container' 

 </pre> 

 h3. Proposed solution 

 Load endpoints conditionally. 
 E.g. If it's possible to import from the pulp3 plugin module, then load the corresponding endpoint, otherwise skip it.

Back