Issue #5820
Updated by ttereshc almost 5 years ago
Support for different plugins is added through the endpoints. All endpoints are always loaded, which basically means that all the supported plugins have has to be installed. If one of the supported plugins is not installed, workers won't be able to start: <pre> Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com systemd[1]: pulpcore-worker@2.service: main process exited, code=exited, status=2/INVALIDARGUMENT Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com systemd[1]: Unit pulpcore-worker@2.service entered failed state. Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com systemd[1]: pulpcore-worker@2.service failed. Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: [2019-11-28 14:00:50 +0000] [6309] [ERROR] Exception in worker process Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: Traceback (most recent call last): Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: worker.init_process() Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py", line 119, in init_process Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.load_wsgi() Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.wsgi = self.app.wsgi() Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.callable = self.load() Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 49, in load Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return self.load_wsgiapp() Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return util.import_app(self.app_uri) Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/util.py", line 358, in import_app Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: mod = importlib.import_module(module) Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return _bootstrap._gcd_import(name[level:], package, level) Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 994, in _gcd_import Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 971, in _find_and_load Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 665, in _load_unlocked Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap_external>", line 678, in exec_module Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/wsgi.py", line 17, in <module> Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: application = get_wsgi_application() Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: django.setup(set_prefix=False) Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py", line 24, in setup Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: apps.populate(settings.INSTALLED_APPS) Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py", line 122, in populate Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: app_config.ready() Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/apps.py", line 75, in ready Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.import_viewsets() Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/apps.py", line 108, in import_viewsets Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.viewsets_module = import_module(viewsets_module_name) Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return _bootstrap._gcd_import(name[level:], package, level) Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 994, in _gcd_import Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 971, in _find_and_load Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 665, in _load_unlocked Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap_external>", line 678, in exec_module Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/viewsets.py", line 22, in <module> Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from .serializers import ( Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/serializers.py", line 9, in <module> Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from pulp_2to3_migration.app.plugin import PLUGIN_MIGRATORS Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/plugin/__init__.py", line 8, in <module> Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: PLUGIN_MIGRATORS[entry_point.name] = entry_point.load() Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py", line 2318, in load Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return self.resolve() Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in resolve Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: module = __import__(self.module_name, fromlist=['__name__'], level=0) Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/plugin/docker/migrator.py", line 7, in <module> Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from .pulp_2to3_models import ( Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/plugin/docker/pulp_2to3_models.py", line 6, in <module> Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from pulp_container.constants import MEDIA_TYPE Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: ModuleNotFoundError: No module named 'pulp_container' </pre> h3. Proposed solution Load endpoints conditionally. E.g. If it's possible to import from the pulp3 plugin module, then load the corresponding endpoint, otherwise skip it.
.