Migration Plugin

Support for different plugins is added through the endpoints.
All endpoints are always loaded, which basically means that all the supported plugins have to be installed.

If one of the supported plugins is not installed, workers won't be able to start:

Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com systemd[1]: pulpcore-worker@2.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com systemd[1]: Unit pulpcore-worker@2.service entered failed state.
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com systemd[1]: pulpcore-worker@2.service failed.
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: [2019-11-28 14:00:50 +0000] [6309] [ERROR] Exception in worker process
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: Traceback (most recent call last):
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: worker.init_process()
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py", line 119, in init_process
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.load_wsgi()
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.wsgi = self.app.wsgi()
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.callable = self.load()
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return self.load_wsgiapp()
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return util.import_app(self.app_uri)
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/util.py", line 358, in import_app
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: mod = importlib.import_module(module)
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return _bootstrap._gcd_import(name[level:], package, level)
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 994, in _gcd_import
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 971, in _find_and_load
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap_external>", line 678, in exec_module
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/wsgi.py", line 17, in <module>
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: application = get_wsgi_application()
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: django.setup(set_prefix=False)
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py", line 24, in setup
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: apps.populate(settings.INSTALLED_APPS)
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py", line 122, in populate
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: app_config.ready()
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/apps.py", line 75, in ready
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.import_viewsets()
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/apps.py", line 108, in import_viewsets
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: self.viewsets_module = import_module(viewsets_module_name)
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return _bootstrap._gcd_import(name[level:], package, level)
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 994, in _gcd_import
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 971, in _find_and_load
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap_external>", line 678, in exec_module
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/viewsets.py", line 22, in <module>
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from .serializers import (
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/serializers.py", line 9, in <module>
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from pulp_2to3_migration.app.plugin import PLUGIN_MIGRATORS
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/plugin/__init__.py", line 8, in <module>
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: PLUGIN_MIGRATORS[entry_point.name] = entry_point.load()
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py", line 2318, in load
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: return self.resolve()
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in resolve
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: module = __import__(self.module_name, fromlist=['__name__'], level=0)
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/plugin/docker/migrator.py", line 7, in <module>
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from .pulp_2to3_models import (
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: File "/home/vagrant/devel/pulp-2to3-migrate/pulp_2to3_migration/app/plugin/docker/pulp_2to3_models.py", line 6, in <module>
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: from pulp_container.constants import MEDIA_TYPE
Nov 28 14:00:51 pulp2-nightly-pulp3-source-centos7.rhgoose.example.com gunicorn[6295]: ModuleNotFoundError: No module named 'pulp_container'

Proposed solution¶

Load endpoints conditionally.
E.g. If it's possible to import from the pulp3 plugin module, then load the corresponding endpoint, otherwise skip it.