Project

Profile

Help

Issue #7954

pulpcore-api can't start after upgrade

Added by dkliban@redhat.com 4 months ago. Updated 2 days ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
Installer
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 95
Quarter:

Description

[15070]: [2020-12-08 20:49:16 +0000] [15070] [INFO] Starting gunicorn 20.0.4
[15070]: [2020-12-08 20:49:16 +0000] [15070] [INFO] Listening at: http://127.0.0.1:24817 (15070)
[15070]: [2020-12-08 20:49:16 +0000] [15070] [INFO] Using worker: sync
[15070]: [2020-12-08 20:49:16 +0000] [15073] [INFO] Booting worker with pid: 15073
[15070]: [2020-12-08 20:49:19 +0000] [15073] [ERROR] Exception in worker process
[15070]: Traceback (most recent call last):
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
[15070]: worker.init_process()
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py", line 119, in init_process
[15070]: self.load_wsgi()
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
[15070]: self.wsgi = self.app.wsgi()
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
[15070]: self.callable = self.load()
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
[15070]: return self.load_wsgiapp()
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
[15070]: return util.import_app(self.app_uri)
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/util.py", line 358, in import_app
[15070]: mod = importlib.import_module(module)
[15070]: File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
[15070]: return _bootstrap._gcd_import(name[level:], package, level)
[15070]: File "", line 994, in _gcd_import
[15070]: File "", line 971, in _find_and_load
[15070]: File "", line 955, in _find_and_load_unlocked
[15070]: File "", line 665, in _load_unlocked
[15070]: File "", line 678, in exec_module
[15070]: File "", line 219, in _call_with_frames_removed
[15070]: File "/home/vagrant/devel/pulpcore/pulpcore/app/wsgi.py", line 17, in 
[15070]: application = get_wsgi_application()
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
[15070]: return WSGIHandler()
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/wsgi.py", line 135, in __init__
[15070]: self.load_middleware()
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py", line 37, in load_middleware
[15070]: mw_instance = middleware(handler)
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/middleware.py", line 50, in __init__
[15070]: self.add_files(self.static_root, prefix=self.static_prefix)
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py", line 113, in add_files
[15070]: self.update_files_dictionary(root, prefix)
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py", line 120, in update_files_dictionary
[15070]: stat_cache = dict(scantree(root))
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py", line 271, in scantree
[15070]: yield from scantree(entry.path)
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py", line 271, in scantree
[15070]: yield from scantree(entry.path)
[15070]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py", line 273, in scantree
[15070]: yield entry.path, entry.stat()
[15070]: FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/pulp/assets/rest_framework/js/jquery-3.5.1.min.js'

This occurs when one of the dependencies, such as rest_framework, is updated and the symlinks pointing to the older version are no longer pointing to a file that exists. Users can work around this problem by deleting the broken symlinks in /var/lib/pulp/assets/.

The installer should stop using --link option when running 'collectstatic'[0] or another task needs to be added to clean up broken symlinks.

[0] https://github.com/pulp/pulp_installer/blob/ba4b60bb125b76136f7751ef1821870ab76720cb/roles/pulp_common/handlers/main.yml#L35

History

#1 Updated by dkliban@redhat.com 3 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 88

#2 Updated by mdellweg 3 months ago

From discussion we figured, that django collect static will not clean up old links to static assets. Maybe we can add a task to the collect static handler to clean up all broken symlinks.

#3 Updated by rchan 3 months ago

  • Sprint changed from Sprint 88 to Sprint 89

#4 Updated by rchan 2 months ago

  • Sprint changed from Sprint 89 to Sprint 90

#5 Updated by rchan about 2 months ago

  • Sprint changed from Sprint 90 to Sprint 91

#6 Updated by rchan about 1 month ago

  • Sprint changed from Sprint 91 to Sprint 92

#7 Updated by rchan 30 days ago

  • Sprint changed from Sprint 92 to Sprint 93

#8 Updated by rchan 12 days ago

  • Sprint changed from Sprint 93 to Sprint 94

#9 Updated by rchan 2 days ago

  • Sprint changed from Sprint 94 to Sprint 95

Please register to edit this issue

Also available in: Atom PDF