Project

Profile

Help

Backport #9282

closed

Backport "Content app db connection can be closed while matching a distribution" to 3.14.z

Added by ttereshc over 2 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
-
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Triaged:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Quarter:

Description

To reproduce (not reliably):

  • sync/publish/distribute an on_demand kickstart repo
  • try to browse this repo
  • you likely need to have many other repositories in pulp.
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: [2021-08-18 08:42:49 +0000] [27629] [ERROR] Error handling request
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: Traceback (most recent call last):
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 235, in _cursor
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: return self._prepare_cursor(self.create_cursor(name))
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 223, in create_cursor
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: cursor = self.connection.cursor()
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: psycopg2.InterfaceError: connection already closed
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: The above exception was the direct cause of the following exception:
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: Traceback (most recent call last):
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib64/python3.6/site-packages/aiohttp/web_protocol.py", line 422, in _handle_request
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: resp = await self._request_handler(request)
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib64/python3.6/site-packages/aiohttp/web_app.py", line 499, in _handle
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: resp = await handler(request)
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/pulpcore/content/handler.py", line 212, in stream_content
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: return await self._match_and_stream(path, request)
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/pulpcore/content/handler.py", line 443, in _match_and_stream
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: distro = await loop.run_in_executor(None, match_distribution_blocking)
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: result = self.fn(*self.args, **self.kwargs)
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/pulpcore/content/handler.py", line 441, in match_distribution_blocking
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: return self._match_distribution(path)
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/pulpcore/content/handler.py", line 256, in _match_distribution
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: .get(base_path__in=base_paths)
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 402, in get
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: num = len(clone)
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __len__
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: self._fetch_all()
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1242, in _fetch_all
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: self._result_cache = list(self._iterable_class(self))
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 55, in __iter__
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1140, in execute_sql
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: cursor = self.connection.cursor()
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 256, in cursor
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: return self._cursor()
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 235, in _cursor
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: return self._prepare_cursor(self.create_cursor(name))
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: raise dj_exc_value.with_traceback(traceback) from exc_value
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/backends/base/base.py", line 235, in _cursor
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: return self._prepare_cursor(self.create_cursor(name))
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: File "/usr/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 223, in create_cursor
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: cursor = self.connection.cursor()
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: django.db.utils.InterfaceError: connection already closed
Aug 18 09:42:49 ktordeur-testathon pulpcore-content: [18/Aug/2021:08:42:49 +0000] "GET /pulp/content/Default_Organization/Library/content/dist/rhel8/8.4/x86_64/appstream/kickstart/ HTTP/1.1" 500 334 "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"

Related issues

Copied from Pulp - Issue #9275: Content app db connection can be closed while matching a distributionCLOSED - CURRENTRELEASEActions
Actions #1

Updated by ttereshc over 2 years ago

  • Copied from Issue #9275: Content app db connection can be closed while matching a distribution added
Actions #2

Updated by ttereshc over 2 years ago

  • Sprint/Milestone set to 3.14.5

Added by dalley over 2 years ago

Revision 082f4ee1 | View on GitHub

Reset a db connection in content app distribution matching code

It's a workaround for yet another connection already closed issue. backports #9275 https://pulp.plan.io/issues/9275

https://pulp.plan.io/issues/9276 is filed to investigate that issue more.

fixes #9282

(cherry picked from commit d603476240edb942e1505c2deea5dc697fedd4a2)

Actions #3

Updated by dalley over 2 years ago

  • Status changed from NEW to MODIFIED
  • % Done changed from 0 to 100
Actions #4

Updated by pulpbot over 2 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF