Project

Profile

Help

Issue #2849

closed

Worker terminates abnormally while processing task

Added by ipanova@redhat.com over 6 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
3. High
Version - Docker:
Platform Release:
2.18.0
Target Release - Docker:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 45
Quarter:

Description

When i am syncing manifests that have foreign layers, i eventually get worker get killed.


$ pulp-admin docker repo  sync run --repo-id bz
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/

+----------------------------------------------------------------------+
                     Synchronizing Repository [bz]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading manifests
[\]
... completed

Copying units already in pulp
[-]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[=                                                 ] 3%
8 of 221 items

Task Failed

Worker terminated abnormally while processing task
42fabc46-17b3-419f-a79c-ccd0bcba1474.  Check the logs for details

Pulp starts to panic when it hits 404.
We need to check if the registry closes the connection prematurely and investigate what causes the worker termination. It is not OOM kill.

Jun 28 12:34:14 ina pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://registry-1.docker.io/v2/portainer/portainer/blobs/sha256:6330793656b1565cd8b5d5c1e2a736a351dec39f7250daf31eb91e3a76cc872b failed with code 404: Not Found
Jun 28 12:34:14 ina pulp: celery.worker.job:ERROR: (11813-08608) Task pulp.server.managers.repo.sync.sync[b28a4117-b647-484f-a9da-45a15e06f09e] raised unexpected: WorkerLostError('Worker exited prematurely: signal 9 (SIGKILL).',)

Related issues

Related to Docker Support - Issue #2966: Unable to sync docker repo because worker diesCLOSED - CURRENTRELEASEamacdona@redhat.comActions
Related to Docker Support - Test #4128: Test sync of a repository that is missing blobsCLOSED - COMPLETEbherringActions

Also available in: Atom PDF