Project

Profile

Help

Issue #6459

Restarting pulpcore-worker and pulpcore-resource-manager processes leaves old redis queues forever

Added by bmbouter 6 months ago. Updated 6 months ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Worker names are driven by the pid of the process they starts as and so they change ever restart. This is by design. Due to this, a Pulp system with the workers restarted many times will accumulate an unreasonable number of RQ queues in Redis which will likely never be used again.

For example here's a dev system where I've restarted the processes many times. Here is the rq info output:

36075@pulp3-source-fedora31.localhost.example.com | 0
60601@pulp3-source-fedora31.localhost.example.com | 0
86668@pulp3-source-fedora31.localhost.example.com | 0
28656@pulp3-source-fedora31.localhost.example.com | 0
82464@pulp3-source-fedora31.localhost.example.com | 0
115968@pulp3-source-fedora31.localhost.example.com | 0
142415@pulp3-source-fedora31.localhost.example.com | 0
120333@pulp3-source-fedora31.localhost.example.com | 0
120760@pulp3-source-fedora31.localhost.example.com | 0
141871@pulp3-source-fedora31.localhost.example.com | 0
130664@pulp3-source-fedora31.localhost.example.com | 0
117873@pulp3-source-fedora31.localhost.example.com | 0
49515@pulp3-source-fedora31.localhost.example.com | 0
61176@pulp3-source-fedora31.localhost.example.com | 0
57742@pulp3-source-fedora31.localhost.example.com | 0
32907@pulp3-source-fedora31.localhost.example.com | 0
85030@pulp3-source-fedora31.localhost.example.com | 0
128174@pulp3-source-fedora31.localhost.example.com | 0
31505@pulp3-source-fedora31.localhost.example.com | 0
143329@pulp3-source-fedora31.localhost.example.com | 0
91892@pulp3-source-fedora31.localhost.example.com | 0
63839@pulp3-source-fedora31.localhost.example.com | 0
72609@pulp3-source-fedora31.localhost.example.com | 0
119400@pulp3-source-fedora31.localhost.example.com | 0
51828@pulp3-source-fedora31.localhost.example.com | 0
53526@pulp3-source-fedora31.localhost.example.com | 0
53846@pulp3-source-fedora31.localhost.example.com | 0
52975@pulp3-source-fedora31.localhost.example.com | 0
142182@pulp3-source-fedora31.localhost.example.com | 0
87528@pulp3-source-fedora31.localhost.example.com | 0
129849@pulp3-source-fedora31.localhost.example.com | 0
52727@pulp3-source-fedora31.localhost.example.com | 0
115291@pulp3-source-fedora31.localhost.example.com | 0
127930@pulp3-source-fedora31.localhost.example.com | 0
83454@pulp3-source-fedora31.localhost.example.com | 0
120335@pulp3-source-fedora31.localhost.example.com | 0
116775@pulp3-source-fedora31.localhost.example.com | 0
52426@pulp3-source-fedora31.localhost.example.com | 0
115965@pulp3-source-fedora31.localhost.example.com | 0
60366@pulp3-source-fedora31.localhost.example.com | 0
60086@pulp3-source-fedora31.localhost.example.com | 0
130663@pulp3-source-fedora31.localhost.example.com | 0
131373@pulp3-source-fedora31.localhost.example.com | 0
143545@pulp3-source-fedora31.localhost.example.com | 0
50656@pulp3-source-fedora31.localhost.example.com | 0
83712@pulp3-source-fedora31.localhost.example.com | 0
85031@pulp3-source-fedora31.localhost.example.com | 0
84904@pulp3-source-fedora31.localhost.example.com | 0
140946@pulp3-source-fedora31.localhost.example.com | 0
110739@pulp3-source-fedora31.localhost.example.com | 0
110738@pulp3-source-fedora31.localhost.example.com | 0
122919@pulp3-source-fedora31.localhost.example.com | 0
90982@pulp3-source-fedora31.localhost.example.com | 0
34205@pulp3-source-fedora31.localhost.example.com | 0
120488@pulp3-source-fedora31.localhost.example.com | 0
87529@pulp3-source-fedora31.localhost.example.com | 0
53278@pulp3-source-fedora31.localhost.example.com | 0
72352@pulp3-source-fedora31.localhost.example.com | 0
84520@pulp3-source-fedora31.localhost.example.com | 0
36955@pulp3-source-fedora31.localhost.example.com | 0
82947@pulp3-source-fedora31.localhost.example.com | 0
141509@pulp3-source-fedora31.localhost.example.com | 0
32632@pulp3-source-fedora31.localhost.example.com | 0
36954@pulp3-source-fedora31.localhost.example.com | 0
58403@pulp3-source-fedora31.localhost.example.com | 0
60600@pulp3-source-fedora31.localhost.example.com | 0
119399@pulp3-source-fedora31.localhost.example.com | 0
143755@pulp3-source-fedora31.localhost.example.com | 0
119696@pulp3-source-fedora31.localhost.example.com | 0
48028@pulp3-source-fedora31.localhost.example.com | 0
84521@pulp3-source-fedora31.localhost.example.com | 0
52121@pulp3-source-fedora31.localhost.example.com | 0
33891@pulp3-source-fedora31.localhost.example.com | 0
122918@pulp3-source-fedora31.localhost.example.com | 0
42777@pulp3-source-fedora31.localhost.example.com | 0
71920@pulp3-source-fedora31.localhost.example.com | 0
83240@pulp3-source-fedora31.localhost.example.com | 0
37388@pulp3-source-fedora31.localhost.example.com | 0
120489@pulp3-source-fedora31.localhost.example.com | 0
72351@pulp3-source-fedora31.localhost.example.com | 0
144388@pulp3-source-fedora31.localhost.example.com | 0
61712@pulp3-source-fedora31.localhost.example.com | 0
127929@pulp3-source-fedora31.localhost.example.com | 0
53277@pulp3-source-fedora31.localhost.example.com | 0
143018@pulp3-source-fedora31.localhost.example.com | 0
83951@pulp3-source-fedora31.localhost.example.com | 0
resource-manager | 0
42776@pulp3-source-fedora31.localhost.example.com | 0
52726@pulp3-source-fedora31.localhost.example.com | 0
61177@pulp3-source-fedora31.localhost.example.com | 0
53525@pulp3-source-fedora31.localhost.example.com | 0
58870@pulp3-source-fedora31.localhost.example.com | 0
71919@pulp3-source-fedora31.localhost.example.com | 0
48029@pulp3-source-fedora31.localhost.example.com | 0
82948@pulp3-source-fedora31.localhost.example.com | 0
119118@pulp3-source-fedora31.localhost.example.com | 0
140945@pulp3-source-fedora31.localhost.example.com | 0
116985@pulp3-source-fedora31.localhost.example.com | 0
143756@pulp3-source-fedora31.localhost.example.com | 0
142414@pulp3-source-fedora31.localhost.example.com | 0
60085@pulp3-source-fedora31.localhost.example.com | 0
38378@pulp3-source-fedora31.localhost.example.com | 0
53845@pulp3-source-fedora31.localhost.example.com | 0
28657@pulp3-source-fedora31.localhost.example.com | 0
82854@pulp3-source-fedora31.localhost.example.com | 0
83451@pulp3-source-fedora31.localhost.example.com | 0
52120@pulp3-source-fedora31.localhost.example.com | 0
118184@pulp3-source-fedora31.localhost.example.com | 0
129850@pulp3-source-fedora31.localhost.example.com | 0
141201@pulp3-source-fedora31.localhost.example.com | 0
122693@pulp3-source-fedora31.localhost.example.com | 0
32103@pulp3-source-fedora31.localhost.example.com | 0
131372@pulp3-source-fedora31.localhost.example.com | 0
61710@pulp3-source-fedora31.localhost.example.com | 0
82853@pulp3-source-fedora31.localhost.example.com | 0
51829@pulp3-source-fedora31.localhost.example.com | 0
63840@pulp3-source-fedora31.localhost.example.com | 0
72608@pulp3-source-fedora31.localhost.example.com | 0
91894@pulp3-source-fedora31.localhost.example.com | 0
36306@pulp3-source-fedora31.localhost.example.com | 0
36305@pulp3-source-fedora31.localhost.example.com | 0
63355@pulp3-source-fedora31.localhost.example.com | 0
36074@pulp3-source-fedora31.localhost.example.com | 0
37387@pulp3-source-fedora31.localhost.example.com | 0
64588@pulp3-source-fedora31.localhost.example.com | 0
123378@pulp3-source-fedora31.localhost.example.com | 0
115618@pulp3-source-fedora31.localhost.example.com | 0
119117@pulp3-source-fedora31.localhost.example.com | 0
71707@pulp3-source-fedora31.localhost.example.com | 0
86667@pulp3-source-fedora31.localhost.example.com | 0
31506@pulp3-source-fedora31.localhost.example.com | 0
116774@pulp3-source-fedora31.localhost.example.com | 0
128175@pulp3-source-fedora31.localhost.example.com | 0
141872@pulp3-source-fedora31.localhost.example.com | 0
34206@pulp3-source-fedora31.localhost.example.com | 0
71708@pulp3-source-fedora31.localhost.example.com | 0
31867@pulp3-source-fedora31.localhost.example.com | 0
82674@pulp3-source-fedora31.localhost.example.com | 0
32102@pulp3-source-fedora31.localhost.example.com | 0
90981@pulp3-source-fedora31.localhost.example.com | 0
119695@pulp3-source-fedora31.localhost.example.com | 0
66571@pulp3-source-fedora31.localhost.example.com | 0
57743@pulp3-source-fedora31.localhost.example.com | 0
32631@pulp3-source-fedora31.localhost.example.com | 0
38377@pulp3-source-fedora31.localhost.example.com | 0
143544@pulp3-source-fedora31.localhost.example.com | 0
120100@pulp3-source-fedora31.localhost.example.com | 0
64587@pulp3-source-fedora31.localhost.example.com | 0
65349@pulp3-source-fedora31.localhost.example.com | 0
122694@pulp3-source-fedora31.localhost.example.com | 0
121716@pulp3-source-fedora31.localhost.example.com | 0
144387@pulp3-source-fedora31.localhost.example.com | 0
115617@pulp3-source-fedora31.localhost.example.com | 0
32343@pulp3-source-fedora31.localhost.example.com | 0
50937@pulp3-source-fedora31.localhost.example.com | 0
33572@pulp3-source-fedora31.localhost.example.com | 0
50938@pulp3-source-fedora31.localhost.example.com | 0
123598@pulp3-source-fedora31.localhost.example.com | 0
32906@pulp3-source-fedora31.localhost.example.com | 0
143328@pulp3-source-fedora31.localhost.example.com | 0
123599@pulp3-source-fedora31.localhost.example.com | 0
129042@pulp3-source-fedora31.localhost.example.com | 0
117738@pulp3-source-fedora31.localhost.example.com | 0
50657@pulp3-source-fedora31.localhost.example.com | 0
33324@pulp3-source-fedora31.localhost.example.com | 0
52425@pulp3-source-fedora31.localhost.example.com | 0
123828@pulp3-source-fedora31.localhost.example.com | 0
117874@pulp3-source-fedora31.localhost.example.com | 0
143973@pulp3-source-fedora31.localhost.example.com | 0
121570@pulp3-source-fedora31.localhost.example.com | 0
59150@pulp3-source-fedora31.localhost.example.com | 0
118183@pulp3-source-fedora31.localhost.example.com | 0
143974@pulp3-source-fedora31.localhost.example.com | 0
83713@pulp3-source-fedora31.localhost.example.com | 0
117739@pulp3-source-fedora31.localhost.example.com | 0
117420@pulp3-source-fedora31.localhost.example.com | 0
73489@pulp3-source-fedora31.localhost.example.com | 0
33892@pulp3-source-fedora31.localhost.example.com | 0
141202@pulp3-source-fedora31.localhost.example.com | 0
85831@pulp3-source-fedora31.localhost.example.com | 0
58404@pulp3-source-fedora31.localhost.example.com | 0
141510@pulp3-source-fedora31.localhost.example.com | 0
117598@pulp3-source-fedora31.localhost.example.com | 0
32342@pulp3-source-fedora31.localhost.example.com | 0
33323@pulp3-source-fedora31.localhost.example.com | 0
82673@pulp3-source-fedora31.localhost.example.com | 0
123829@pulp3-source-fedora31.localhost.example.com | 0
144693@pulp3-source-fedora31.localhost.example.com | 0
115292@pulp3-source-fedora31.localhost.example.com | 0
144200@pulp3-source-fedora31.localhost.example.com | 0
143017@pulp3-source-fedora31.localhost.example.com | 0
144692@pulp3-source-fedora31.localhost.example.com | 0
142181@pulp3-source-fedora31.localhost.example.com | 0
66572@pulp3-source-fedora31.localhost.example.com | 0
116984@pulp3-source-fedora31.localhost.example.com | 0
129041@pulp3-source-fedora31.localhost.example.com | 0
144199@pulp3-source-fedora31.localhost.example.com | 0
31866@pulp3-source-fedora31.localhost.example.com | 0
120761@pulp3-source-fedora31.localhost.example.com | 0
60367@pulp3-source-fedora31.localhost.example.com | 0
117287@pulp3-source-fedora31.localhost.example.com | 0
49514@pulp3-source-fedora31.localhost.example.com | 0
73488@pulp3-source-fedora31.localhost.example.com | 0
117419@pulp3-source-fedora31.localhost.example.com | 0
85830@pulp3-source-fedora31.localhost.example.com | 0
120101@pulp3-source-fedora31.localhost.example.com | 0
65350@pulp3-source-fedora31.localhost.example.com | 0
123379@pulp3-source-fedora31.localhost.example.com | 0
43058@pulp3-source-fedora31.localhost.example.com | 0
33571@pulp3-source-fedora31.localhost.example.com | 0
43057@pulp3-source-fedora31.localhost.example.com | 0
121715@pulp3-source-fedora31.localhost.example.com | 0
59151@pulp3-source-fedora31.localhost.example.com | 0
52974@pulp3-source-fedora31.localhost.example.com | 0
83950@pulp3-source-fedora31.localhost.example.com | 0
63356@pulp3-source-fedora31.localhost.example.com | 0
58871@pulp3-source-fedora31.localhost.example.com | 0
217 queues, 0 jobs total

resource-manager (pulp3-source-fedora31.localhost.example.com 149281): idle resource-manager
1 workers, 217 queues

Updated: 2020-04-07 14:17:19.544967

History

#1 Updated by fao89 6 months ago

  • Triaged changed from No to Yes

Please register to edit this issue

Also available in: Atom PDF