Pulp

I believe this to be directly related to the 'pulp-admin rpm repo copy all' command that I'm running as mhrivnak had mentioned in IRC. Per his suggestion, I converted that command to several commands, each specifying a type of content to copy between the repos, and was able to complete the script with a max RAM usage of ~2GB. I believe that ~1GB of that usage (which is still left even after all of the tasks have finished) is directly related to celery. Below is the top output after running the script with mhrivnak's suggested changes.

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                                                                                  
 3262 apache    20   0 1240244 593036   3600 S   0.0 10.0 145:13.20 celery                                                                                                                                                                   
 2933 mongodb   20   0 12.544g 223620 115664 S   0.0  3.8  14:32.96 mongod                                                                                                                                                                   
 3261 apache    20   0  774876 165176   2104 S   0.0  2.8   3:19.60 celery

This issue should track the memory improvements of the copy all which it does well.

@sysadmind would you be interested in opening a separate issue to track the "celery does not free memory after work is completed issue?" Posting some pre and post task process-memory usage information would be great along with pulp-admin commands that you used in between the pre and post run. That way another user/dev could try to compare their results.

I can certainly appreciate having a separate issue for the memory freeing issue that I believe I am having. If I can get some solid details together, I'll do just that. Thanks

Hello I am affected as well on a CentOS 6.7 System with Pulp 2.8.2. In my case the worker took 8GB RAM and about 20GB Swap-Space. At the moment I an trying to run the Workaround, reporting later if this was working or not.

Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.