Actions
Issue #1821
closed
2 GB is not enough RAM to publish the RHEL 5 repository
Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Master
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:
Description
My dev box has 2 GB of RAM and runs out of RAM while publishing x86_64 RHEL 5:
Apr 06 15:20:11 dev pulp[9658]: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[ec4851bd-e21c-42b9-a2ca-4d64dc534f56]
Apr 06 15:20:11 dev pulp[9658]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[ee349f6b-670f-45d5-8703-479f6d77dceb]
Apr 06 15:20:11 dev pulp[9654]: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[2c206f4e-9020-4b47-a1c9-369ad51528fa] succeeded in 0.0422192100013s: None
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) Exception caught from plugin during publish for repo [rhel-5-server]
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) Traceback (most recent call last):
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1023, in _do_publish
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) publish_report = publish_repo(transfer_repo, conduit, call_config)
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 658, in wrap_f
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) return f(*args, **kwargs)
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) return self._publisher.process_lifecycle()
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 562, in process_lifecycle
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) super(PluginStep, self).process_lifecycle()
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 159, in process_lifecycle
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) step.process()
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 256, in process
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) self.finalize()
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/publish.py", line 424, in finalize
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) self.file_lists_context.checksum)
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 114, in add_metadata_file_metadata
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) content = file_handle.read()
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/usr/lib64/python2.7/gzip.py", line 261, in read
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) self._read(readsize)
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/usr/lib64/python2.7/gzip.py", line 320, in _read
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) self._add_read_data( uncompress )
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) File "/usr/lib64/python2.7/gzip.py", line 338, in _add_read_data
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) self.extrabuf = self.extrabuf[offset:] + data
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) MemoryError
Apr 06 15:24:57 dev pulp[9914]: pulp.server.async.tasks:INFO: Task failed : [ec4851bd-e21c-42b9-a2ca-4d64dc534f56]
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) Task pulp.server.managers.repo.publish.publish[ec4851bd-e21c-42b9-a2ca-4d64dc534f56] raised unexpected: MemoryError()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) Traceback (most recent call last):
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) R = retval = fun(*args, **kwargs)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 473, in __call__
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) return super(Task, self).__call__(*args, **kwargs)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 103, in __call__
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) return super(PulpTask, self).__call__(*args, **kwargs)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) return self.run(*args, **kwargs)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 971, in publish
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1023, in _do_publish
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) publish_report = publish_repo(transfer_repo, conduit, call_config)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 658, in wrap_f
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) return f(*args, **kwargs)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) return self._publisher.process_lifecycle()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 562, in process_lifecycle
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) super(PluginStep, self).process_lifecycle()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 159, in process_lifecycle
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) step.process()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 256, in process
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) self.finalize()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/publish.py", line 424, in finalize
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) self.file_lists_context.checksum)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 114, in add_metadata_file_metadata
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) content = file_handle.read()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/usr/lib64/python2.7/gzip.py", line 261, in read
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) self._read(readsize)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/usr/lib64/python2.7/gzip.py", line 320, in _read
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) self._add_read_data( uncompress )
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) File "/usr/lib64/python2.7/gzip.py", line 338, in _add_read_data
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) self.extrabuf = self.extrabuf[offset:] + data
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) MemoryError
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[ee349f6b-670f-45d5-8703-479f6d77dceb] succeeded in 0.0173126230002s: None
Related issues
Updated by rbarlow about 8 years ago
I restarted my dev box with 4 GB of RAM and was able to get past that part of the publish operation. I observed the celery process peak at around 700 MB of RAM during that publish, which does seem excessive since it uses under 100 MB during the rest of the publish.
The CLI output shows the failure just after the "publishing rpms" step hits 100%, so whatever it does immediately after writing each RPM is what I'd suspect.
Updated by mhrivnak about 8 years ago
- Triaged changed from No to Yes
This is likely a duplicate of 1716, but needs additional investigation.
Updated by mhrivnak about 8 years ago
- Related to Story #1716: As a user, I can have better memory performance on Publish by using SAX instead of etree for comps and updateinfo XML production added
Updated by bmbouter about 5 years ago
- Status changed from NEW to CLOSED - WONTFIX
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
Actions
.