Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #1821

closed

2 GB is not enough RAM to publish the RHEL 5 repository

Added by rbarlow about 8 years ago. Updated almost 3 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Master
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

My dev box has 2 GB of RAM and runs out of RAM while publishing x86_64 RHEL 5:

Apr 06 15:20:11 dev pulp[9658]: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[ec4851bd-e21c-42b9-a2ca-4d64dc534f56]
Apr 06 15:20:11 dev pulp[9658]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[ee349f6b-670f-45d5-8703-479f6d77dceb]
Apr 06 15:20:11 dev pulp[9654]: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[2c206f4e-9020-4b47-a1c9-369ad51528fa] succeeded in 0.0422192100013s: None
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) Exception caught from plugin during publish for repo [rhel-5-server]
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) Traceback (most recent call last):
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1023, in _do_publish
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 658, in wrap_f
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     return f(*args, **kwargs)
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     return self._publisher.process_lifecycle()
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 562, in process_lifecycle
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     super(PluginStep, self).process_lifecycle()
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 159, in process_lifecycle
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     step.process()
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 256, in process
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     self.finalize()
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/publish.py", line 424, in finalize
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     self.file_lists_context.checksum)
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 114, in add_metadata_file_metadata
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     content = file_handle.read()
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/usr/lib64/python2.7/gzip.py", line 261, in read
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     self._read(readsize)
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/usr/lib64/python2.7/gzip.py", line 320, in _read
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     self._add_read_data( uncompress )
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)   File "/usr/lib64/python2.7/gzip.py", line 338, in _add_read_data
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760)     self.extrabuf = self.extrabuf[offset:] + data
Apr 06 15:24:57 dev pulp[9914]: pulp.server.controllers.repository:ERROR: (9914-85760) MemoryError
Apr 06 15:24:57 dev pulp[9914]: pulp.server.async.tasks:INFO: Task failed : [ec4851bd-e21c-42b9-a2ca-4d64dc534f56]
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) Task pulp.server.managers.repo.publish.publish[ec4851bd-e21c-42b9-a2ca-4d64dc534f56] raised unexpected: MemoryError()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) Traceback (most recent call last):
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     R = retval = fun(*args, **kwargs)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 473, in __call__
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     return super(Task, self).__call__(*args, **kwargs)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 103, in __call__
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     return super(PulpTask, self).__call__(*args, **kwargs)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     return self.run(*args, **kwargs)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 971, in publish
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1023, in _do_publish
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 658, in wrap_f
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     return f(*args, **kwargs)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     return self._publisher.process_lifecycle()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 562, in process_lifecycle
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     super(PluginStep, self).process_lifecycle()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 159, in process_lifecycle
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     step.process()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 256, in process
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     self.finalize()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/publish.py", line 424, in finalize
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     self.file_lists_context.checksum)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 114, in add_metadata_file_metadata
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     content = file_handle.read()
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/usr/lib64/python2.7/gzip.py", line 261, in read
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     self._read(readsize)
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/usr/lib64/python2.7/gzip.py", line 320, in _read
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     self._add_read_data( uncompress )
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)   File "/usr/lib64/python2.7/gzip.py", line 338, in _add_read_data
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760)     self.extrabuf = self.extrabuf[offset:] + data
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:ERROR: (9658-85760) MemoryError
Apr 06 15:24:58 dev pulp[9658]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[ee349f6b-670f-45d5-8703-479f6d77dceb] succeeded in 0.0173126230002s: None

Related issues

Related to RPM Support - Story #1716: As a user, I can have better memory performance on Publish by using SAX instead of etree for comps and updateinfo XML productionCLOSED - CURRENTRELEASEttereshc

Actions
Actions
Send by e-mail Copy link

Also available in: Atom PDF