Project

Profile

Help

Issue #3344

closed

Copy content from one repo to another fails

Added by balonik about 6 years ago. Updated almost 5 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Hi,
I am hitting an issue while trying to copy content between RPM repositories.
I call Pulp API and send multiple tasks to copy content from multiple (3+) RPM repositories to 1 destination.
API calls are done immediatelly. I don't wait for the previous copy tasks to finish.

I get the below error for random source repos each try. I can reproduce it everytime.
Sometimes only 1 task fail, sometimes more tasks fail.

Environment:
RHEL 7.4
Pulp 2.15

API call:
url: /pulp/api/v2/repositories/destination_repo/actions/associate/
json body: {'source_repo_id': source, 'override_config': {'recursive': True}, 'criteria': { 'type_ids' : ['rpm','drpm','package_group','package_category','package_environment']}}

Logs:
Feb 5 11:28:51 pulp-server pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.unit_association.associate_from_repo[6fb6d4ac-8472-4a41-9cb0-92eccb08444d]
Feb 5 11:31:05 pulp-server pulp: py.warnings:WARNING: [6fb6d4ac] (12260-91776) /usr/lib64/python2.7/site-packages/pymongo/topology.py:74: UserWarning: MongoClient opened before fork. Create MongoClient with connect=False, or create client after forking. See PyMongo's documentation for details: http://api.mongodb.org/python/current/faq.html#using-pymongo-with-multiprocessing>
Feb 5 11:31:05 pulp-server pulp: py.warnings:WARNING: [6fb6d4ac] (12260-91776) "MongoClient opened before fork. Create MongoClient "
Feb 5 11:31:05 pulp-server pulp: py.warnings:WARNING: [6fb6d4ac] (12260-91776)
Feb 5 11:31:06 pulp-server pulp: py.warnings:WARNING: [6fb6d4ac] (12260-91776) /usr/lib/python2.7/site-packages/mongoengine/document.py:324: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
Feb 5 11:31:06 pulp-server pulp: py.warnings:WARNING: [6fb6d4ac] (12260-91776) object_id = collection.save(doc, **write_concern)
Feb 5 11:31:06 pulp-server pulp: py.warnings:WARNING: [6fb6d4ac] (12260-91776)
Feb 5 11:31:13 pulp-server pulp: py.warnings:WARNING: [6fb6d4ac] (12260-91776) /usr/lib/python2.7/site-packages/mongoengine/document.py:367: DeprecationWarning: update is deprecated. Use replace_one, update_one or update_many instead.
Feb 5 11:31:13 pulp-server pulp: py.warnings:WARNING: [6fb6d4ac] (12260-91776) upsert=upsert, **write_concern)
Feb 5 11:31:13 pulp-server pulp: py.warnings:WARNING: [6fb6d4ac] (12260-91776)
Feb 5 11:31:16 pulp-server pulp: pulp.server.managers.repo.unit_association:ERROR: [6fb6d4ac] (12260-91776) Exception from importer [yum_importer] while importing units into repository [destination_repo]
Feb 5 11:31:16 pulp-server pulp: pulp.server.managers.repo.unit_association:ERROR: [6fb6d4ac] (12260-91776) Traceback (most recent call last):
Feb 5 11:31:16 pulp-server pulp: pulp.server.managers.repo.unit_association:ERROR: [6fb6d4ac] (12260-91776) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 273, in associate_from_repo
Feb 5 11:31:16 pulp-server pulp: pulp.server.managers.repo.unit_association:ERROR: [6fb6d4ac] (12260-91776) units=transfer_units)
Feb 5 11:31:16 pulp-server pulp: pulp.server.managers.repo.unit_association:ERROR: [6fb6d4ac] (12260-91776) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 58, in import_units
Feb 5 11:31:16 pulp-server pulp: pulp.server.managers.repo.unit_association:ERROR: [6fb6d4ac] (12260-91776) return associate.associate(source_repo, dest_repo, import_conduit, config, units)
Feb 5 11:31:16 pulp-server pulp: pulp.server.managers.repo.unit_association:ERROR: [6fb6d4ac] (12260-91776) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 88, in associate
Feb 5 11:31:16 pulp-server pulp: pulp.server.managers.repo.unit_association:ERROR: [6fb6d4ac] (12260-91776) associate(source_repo, dest_repo, import_conduit, config, group_units))
Feb 5 11:31:16 pulp-server pulp: pulp.server.managers.repo.unit_association:ERROR: [6fb6d4ac] (12260-91776) TypeError: unhashable type: 'list'
Feb 5 11:31:16 pulp-server pulp: pulp.server.async.tasks:INFO: [6fb6d4ac] Task failed : [6fb6d4ac-8472-4a41-9cb0-92eccb08444d]
Feb 5 11:31:16 pulp-server pulp: celery.worker.job:ERROR: (9966-91776) Task pulp.server.managers.repo.unit_association.associate_from_repo[6fb6d4ac-8472-4a41-9cb0-92eccb08444d] raised unexpected: TypeError("unhashable type: 'list'",)

Also available in: Atom PDF