Project

Profile

Help

Issue #5323

closed

syncing all collections results in 'too many open files'

Added by jsherril@redhat.com over 4 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Quarter:

Description

I run these steps:


http -a admin:password  POST $BASE_ADDR/pulp/api/v3/repositories/ name=foo
http -a admin:password POST $BASE_ADDR/pulp/api/v3/remotes/ansible/collection/     name='bar'     url='https://galaxy-dev.ansible.com/api/v2/collections'

export REPO_HREF=/pulp/api/v3/repositories/302f1759-c04a-43fd-be54-799e0a7fe86c/
export REMOTE_HREF=/pulp/api/v3/remotes/ansible/collection/290bf232-ec75-432c-9925-18f34e118f24/

http -a admin:password POST $BASE_ADDR$REMOTE_HREF'sync/' repository=$REPO_HREF 
http -a admin:password $BASE_ADDR//pulp/api/v3/tasks/b558fee1-c014-4396-a2a4-a976f24c9e7f/

and i get:


{
    "_created": "2019-08-21T16:08:01.444530Z", 
    "_href": "/pulp/api/v3/tasks/b558fee1-c014-4396-a2a4-a976f24c9e7f/", 
    "created_resources": [], 
    "error": {
        "code": null, 
        "description": "[Errno 24] Too many open files: '/var/lib/pulp/tmp/reserved-resource-worker-2@devel.localhost.example.com/b558fee1-c014-4396-a2a4-a976f24c9e7f'", 
        "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 822, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 605, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 611, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/local/lib/pulp/src/pulp-ansible/pulp_ansible/app/tasks/collections.py\", line 70, in sync\n    d_version.create()\n  File \"/usr/local/lib/pulp/src/pulpcore-plugin/pulpcore/plugin/stages/declarative_version.py\", line 169, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/tasking/services/storage.py\", line 169, in __exit__\n    self.delete()\n  File \"/usr/local/lib/pulp/src/pulpcore/pulpcore/tasking/services/storage.py\", line 78, in delete\n    shutil.rmtree(self.path)\n  File \"/usr/lib64/python3.6/shutil.py\", line 486, in rmtree\n    _rmtree_safe_fd(fd, path, onerror)\n  File \"/usr/lib64/python3.6/shutil.py\", line 408, in _rmtree_safe_fd\n    onerror(os.listdir, path, sys.exc_info())\n  File \"/usr/lib64/python3.6/shutil.py\", line 405, in _rmtree_safe_fd\n    names = os.listdir(topfd)\n"
    }, 
    "finished_at": "2019-08-21T16:08:02.226437Z", 
    "name": "pulp_ansible.app.tasks.collections.sync", 
    "non_fatal_errors": [], 
    "parent": null, 
    "progress_reports": [
        {
            "done": 0, 
            "message": "Parsing Galaxy Collections API", 
            "state": "failed", 
            "suffix": null, 
            "total": 14734
        }, 
        {
            "done": 0, 
            "message": "Parsing Collection Metadata", 
            "state": "failed", 
            "suffix": null, 
            "total": null
        }, 
        {
            "done": 0, 
            "message": "Downloading Artifacts", 
            "state": "canceled", 
            "suffix": null, 
            "total": null
        }, 
        {
            "done": 0, 
            "message": "Associating Content", 
            "state": "canceled", 
            "suffix": null, 
            "total": null
        }
    ], 
    "reserved_resources_record": [
        "/pulp/api/v3/remotes/ansible/collection/290bf232-ec75-432c-9925-18f34e118f24/", 
        "/pulp/api/v3/repositories/302f1759-c04a-43fd-be54-799e0a7fe86c/"
    ], 
    "spawned_tasks": [], 
    "started_at": "2019-08-21T16:08:01.550620Z", 
    "state": "failed", 
    "worker": "/pulp/api/v3/workers/75176664-9633-46f5-8989-456923ba333a/"
}

Also available in: Atom PDF