Project

Profile

Help

Issue #9634

closed

Uploading large files fails on the "Too many open files" error

Added by lmjachky about 3 years ago. Updated almost 3 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 111
Quarter:

Description

Ticket moved to GitHub: "pulp/pulpcore/2087":https://github.com/pulp/pulpcore/issues/2087


Steps to reproduce the behaviour:

(pulp) [vagrant@pulp3-source-fedora34 pulp_ostree]$  truncate -s 3G gentoo_root.img

(pulp) [vagrant@pulp3-source-fedora34 pulp_ostree]$ pulp artifact upload --file gentoo_root.img
Uploading file gentoo_root.img
................[truncated the number of dots]...................Upload complete. Creating artifact.
Started background task /pulp/api/v3/tasks/2d3cf569-2d5c-449d-9ac6-7a53ab5f3a30/
........Error: Task /pulp/api/v3/tasks/2d3cf569-2d5c-449d-9ac6-7a53ab5f3a30/ failed: '[Errno 24] Too many open files: '/var/lib/pulp/media/upload/ec2d5c92-57e9-4191-ae95-ff4b9ec94353''

(pulp) [vagrant@pulp3-source-fedora34 pulp_ostree]$ ls -la gentoo_root.img
-rw-r--r--. 1 vagrant vagrant 3221225472 Dec 13 11:32 gentoo_root.img

Traceback:

Task 2d3cf569-2d5c-449d-9ac6-7a53ab5f3a30 failed ([Errno 24] Too many open files: '/var/lib/pulp/media/upload/ec2d5c92-57e9-4191-ae95-ff4b9ec94353')
pulp [3a3a9ea662994f609eea7d43ac8f30aa]: pulpcore.tasking.pulpcore_worker:INFO:   File "/home/vagrant/devel/pulpcore/pulpcore/tasking/pulpcore_worker.py", line 362, in _perform_task
    result = func(*args, **kwargs)
  File "/home/vagrant/devel/pulpcore/pulpcore/app/tasks/upload.py", line 31, in commit
    temp_file.write(chunk.file.read())
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/django/core/files/utils.py", line 42, in <lambda>
    read = property(lambda self: self.file.read)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/django/db/models/fields/files.py", line 45, in _get_file
    self._file = self.storage.open(self.name, 'rb')
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/django/core/files/storage.py", line 38, in open
    return self._open(name, mode)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/django/core/files/storage.py", line 238, in _open
    return File(open(self.path(name), mode))
Actions #1

Updated by jsherril@redhat.com about 3 years ago

  • Tags Katello added
Actions #2

Updated by dkliban@redhat.com about 3 years ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 111
Actions #4

Updated by pulpbot almost 3 years ago

  • Status changed from NEW to POST

Added by ipanova@redhat.com almost 3 years ago

Revision dcbf9357 | View on GitHub

Fixed "Too many open files" error during upload.

closes #9634

Actions #5

Updated by ipanova@redhat.com almost 3 years ago

  • Status changed from POST to MODIFIED
Actions #6

Updated by ipanova@redhat.com almost 3 years ago

  • Assignee set to ipanova@redhat.com
Actions #7

Updated by fao89 almost 3 years ago

  • Description updated (diff)
  • Status changed from MODIFIED to CLOSED - DUPLICATE

Also available in: Atom PDF