Project

Profile

Help

Issue #5087

closed

Creating artifact in pulp3 fails for big files

Added by jcabrera over 5 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 56
Quarter:

Description

Used Version

pulp_source_dir: "git+https://github.com/pulp/pulpcore.git@3.0.0rc3"
pulp_plugin_source_dir:
"git+https://github.com/pulp/pulpcore-plugin.git@0.1.0rc3"
pulp_install_plugins:
pulp-rpm:
app_label: "rpm"
source_dir: "git+https://github.com/pulp/pulp_rpm.git@3.0.0b4"

Steps to reproduce

I created different size files

dd if=/dev/zero of=500m.bin bs=256M count=2
dd if=/dev/zero of=750m.bin bs=256M count=3
dd if=/dev/zero of=1g.bin bs=256M count=4
dd if=/dev/zero of=1.5g.bin bs=256M count=6
dd if=/dev/zero of=5.5g.bin bs=256M count=22

Using script test-chunk.sh join to this ticket I do:

./test-chunk.sh 500m.bin # OK
./test-chunk.sh 750m.bin  # OK
./test-chunk.sh 1g.bin  # OK
./test-chunk.sh 1.5g.bin # Fails with error

Creating artifact

http: error: Request timed out (30s).

Changing the script and adding bigger timeout

http --timeout=120 POST $PORT/pulp/api/v3/artifacts/ upload=$UPLOAD

I get the error:

Creating artifact

http: error: ConnectionError: ('Connection aborted.', BadStatusLine("''",)) while doing POST request to URL: http://dev-pulp-server.ptci.dev:24817/pulp/api/v3/artifacts/

Trynig the bigest file 5.5g.bin I get the error:

./test-chunk.sh 5.5g.bin
...
...
Creating artifact
HTTP/1.1 500 Internal Server Error
Connection: close
Content-Length: 27
Content-Type: text/html
Date: Fri, 05 Jul 2019 09:59:10 GMT
Server: gunicorn/19.9.0
Vary: Cookie
X-Frame-Options: SAMEORIGIN

<h1>Server Error (500)</h1>

In the server the upload files seems OK

[root@dev-pulp-server upload]# pwd
/var/lib/pulp/upload
[root@dev-pulp-server upload]# ls -lhs
total 9.5G
1.5G -rw-r--r--. 1 pulp pulp 1.5G Jul  5 11:44 3259c600-29ad-4629-a7f4-fa56add68b7d
5.5G -rw-r--r--. 1 pulp pulp 5.5G Jul  5 11:58 5bbe89e6-2f86-4738-a196-b3ed4c88d8de
1.0G -rw-r--r--. 1 pulp pulp 1.0G Jul  5 11:35 66d19833-0eea-4bfb-af8d-54bb6840d9cb
1.5G -rw-r--r--. 1 pulp pulp 1.5G Jul  5 11:38 90af4a0d-6f1a-4f14-9b47-67f7327fe067
[root@dev-pulp-server upload]# sha256sum 5bbe89e6-2f86-4738-a196-b3ed4c88d8de
4da89f41df88aa946bee824842471f89ac378b337dcf5cef2dafa53bb1e82cc6  5bbe89e6-2f86-4738-a196-b3ed4c88d8de

In the client

[vagrant@dev-pulp-client scripts]$ sha256sum 5.5g.bin
4da89f41df88aa946bee824842471f89ac378b337dcf5cef2dafa53bb1e82cc6  5.5g.bin

Files

test-chunk.sh (1017 Bytes) test-chunk.sh jcabrera, 07/05/2019 11:48 AM

Related issues

Related to Pulp - Issue #4998: Artifact size is limited to 2 GBCLOSED - CURRENTRELEASEdaviddavisActions

Also available in: Atom PDF