Story #2843
closed
As an authenticated user, I can create an Artifact by uploading a file
100%
Description
For an API user to create an Artifact, Pulp 3 needs to have the following:
- updated Artifact[0] model without a 'content' foreign key nor the 'relative_path' field. A uniqueness constraint should be added on the sha256 field.
- viewset that can handle CRD operations for Artifacts. It most likely needs to use the FileUploadParser[1] and a custom Django Upload Handler[2].
- serializer for the viewset which will return all serialized fields of the Artifact model. The file field should be it's full path on disk.
- API endpoint at /api/v3/content/artifacts
- POST request to the /api/v3/content/artifacts/ endpoint creates an Artifact. The body of the request contains multipart form data with the following:
file - The file being uploaded
size - The size of the file in bytes.
md5 - The MD5 checksum of the file.
sha1 - The SHA-1 checksum of the file.
sha224 - The SHA-224 checksum of the file.
sha256 - The SHA-256 checksum of the file.
sha384 - The SHA-384 checksum of the file.
sha512 - The SHA-512 checksum of the file.
Before the model is saved, a SHA-256 checksum (digest) of the uploaded file is generated.
If a sha256 was specified in POST parameters, the generated hash is validated against the value specified as the POST parameter. If the values don't match a validation exception is raised.
If an Artifact with the same sha256 checksum already exists, a 400 response is returned to the user.
When the model is saved, the file is written to MEDIA_ROOT/content/units/digest[0:2]/digest[2:]
After a successful save, a serialized version of the Artifact is returned.
- GET request to the /api/v3/content/artifacts/<artifact id> returns serialized Artifact
- DELETE request to the /api/v3/content/artifacts/<artifact id>/ raises an exception if the Artifact is still associated with any ContentUnit. The exception should provide a list of content units still associated with the Artifact. If an exception is not raised, Artifact is removed from the database and the file is removed from disk.
[0] https://github.com/pulp/pulp/blob/3.0-dev/platform/pulpcore/app/models/content.py#L72
[1] http://www.django-rest-framework.org/api-guide/parsers/#fileuploadparser
[2] https://docs.djangoproject.com/en/1.11/topics/http/file-uploads/#upload-handlers
Updated by amacdona@redhat.com over 7 years ago
- Description updated (diff)
There is a small gotcha here. NamedModelViewSets come with an update method.
The normal DRF way to create a ViewSet without update would be to disinclude the update mixin. Since NamedModelViewSet uses the update mixin, it might be simpler to disable PUT/PATCH. https://stackoverflow.com/a/31450643/3154930
Updated by dkliban@redhat.com over 7 years ago
- Subject changed from As an authenticated user, I can request a file ID from the server to upload a file with to As an authenticated user, I can create an Artifact by uploading a file
- Description updated (diff)
Updated by dkliban@redhat.com over 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to dkliban@redhat.com
Updated by dkliban@redhat.com over 7 years ago
- Status changed from ASSIGNED to POST
https://github.com/pulp/pulp/pull/3080
I marked it as a work in progress for now. I just realized that the story lists GET parameters and my initial implementation uses POST. I will adjust it to use GET params.
Updated by dkliban@redhat.com over 7 years ago
- Tags Pulp 3 Plugin Writer Alpha added
Updated by mhrivnak over 7 years ago
When the story says "GET parameters", I assume that means "query parameters" since they're being used with a POST request.
I appears that django and DRF have out-of-the-box support for multipart form data. Why not just use that? I think it's much more common than adding query parameters to a POST request.
Updated by dkliban@redhat.com over 7 years ago
I implemented the story using multipart form data. I was planning on emailing pulp-dev with that info later today.
Updated by dkliban@redhat.com over 7 years ago
Talked with jortel earlier today and we determined that an Artifact requires a file to exist on disk. Content created with a deferred download policy would have ContentArtifacts that don't have the Artifact foreign key set.
Added by dkliban@redhat.com over 7 years ago
Added by dkliban@redhat.com over 7 years ago
Revision 884ae9d7 | View on GitHub
Problem: Artifacts can't be created via REST API
Solution: add a viewset for Artifacts
The REST API user can now POST to /api/v3/artifacts endpoint with a file, size, and any number of checksums. During the upload of the file, each possible checksum is calculated. If user provides size or any checksums, they are verified after the upload finishes. If one of the values does not match a ValidationError is raised.
The user can also list all the the Artifacts by performing a GET request to /api/v3/artifacts. An Artifact can be deleted by making a DELETE request to /api/v3/artifacts/.
The user can not update Artifacts via REST API after one is created.
The 'django.core.files.uploadhandler.MemoryFileUploadHandler' is disabled so all uploads are handled by 'pulpcore.app.upload.PulpFileUploadHandler' and are written to '/var/lib/pulp/tmp' as the file is being uploaded.
Updated by dkliban@redhat.com over 7 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp|884ae9d7cd20ab612e3a635302c9bd963e07fb47.
Updated by bmbouter about 7 years ago
- Tags deleted (
Pulp 3 Plugin Writer Alpha)
Cleaning up Redmine tags
Updated by bmbouter about 5 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
.
Problem: Artifacts can't be created via REST API
Solution: add a viewset for Artifacts
The REST API user can now POST to /api/v3/artifacts endpoint with a file, size, and any number of checksums. During the upload of the file, each possible checksum is calculated. If user provides size or any checksums, they are verified after the upload finishes. If one of the values does not match a ValidationError is raised.
The user can also list all the the Artifacts by performing a GET request to /api/v3/artifacts. An Artifact can be deleted by making a DELETE request to /api/v3/artifacts/.
The user can not update Artifacts via REST API after one is created.
The 'django.core.files.uploadhandler.MemoryFileUploadHandler' is disabled so all uploads are handled by 'pulpcore.app.upload.PulpFileUploadHandler' and are written to '/var/lib/pulp/tmp' as the file is being uploaded.
closes #2843 https://pulp.plan.io/issues/2843