Docker Support

What are the reasons behind for this request?

when you say' blob requests' do you mean layers listed on the image manifests? because there is no information of blobs on the manifest list.

It pertains to this call: '/pulp/api/v2/repositories/<repo-id>/search/units/'
with the criteria being

 'criteria': {                                                                        
    'type_ids': ['docker_manifest_list'],                   
    'filters': {                                                                     
        'unit': {}                                                                   
     }                                                                                
} 

Right now this returns information about the manifest list, including the ids of every manifest in the manifest list. If arch information was added for each manifest it would be possible to keep track of which arch each manifest is intended for through the API, which as I understand it isn't possible right now.

By blob requests I meant that same call but with 'type_ids': ['docker_blob'] - this currently returns basic information about the blob, we'd like the size of the blob added to that output as well.

Thank you for providing information @breailly. I understand where these changes needs to go, but i still don't understand the reason

what is the purpose of keeping the track of specific arches?
what is the reason of the having the size of each blob?

Specific arch info is needed in the case of arch specific problems arising with available content - being able to see which manifests / blobs correspond to which arches via the API would allow for better diagnosis of the issue.

The size information was simply a 'nice to have'; it could be dropped from this request.

I would like size information. In registry UI (foreman in my case) it would be useful to be able to see storage consumption. Users have asked for this size info to help them in choosing where to clean up disk space as well as to determine how much space is needed on a second registry (smart proxy).
Thanks!

on the manifest list model, besides the digest for each image manifest we will display its' arch and os( if those are available in the manifest_list.json file)

before:

"manifests" : [
            "digest" : "sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4",
            "digest" : "sha256:ff3a5c916c92643ff77519ffa742d3ec61b7f591b6b7504599d95a4a41134e28",

    ],

After:

"manifests" : [
        {
            "digest" : "sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4",
            "os" : "linux",
            "arch" : "ppc64le"
        },
        {
            "digest" : "sha256:ff3a5c916c92643ff77519ffa742d3ec61b7f591b6b7504599d95a4a41134e28"
            "os" : "linux",
            "arch : "amd64"
        }
    ],

blob model can have an additional field size, which will be populated with the result of os.stat()

 > db.units_docker_blob.find()[0]
{
    "_id" : "30df8e76-b8be-4183-be2e-6b64ceaa54a5",
    "pulp_user_metadata" : {

    },
    "_last_updated" : 1525444105,
    "_storage_path" : "/var/lib/pulp/content/units/docker_blob/86/0ac1a2efc27973c9cc967bd2c62fbab1355aa0574b8d8f7184935d80094195/sha256:daa414b19dcffa0ba7b80abb50a6b31156f1efd7bb1b8cdaeeb848da367afa38",
    "downloaded" : true,
    "digest" : "sha256:daa414b19dcffa0ba7b80abb50a6b31156f1efd7bb1b8cdaeeb848da367afa38",
    "size" : 1760,
    "_ns" : "units_docker_blob",
    "_content_type_id" : "docker_blob"

or manifest model can have that info, but it would be extracted from manifest.json metadata file and it would look like:

> db.units_docker_manifest.find()[0]
{
    "_id" : "6bd94c83-eb02-43d5-815a-27700d87e7e5",
    "pulp_user_metadata" : {

    },
    "_last_updated" : 1525444105,
    "_storage_path" : "/var/lib/pulp/content/units/docker_manifest/ba/7018fa3216c80b6c4e7f75580b7ff7f7d46c172f672ee2db006d9f6a0c757a/sha256:e9cec9aec697d8b9d450edd32860ecd363f2f3174c8338beb5f809422d182c63",
    "downloaded" : true,
    "digest" : "sha256:e9cec9aec697d8b9d450edd32860ecd363f2f3174c8338beb5f809422d182c63",
    "schema_version" : 2,
    "fs_layers" : [
        {
            "blob_sum" : "sha256:2a3ebcb7fbcc29bf40c4f62863008bb573acdea963454834d9483b3e5300c45d"
            "size": 123455
        }
    ],
    "_ns" : "units_docker_manifest",
    "_content_type_id" : "docker_manifest"
}

The problem about 2 approach is that shema1 does not have blob sizes on the manifest.json metadata filee at all, so we'd end up to have some manifests (like schema2) providing blob sizes from metadata and some(like schema1 not providing

We could implement both approaches where the size could be stored on the blob model after os.stat and where available( from manifest metadata json) on the manifest model as well.

based on some rmore eflection and discussion - we'll keep the size info of the blob on the blob model doing os.stat() before saving it into the db