Crane

We are working on a Docker registry that uses the same metadata as Crane, and there is also some interest to make this registry a read-write registry. Some modifications to the existing metadata file structure would make this easier.

1) Use a directory structure to store metadata files that is similar to Docker registry. For instance:

rhel7 would be stored as:
    .../rjel7/pulpmd.json
rhel7_rsyslog would be stored as:
    .../rhel7/rsyslog/pulpmd.json

If this is done, we don't need to pre-load all metadata files on startup, and then watch the directory for changes.

2) Include tag and blob information in metadata.

-Example:
   .../rhel7/rsyslog/pulpmd.json
   .../rhel7/rsyslog/tags/latest.json: contains blob digests (manifest and layers)-

This would be really useful for the Docker registry implementation, because at startup, it downloads all manifests to create a blob index. If all blobs are available, we can construct the blob index from the metadata.
Moved to a separate story #2862.