Docker Support

I'm unable to perform a docker repo sync with an Integrated Openshift Registry.
I modified the "docker_importer" config to contain "basic_auth_username" and "basic_auth_password" attributes as suggested in https://pulp.plan.io/issues/1975.

Authentication seems to work properly as some of the task steps are successful.

However, it seems to fail during the "Downloading remote files" stage.

$ pulp-admin docker repo sync run --repo-id devops-osbs-openshift
+----------------------------------------------------------------------+
            Synchronizing Repository [devops-osbs-openshift]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading manifests
[\]
... completed

Copying units already in pulp
[-]
... completed

Copying units already in pulp
[-]
... completed

Downloading remote files
[                                                  ] 0%
0 of 4 items

Task Failed

Content import of
/var/cache/pulp/reserved_resource_worker-2@brew-pulp-docker01.web.qa.ext.phx1.re
dhat.com/fb14c674-0d47-4948-9662-3000bcb10f2f/sha256:478be19d61cd9d6871c440ac574
f7ce9b966c35d51a392dabf99a8dc01a2e924 failed - must be an existing file.

We've identified the file mentioned above is an image layer.

Debugging information indicates that download failed but file was still attempted to be copied, which seems odd. All error messages/tracebacks available are from this copy step which does not appear to be the root cause of the issue considering download failed.

{
"num_success": 0,
"description": "Downloading remote files",
"step_type": "sync_step_download",
"items_total": 3,
"state": "FAILED",
"error_details": [],
"details": "",
"num_failures": 3,
"step_id": "88062216-6722-4522-aebb-f029737450cf",
"num_processed": 3
},
{
"num_success": 0,
"description": "Saving Manifests and Blobs",
"step_type": "sync_step_save",
"items_total": 1,
"state": "FAILED",
"error_details": [
{
    "traceback": "  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 232, in process\n    self._process_block(item=item)\n\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.
py\", line 291, in _process_block\n    self.process_main(item=item)\n\n  File \"/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py\", line 282, in process_main\n    item.save_and_import_content(os.path.join(self.get_wo
rking_dir(), item.digest))\n\n  File \"/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py\", line 801, in save_and_import_content\n    self.safe_import_content(path, location)\n\n  File \"/usr/lib/python2.7/site-packages/pu
lp/server/db/model/__init__.py\", line 814, in safe_import_content\n    self.import_content(path, location)\n\n  File \"/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py\", line 786, in import_content\n    raise exceptions
.PulpCodedException(error_code=error_codes.PLP0037, path=path)\n",
            "error": "Content import of /var/cache/pulp/reserved_resource_worker-2@brew-pulp-docker01.web.qa.ext.phx1.redhat.com/121483e4-7453-454d-ba83-8bc83dba115f/sha256:18c96d12bd777da143dbdac5daaaaa5abb8b77d5697c43611b5be58a5097e220
failed - must be an existing file."
}
],
"details": "",
"num_failures": 1,
"step_id": "755c0fa6-6e9f-48ed-a5fd-212a7ea67028",
"num_processed": 1
},

Importer config being used:

{
"_href": "/pulp/api/v2/repositories/devops-osbs-openshift/importers/docker_importer/",
"_id": {
    "$oid": "57582fed11573f410613dba6"
},
"_ns": "repo_importers",
"config": {
    "basic_auth_password": "*****",
    "basic_auth_username": "lucarval",
    "enable_v1": false,
    "enable_v2": true,
    "feed": "https://my-registry-url",
    "upstream_name": "osbs-qa01/helloworld"
},
"id": "docker_importer",
"importer_type_id": "docker_importer",
"last_sync": null,
"repo_id": "devops-osbs-openshift",
"scratchpad": null
}