Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #4328

closed

fsLayers error during sync

Added by EppO almost 6 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
ipanova@redhat.com
Sprint/Milestone:
Pulp - 2.19.0
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Docker:
Platform Release:
2.19.0
Target Release - Docker:
OS:
CentOS 7
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 48
Quarter:

Description

Hello,

trying to mirror gcr.io/google-containers/etcd, I encountered that issue during a sync:

2019-01-15 12:54:13,606 - INFO - Response body :
 {
  "exception": null,
  "task_type": "pulp.server.managers.repo.sync.sync",
  "_href": "/pulp/api/v2/tasks/291a8d9d-4b84-4aeb-af0f-2772e8ade987/",
  "task_id": "291a8d9d-4b84-4aeb-af0f-2772e8ade987",
  "tags": [
    "pulp:repository:etcd-gcr2",
    "pulp:action:sync"
  ],
  "finish_time": "2019-01-15T20:54:12Z",
  "_ns": "task_status",
  "start_time": "2019-01-15T20:54:09Z",
  "traceback": "Traceback (most recent call last):\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 367, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 529, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 107, in __call__\n    return super(PulpTask, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 622, in __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 770, in sync\n    sync_report = sync_repo(transfer_repo, conduit, call_config)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 737, in wrap_f\n    return f(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/importer.py\", line 85, in sync_repo\n    return self.sync_step.process_lifecycle()\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 572, in process_lifecycle\n    super(PluginStep, self).process_lifecycle()\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 163, in process_lifecycle\n    step.process()\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 256, in process\n    self._process_block()\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 303, in _process_block\n    self.process_main()\n  File \"/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py\", line 237, in process_main\n    self._process_manifest_list(manifest, digest, available_blobs, tag)\n  File \"/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py\", line 283, in _process_manifest_list\n    self._process_manifest(manifest, digest, available_blobs, tag=tag)\n  File \"/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py\", line 313, in _process_manifest\n    manifest = models.Manifest.from_json(manifest, digest)\n  File \"/usr/lib/python2.7/site-packages/pulp_docker/plugins/models.py\", line 219, in from_json\n    fs_layers = [FSLayer(blob_sum=layer['blobSum']) for layer in manifest['fsLayers']]\nKeyError: 'fsLayers'\n",
  "spawned_tasks": [],
  "progress_report": {
    "docker_importer": [
      {
        "num_success": 5,
        "description": "Downloading manifests",
        "step_type": "sync_step_metadata",
        "items_total": 1,
        "state": "FAILED",
        "error_details": [
          {
            "traceback": "  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 256, in process\n    self._process_block()\n\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 303, in _process_block\n    self.process_main()\n\n  File \"/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py\", line 237, in process_main\n    self._process_manifest_list(manifest, digest, available_blobs, tag)\n\n  File \"/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py\", line 283, in _process_manifest_list\n    self._process_manifest(manifest, digest, available_blobs, tag=tag)\n\n  File \"/usr/lib/python2.7/site-packages/pulp_docker/plugins/importers/sync.py\", line 313, in _process_manifest\n    manifest = models.Manifest.from_json(manifest, digest)\n\n  File \"/usr/lib/python2.7/site-packages/pulp_docker/plugins/models.py\", line 219, in from_json\n    fs_layers = [FSLayer(blob_sum=layer['blobSum']) for layer in manifest['fsLayers']]\n",
            "error": "'fsLayers'"
          }
        ],
        "details": "",
        "num_failures": 1,
        "step_id": "77747e24-bc58-46ed-b5a2-60c21ba5c5ad",
        "num_processed": 6
      },
      {
        "num_success": 0,
        "description": "Copying units already in pulp",
        "step_type": "get_local",
        "items_total": 1,
        "state": "NOT_STARTED",
        "error_details": [],
        "details": "",
        "num_failures": 0,
        "step_id": "695588b1-9004-4bce-ac6a-4f20bb1209c8",
        "num_processed": 0
      },
      {
        "num_success": 0,
        "description": "Copying units already in pulp",
        "step_type": "get_local",
        "items_total": 1,
        "state": "NOT_STARTED",
        "error_details": [],
        "details": "",
        "num_failures": 0,
        "step_id": "d211ca5e-bcf5-49df-a390-0e0016b4246f",
        "num_processed": 0
      },
      {
        "num_success": 0,
        "description": "Downloading remote files",
        "step_type": "sync_step_download",
        "items_total": 1,
        "state": "NOT_STARTED",
        "error_details": [],
        "details": "",
        "num_failures": 0,
        "step_id": "d627e410-d608-4a89-bec1-9709a1172821",
        "num_processed": 0
      },
      {
        "num_success": 0,
        "description": "Saving Manifests and Blobs",
        "step_type": "sync_step_save",
        "items_total": 1,
        "state": "NOT_STARTED",
        "error_details": [],
        "details": "",
        "num_failures": 0,
        "step_id": "23d120bd-01d3-40b8-ac6c-2cd434aed12f",
        "num_processed": 0
      },
      {
        "num_success": 0,
        "description": "Saving Tags",
        "step_type": "sync_step_save",
        "items_total": 1,
        "state": "NOT_STARTED",
        "error_details": [],
        "details": "",
        "num_failures": 0,
        "step_id": "97e64823-ef75-4406-ba71-fe7b8a081e61",
        "num_processed": 0
      }
    ]
  },

Changing the source repository to quay.io/etcd works though.
I tried deleting the repo in pulp and re-create it or whitelisting a single tag, no luck.

Here are the repository details:

Importers:
  Config:
    Feed:          https://gcr.io
    Tags:          3.2.24
    Upstream Name: google-containers/etcd
  Id:                   docker_importer
  Importer Type Id:     docker_importer
  Last Override Config:
  Last Sync:            2019-01-15T20:54:12Z
  Last Updated:         2019-01-15T20:53:58Z
  Repo Id:              etcd-gcr2
  Scratchpad:           None
Distributors:
  Auto Publish:         True
  Config:
    Repo-registry-id: google-containers/etcd2
  Distributor Type Id:  docker_distributor_web
  Id:                   docker_web_distributor_name_cli
  Last Override Config:
  Last Publish:         None
  Last Updated:         2019-01-15T20:53:58Z
  Repo Id:              etcd-gcr2
  Scratchpad:
  Auto Publish:         False
  Config:
    Repo-registry-id: google-containers/etcd2
  Distributor Type Id:  docker_distributor_export
  Id:                   docker_export_distributor_name_cli
  Last Override Config:
  Last Publish:         None
  Last Updated:         2019-01-15T20:53:58Z
  Repo Id:              etcd-gcr2
Actions
Copy link
 #1

Updated by ipanova@redhat.com almost 6 years ago

can you please share the version of pulp and pulp_docker you have installed?thank you.

if it is older than 2.13 then i advise you to upgrade https://pulp.plan.io/issues/2099
reason why quay works, because it supports schema1

in your particular case, the tag you query goes not support schema1

$ curl -L -H "Accept: application/vnd.docker.distribution.manifest.v1+json"  -X GET 'https://gcr.io/v2/google-containers/etcd/manifests/3.2.24'
{"errors":[{"code":"MANIFEST_UNKNOWN","message":"Manifest with tag '3.2.24' has media type 'application/vnd.docker.distribution.manifest.list.v2+json', but client accepts 'application/vnd.docker.distribution.manifest.v1+json'."}]}
Actions
Copy link
 #2

Updated by ttereshc almost 6 years ago

  • Project changed from Pulp to Docker Support
  • Category changed from 15 to 17
Actions
Copy link
 #3

Updated by ttereshc almost 6 years ago

  • Category deleted (17)
Actions
Copy link
 #4

Updated by ipanova@redhat.com almost 6 years ago

  • Status changed from NEW to CLOSED - NOTABUG

please re-open in case c#1 does not cover your case.

Actions
Copy link
 #5

Updated by EppO almost 6 years ago

Here is my pulp stack:

  • pulp-admin-client-2.18.0-2.el7.noarch
  • pulp-docker-admin-extensions-3.2.2-1.el7.noarch
  • pulp-docker-plugins-3.2.2-1.el7.noarch
  • pulp-nodes-admin-extensions-2.18.0-2.el7.noarch
  • pulp-nodes-common-2.18.0-2.el7.noarch
  • pulp-ostree-admin-extensions-1.3.1-1.el7.noarch
  • pulp-ostree-plugins-1.3.1-1.el7.noarch
  • pulp-puppet-admin-extensions-2.18.0-1.el7.noarch
  • pulp-puppet-plugins-2.18.0-1.el7.noarch
  • pulp-puppet-tools-2.18.0-1.el7.noarch
  • pulp-python-admin-extensions-2.0.3-1.el7.noarch
  • pulp-python-plugins-2.0.3-1.el7.noarch
  • pulp-rpm-admin-extensions-2.18.0-1.el7.noarch
  • pulp-rpm-plugins-2.18.0-1.el7.noarch
  • pulp-selinux-2.18.0-2.el7.noarch
  • pulp-server-2.18.0-2.el7.noarch
  • python-pulp-bindings-2.18.0-2.el7.noarch
  • python-pulp-client-lib-2.18.0-2.el7.noarch
  • python-pulp-common-2.18.0-2.el7.noarch
  • python-pulp-docker-common-3.2.2-1.el7.noarch
  • python-pulp-oid_validation-2.18.0-2.el7.noarch
  • python-pulp-ostree-common-1.3.1-1.el7.noarch
  • python-pulp-puppet-common-2.18.0-1.el7.noarch
  • python-pulp-python-common-2.0.3-1.el7.noarch
  • python-pulp-repoauth-2.18.0-2.el7.noarch
  • python-pulp-rpm-common-2.18.0-1.el7.noarch
  • python-pulp-streamer-2.18.0-2.el7.noarch

I find it surprising that gcr.io doesn't support schema v2. It works for other images on gcr.io

Actions
Copy link
 #6

Updated by EppO almost 6 years ago

Can't reopen myself the issue, but it's still happening and I have latest release of pulp 2.x

Actions
Copy link
 #7

Updated by ipanova@redhat.com almost 6 years ago

re-opened on request. We need to test and try to reproduce

@EppO
gcr as a registry does support schema1. It is some of the images which are stored only in schema2 format and afiak gcr does not do conversion on the fly of schema2-->schema1

Actions
Copy link
 #8

Updated by ipanova@redhat.com almost 6 years ago

  • Status changed from CLOSED - NOTABUG to NEW
 pulp-admin docker repo create --repo-id=gcr --feed=https://gcr.io --upstream-name=google-containers/etcd
pulp-admin -vv docker  repo sync run --repo-id gcr

$ pulp-admin -vv tasks details --task-id 35522fa6-955f-4224-90ac-f7a2c976464a
Warning: path should have mode 0700 because it may contain sensitive information: /home/ipanova/.pulp/

+----------------------------------------------------------------------+
                              Task Details
+----------------------------------------------------------------------+

2019-02-08 15:40:02,767 - DEBUG - sending GET request to /pulp/api/v2/tasks/35522fa6-955f-4224-90ac-f7a2c976464a/
2019-02-08 15:40:02,862 - INFO - GET request to /pulp/api/v2/tasks/35522fa6-955f-4224-90ac-f7a2c976464a/ with parameters None
2019-02-08 15:40:02,862 - INFO - Response status : 200 

2019-02-08 15:40:02,862 - INFO - Response body :
 {
  "exception": null, 
  "task_type": "pulp.server.managers.repo.sync.sync", 
  "_href": "/pulp/api/v2/tasks/35522fa6-955f-4224-90ac-f7a2c976464a/", 
  "task_id": "35522fa6-955f-4224-90ac-f7a2c976464a", 
  "tags": [
    "pulp:repository:gcr", 
    "pulp:action:sync"
  ], 
  "finish_time": "2019-02-08T14:37:03Z", 
  "_ns": "task_status", 
  "start_time": "2019-02-08T14:36:16Z", 
  "traceback": "Traceback (most recent call last):\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 367, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/home/ipanova/pulp_development/pulp/server/pulp/server/async/tasks.py\", line 529, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n  File \"/home/ipanova/pulp_development/pulp/server/pulp/server/async/tasks.py\", line 107, in __call__\n    return super(PulpTask, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 622, in __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/home/ipanova/pulp_development/pulp/server/pulp/server/controllers/repository.py\", line 770, in sync\n    sync_report = sync_repo(transfer_repo, conduit, call_config)\n  File \"/home/ipanova/pulp_development/pulp/server/pulp/server/async/tasks.py\", line 737, in wrap_f\n    return f(*args, **kwargs)\n  File \"/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker/plugins/importers/importer.py\", line 85, in sync_repo\n    return self.sync_step.process_lifecycle()\n  File \"/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/publish_step.py\", line 572, in process_lifecycle\n    super(PluginStep, self).process_lifecycle()\n  File \"/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/publish_step.py\", line 163, in process_lifecycle\n    step.process()\n  File \"/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/publish_step.py\", line 256, in process\n    self._process_block()\n  File \"/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/publish_step.py\", line 303, in _process_block\n    self.process_main()\n  File \"/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker/plugins/importers/sync.py\", line 237, in process_main\n    self._process_manifest_list(manifest, digest, available_blobs, tag)\n  File \"/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker/plugins/importers/sync.py\", line 283, in _process_manifest_list\n    self._process_manifest(manifest, digest, available_blobs, tag=tag)\n  File \"/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker/plugins/importers/sync.py\", line 313, in _process_manifest\n    manifest = models.Manifest.from_json(manifest, digest)\n  File \"/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker/plugins/models.py\", line 219, in from_json\n    fs_layers = [FSLayer(blob_sum=layer['blobSum']) for layer in manifest['fsLayers']]\nKeyError: 'fsLayers'\n", 
  "spawned_tasks": [], 
  "progress_report": {
    "docker_importer": [
      {
        "num_success": 100, 
        "description": "Downloading manifests", 
        "step_type": "sync_step_metadata", 
        "items_total": 71, 
        "state": "FAILED", 
        "error_details": [
          {
            "traceback": "  File \"/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/publish_step.py\", line 256, in process\n    self._process_block()\n\n  File \"/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/publish_step.py\", line 303, in _process_block\n    self.process_main()\n\n  File \"/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker/plugins/importers/sync.py\", line 237, in process_main\n    self._process_manifest_list(manifest, digest, available_blobs, tag)\n\n  File \"/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker/plugins/importers/sync.py\", line 283, in _process_manifest_list\n    self._process_manifest(manifest, digest, available_blobs, tag=tag)\n\n  File \"/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker/plugins/importers/sync.py\", line 313, in _process_manifest\n    manifest = models.Manifest.from_json(manifest, digest)\n\n  File \"/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker/plugins/models.py\", line 219, in from_json\n    fs_layers = [FSLayer(blob_sum=layer['blobSum']) for layer in manifest['fsLayers']]\n", 
            "error": "'fsLayers'"
          }
        ], 
        "details": "", 
        "num_failures": 1, 
        "step_id": "3358cefd-f518-44ac-b9e3-c62504a36bc4", 
        "num_processed": 101
      }, 
      {
        "num_success": 0, 
        "description": "Copying units already in pulp", 
        "step_type": "get_local", 
        "items_total": 1, 
        "state": "NOT_STARTED", 
        "error_details": [], 
        "details": "", 
        "num_failures": 0, 
        "step_id": "174fc073-2fe5-43d1-ab17-8aacc0edc617", 
        "num_processed": 0
      }, 
      {
        "num_success": 0, 
        "description": "Copying units already in pulp", 
        "step_type": "get_local", 
        "items_total": 1, 
        "state": "NOT_STARTED", 
        "error_details": [], 
        "details": "", 
        "num_failures": 0, 
        "step_id": "94ea98c6-e665-4b37-8a3c-e88a6cc9ae07", 
        "num_processed": 0
      }, 
      {
        "num_success": 0, 
        "description": "Downloading remote files", 
        "step_type": "sync_step_download", 
        "items_total": 1, 
        "state": "NOT_STARTED", 
        "error_details": [], 
        "details": "", 
        "num_failures": 0, 
        "step_id": "796fff61-a4b4-492c-9dca-e899f6265251", 
        "num_processed": 0
      }, 
      {
        "num_success": 0, 
        "description": "Saving Manifests and Blobs", 
        "step_type": "sync_step_save", 
        "items_total": 1, 
        "state": "NOT_STARTED", 
        "error_details": [], 
        "details": "", 
        "num_failures": 0, 
        "step_id": "b0ae0015-53f9-4315-8302-cc26ac6a93fd", 
        "num_processed": 0
      }, 
      {
        "num_success": 0, 
        "description": "Saving Tags", 
        "step_type": "sync_step_save", 
        "items_total": 1, 
        "state": "NOT_STARTED", 
        "error_details": [], 
        "details": "", 
        "num_failures": 0, 
        "step_id": "419fcd20-322e-4af6-a518-6199b224a0e5", 
        "num_processed": 0
      }
    ]
  }, 
  "queue": "reserved_resource_worker-0@ina.usersys.redhat.com.dq2", 
  "state": "error", 
  "worker_name": "reserved_resource_worker-0@ina.usersys.redhat.com", 
  "result": null, 
  "error": {
    "code": "PLP0000", 
    "data": {}, 
    "description": "'fsLayers'", 
    "sub_errors": []
  }, 
  "_id": {
    "$oid": "5c5d93e00c8a45468e11f8eb"
  }, 
  "id": "5c5d93e00c8a45468e11f8eb"
}

Operations:       sync
Resources:        gcr (repository)
State:            Failed
Start Time:       2019-02-08T14:36:16Z
Finish Time:      2019-02-08T14:37:03Z
Result:           N/A
Task Id:          35522fa6-955f-4224-90ac-f7a2c976464a
Worker Name:      reserved_resource_worker-0@ina.usersys.redhat.com
Progress Report:  
  Docker Importer: 
    Description:   Downloading manifests
    Details:       
    Error Details: 
      Error:     'fsLayers'
      Traceback:   File
                 "/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/p
                 ublish_step.py", line 256, in process     self._process_block()
                 File
                 "/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/p
                 ublish_step.py", line 303, in _process_block    
                 self.process_main()    File
                 "/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker
                 /plugins/importers/sync.py", line 237, in process_main    
                 self._process_manifest_list(manifest, digest, available_blobs,
                 tag)    File
                 "/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker
                 /plugins/importers/sync.py", line 283, in
                 _process_manifest_list     self._process_manifest(manifest,
                 digest, available_blobs, tag=tag)    File
                 "/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker
                 /plugins/importers/sync.py", line 313, in _process_manifest    
                 manifest = models.Manifest.from_json(manifest, digest)    File
                 "/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docker
                 /plugins/models.py", line 219, in from_json     fs_layers =
                 [FSLayer(blob_sum=layer['blobSum']) for layer in
                 manifest['fsLayers']] 
    Items Total:   71
    Num Failures:  1
    Num Processed: 101
    Num Success:   100
    State:         FAILED
    Step Id:       3358cefd-f518-44ac-b9e3-c62504a36bc4
    Step Type:     sync_step_metadata
    Description:   Copying units already in pulp
    Details:       
    Error Details: 
    Items Total:   1
    Num Failures:  0
    Num Processed: 0
    Num Success:   0
    State:         NOT_STARTED
    Step Id:       174fc073-2fe5-43d1-ab17-8aacc0edc617
    Step Type:     get_local
    Description:   Copying units already in pulp
    Details:       
    Error Details: 
    Items Total:   1
    Num Failures:  0
    Num Processed: 0
    Num Success:   0
    State:         NOT_STARTED
    Step Id:       94ea98c6-e665-4b37-8a3c-e88a6cc9ae07
    Step Type:     get_local
    Description:   Downloading remote files
    Details:       
    Error Details: 
    Items Total:   1
    Num Failures:  0
    Num Processed: 0
    Num Success:   0
    State:         NOT_STARTED
    Step Id:       796fff61-a4b4-492c-9dca-e899f6265251
    Step Type:     sync_step_download
    Description:   Saving Manifests and Blobs
    Details:       
    Error Details: 
    Items Total:   1
    Num Failures:  0
    Num Processed: 0
    Num Success:   0
    State:         NOT_STARTED
    Step Id:       b0ae0015-53f9-4315-8302-cc26ac6a93fd
    Step Type:     sync_step_save
    Description:   Saving Tags
    Details:       
    Error Details: 
    Items Total:   1
    Num Failures:  0
    Num Processed: 0
    Num Success:   0
    State:         NOT_STARTED
    Step Id:       419fcd20-322e-4af6-a518-6199b224a0e5
    Step Type:     sync_step_save
Traceback:        Traceback (most recent call last):   File
                  "/usr/lib/python2.7/site-packages/celery/app/trace.py", line
                  367, in trace_task     R = retval = fun(*args, **kwargs)  
                  File
                  "/home/ipanova/pulp_development/pulp/server/pulp/server/async/
                  tasks.py", line 529, in __call__     return super(Task,
                  self).__call__(*args, **kwargs)   File
                  "/home/ipanova/pulp_development/pulp/server/pulp/server/async/
                  tasks.py", line 107, in __call__     return super(PulpTask,
                  self).__call__(*args, **kwargs)   File
                  "/usr/lib/python2.7/site-packages/celery/app/trace.py", line
                  622, in __protected_call__     return self.run(*args,
                  **kwargs)   File
                  "/home/ipanova/pulp_development/pulp/server/pulp/server/contro
                  llers/repository.py", line 770, in sync     sync_report =
                  sync_repo(transfer_repo, conduit, call_config)   File
                  "/home/ipanova/pulp_development/pulp/server/pulp/server/async/
                  tasks.py", line 737, in wrap_f     return f(*args, **kwargs)  
                  File
                  "/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docke
                  r/plugins/importers/importer.py", line 85, in sync_repo    
                  return self.sync_step.process_lifecycle()   File
                  "/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/
                  publish_step.py", line 572, in process_lifecycle    
                  super(PluginStep, self).process_lifecycle()   File
                  "/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/
                  publish_step.py", line 163, in process_lifecycle    
                  step.process()   File
                  "/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/
                  publish_step.py", line 256, in process    
                  self._process_block()   File
                  "/home/ipanova/pulp_development/pulp/server/pulp/plugins/util/
                  publish_step.py", line 303, in _process_block    
                  self.process_main()   File
                  "/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docke
                  r/plugins/importers/sync.py", line 237, in process_main    
                  self._process_manifest_list(manifest, digest, available_blobs,
                  tag)   File
                  "/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docke
                  r/plugins/importers/sync.py", line 283, in
                  _process_manifest_list     self._process_manifest(manifest,
                  digest, available_blobs, tag=tag)   File
                  "/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docke
                  r/plugins/importers/sync.py", line 313, in _process_manifest  
                  manifest = models.Manifest.from_json(manifest, digest)   File
                  "/home/ipanova/pulp_development/pulp_docker/plugins/pulp_docke
                  r/plugins/models.py", line 219, in from_json     fs_layers =
                  [FSLayer(blob_sum=layer['blobSum']) for layer in
                  manifest['fsLayers']] KeyError: 'fsLayers' 

[ipanova@ina pulp]$
Actions
Copy link
 #9

Updated by ipanova@redhat.com almost 6 years ago

the issue is here https://github.com/pulp/pulp_docker/blob/2-master/plugins/pulp_docker/plugins/importers/sync.py#L281

Registry is supposed to return the the older format( if available) in case in accept header nothing is specified. GCR instead returns manifest list.

$curl -i -L -X GET 'https://gcr.io/v2/google-containers/etcd/manifests/3.2.24'
HTTP/2 200 
docker-distribution-api-version: registry/2.0
content-type: application/vnd.docker.distribution.manifest.list.v2+json
content-length: 1665
docker-content-digest: sha256:905d7ca17fd02bc24c0eba9a062753aba15db3e31422390bc3238eb762339b20
date: Fri, 08 Feb 2019 15:15:19 GMT
server: Docker Registry
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
alt-svc: quic=":443"; ma=2592000; v="44,43,39"

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 950,
         "digest": "sha256:7b073bdab8c52dc23dfb3e2101597d30304437869ad8c0b425301e96a066c408",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 950,
         "digest": "sha256:4f93210747a2150605fdbc764a86b549824f322f2168b7c762bd4e9a9a264860",
         "platform": {
            "architecture": "arm",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 950,
         "digest": "sha256:f1dcb048bb5a5c6d74e07db03f0a006a8bdcea4dc034722d1d9e24da15901390",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 951,
         "digest": "sha256:bf275fe898f9efda3db428e3287970b348a7e60d1086af36770aa78c9caec886",
         "platform": {
            "architecture": "ppc64le",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 951,
         "digest": "sha256:857376ef4d252c5ce1817902216dc92fedcc133a225144b3a427e5f89c88a17f",
         "platform": {
            "architecture": "s390x",
            "os": "linux"
         }
      }
   ]

check for the reference https://docs.docker.com/registry/spec/manifest-v2-2/#backward-compatibility

If the manifest being requested uses the new format, and the appropriate media type is not present in an Accept header, the registry will assume that the client cannot handle the manifest as-is, and rewrite it on the fly into the old format. If the object that would otherwise be returned is a manifest list, the registry will look up the appropriate manifest for the amd64 platform and linux OS, rewrite that manifest into the old format if necessary, and return the result to the client. If no suitable manifest is found in the manifest list, the registry will return a 404 error.
Actions
Copy link
 #10

Updated by ipanova@redhat.com almost 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ipanova@redhat.com
  • Triaged changed from No to Yes
  • Sprint set to Sprint 48

Added by ipanova@redhat.com almost 6 years ago

Revision a7f704ff | View on GitHub

Check for returned mediatype from the registry

closes #4328 https://pulp.plan.io/issues/4328

Added by ipanova@redhat.com almost 6 years ago

Revision a7f704ff | View on GitHub

Check for returned mediatype from the registry

closes #4328 https://pulp.plan.io/issues/4328

Actions
Copy link
 #11

Updated by amacdona@redhat.com almost 6 years ago

  • Status changed from ASSIGNED to POST
Actions
Copy link
 #12

Updated by ipanova@redhat.com almost 6 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #14

Updated by ttereshc over 5 years ago

  • Platform Release set to 2.19.0
Actions
Copy link
 #15

Updated by ttereshc over 5 years ago

  • Sprint/Milestone set to 2.19.0

Added by ipanova@redhat.com over 5 years ago

Revision d09e8b48 | View on GitHub

Check for returned mediatype from the registry

closes #4328 https://pulp.plan.io/issues/4328

(cherry picked from commit a7f704ff5a7de13be2a17735a3c22f5d5aa00e61)

Added by ipanova@redhat.com over 5 years ago

Revision d09e8b48 | View on GitHub

Check for returned mediatype from the registry

closes #4328 https://pulp.plan.io/issues/4328

(cherry picked from commit a7f704ff5a7de13be2a17735a3c22f5d5aa00e61)

Actions
Copy link
 #17

Updated by ttereshc over 5 years ago

  • Status changed from MODIFIED to 5
Actions
Copy link
 #18

Updated by ipanova@redhat.com over 5 years ago

  • Status changed from 5 to MODIFIED
Actions
Copy link
 #19

Updated by ttereshc over 5 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions
Copy link
 #21

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added
Actions
Send by e-mail Copy link

Also available in: Atom PDF