RPM Support

Let's say you create a Pulp repository and set its feed URL to a mirrorlist like the following: https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-mirrorlist-mixed

This mirrorlist file contains both invalid and valid entries, and sane behaviour is to try each mirrorlist entry (in some arbitrary order) until one works or all entries have been tried. Instead, Pulp appears to pick only one arbitrary entry, and will never try an additional entry. This results in log entries like the following:

Oct 04 15:38:52 pulp.example.com pulp[8987]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[58ea169d-dd37-405a-bbfc-80421aedb52a]
Oct 04 15:38:53 pulp.example.com pulp[8945]: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[eb51df16-8223-4065-9004-19f26f1e2db5]
Oct 04 15:38:53 pulp.example.com pulp[8945]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[aecf2c84-038a-4eaf-aac2-9d5098d67b2c]
Oct 04 15:38:53 pulp.example.com pulp[9004]: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-mirrorlist-mixed/.
Oct 04 15:38:53 pulp.example.com pulp[9004]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): repos.fedorapeople.org
Oct 04 15:38:53 pulp.example.com pulp[8987]: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[58ea169d-dd37-405a-bbfc-80421aedb52a] succeeded in 0.0361693099985s: None
Oct 04 15:38:54 pulp.example.com pulp[9004]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): repos.fedorapeople.org
Oct 04 15:38:54 pulp.example.com pulp[9004]: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-mirrorlist-mixed/.
Oct 04 15:38:54 pulp.example.com pulp[9004]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): repos.fedorapeople.org
Oct 04 15:38:55 pulp.example.com pulp[9004]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): repos.fedorapeople.org
Oct 04 15:38:55 pulp.example.com pulp[9004]: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from https://repos.fedorapeople.org/pulp/pulp/fixtures/rpmm-unsigned/.
Oct 04 15:38:55 pulp.example.com pulp[9004]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): repos.fedorapeople.org
Oct 04 15:38:56 pulp.example.com pulp[9004]: pulp_rpm.plugins.importers.yum.sync:ERROR: (9004-47584) 'NoneType' object has no attribute 'downloader'
Oct 04 15:38:56 pulp.example.com pulp[9004]: pulp_rpm.plugins.importers.yum.sync:ERROR: (9004-47584) Traceback (most recent call last):
Oct 04 15:38:56 pulp.example.com pulp[9004]: pulp_rpm.plugins.importers.yum.sync:ERROR: (9004-47584)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 248, in run
Oct 04 15:38:56 pulp.example.com pulp[9004]: pulp_rpm.plugins.importers.yum.sync:ERROR: (9004-47584)     metadata_files = self.get_metadata(metadata_files)
Oct 04 15:38:56 pulp.example.com pulp[9004]: pulp_rpm.plugins.importers.yum.sync:ERROR: (9004-47584)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 400, in get_metadata
Oct 04 15:38:56 pulp.example.com pulp[9004]: pulp_rpm.plugins.importers.yum.sync:ERROR: (9004-47584)     self.downloader = metadata_files.downloader
Oct 04 15:38:56 pulp.example.com pulp[9004]: pulp_rpm.plugins.importers.yum.sync:ERROR: (9004-47584) AttributeError: 'NoneType' object has no attribute 'downloader'
Oct 04 15:38:56 pulp.example.com pulp[9004]: pulp.server.async.tasks:INFO: Task failed : [eb51df16-8223-4065-9004-19f26f1e2db5]
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584) Task pulp.server.managers.repo.sync.sync[eb51df16-8223-4065-9004-19f26f1e2db5] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584) Traceback (most recent call last):
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584)     R = retval = fun(*args, **kwargs)
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 488, in __call__
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584)     return super(Task, self).__call__(*args, **kwargs)
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584)     return super(PulpTask, self).__call__(*args, **kwargs)
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584)     return self.run(*args, **kwargs)
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 816, in sync
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:ERROR: (8945-47584) PulpExecutionException: Importer indicated a failed response
Oct 04 15:38:56 pulp.example.com pulp[8945]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[aecf2c84-038a-4eaf-aac2-9d5098d67b2c] succeeded in 0.007053248999s: None

This appears to an API as a failed task:

{'_href': '/pulp/api/v2/tasks/eb51df16-8223-4065-9004-19f26f1e2db5/',
 '_id': {'$oid': '57f4054caf9e9e4fefaf3f27'},
 '_ns': 'task_status',
 'error': {'code': 'PLP0000',
           'data': {},
           'description': 'Importer indicated a failed response',
           'sub_errors': []},
 'exception': None,
 'finish_time': '2016-10-04T19:38:56Z',
 'id': '57f4054caf9e9e4fefaf3f27',
 'progress_report': {'yum_importer': {'comps': {'state': 'NOT_STARTED'},
                                      'content': {'details': {'drpm_done': 0,
                                                              'drpm_total': 0,
                                                              'rpm_done': 0,
                                                              'rpm_total': 0},
                                                  'error_details': [],
                                                  'items_left': 0,
                                                  'items_total': 0,
                                                  'size_left': 0,
                                                  'size_total': 0,
                                                  'state': 'NOT_STARTED'},
                                      'distribution': {'error_details': [],
                                                       'items_left': 0,
                                                       'items_total': 0,
                                                       'state': 'NOT_STARTED'},
                                      'errata': {'state': 'NOT_STARTED'},
                                      'metadata': {'error': "'NoneType' object "
                                                            'has no attribute '
                                                            "'downloader'",
                                                   'state': 'FAILED'},
                                      'purge_duplicates': {'state': 'NOT_STARTED'}}},
 'queue': 'reserved_resource_worker-0@pulp.example.com.dq',
 'result': None,
 'spawned_tasks': [],
 'start_time': '2016-10-04T19:38:53Z',
 'state': 'error',
 'tags': ['pulp:repository:fad0a931-19d4-4f76-b9b3-bc5f38d5fd92',
          'pulp:action:sync'],
 'task_id': 'eb51df16-8223-4065-9004-19f26f1e2db5',
 'task_type': 'pulp.server.managers.repo.sync.sync',
 'traceback': 'Traceback (most recent call last):\n'
              '  File "/usr/lib/python2.7/site-packages/celery/app/trace.py", '
              'line 240, in trace_task\n'
              '    R = retval = fun(*args, **kwargs)\n'
              '  '
              'File"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", '
              'line 488, in __call__\n'
              '    return super(Task, self).__call__(*args, **kwargs)\n'
              '  File '
              '"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", '
              'line 103, in __call__\n'
              ' return super(PulpTask, self).__call__(*args, **kwargs)\n'
              '  File "/usr/lib/python2.7/site-packages/celery/app/trace.py", '
              'line 437, in __protected_call__\n'
              '    return self.run(*args, **kwargs)\n'
              '  File '
              '"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", '
              'line 816, in sync\n'
              "    raise pulp_exceptions.PulpExecutionException(_('Importer "
              "indicated a failed response'))\n"
              'PulpExecutionException: Importer indicated a failed response\n',
 'worker_name': 'reserved_resource_worker-0@pulp.example.com'}

This issue is apparent in the current development releases of Pulp 2.11. It doesn't appear to happen in Pulp 2.10.1.