Project

Profile

Help

Issue #4424

Updated by ttereshc over 5 years ago

Pulp 3 does not handle RPM RPMs repos with invalid metadata. 

 When syncing a repository that's missing its ``filelists.xml`` file.    Like: "missing file list":https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-missing-filelists/ 


 Traceback: 

 <pre><code class="text"> 

 ("Task report /pulp/api/v3/tasks/213/ contains a error: {'code': None, " 
  '\'description\': "404, message=\'Not Found\'", \'traceback\': \'    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, ' 
  'in perform_job\\n      rv = job.perform()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 600, in ' 
  'perform\\n      self._result = self._execute()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 606, in ' 
  '_execute\\n      return self.func(*self.args, **self.kwargs)\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/tasks/synchronizing.py", ' 
  'line 79, in synchronize\\n      loop.run_until_complete(pipeline)\\n    File ' 
  '"/usr/lib64/python3.7/asyncio/base_events.py", line 584, in ' 
  'run_until_complete\\n      return future.result()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py", ' 
  'line 209, in create_pipeline\\n      await asyncio.gather(*futures)\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py", ' 
  'line 43, in __call__\\n      await self.run()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/tasks/synchronizing.py", ' 
  'line 234, in run\\n      results = downloader.result()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/download/base.py", ' 
  'line 212, in run\\n      return await self._run(extra_data=extra_data)\\n    ' 
  'File "/usr/local/lib/pulp/lib64/python3.7/site-packages/backoff/_async.py", ' 
  'line 131, in retry\\n      ret = await target(*args, **kwargs)\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/download/http.py", ' 
  'line 183, in _run\\n      response.raise_for_status()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/aiohttp/client_reqrep.py", ' 
  "line 942, in raise_for_status\\n      headers=self.headers)\\n'}\n" 
  "Full task report: {'_href': '/pulp/api/v3/tasks/213/', '_created': " 
  "'2018-12-21T00:58:21.054706Z', 'job_id': " 
  "'5d967c5e-4903-43e9-a171-9fbad97bae9e', 'state': 'failed', 'name': " 
  "'pulp_rpm.app.tasks.synchronizing.synchronize', 'started_at': " 
  "'2018-12-21T00:58:21.130936Z', 'finished_at': '2018-12-21T00:58:21.589230Z', " 
  '\'non_fatal_errors\': [], \'error\': {\'code\': None, \'description\': "404, ' 
  'message=\'Not Found\'", \'traceback\': \'    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, ' 
  'in perform_job\\n      rv = job.perform()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 600, in ' 
  'perform\\n      self._result = self._execute()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 606, in ' 
  '_execute\\n      return self.func(*self.args, **self.kwargs)\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/tasks/synchronizing.py", ' 
  'line 79, in synchronize\\n      loop.run_until_complete(pipeline)\\n    File ' 
  '"/usr/lib64/python3.7/asyncio/base_events.py", line 584, in ' 
  'run_until_complete\\n      return future.result()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py", ' 
  'line 209, in create_pipeline\\n      await asyncio.gather(*futures)\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py", ' 
  'line 43, in __call__\\n      await self.run()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/tasks/synchronizing.py", ' 
  'line 234, in run\\n      results = downloader.result()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/download/base.py", ' 
  'line 212, in run\\n      return await self._run(extra_data=extra_data)\\n    ' 
  'File "/usr/local/lib/pulp/lib64/python3.7/site-packages/backoff/_async.py", ' 
  'line 131, in retry\\n      ret = await target(*args, **kwargs)\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/download/http.py", ' 
  'line 183, in _run\\n      response.raise_for_status()\\n    File ' 
  '"/usr/local/lib/pulp/lib64/python3.7/site-packages/aiohttp/client_reqrep.py", ' 
  "line 942, in raise_for_status\\n      headers=self.headers)\\n'}, 'worker': " 
  "'/pulp/api/v3/workers/1/', 'parent': None, 'spawned_tasks': [], " 
  "'progress_reports': [{'message': 'Downloading and Parsing Metadata', " 
  "'state': 'failed', 'total': 2, 'done': 2, 'suffix': '', 'task': " 
  "'/pulp/api/v3/tasks/213/'}, {'message': 'Downloading Artifacts', 'state': " 
  "'canceled', 'total': None, 'done': 0, 'suffix': '', 'task': " 
  "'/pulp/api/v3/tasks/213/'}, {'message': 'Associating Content', 'state': " 
  "'canceled', 'total': None, 'done': 0, 'suffix': '', 'task': " 
  "'/pulp/api/v3/tasks/213/'}], 'created_resources': []}", 
  {'_created': '2018-12-21T00:58:21.054706Z', 
   '_href': '/pulp/api/v3/tasks/213/', 
   'created_resources': [], 
   'error': {'code': None, 
             'description': "404, message='Not Found'", 
             'traceback': '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", ' 
                          'line 799, in perform_job\n' 
                          '      rv = job.perform()\n' 
                          '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", ' 
                          'line 600, in perform\n' 
                          '      self._result = self._execute()\n' 
                          '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", ' 
                          'line 606, in _execute\n' 
                          '      return self.func(*self.args, **self.kwargs)\n' 
                          '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/tasks/synchronizing.py", ' 
                          'line 79, in synchronize\n' 
                          '      loop.run_until_complete(pipeline)\n' 
                          '    File ' 
                          '"/usr/lib64/python3.7/asyncio/base_events.py", line ' 
                          '584, in run_until_complete\n' 
                          '      return future.result()\n' 
                          '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py", ' 
                          'line 209, in create_pipeline\n' 
                          '      await asyncio.gather(*futures)\n' 
                          '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/stages/api.py", ' 
                          'line 43, in __call__\n' 
                          '      await self.run()\n' 
                          '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulp_rpm/app/tasks/synchronizing.py", ' 
                          'line 234, in run\n' 
                          '      results = downloader.result()\n' 
                          '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/download/base.py", ' 
                          'line 212, in run\n' 
                          '      return await self._run(extra_data=extra_data)\n' 
                          '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/backoff/_async.py", ' 
                          'line 131, in retry\n' 
                          '      ret = await target(*args, **kwargs)\n' 
                          '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/pulpcore/plugin/download/http.py", ' 
                          'line 183, in _run\n' 
                          '      response.raise_for_status()\n' 
                          '    File ' 
                          '"/usr/local/lib/pulp/lib64/python3.7/site-packages/aiohttp/client_reqrep.py", ' 
                          'line 942, in raise_for_status\n' 
                          '      headers=self.headers)\n'}, 
   'finished_at': '2018-12-21T00:58:21.589230Z', 
   'job_id': '5d967c5e-4903-43e9-a171-9fbad97bae9e', 
   'name': 'pulp_rpm.app.tasks.synchronizing.synchronize', 
   'non_fatal_errors': [], 
   'parent': None, 
   'progress_reports': [{'done': 2, 
                         'message': 'Downloading and Parsing Metadata', 
                         'state': 'failed', 
                         'suffix': '', 
                         'task': '/pulp/api/v3/tasks/213/', 
                         'total': 2}, 
                        {'done': 0, 
                         'message': 'Downloading Artifacts', 
                         'state': 'canceled', 
                         'suffix': '', 
                         'task': '/pulp/api/v3/tasks/213/', 
                         'total': None}, 
                        {'done': 0, 
                         'message': 'Associating Content', 
                         'state': 'canceled', 
                         'suffix': '', 
                         'task': '/pulp/api/v3/tasks/213/', 
                         'total': None}], 
   'spawned_tasks': [], 
   'started_at': '2018-12-21T00:58:21.130936Z', 
   'state': 'failed', 
   'worker': '/pulp/api/v3/workers/1/'}) 

 </code></pre> 


 Same happen using the following test repos: 
 "missing other":https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-missing-other/ 
 "missing primary":https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-missing-primary/ 

 A descriptive error should be returned. 

 Pulp 2 does fail before any content being synced. Related issue: https://pulp.plan.io/issues/1287 

Back