Project

Profile

Help

Issue #1455

closed

pulp_rpm repository syncs fail

Added by Ichimonji10 over 8 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Master
Platform Release:
2.8.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Pulp cannot currently sync RPM repositories from feeds. To reproduce this failure, install pulp and pulp_rpm from source, and execute a Pulp Smash test case like so:

python -m unittest2 pulp_smash.tests.rpm.api_v2.test_sync_publish.SyncValidFeedTestCase

The following tests will fail:

  • test_task_error
  • test_task_traceback
  • test_unit_count_on_repo

These tests fail because of unwanted results in the response that Pulp hands back. Pulp's response is a task progress report. The `test_task_error` test fails because the progress report's "error" field contains the following:

{
    "sub_errors": [],
    "data": {},
    "description": "Importer indicated a failed response",
    "code": "PLP0000"
}

The `test_task_traceback` test fails because the progress report's "traceback" field contains the following:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 473, in __call__
    return super(Task, self).__call__(*args, **kwargs)
  File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 103, in __call__
    return super(PulpTask, self).__call__(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 755, in sync
    raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
PulpExecutionException: Importer indicated a failed response

If one logs into the system hosting Pulp and examines the logs (with journalctl), the following is present:

Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752) ValidationError (Errata:2a8187c4-803c-4474-a89e-4036a2ae5ec4) ( could not be converted to int: ['pushcount'])
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752) Traceback (most recent call last):
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 239, in run
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752)     self.get_errata(metadata_files)
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 757, in get_errata
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752)     updateinfo.process_package_element, additive_type=True)
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/importers/yum/sync.py", line 841, in save_fileless_units
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752)     model.save()
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752)   File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 287, in save
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752)     self.validate(clean=clean)
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752)   File "/usr/lib/python2.7/site-packages/mongoengine/base/document.py", line 411, in validate
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752)     raise ValidationError(message, errors=errors)
Dec 22 14:05:40 dev pulp[20724]: pulp_rpm.plugins.importers.yum.sync:ERROR: (20724-82752) ValidationError: ValidationError (Errata:2a8187c4-803c-4474-a89e-4036a2ae5ec4) ( could not be converted to int: ['pushcount'])
Dec 22 14:05:40 dev pulp[20724]: pulp.server.async.tasks:INFO: Task failed : [889ed0cb-b9c0-46a5-b347-8247b5928be7]
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752) Task pulp.server.managers.repo.sync.sync[889ed0cb-b9c0-46a5-b347-8247b5928be7] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752) Traceback (most recent call last):
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752)     R = retval = fun(*args, **kwargs)
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 473, in __call__
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752)     return super(Task, self).__call__(*args, **kwargs)
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 103, in __call__
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752)     return super(PulpTask, self).__call__(*args, **kwargs)
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752)     return self.run(*args, **kwargs)
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 755, in sync
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Dec 22 14:05:40 dev pulp[20597]: celery.worker.job:ERROR: (20597-82752) PulpExecutionException: Importer indicated a failed response
Actions #1

Updated by jcline@redhat.com over 8 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jcline@redhat.com
Actions #2

Updated by jcline@redhat.com over 8 years ago

  • Status changed from ASSIGNED to POST
Actions #3

Updated by bmbouter over 8 years ago

  • Status changed from POST to ASSIGNED
  • Assignee changed from jcline@redhat.com to bmbouter

The previous PR didn't handle a few other cases, so we're fixing this with a new PR.

Actions #4

Updated by bmbouter over 8 years ago

  • Status changed from ASSIGNED to POST

Added by bmbouter over 8 years ago

Revision b3af75eb | View on GitHub

Fixes related to units_erratum.pushcount field

  • Reverts units_erratum.pushcount to be an StringField

  • Adds migration 24 which converts all int or float units_erratum.pushcount values to a string. Any pushcount fields with a value of null are $unset.

  • Adds tests for migration 24

  • Adds release notes about field data quality changes

  • Adjusts pulp-admin to have it submit pushcount as a string

https://pulp.plan.io/issues/1455 closes #1455

Actions #5

Updated by bmbouter over 8 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #6

Updated by mhrivnak about 8 years ago

  • Platform Release set to 2.8.0
  • Triaged changed from No to Yes
Actions #7

Updated by rbarlow about 8 years ago

  • Project changed from Pulp to RPM Support
  • Category deleted (14)
Actions #8

Updated by Ichimonji10 about 8 years ago

I can confirm that this issue does not affect the current master version of Pulp.

Actions #9

Updated by dkliban@redhat.com about 8 years ago

  • Status changed from MODIFIED to 5
Actions #10

Updated by dkliban@redhat.com about 8 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #11

Updated by bmbouter almost 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF