Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #1743

closed

Repo sync fails during distribution sync on repos created pre upgrade

Added by pthomas@redhat.com over 7 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
bmbouter
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.8.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Syncing a rhel6 repo that was created and synced pre upgrade fails after the upgrade

Steps
1. Setup a 2.7 pulp with rhel6 kickstart repo synced

pulp-admin rpm repo create --repo-id rhel6 --feed http://download.devel.redhat.com/cds/prod/content/dist/rhel/rhui/server/6/6.7/x86_64/kickstart/
pulp-admin rpm repo sync run --repo-id rhel6
2. Upgrade to 2.8

3. Resync the repo


[root@rhsm-jenkins ~]# rpm -qa |grep pulp
python-pulp-client-lib-2.8.0-0.7.beta.el6.noarch
pulp-server-2.8.0-0.7.beta.el6.noarch
pulp-rpm-admin-extensions-2.8.0-0.7.beta.el6.noarch
python-pulp-agent-lib-2.8.0-0.7.beta.el6.noarch
pulp-agent-2.8.0-0.7.beta.el6.noarch
python-pulp-common-2.8.0-0.7.beta.el6.noarch
python-pulp-rpm-common-2.8.0-0.7.beta.el6.noarch
pulp-admin-client-2.8.0-0.7.beta.el6.noarch
python-pulp-oid_validation-2.8.0-0.7.beta.el6.noarch
pulp-rpm-plugins-2.8.0-0.7.beta.el6.noarch
pulp-docker-admin-extensions-2.0.0-0.7.beta.el6.noarch
pulp-rpm-consumer-extensions-2.8.0-0.7.beta.el6.noarch
pulp-rpm-yumplugins-2.8.0-0.7.beta.el6.noarch
python-isodate-0.5.0-4.pulp.el6.noarch
mod_wsgi-3.4-2.pulp.el6.x86_64
python-pulp-bindings-2.8.0-0.7.beta.el6.noarch
python-pulp-puppet-common-2.8.0-0.7.beta.el6.noarch
python-pulp-docker-common-2.0.0-0.7.beta.el6.noarch
pulp-puppet-plugins-2.8.0-0.7.beta.el6.noarch
pulp-puppet-admin-extensions-2.8.0-0.7.beta.el6.noarch
pulp-rpm-handlers-2.8.0-0.7.beta.el6.noarch
pulp-selinux-2.8.0-0.7.beta.el6.noarch
python-pulp-repoauth-2.8.0-0.7.beta.el6.noarch
python-kombu-3.0.33-4.pulp.el6.noarch
pulp-puppet-handlers-2.8.0-0.7.beta.el6.noarch
pulp-consumer-client-2.8.0-0.7.beta.el6.noarch
pulp-docker-plugins-2.0.0-0.7.beta.el6.noarch
pulp-puppet-consumer-extensions-2.8.0-0.7.beta.el6.noarch
[root@rhsm-jenkins ~]# 

[root@rhsm-jenkins ~]# pulp-admin rpm repo sync run --repo-id rhel6 +----------------------------------------------------------------------+
                    Synchronizing Repository [rhel6]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... completed

Downloading repository content...
[-]
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items

Task Failed

Importer indicated a failed response

Mar  4 12:50:02 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.7/x86_64/kickstart/.
Mar  4 12:50:02 rhsm-jenkins pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Mar  4 12:50:03 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Parsing metadata.
Mar  4 12:50:03 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.7/x86_64/kickstart/.
Mar  4 12:50:03 rhsm-jenkins pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Mar  4 12:50:04 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Parsing metadata.
Mar  4 12:50:04 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:INFO: upstream repo metadata has not changed. Skipping steps.
Mar  4 12:50:04 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading additional units.
Mar  4 12:50:04 rhsm-jenkins pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Mar  4 12:50:05 rhsm-jenkins pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160) The field 'pulp_distribution_xml_file' does not exist on the document 'Distribution'
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160) Traceback (most recent call last):
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 228, in run
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)     dist_sync.run()
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 125, in run
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)     self._run(tmp_dir)
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 142, in _run
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)     unit, files = self.parse_treeinfo_file(treeinfo_path)
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 507, in parse_treeinfo_file
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)     arch=new_dist.arch
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)   File "/usr/lib/python2.6/site-packages/mongoengine/queryset/base.py", line 264, in first
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)     result = queryset[0]
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)   File "/usr/lib/python2.6/site-packages/mongoengine/queryset/base.py", line 163, in __getitem__
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)     only_fields=self.only_fields)
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)   File "/usr/lib/python2.6/site-packages/mongoengine/base/document.py", line 730, in _from_son
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)     obj = cls(__auto_convert=False, _created=created, __only_fields=only_fields, **data)
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/db/models.py", line 226, in __init__
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)     super(Distribution, self).__init__(*args, **kwargs)
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)   File "/usr/lib/python2.6/site-packages/pulp/server/db/model/__init__.py", line 52, in __init__
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)     super(AutoRetryDocument, self).__init__(*args, **kwargs)
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)   File "/usr/lib/python2.6/site-packages/mongoengine/base/document.py", line 80, in __init__
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160)     raise FieldDoesNotExist(msg)
Mar  4 12:50:06 rhsm-jenkins pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (17665-68160) FieldDoesNotExist: The field 'pulp_distribution_xml_file' does not exist on the document 'Distribution'
Mar  4 12:50:07 rhsm-jenkins pulp: pulp.server.async.tasks:INFO: Task failed : [8dd53cd1-023a-4e1e-a6b0-e053b1d88976]
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160) Task pulp.server.managers.repo.sync.sync[8dd53cd1-023a-4e1e-a6b0-e053b1d88976] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160) Traceback (most recent call last):
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160)   File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160)     R = retval = fun(*args, **kwargs)
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160)   File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 473, in __call__
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160)     return super(Task, self).__call__(*args, **kwargs)
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160)   File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160)     return super(PulpTask, self).__call__(*args, **kwargs)
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160)   File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160)     return self.run(*args, **kwargs)
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160)   File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 807, in sync
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:ERROR: (17592-68160) PulpExecutionException: Importer indicated a failed response
Mar  4 12:50:07 rhsm-jenkins pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[8ed5a17e-ec62-40e0-9a60-1f63435b1a52] succeeded in 0.0124898170034s: None
Actions
Copy link
 #1

Updated by pthomas@redhat.com over 7 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by bmbouter over 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bmbouter

Added by bmbouter over 7 years ago

Revision 6f575de8 | View on GitHub

Adds migration 0026 which removes a field from Distribution units

The pulp_distribution_xml_file was present in earlier versions of Pulp but is no longer used. It's not included in the mongoengine Distribution model so it fails to hydrate when it is present in the database.

closes #1742 https://pulp.plan.io/issues/1743

Actions
Copy link
 #4

Updated by bmbouter over 7 years ago

  • Status changed from ASSIGNED to POST
Actions
Copy link
 #5

Updated by bmbouter over 7 years ago

  • Platform Release set to 2.8.0
  • Triaged changed from No to Yes
Actions
Copy link
 #6

Updated by bmbouter over 7 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #7

Updated by dkliban@redhat.com over 7 years ago

  • Status changed from MODIFIED to 5
Actions
Copy link
 #8

Updated by pthomas@redhat.com over 7 years ago

  • Status changed from 5 to 6

verified

[root@mgmt5 ~]# rpm -qa pulp-server
pulp-server-2.8.0-0.9.rc.el7.noarch
[root@mgmt5 ~]#

[root@mgmt5 ~]# pulp-admin rpm repo sync run --repo-id rhel6
--------------------------------------------------------------------
Synchronizing Repository [rhel6]
--------------------------------------------------------------------

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[\]
... completed

Downloading repository content...
[-]
[==================================================] 100%
RPMs: 0/0 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Cleaning duplicate packages...
[-]
... completed

Task Succeeded

Copying files
[/]
... completed

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[|]
... completed

Publishing RPMs
[\]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[-]
... completed

Publishing Comps file
[==================================================] 100%
212 of 212 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[|]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

Actions
Copy link
 #9

Updated by dkliban@redhat.com over 7 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE
Actions
Copy link
 #11

Updated by bmbouter over 4 years ago

  • Tags Pulp 2 added
Actions
Send by e-mail Copy link

Also available in: Atom PDF