Actions
Issue #635
closed
Scientific Linux Errata Data does not get imported
Added by dlevene@advisns.com.au about 9 years ago. Updated almost 5 years ago.
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.4.0
Platform Release:
2.6.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:
Description
Description of problem: Errata Data is not synced from the Scientific Linux Repos when it is available;
http://mirror.aarnet.edu.au/pub/scientific/6.6/x86_64/os/repodata/repomd.xml
Version-Release number of selected component (if applicable): This importing is being done via the latest version of katello
pulp-server-2.4.0-1.el6.noarch / pulp-katello-0.3-3.el6.noarch / katello-2.0.0-0.el6.noarch
How reproducible: always
Steps to Reproduce:
1. Log into Katello and setup a new product
2. Click repo discovery and use this URL http://mirror.aarnet.edu.au/pub/scientific/6.6/x86_64/
3. Select OS (or any of the other 3
Actual results: All packages are successfully synced, but no errata data is pulled in
Expected results: Packages & errata data are seen in katello
Additional info: http://mirror.aarnet.edu.au/pub/scientific/6.6/x86_64/os/ is the single mirror that can be used.
Also on a side note, can you import errata data manually or manually modify a pulp sync to provide it an alternative location for errata data?
+ This bug was cloned from Bugzilla Bug #1172887 +
Updated by bcourt about 9 years ago
The problem was that the mirror uses bz2 compression for the metadata files instead of the standard gzip. PR is to add support for bz2.
https://github.com/pulp/pulp_rpm/pull/627
+ This comment was cloned from Bugzilla #1172887 comment 1 +
Updated by pthomas@redhat.com about 9 years ago
- Status changed from 5 to ASSIGNED
Fails-qa
[root@qe-blade-10 ~]# rpm -qa pulp-server
pulp-server-2.6.1-0.2.beta.el6.noarch
[root@qe-blade-10 ~]#
Importing errata seems to be failing still
[root@qe-blade-10 ~]# pulp-admin rpm repo create --repo-id scientific --feed http://mirror.aarnet.edu.au/pub/scientific/6.6/x86_64/
Warning: path should have mode 0700 because it may contain sensitive information: /root/.pulp/
Successfully created repository [scientific]
[root@qe-blade-10 ~]#
[root@qe-blade-10 ~]#
[root@qe-blade-10 ~]# pulp-admin rpm repo sync run --repo-id scientific
Warning: path should have mode 0700 because it may contain sensitive information: /root/.pulp/
+----------------------------------------------------------------------+
Synchronizing Repository [scientific]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[\]
... failed
Not Found
Task Failed
Importer indicated a failed response
Mar 31 05:57:09 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading 0 RPMs.
Mar 31 05:57:31 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: The content container reported: {'downloads': {}, 'total_sources': 0} for base URL: http://mirror.aarnet.edu.au/pub/scientific/6.6/x86_64/os/
Mar 31 05:57:31 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading additional units.
Mar 31 05:57:31 qe-blade-10 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.aarnet.edu.au
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) sync failed
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) Traceback (most recent call last):
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 135, in run
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) self.get_errata(metadata_files)
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 483, in get_errata
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) updateinfo.process_package_element, additive_type=True)
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 563, in save_fileless_units
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) for model in package_info_generator:
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/packages.py", line 63, in package_list_generator
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) package_info = process_func(element)
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/updateinfo.py", line 27, in process_package_element
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) 'description': element.find('description').text,
Mar 31 05:57:41 qe-blade-10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (27703-77280) AttributeError: 'NoneType' object has no attribute 'text'
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) Task pulp.server.managers.repo.sync.sync[f7d421b1-1149-444e-9d54-356f501f938d] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) Traceback (most recent call last):
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) R = retval = fun(*args, **kwargs)
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 328, in __call__
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) return super(Task, self).__call__(*args, **kwargs)
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) return self.run(*args, **kwargs)
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sync.py", line 114, in sync
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) raise PulpExecutionException(_('Importer indicated a failed response'))
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:ERROR: (27685-77280) PulpExecutionException: Importer indicated a failed response
Mar 31 05:57:41 qe-blade-10 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[ec62b7e5-5602-442f-be3d-d19049efede7] succeeded in 0.0082434789947s: None
<\pre>
Updated by bcourt about 9 years ago
- Status changed from ASSIGNED to POST
Added by bcourt about 9 years ago
Updated by bcourt about 9 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset c53af27a60ca9cf18e4cedbdfc41f3dfc629a6a7.
Updated by pthomas@redhat.com almost 9 years ago
- Status changed from 5 to ASSIGNED
still failing to sync scientific linux
[root@qe-blade-11 ~]# pulp-admin rpm repo create --repo-id scientific --feed http://mirror.aarnet.edu.au/pub/scientific/6.6/x86_64/
Warning: path should have mode 0700 because it may contain sensitive information: /root/.pulp/
Successfully created repository [scientific]
[root@qe-blade-11 ~]# pulp-admin rpm repo sync run --repo-id scientific
Warning: path should have mode 0700 because it may contain sensitive information: /root/.pulp/
+----------------------------------------------------------------------+
Synchronizing Repository [scientific]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[|]
... failed
Not Found
Task Failed
Importer indicated a failed response
[root@qe-blade-11 ~]# tail -f /var/log/messages
Apr 28 12:42:17 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Apr 28 12:42:17 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) R = retval = fun(*args, **kwargs)
Apr 28 12:42:17 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 328, in __call__
Apr 28 12:42:17 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) return super(Task, self).__call__(*args, **kwargs)
Apr 28 12:42:17 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Apr 28 12:42:17 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) return self.run(*args, **kwargs)
Apr 28 12:42:17 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py", line 114, in sync
Apr 28 12:42:17 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) raise PulpExecutionException(_('Importer indicated a failed response'))
Apr 28 12:42:17 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) PulpExecutionException: Importer indicated a failed response
Apr 28 12:42:17 qe-blade-11 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[06a81428-d793-458e-8ac7-15262a2404b7] succeeded in 0.00686540501192s: None
Apr 28 13:01:01 qe-blade-11 systemd: Starting Session 100 of user root.
Apr 28 13:01:01 qe-blade-11 systemd: Started Session 100 of user root.
Apr 28 13:05:44 qe-blade-11 NetworkManager[777]: <error> [1430240744.751764] [devices/nm-device.c:2303] activation_source_schedule(): (enp4s0): activation stage already scheduled
Updated by pthomas@redhat.com almost 9 years ago
Apr 28 13:36:13 qe-blade-11 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[f17bc5a8-4e34-456c-a863-661272ac042d]
Apr 28 13:36:14 qe-blade-11 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[e42708ea-b1f3-4a2e-b910-c3a64bd49c75]
Apr 28 13:36:14 qe-blade-11 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[10cf4534-037a-4817-ae95-db1565552f8f]
Apr 28 13:36:14 qe-blade-11 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from http://mirror.aarnet.edu.au/pub/scientific/6.6/x86_64/.
Apr 28 13:36:14 qe-blade-11 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.aarnet.edu.au
Apr 28 13:36:14 qe-blade-11 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[f17bc5a8-4e34-456c-a863-661272ac042d] succeeded in 0.0333014460048s: None
Apr 28 13:36:15 qe-blade-11 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (25280-69632) sync failed
Apr 28 13:36:15 qe-blade-11 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (25280-69632) Traceback (most recent call last):
Apr 28 13:36:15 qe-blade-11 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (25280-69632) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 105, in run
Apr 28 13:36:15 qe-blade-11 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (25280-69632) metadata_files = self.get_metadata()
Apr 28 13:36:15 qe-blade-11 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (25280-69632) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 199, in get_metadata
Apr 28 13:36:15 qe-blade-11 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (25280-69632) raise FailedException(str(e))
Apr 28 13:36:15 qe-blade-11 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (25280-69632) FailedException: Not Found
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) Task pulp.server.managers.repo.sync.sync[e42708ea-b1f3-4a2e-b910-c3a64bd49c75] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) Traceback (most recent call last):
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) R = retval = fun(*args, **kwargs)
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 328, in __call__
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) return super(Task, self).__call__(*args, **kwargs)
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) return self.run(*args, **kwargs)
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py", line 114, in sync
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) raise PulpExecutionException(_('Importer indicated a failed response'))
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:ERROR: (25170-69632) PulpExecutionException: Importer indicated a failed response
Apr 28 13:36:15 qe-blade-11 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[10cf4534-037a-4817-ae95-db1565552f8f] succeeded in 0.00619069201639s: None
Updated by bcourt almost 9 years ago
- Platform Release changed from 2.6.1 to 2.6.2
Updated by bcourt almost 9 years ago
- Status changed from ASSIGNED to 5
- Platform Release changed from 2.6.2 to 2.6.1
It appears that this was tested using a bad repository feed url. Closer mirrors that could be used for this are:
pulp-admin rpm repo create --repo-id sci-security --feed http://www.gtlib.gatech.edu/pub/scientific/7/x86_64/updates/security/
or
pulp-admin rpm repo create --repo-id sci-security --feed http://www.gtlib.gatech.edu/pub/scientific/7/x86_64/os/
Updated by pthomas@redhat.com almost 9 years ago
- Status changed from 5 to 6
verified
[root@qe-blade-13 ~]# pulp-admin rpm repo sync run --repo-id sci-security
Warning: path should have mode 0700 because it may contain sensitive information: /root/.pulp/
+----------------------------------------------------------------------+
Synchronizing Repository [sci-security]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[/]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 8685/8685 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[========================================= ] 83%
Distributions: 5/6 items
Errors encountered during distribution sync:
File:
http://www.gtlib.gatech.edu/pub/scientific/7/x86_64/os/images/boot.iso
Error
Code: None
Error Message: None
Importing errata...
[-]
... completed
Importing package groups/categories...
[\]
... completed
Task Succeeded
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[==================================================] 100%
8685 of 8685 items
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[==================================================] 100%
3021 of 3021 items
... completed
Publishing Comps file
[==================================================] 100%
105 of 105 items
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Publishing files to web
[\]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
<\pre>
Updated by dkliban@redhat.com over 8 years ago
- Status changed from 6 to CLOSED - CURRENTRELEASE
Actions
.
Fix import of scientific linux erratum that don't contain a description field.
fixes #635