Actions
Issue #2012
closed
alternate content source refresh fails with json serialization error
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.8.4
Platform Release:
2.8.6
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 4
Quarter:
Description
When an alternate content source is defined, an error like this one will be seen in the log:
pulp.server.content.sources.model:ERROR: Refresh [XXXXX] url: http://zfs1.virt/pulp/content/beta/rhel/server/7/x86_64/sat-tools/6/os/, failed: <mongoengine.fields.StringField object at 0x3118510> is not JSON serializable
User Billy Holmes, who originally reported the issue, suggested this patch:
--- /usr/lib/python2.7/site-packages/pulp_rpm/plugins/catalogers/yum.py 2016-04-19 17:54:46.000000000 -0400
+++ /usr/lib/python2.7/site-packages/pulp_rpm/plugins/catalogers/yum.py.new 2016-05-05 01:15:39.220012053 -0400
@@ -50,8 +50,9 @@
fp, primary.PACKAGE_TAG, primary.process_package_element)
for model in _packages:
unit_key = model.unit_key
+ type_id = model._content_type_id
url = urljoin(base_url, model.download_path)
- conduit.add_entry(models.RPM._content_type_id, unit_key, url)
+ conduit.add_entry(type_id, unit_key, url)
finally:
fp.close()
Related issues
Updated by jcline@redhat.com over 8 years ago
(This comment was meant for #2013)
Updated by mhrivnak over 8 years ago
- Sprint/Milestone set to 22
Adding to sprint 4 after discussion with jalberts.
Updated by mhrivnak over 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to mhrivnak
I reproduced by doing this:
1. create file /etc/pulp/content/sources/conf.d/zoo.conf with these contents:
[content-world]
enabled: 1
priority: 0
expires: 3d
name: Content World
type: yum
base_url: https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/
paths: rpm/
2. Run these commands:
pulp-admin rpm repo create --repo-id=zoo --feed=https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
pulp-admin rpm repo sync run --repo-id=zoo
3. Look in the system log for this error:
pulp.server.content.sources.model:ERROR: Refresh [content-world] url: https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/rpm/, failed: <mongoengine.fields.StringField object at 0x7fa3b448f190> is not JSON serializable
Note that if you then run another sync, the error does not appear. You have to delete and re-create the repo before the next sync if you want to re-produce the error.
Added by mhrivnak over 8 years ago
Updated by mhrivnak over 8 years ago
- Status changed from ASSIGNED to POST
Updated by mhrivnak over 8 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp_rpm:70cd7e45fd0ff24c0c8d8ddd0f572018982a5e2e.
Updated by pthomas@redhat.com over 8 years ago
- Status changed from MODIFIED to 6
verified
[root@~]# cat /etc/pulp/content/sources/conf.d/zoo.conf
[content-world]
enabled: 1
priority: 0
expires: 3d
name: Content World
type: yum
base_url: https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/
paths: rpm/
[root@ ~]#
[i~]#
[root@ ~]# pulp-admin rpm repo create --repo-id=zoo --feed=https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
Successfully created repository [zoo]
[root@ ~]# pulp-admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
Synchronizing Repository [zoo]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[|]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 32/32 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Cleaning duplicate packages...
[-]
... completed
Task Succeeded
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[==================================================] 100%
32 of 32 items
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[==================================================] 100%
4 of 4 items
... completed
Publishing Comps file
[==================================================] 100%
3 of 3 items
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Publishing files to web
[-]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
[root@ibm-x3550m3-11 ~]#
Jun 24 14:55:23 ibm-x3550m3-11.lab.eng.brq.redhat.com pulp[15977]: pulp.server.content.sources.model:INFO: Refreshing [content-world] url:https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/rpm/
Jun 24 14:55:28 ibm-x3550m3-11.lab.eng.brq.redhat.com pulp[15977]: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: The content container reported: {'downloads': {'content-world': {'total_failed': 0, 'total_succeeded': 32}}, 'total_sources': 1} for base URL: https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
Updated by semyers over 8 years ago
- Status changed from 5 to 6
I accidentally moved this from VERIFIED back to ON_QA. Sorry for the ticket noise!
Updated by semyers over 8 years ago
- Status changed from 6 to CLOSED - CURRENTRELEASE
Updated by semyers over 8 years ago
- Related to Task #2083: Issues common to 2.9.1 and 2.8 stream added
Actions
.
fixes json serialization error when adding catalog entries
https://pulp.plan.io/issues/2012 fixes #2012