Project

Profile

Help

Issue #2012

alternate content source refresh fails with json serialization error

Added by mhrivnak over 4 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.8.4
Platform Release:
2.8.6
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 4
Quarter:

Description

When an alternate content source is defined, an error like this one will be seen in the log:

pulp.server.content.sources.model:ERROR: Refresh [XXXXX] url: http://zfs1.virt/pulp/content/beta/rhel/server/7/x86_64/sat-tools/6/os/, failed: <mongoengine.fields.StringField object at 0x3118510> is not JSON serializable

User Billy Holmes, who originally reported the issue, suggested this patch:

--- /usr/lib/python2.7/site-packages/pulp_rpm/plugins/catalogers/yum.py    2016-04-19 17:54:46.000000000 -0400
+++ /usr/lib/python2.7/site-packages/pulp_rpm/plugins/catalogers/yum.py.new    2016-05-05 01:15:39.220012053 -0400
@@ -50,8 +50,9 @@
                 fp, primary.PACKAGE_TAG, primary.process_package_element)
             for model in _packages:
                 unit_key = model.unit_key
+                type_id = model._content_type_id
                 url = urljoin(base_url, model.download_path)
-                conduit.add_entry(models.RPM._content_type_id, unit_key, url)
+                conduit.add_entry(type_id, unit_key, url)
         finally:
             fp.close()

Related issues

Related to Packaging - Task #2083: Issues common to 2.9.1 and 2.8 streamCLOSED - NOTABUG

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 70cd7e45 View on GitHub
Added by mhrivnak over 4 years ago

fixes json serialization error when adding catalog entries

https://pulp.plan.io/issues/2012 fixes #2012

History

#1 Updated by amacdona@redhat.com over 4 years ago

  • Triaged changed from No to Yes

#2 Updated by jcline@redhat.com over 4 years ago

(This comment was meant for #2013)

#3 Updated by mhrivnak over 4 years ago

  • Sprint/Milestone set to 22

Adding to sprint 4 after discussion with jalberts.

#4 Updated by mhrivnak over 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to mhrivnak

I reproduced by doing this:

1. create file /etc/pulp/content/sources/conf.d/zoo.conf with these contents:

[content-world]
enabled: 1
priority: 0
expires: 3d
name: Content World
type: yum
base_url: https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/
paths: rpm/

2. Run these commands:

pulp-admin rpm repo create --repo-id=zoo --feed=https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/ 
pulp-admin rpm repo sync run --repo-id=zoo

3. Look in the system log for this error:

pulp.server.content.sources.model:ERROR: Refresh [content-world] url: https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/rpm/, failed: <mongoengine.fields.StringField object at 0x7fa3b448f190> is not JSON serializable

Note that if you then run another sync, the error does not appear. You have to delete and re-create the repo before the next sync if you want to re-produce the error.

#5 Updated by mhrivnak over 4 years ago

  • Status changed from ASSIGNED to POST

#6 Updated by mhrivnak over 4 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#7 Updated by mhrivnak over 4 years ago

  • Platform Release set to 2.8.6

#8 Updated by pthomas@redhat.com over 4 years ago

  • Status changed from MODIFIED to 6

verified


[root@~]# cat /etc/pulp/content/sources/conf.d/zoo.conf
[content-world]
enabled: 1
priority: 0
expires: 3d
name: Content World
type: yum
base_url: https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/
paths: rpm/
[root@ ~]# 
[i~]# 
[root@ ~]# pulp-admin rpm repo create --repo-id=zoo --feed=https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/ 
Successfully created repository [zoo]

[root@ ~]# pulp-admin rpm repo sync run --repo-id zoo 
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       32/32 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Cleaning duplicate packages...
[-]
... completed

Task Succeeded

Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
32 of 32 items
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[==================================================] 100%
4 of 4 items
... completed

Publishing Comps file
[==================================================] 100%
3 of 3 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed

Task Succeeded

[root@ibm-x3550m3-11 ~]# 

Jun 24 14:55:23 ibm-x3550m3-11.lab.eng.brq.redhat.com pulp[15977]: pulp.server.content.sources.model:INFO: Refreshing [content-world] url:https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/rpm/

Jun 24 14:55:28 ibm-x3550m3-11.lab.eng.brq.redhat.com pulp[15977]: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: The content container reported: {'downloads': {'content-world': {'total_failed': 0, 'total_succeeded': 32}}, 'total_sources': 1} for base URL: https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/

#9 Updated by semyers over 4 years ago

  • Status changed from 6 to 5

#10 Updated by semyers over 4 years ago

  • Status changed from 5 to 6

I accidentally moved this from VERIFIED back to ON_QA. Sorry for the ticket noise!

#11 Updated by semyers over 4 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE

#12 Updated by semyers over 4 years ago

  • Related to Task #2083: Issues common to 2.9.1 and 2.8 stream added

#14 Updated by bmbouter over 2 years ago

  • Sprint set to Sprint 4

#15 Updated by bmbouter over 2 years ago

  • Sprint/Milestone deleted (22)

#16 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF