Issue #5989
closedIssue when syncing from Pulp to Pulp
Description
Hi
I am experiencing an issue regarding the .treeinfo file when doing sync from one Pulp3 install to another.
I have a repo for Centos7, this is a complete mirror of an official Centos 7 repo. It creates the following .treeinfo:
-----------------------------------------------
[general]
arch = x86_64
family = CentOS
name = CentOS 7
packagedir = .
platforms = x86_64, xen
repository = .
timestamp = 1567770561.0
variant = CentOS
version = 7
[header]
type = productmd.treeinfo
version = 1.2
[images- xen]
initrd = images/pxeboot/initrd.img
kernel = images/pxeboot/vmlinuz
[images-x86_64]
boot.iso = images/boot.iso
initrd = images/pxeboot/initrd.img
kernel = images/pxeboot/vmlinuz
[release]
is_layered = False
name = CentOS
short = CentOS
version = 7
[stage2]
mainimage = LiveOS/squashfs.img
[tree]
arch = x86_64
build_timestamp = 1567770561.0
platforms = x86_64, xen
variants = CentOS
[variant-CentOS]
id = CentOS
name = CentOS
packages = .
repository = .
type = variant
uid = CentOS
take notice of the timestamp = 1567770561.0 and build_timestamp = 1567770561.0.
When running a sync from another Pulp3 install in our environment towards this repo, I get the following error:
-----------------------------------------------------------------
description: 'invalid literal for int() with base 10: ''1567770561.0'''
traceback: |2-
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 822, in perform_job
rv = job.perform()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 605, in perform
self._result = self._execute()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 611, in _execute
return self.func(*self.args, **self.kwargs)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 125, in synchronize
treeinfo = get_treeinfo_data(remote)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 33, in get_treeinfo_data
treeinfo.load(f=result.path)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 185, in load
super().load(f)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/productmd/common.py", line 263, in load
self.deserialize(parser)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 207, in deserialize
current.deserialize(parser)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/productmd/treeinfo.py", line 316, in deserialize
self.deserialize_1_0(parser)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/productmd/treeinfo.py", line 341, in deserialize_1_0
self.build_timestamp = parser.getint(self._section, "build_timestamp")
File "/usr/lib64/python3.6/configparser.py", line 819, in getint
fallback=fallback, **kwargs)
File "/usr/lib64/python3.6/configparser.py", line 809, in _get_conv
**kwargs)
File "/usr/lib64/python3.6/configparser.py", line 803, in _get
return conv(self.get(section, option, **kwargs))
When looking at the Timestamp value of the originating external repository, the timestamp value is "timestamp = 1567770561.47"
Summary:
External repo (timestamp = 1567770561.47) -> SYNC -> internal repo 1 (timestamp = 1567770561.0) -> SYNC -> internal repo 2 FAILS.
Stop creating treeinfo file
productmd instroduces some data when parsing a treeinfo, it sets the header version as 1.2 for example. Creating a treeinfo file could lead to some errors, so with this commit we stop to create a treeinfo file and just store the original treeinfo file from the repository link.
https://pulp.plan.io/issues/5989 closes #5989