Project

Profile

Help

Issue #5989

closed

Issue when syncing from Pulp to Pulp

Added by toffe about 4 years ago. Updated about 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
CentOS 7
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 66
Quarter:

Description

Hi

I am experiencing an issue regarding the .treeinfo file when doing sync from one Pulp3 install to another.

I have a repo for Centos7, this is a complete mirror of an official Centos 7 repo. It creates the following .treeinfo:

-----------------------------------------------
[general]
arch = x86_64
family = CentOS
name = CentOS 7
packagedir = .
platforms = x86_64, xen
repository = .
timestamp = 1567770561.0
variant = CentOS
version = 7

[header]
type = productmd.treeinfo
version = 1.2

[images- xen]
initrd = images/pxeboot/initrd.img
kernel = images/pxeboot/vmlinuz

[images-x86_64]
boot.iso = images/boot.iso
initrd = images/pxeboot/initrd.img
kernel = images/pxeboot/vmlinuz

[release]
is_layered = False
name = CentOS
short = CentOS
version = 7

[stage2]
mainimage = LiveOS/squashfs.img

[tree]
arch = x86_64
build_timestamp = 1567770561.0
platforms = x86_64, xen
variants = CentOS

[variant-CentOS]
id = CentOS
name = CentOS
packages = .
repository = .
type = variant
uid = CentOS


take notice of the timestamp = 1567770561.0 and build_timestamp = 1567770561.0.

When running a sync from another Pulp3 install in our environment towards this repo, I get the following error:

-----------------------------------------------------------------
description: 'invalid literal for int() with base 10: ''1567770561.0'''
traceback: |2-
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 822, in perform_job
rv = job.perform()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 605, in perform
self._result = self._execute()
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 611, in _execute
return self.func(*self.args, **self.kwargs)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 125, in synchronize
treeinfo = get_treeinfo_data(remote)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 33, in get_treeinfo_data
treeinfo.load(f=result.path)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 185, in load
super().load(f)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/productmd/common.py", line 263, in load
self.deserialize(parser)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 207, in deserialize
current.deserialize(parser)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/productmd/treeinfo.py", line 316, in deserialize
self.deserialize_1_0(parser)
File "/usr/local/lib/pulp/lib64/python3.6/site-packages/productmd/treeinfo.py", line 341, in deserialize_1_0
self.build_timestamp = parser.getint(self._section, "build_timestamp")
File "/usr/lib64/python3.6/configparser.py", line 819, in getint
fallback=fallback, **kwargs)
File "/usr/lib64/python3.6/configparser.py", line 809, in _get_conv
**kwargs)
File "/usr/lib64/python3.6/configparser.py", line 803, in _get
return conv(self.get(section, option, **kwargs))


When looking at the Timestamp value of the originating external repository, the timestamp value is "timestamp = 1567770561.47"

Summary:

External repo (timestamp = 1567770561.47) -> SYNC -> internal repo 1 (timestamp = 1567770561.0) -> SYNC -> internal repo 2 FAILS.

Actions #1

Updated by fao89 about 4 years ago

  • Project changed from Pulp to RPM Support
  • Triaged changed from No to Yes
  • Sprint set to Sprint 64
Actions #2

Updated by fao89 about 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fao89
Actions #3

Updated by fao89 about 4 years ago

  • Status changed from ASSIGNED to POST
Actions #4

Updated by rchan about 4 years ago

  • Sprint changed from Sprint 64 to Sprint 65

Added by Fabricio Aguiar about 4 years ago

Revision b0628307 | View on GitHub

Stop creating treeinfo file

productmd instroduces some data when parsing a treeinfo, it sets the header version as 1.2 for example. Creating a treeinfo file could lead to some errors, so with this commit we stop to create a treeinfo file and just store the original treeinfo file from the repository link.

https://pulp.plan.io/issues/5989 closes #5989

Actions #5

Updated by rchan about 4 years ago

  • Sprint changed from Sprint 65 to Sprint 66

Added by Fabricio Aguiar about 4 years ago

Revision 2ef95c99 | View on GitHub

Handling float timestamp on treeinfo file

https://pulp.plan.io/issues/5989 closes #5989

Actions #6

Updated by Anonymous about 4 years ago

  • Status changed from POST to MODIFIED
Actions #7

Updated by Anonymous about 4 years ago

Actions #8

Updated by ttereshc about 4 years ago

  • Sprint/Milestone set to Pulp RPM 3.2.0
Actions #9

Updated by ttereshc about 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF