Project

Profile

Help

Issue #2912

closed

parsing manifest fails during sync of file repo

Added by dkliban@redhat.com over 7 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 22
Quarter:

Description

[vagrant@dev devel]$ http POST http://localhost:1234/api/v3/repositories/test-repo/importers/file/test-importer/sync/
2017-07-15 22:12:18,525: ERROR/PoolWorker-1] Task failed : [f53993da-c30d-4ec9-b299-1163a1d02394]
2017-07-15 22:12:18,555: ERROR/PoolWorker-1] Task pulpcore.app.tasks.importer.sync[f53993da-c30d-4ec9-b299-1163a1d02394] raised unexpected: ValueError("invalid literal for int() with base 10: '05afba0da8b8c7d52cad9e04355d3efa30696936767d9cb0130c90c6515d934b'",)
Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/pulp/lib/python3.5/site-packages/celery/app/trace.py", line 367, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/vagrant/devel/pulp/platform/pulpcore/tasking/tasks.py", line 272, in __call__
    return super(UserFacingTask, self).__call__(*args, **kwargs)
  File "/home/vagrant/.virtualenvs/pulp/lib/python3.5/site-packages/celery/app/trace.py", line 622, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/vagrant/devel/pulp/platform/pulpcore/app/tasks/importer.py", line 75, in sync
    importer.sync()
  File "/home/vagrant/devel/pulp_file/pulp_file/app/models.py", line 85, in sync
    changeset = self._build_changeset()
  File "/home/vagrant/devel/pulp_file/pulp_file/app/models.py", line 226, in _build_changeset
    delta = self._find_delta(manifest, inventory) 
  File "/home/vagrant/devel/pulp_file/pulp_file/app/models.py", line 150, in _find_delta
    for entry in manifest.read():
  File "/home/vagrant/devel/pulp_file/pulp_file/manifest.py", line 92, in read
    yield Entry.parse(Line(number=n, content=line))
  File "/home/vagrant/devel/pulp_file/pulp_file/manifest.py", line 45, in parse
    size=int(part[1]),
ValueError: invalid literal for int() with base 10: '05afba0da8b8c7d52cad9e04355d3efa30696936767d9cb0130c90c6515d934b'

The manifest parsing is out of order. Following diff solves the issue:

[dkliban@localhost pulp_file]$ git diff
diff --git a/pulp_file/manifest.py b/pulp_file/manifest.py
index 3a83376..0a8e744 100644
--- a/pulp_file/manifest.py
+++ b/pulp_file/manifest.py
@@ -40,8 +40,8 @@ class Entry:
                   'must be: <path>, <size>, <digest>').format(
                     n=line.number))
         return Entry(path=part[0],
-                     size=int(part[1]),
-                     digest=part[2])
+                     size=int(part[2]),
+                     digest=part[1])

     def __init__(self, path, size, digest):
Actions #1

Updated by jortel@redhat.com over 7 years ago

We identified in the last MVP that the file plugin manifest format size and digest field ordering need to be switched to support pulp2 PULP_MANIFEST files. Should be <path>,<digest>,<size>.

Actions #2

Updated by jortel@redhat.com over 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jortel@redhat.com
  • Sprint/Milestone set to 41
  • Triaged changed from No to Yes

Added by jortel@redhat.com over 7 years ago

Revision 98e94ec6 | View on GitHub

Align manifest field ordering with pulp2 PULP_MANIFEST. closes #2912

Actions #3

Updated by mhrivnak over 7 years ago

Just because I like primary sources, here are the docs defining the PULP_MANIFEST:

http://docs.pulpproject.org/plugins/pulp_rpm/tech-reference/iso-plugins.html?highlight=pulp_manifest#iso-importer

Actions #4

Updated by jortel@redhat.com over 7 years ago

  • Project changed from Pulp to File Support
Actions #5

Updated by jortel@redhat.com over 7 years ago

  • Status changed from ASSIGNED to MODIFIED
Actions #7

Updated by bmbouter about 7 years ago

  • Tags deleted (Pulp 3 Plugin Writer Alpha)

Cleaning up Redmine tags

Actions #8

Updated by bmbouter almost 7 years ago

  • Sprint set to Sprint 22
Actions #9

Updated by bmbouter almost 7 years ago

  • Sprint/Milestone deleted (41)
Actions #10

Updated by bmbouter over 5 years ago

  • Tags deleted (Pulp 3)
Actions #11

Updated by bmbouter about 5 years ago

  • Sprint/Milestone set to 0.1.0
Actions #12

Updated by bmbouter about 5 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF