Actions
Issue #8116
closedpulp-import relying on 'next_version' in ways it can't, causing import to fail.
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Quarter:
Description
- Create a repo
- upload a package -> this should generate version 1
- remove a package -> this should generate version 2
- optionally upload another package -> this should generate version 3
- Now export version 1
- copy and extract the generated tar gz
$ cat pulpcore.app.modelresource.RepositoryResource.json [{"pulp_type": "rpm.rpm", "name": "life-531063", "description": "", "next_version": 4, "remote": ""}]
- do an ls
$ ls|grep repo repository-life-531063_1
so notice that next version is pointing to 4 while the directory name ends with _1
- Now wipe the db
- Create a new yum repository called foo
- Create an importer with the mapping ->
{'foo':"life-531063"}
- Try to import the exported tgz
- The operation should fail
- You should see an error along the lines of
an 18 19:57:17 pulpcore-worker-2[1770]: pulp: pulpcore.app.tasks.importer:INFO: Importing file /tmp/tmp47swboli/repository-life-531063_3/pulp_rpm.app.modelresource.PackageResource.json. Jan 18 19:57:17 pulpcore-worker-2[1770]: pulp: rq.worker:ERROR: FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp47swboli/repository-life-531063_3/pulp_rpm.app.modelresource.PackageResource.json' Jan 18 19:57:17 pulpcore-worker-2[1770]: Traceback (most recent call last): Jan 18 19:57:17 pulpcore-worker-2[1770]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 883, in perform_job Jan 18 19:57:17 pulpcore-worker-2[1770]: rv = job.perform() Jan 18 19:57:17 pulpcore-worker-2[1770]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 645, in perform Jan 18 19:57:17 pulpcore-worker-2[1770]: self._result = self._execute() Jan 18 19:57:17 pulpcore-worker-2[1770]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 651, in _execute Jan 18 19:57:17 pulpcore-worker-2[1770]: return self.func(*self.args, **self.kwargs) Jan 18 19:57:17 pulpcore-worker-2[1770]: File "/usr/local/lib/python3.6/site-packages/pulpcore/app/tasks/importer.py", line 150, in import_repository_version Jan 18 19:57:17 pulpcore-worker-2[1770]: a_result = _import_file(os.path.join(rv_path, filename), res_class) Jan 18 19:57:17 pulpcore-worker-2[1770]: File "/usr/local/lib/python3.6/site-packages/pulpcore/app/tasks/importer.py", line 57, in _import_file Jan 18 19:57:17 pulpcore-worker-2[1770]: with open(fpath, "r") as json_file: Jan 18 19:57:17 pulpcore-worker-2[1770]: FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp47swboli/repository-life-531063_3/pulp_rpm.app.modelresource.PackageResource.json' Jan 18 19:57:17 pulpcore-worker-2[1770]: Traceback (most recent call last): Jan 18 19:57:17 pulpcore-worker-2[1770]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 883, in perform_job Jan 18 19:57:17 pulpcore-worker-2[1770]: rv = job.perform() Jan 18 19:57:17 pulpcore-worker-2[1770]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 645, in perform Jan 18 19:57:17 pulpcore-worker-2[1770]: self._result = self._execute() Jan 18 19:57:17 pulpcore-worker-2[1770]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 651, in _execute Jan 18 19:57:17 pulpcore-worker-2[1770]: return self.func(*self.args, **self.kwargs) Jan 18 19:57:17 pulpcore-worker-2[1770]: File "/usr/local/lib/python3.6/site-packages/pulpcore/app/tasks/importer.py", line 150, in import_repository_version Jan 18 19:57:17 pulpcore-worker-2[1770]: a_result = _import_file(os.path.join(rv_path, filename), res_class) Jan 18 19:57:17 pulpcore-worker-2[1770]: File "/usr/local/lib/python3.6/site-packages/pulpcore/app/tasks/importer.py", line 57, in _import_file Jan 18 19:57:17 pulpcore-worker-2[1770]: with open(fpath, "r") as json_file: Jan 18 19:57:17 pulpcore-worker-2[1770]: FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp47swboli/repository-life-531063_3/pulp_rpm.app.modelresource.PackageResource.json'
- The logic for this error is here -> https://github.com/pulp/pulpcore/blob/master/pulpcore/app/tasks/importer.py#L71-L74 . The code looks for
repository-life-531063_3
when it should be looking atrepository-life-531063_1
Not sure if its the importer's fault or exporter's. But we definitely need to be able to handle this.
Files
Updated by paji@redhat.com almost 4 years ago
- File export-dump.tgz export-dump.tgz added
Also uploading my tgz
Updated by fao89 almost 4 years ago
- Triaged changed from No to Yes
- Sprint set to Sprint 88
Updated by fao89 almost 4 years ago
- Triaged changed from Yes to No
- Sprint deleted (
Sprint 88)
Updated by fao89 almost 4 years ago
- Triaged changed from No to Yes
- Sprint set to Sprint 89
Updated by ggainey almost 4 years ago
export is just recording what next_version is on the upstream machine. The problem is that import can't be relying on that for...anything, really, given how selective one can be on-export. Need to rethink the import-algorithm at this point.
Updated by daviddavis almost 4 years ago
- Sprint/Milestone set to 3.12.0
- Sprint deleted (
Sprint 90)
Updated by ggainey over 3 years ago
- Subject changed from Export is generating the incorrect 'next_version' causing import to fail. to pulp-import relying on 'next_version' in ways it can't, causing import to fail.
Updated subject to better reflect the problem. Also, to reproduce, no need to wipe the db, you can just attempt to import into a clean/new repository.
Updated by ggainey over 3 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ggainey
Updated by pulpbot over 3 years ago
- Status changed from ASSIGNED to POST
Added by ggainey over 3 years ago
Updated by ggainey over 3 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulpcore|b10ac5e801db6f35d6074eef0333e1c150c250be.
Updated by pulpbot over 3 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions
Taught PulpImport to not rely on next_version from upstream.
Also taught import and export tests to share more utility code.
fixes #8116