Issue #8400: Migration fails if there is .zck metadata in pulp2 - Migration Plugin - Pulp

Actions

Send by e-mail Copy link

Issue #8400

closed

Migration fails if there is .zck metadata in pulp2

Added by amesa-medina almost 4 years ago. Updated over 3 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

ttereshc

Sprint/Milestone:

0.11.1

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Platform Release:

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Katello

Sprint:

Sprint 96

Quarter:

Description

Hi,

After upgrade foreman and katello, (based on https://theforeman.org/plugins/katello/3.18/upgrade/index.html) the Pulp3 Migration failed.

With this error: ForemanTasks::TaskError: Task 3568639a-17d0-4198-a12a-8b0a7d29cfb0: Katello::Errors::Pulp3Error: 1 subtask(s) failed for task group /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/.

They, Foreman team, have already verified the logs and suggested me that I open a case with you.

pip3 list | grep pulp

pulp-2to3-migration (0.8.0)
pulp-certguard (1.0.3)
pulp-container (2.1.0)
pulp-deb (2.7.0)
pulp-file (1.3.0)
pulp-rpm (3.7.0)
pulpcore (3.7.3)

I show you here, fragments of /var/log/messages around the time of the last error:

…
Mar 11 15:07:10 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:10 +0000] “GET /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/ HTTP/1.1” 200 440 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:07:12 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:12 +0000] “GET /pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/ HTTP/1.1” 200 3960 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:07:12 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:12 +0000] “GET /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/ HTTP/1.1” 200 440 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:07:13 cpt-infra22l-d pulpcore-worker-1[25158]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without an entry in the lazy catalog, pulp2 unit_id: 6b7f5719-ff27-4246-b86b-b53fd996881b
Mar 11 15:07:13 cpt-infra22l-d pulpcore-worker-1[25158]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without an entry in the lazy catalog, pulp2 unit_id: 28d28583-a22b-43c2-b6d7-25263a1d36bf
Mar 11 15:07:13 cpt-infra22l-d pulpcore-worker-1[25158]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without an entry in the lazy catalog, pulp2 unit_id: 51f5d2fd-e1cc-4e19-9389-f4d7da27ee7b
… many similar lines …
Mar 11 15:07:18 cpt-infra22l-d pulpcore-worker-1[25158]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without an entry in the lazy catalog, pulp2 unit_id: edb8dd22-8893-48a2-bf83-7e5f4cc206f9
Mar 11 15:07:19 cpt-infra22l-d pulpcore-resource-manager[25633]: pulp: rq.worker:INFO: resource-manager: 66f04d27-c44d-4eab-b78e-079370434c02
Mar 11 15:07:19 cpt-infra22l-d pulpcore-worker-3[25160]: pulp: rq.worker:INFO: 25160@cpt-infra22l-d.acceo.com: 5e8e263c-daaa-4d45-857e-de4a1698aa51
Mar 11 15:07:19 cpt-infra22l-d pulpcore-resource-manager[25633]: pulp: rq.worker:INFO: resource-manager: Job OK (66f04d27-c44d-4eab-b78e-079370434c02)
Mar 11 15:07:19 cpt-infra22l-d pulpcore-resource-manager[25633]: pulp: rq.worker:INFO: resource-manager: 97a5e73c-794c-486d-8774-948bcd28559f
… many similar lines …
Mar 11 15:07:26 cpt-infra22l-d pulpcore-worker-2[25156]: pulp: rq.worker:INFO: 25156@cpt-infra22l-d.acceo.com: 584131d3-1f44-4927-af32-73bd6f432fe4
Mar 11 15:07:26 cpt-infra22l-d pulpcore-worker-2[25156]: pulp: rq.worker:INFO: 25156@cpt-infra22l-d.acceo.com: Job OK (584131d3-1f44-4927-af32-73bd6f432fe4)
Mar 11 15:07:27 cpt-infra22l-d pulpcore-worker-3[25160]: pulp: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=f5b6756b-bd2e-402f-8269-8847b8bea678, version=3
Mar 11 15:07:29 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:29 +0000] “GET /pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/ HTTP/1.1” 200 11587 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:07:29 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:29 +0000] “GET /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/ HTTP/1.1” 200 440 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:07:33 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:33 +0000] “GET /pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/ HTTP/1.1” 200 11587 “-” “OpenAPI-Generator/3.7.1/ruby”
… many similar lines …
Mar 11 15:08:30 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:08:30 +0000] “GET /pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/ HTTP/1.1” 200 11587 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:08:30 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:08:30 +0000] “GET /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/ HTTP/1.1” 200 440 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: pulp: rq.worker:ERROR: Traceback (most recent call last):
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py”, line 78, in migrate_to_pulp3
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: TypeError: publish() got an unexpected keyword argument ‘sqlite_metadata’
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: During handling of the above exception, another exception occurred:
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: Traceback (most recent call last):
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 936, in perform_job
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: rv = job.perform()
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 684, in perform
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: self._result = self._execute()
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 690, in _execute
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: return self.func(*self.args, **self.kwargs)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py”, line 240, in complex_repo_migration
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: migrated_repo.pulp3_repository_version
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py”, line 360, in migrate_repo_distributor
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: pulp2dist, repo_version)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py”, line 81, in migrate_to_pulp3
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: publish(repo_version.pk, checksum_types=checksum_types)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py”, line 290, in publish
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: metadata_signing_service=metadata_signing_service
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py”, line 483, in create_repomd_xml
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: record.fill(checksum_type)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: OSError: Error while computing stat of compressed content of updateinfo.xml.zck:Cannot open a file updateinfo.xml.zck: createrepo_c wasn’t compiled with zchunk support
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: Traceback (most recent call last):
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py”, line 78, in migrate_to_pulp3
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: TypeError: publish() got an unexpected keyword argument ‘sqlite_metadata’
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: During handling of the above exception, another exception occurred:
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: Traceback (most recent call last):
…
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: record.fill(checksum_type)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: OSError: Error while computing stat of compressed content of updateinfo.xml.zck:Cannot open a file updateinfo.xml.zck: createrepo_c wasn’t compiled with zchunk support
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: pulp: rq.worker:INFO: 25159@cpt-infra22l-d.acceo.com: 88d7a555-20bb-4ce3-9408-dd05005f9fcf
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: pulp: rq.worker:INFO: 25159@cpt-infra22l-d.acceo.com: Job OK (88d7a555-20bb-4ce3-9408-dd05005f9fcf)
Mar 11 15:08:46 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:08:46 +0000] “GET /pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/ HTTP/1.1” 200 11587 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:08:46 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:08:46 +0000] “GET /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/ HTTP/1.1” 200 440 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: pulp: rq.worker:ERROR: Traceback (most recent call last):
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py”, line 78, in migrate_to_pulp3
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: TypeError: publish() got an unexpected keyword argument ‘sqlite_metadata’
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: During handling of the above exception, another exception occurred:
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: Traceback (most recent call last):
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 936, in perform_job
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: rv = job.perform()
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 684, in perform
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: self._result = self._execute()
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 690, in _execute
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: return self.func(*self.args, **self.kwargs)
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py”, l
ine 240, in complex_repo_migration
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: migrated_repo.pulp3_repository_version
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py”, l
ine 360, in migrate_repo_distributor
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: pulp2dist, repo_version)
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repos
itory.py”, line 81, in migrate_to_pulp3
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: publish(repo_version.pk, checksum_types=checksum_types)
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py”, line
290, in publish
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: metadata_signing_service=metadata_signing_service
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py”, line
483, in create_repomd_xml
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: record.fill(checksum_type)
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: OSError: Error while computing stat of compressed content of updateinfo.xml.zck
:Cannot open a file updateinfo.xml.zck: createrepo_c wasn’t compiled with zchunk support
…

Thanks

Actions

Copy link

Updated by ttereshc almost 4 years ago

Description updated (diff)

pre'd

Actions

Copy link

Updated by ttereshc almost 4 years ago

@amesa-medina, what is the reason for this issue to be private? Can I make it public? If not, please, let me know which private information needs to be removed from it.

Actions

Copy link

Updated by ttereshc almost 4 years ago

Subject changed from Pulp3 migration failed to Migration fails if there is .zck metadata in pulp2

Actions

Copy link

Updated by amesa-medina almost 4 years ago

Hi. Ok, we can make it public, please.

Actions

Copy link

Updated by ttereshc almost 4 years ago

Private changed from Yes to No

Making the issue public based on the comment 4.

Actions

Copy link

Updated by ttereshc almost 4 years ago

Triaged changed from No to Yes

Actions

Copy link

Updated by ttereshc almost 4 years ago

Tags Katello added

Actions

Copy link

Updated by amesa-medina over 3 years ago

Hi, Since version 4 of Katello is already available, and the migration from pulp 2 to 3 is a requirement, do you have any news about this issue, please? Thanks.

Actions

Copy link

#10

Updated by florianfa over 3 years ago

Hello,

I'm facing the same issue: OSError: Error while computing stat of compressed content of updateinfo.xml.zck:Cannot open a file updateinfo.xml.zck: createrepo_c wasn't compiled with zchunk support

Version: pulp-2to3-migration (0.11.0)

createrepo_c Package:

Installed Packages
Name        : createrepo_c
Arch        : x86_64
Version     : 0.17.1
Release     : 1.el7
Size        : 159 k
Repo        : installed
From repo   : pulpcore
Summary     : Creates a common metadata repository
URL         : https://github.com/rpm-software-management/createrepo_c
License     : GPLv2+
Description : C implementation of Createrepo.
            : A set of utilities (createrepo_c, mergerepo_c, modifyrepo_c)
            : for generating a common metadata repository from a directory of
            : rpm packages and maintaining it.

pulpcore-worker-7: pulp: rq.worker:ERROR: Traceback (most recent call last): 2021-04-29T14:55:39.713275+02:00 lxcfg1 pulpcore-worker-7: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job 2021-04-29T14:55:39.713494+02:00 lxcfg1 pulpcore-worker-7: rv = job.perform() 2021-04-29T14:55:39.713706+02:00 lxcfg1 pulpcore-worker-7: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform 2021-04-29T14:55:39.713898+02:00 lxcfg1 pulpcore-worker-7: self._result = self._execute() 2021-04-29T14:55:39.714091+02:00 lxcfg1 pulpcore-worker-7: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute 2021-04-29T14:55:39.714280+02:00 lxcfg1 pulpcore-worker-7: return self.func(*self.args, **self.kwargs) 2021-04-29T14:55:39.714506+02:00 lxcfg1 pulpcore-worker-7: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 224, in complex_repo_migration 2021-04-29T14:55:39.714713+02:00 lxcfg1 pulpcore-worker-7: migrated_repo.pulp3_repository_version 2021-04-29T14:55:39.714909+02:00 lxcfg1 pulpcore-worker-7: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 371, in migrate_repo_distributor 2021-04-29T14:55:39.715127+02:00 lxcfg1 pulpcore-worker-7: pulp2dist, repo_version) 2021-04-29T14:55:39.715342+02:00 lxcfg1 pulpcore-worker-7: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py", line 78, in migrate_to_pulp3 2021-04-29T14:55:39.715553+02:00 lxcfg1 pulpcore-worker-7: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite) 2021-04-29T14:55:39.715758+02:00 lxcfg1 pulpcore-worker-7: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 320, in publish 2021-04-29T14:55:39.715951+02:00 lxcfg1 pulpcore-worker-7: metadata_signing_service=metadata_signing_service, 2021-04-29T14:55:39.716143+02:00 lxcfg1 pulpcore-worker-7: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 551, in create_repomd_xml 2021-04-29T14:55:39.716363+02:00 lxcfg1 pulpcore-worker-7: record.fill(checksum_type) 2021-04-29T14:55:39.716560+02:00 lxcfg1 pulpcore-worker-7: OSError: Error while computing stat of compressed content of updateinfo.xml.zck:Cannot open a file updateinfo.xml.zck: createrepo_c wasn't compiled with zchunk support

Florian

Actions

Copy link

#11

Updated by ttereshc over 3 years ago

Status changed from NEW to POST

https://github.com/pulp/pulp-2to3-migration/pull/360

Actions

Copy link

#12

Updated by ttereshc over 3 years ago

The fix should be included in Katello 3.18.z.
You would need to reset the migration (foreman-maintain content migration-reset) before running it again.

Actions

Copy link

#13