Issue #6096
closedAfter using the /associate endpoint, publish of the new repository fails
Description
If you generate a new repository using the associate command, and then attempt to publish that repository, it fails with the following stacktrace as a result of ModulemdDefaults in the new repo not having a checksum:
Jan 31 21:12:20 pulp2.dev pulp[11905]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[8e1ad085-9190-48d8-8b3b-4dd18d00813f]
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) Exception caught from plugin during publish for repo [destination5]
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) Traceback (most recent call last):
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1259, in _do_publish
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) publish_report = publish_repo(transfer_repo, conduit, call_config)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 894, in wrap_f
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) return f(*args, **kwargs)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) return self._publisher.process_lifecycle()
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 573, in process_lifecycle
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) super(PluginStep, self).process_lifecycle()
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 164, in process_lifecycle
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) step.process()
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 240, in process
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) self._process_block(item=item)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 302, in _process_block
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) self.process_main(item=item)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/publish.py", line 896, in process_main
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) self.context.add_document(document, doc_checksum=item.checksum)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/metadata/modules.py", line 37, in add_document
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) raise PulpCodedException(error_codes.RPM1017)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) PulpCodedException: Publish modules.yaml failed: modulemd checksum mismatch detected.
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.async.tasks:INFO: [8d22564d] Task failed : [8d22564d-842a-48d5-870a-db4b166c5614] : Publish modules.yaml failed: modulemd checksum mismatch detected.
You can reproduce this behavior with the following steps:
export BASE1='foo'
export BASE2='bar'
export REMOTE1='https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules-modified/'
export REMOTE2='https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned-modified/'
export DEST='destination'
export ERRATUM='RHEA-2012:0059'
pulp-admin rpm repo create --repo-id=$BASE1 --relative-url=$BASE1 --feed=$REMOTE1
pulp-admin rpm repo sync run --repo-id=$BASE1
pulp-admin rpm repo create --repo-id=$BASE2 --relative-url=$BASE2 --feed=$REMOTE2
pulp-admin rpm repo sync run --repo-id=$BASE2
pulp-admin rpm repo create --repo-id=$DEST
TASK_OUTPUT=$(curl -k -u admin:admin --cert ~/.pulp/user-cert.pem \
-d "{\"source_repo_id\":\"$BASE1\",\"criteria\":{\"type_ids\":[\"erratum\"],\"filters\":{\"unit\":{\"id\":\"$ERRATUM\"}}},\"override_config\":{\"recursive_conservative\":true,\"additional_repos\":{\"$BASE2\": \"$DEST\"}}}" \
-H "Content-Type: application/json" \
-X POST https://localhost/pulp/api/v2/repositories/$DEST/actions/associate/)
TASK_URL=$(echo $TASK_OUTPUT | jq -r '.spawned_tasks[]._href')
echo "Task url : " $TASK_URL
if [ -z "$TASK_URL" ]; then echo "NO TASK?!?"; exit; fi
wait_until_task_finished https://localhost$TASK_URL
pulp-admin rpm repo publish run --repo-id $DEST
Added by ggainey almost 5 years ago
Updated by ggainey almost 5 years ago
- Status changed from POST to MODIFIED
Applied in changeset 4c6be7c35494a7a1fa08b899293547d48495ed5f.
Added by ggainey over 4 years ago
Revision ba20df3a | View on GitHub
Using the /associate endpoint in a way that pulls modules into the detsination breaks at publish-time due to missing checksum on the resulting ModulemdDefaults unit.
Fixed by noticing and fixing the problem at associate-time.
closes #6096 https://pulp.plan.io/issues/6096
(cherry picked from commit 4c6be7c35494a7a1fa08b899293547d48495ed5f)
Updated by ggainey over 4 years ago
Applied in changeset ba20df3ae902d7b4267df180aa22ad968bbe1277.
Updated by ipanova@redhat.com over 4 years ago
- Status changed from MODIFIED to 5
Updated by ipanova@redhat.com over 4 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
Using the /associate endpoint in a way that pulls modules into the detsination breaks at publish-time due to missing checksum on the resulting ModulemdDefaults unit.
Fixed by noticing and fixing the problem at associate-time.
closes #6096 https://pulp.plan.io/issues/6096