Project

Profile

Help

Issue #6096

closed

After using the /associate endpoint, publish of the new repository fails

Added by ggainey almost 5 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.21.0
Platform Release:
2.21.1
OS:
Triaged:
Yes
Groomed:
Yes
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 66
Quarter:

Description

If you generate a new repository using the associate command, and then attempt to publish that repository, it fails with the following stacktrace as a result of ModulemdDefaults in the new repo not having a checksum:

Jan 31 21:12:20 pulp2.dev pulp[11905]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[8e1ad085-9190-48d8-8b3b-4dd18d00813f]
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) Exception caught from plugin during publish for repo [destination5]
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) Traceback (most recent call last):
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1259, in _do_publish
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 894, in wrap_f
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)     return f(*args, **kwargs)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)     return self._publisher.process_lifecycle()
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 573, in process_lifecycle
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)     super(PluginStep, self).process_lifecycle()
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 164, in process_lifecycle
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)     step.process()
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 240, in process
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)     self._process_block(item=item)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 302, in _process_block
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)     self.process_main(item=item)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/publish.py", line 896, in process_main
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)    self.context.add_document(document, doc_checksum=item.checksum)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/metadata/modules.py", line 37, in add_document
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624)     raise PulpCodedException(error_codes.RPM1017)
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.controllers.repository:ERROR: [8d22564d] (13062-22624) PulpCodedException: Publish modules.yaml failed: modulemd checksum mismatch detected.
Jan 31 21:12:20 pulp2.dev pulp[13062]: pulp.server.async.tasks:INFO: [8d22564d] Task failed : [8d22564d-842a-48d5-870a-db4b166c5614] : Publish modules.yaml failed: modulemd checksum mismatch detected.

You can reproduce this behavior with the following steps:

export BASE1='foo'
export BASE2='bar'
export REMOTE1='https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-modules-modified/'
export REMOTE2='https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-unsigned-modified/'
export DEST='destination'
export ERRATUM='RHEA-2012:0059'
pulp-admin rpm repo create --repo-id=$BASE1 --relative-url=$BASE1 --feed=$REMOTE1
pulp-admin rpm repo sync run --repo-id=$BASE1
pulp-admin rpm repo create --repo-id=$BASE2 --relative-url=$BASE2 --feed=$REMOTE2
pulp-admin rpm repo sync run --repo-id=$BASE2
pulp-admin rpm repo create --repo-id=$DEST
TASK_OUTPUT=$(curl -k -u admin:admin --cert ~/.pulp/user-cert.pem \
         -d "{\"source_repo_id\":\"$BASE1\",\"criteria\":{\"type_ids\":[\"erratum\"],\"filters\":{\"unit\":{\"id\":\"$ERRATUM\"}}},\"override_config\":{\"recursive_conservative\":true,\"additional_repos\":{\"$BASE2\": \"$DEST\"}}}" \
         -H "Content-Type: application/json" \
         -X POST https://localhost/pulp/api/v2/repositories/$DEST/actions/associate/)
TASK_URL=$(echo $TASK_OUTPUT | jq -r '.spawned_tasks[]._href')
echo "Task url : " $TASK_URL
if [ -z "$TASK_URL" ]; then echo "NO TASK?!?"; exit; fi
wait_until_task_finished https://localhost$TASK_URL
pulp-admin rpm repo publish run --repo-id $DEST

Actions #1

Updated by ggainey almost 5 years ago

  • Status changed from ASSIGNED to POST
Actions #2

Updated by rchan almost 5 years ago

  • Sprint changed from Sprint 65 to Sprint 66

Added by ggainey almost 5 years ago

Revision 4c6be7c3 | View on GitHub

Using the /associate endpoint in a way that pulls modules into the detsination breaks at publish-time due to missing checksum on the resulting ModulemdDefaults unit.

Fixed by noticing and fixing the problem at associate-time.

closes #6096 https://pulp.plan.io/issues/6096

Actions #4

Updated by ggainey almost 5 years ago

  • Status changed from POST to MODIFIED
Actions #5

Updated by ipanova@redhat.com almost 5 years ago

  • Platform Release set to 2.21.1

Added by ggainey over 4 years ago

Revision ba20df3a | View on GitHub

Using the /associate endpoint in a way that pulls modules into the detsination breaks at publish-time due to missing checksum on the resulting ModulemdDefaults unit.

Fixed by noticing and fixing the problem at associate-time.

closes #6096 https://pulp.plan.io/issues/6096

(cherry picked from commit 4c6be7c35494a7a1fa08b899293547d48495ed5f)

Actions #6

Updated by ggainey over 4 years ago

Actions #7

Updated by ipanova@redhat.com over 4 years ago

  • Status changed from MODIFIED to 5
Actions #8

Updated by ipanova@redhat.com over 4 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

Also available in: Atom PDF