Project

Profile

Help

Issue #4661

incremental publish of yum_repo_metadata_repo fails after upgrade from 2.18/2.19 to 2-master

Added by ttereshc 7 months ago. Updated 6 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Severity:
2. Medium
Version:
Master
Platform Release:
2.19.1
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 51

Description

Steps to reproduce:
1. use Pulp 2.18
2. sync repo with yum_repo_metadata_file (e.g any RHEL repo contains productid which is of the type we need)
3. publish it (in the publish directory there should be a symlink to /var/lib/pulp/published/../../../productid)
4. upgrade to the 2-master (commit c86c4339b9c1b4f158af1e961e8d68492dd2a760)
5. upload any rpm to the repo (to make publish incremental)
6. publish the repo, see the error below:

Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096) Traceback (most recent call last):
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     R = retval = fun(*args, **kwargs)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 529, in __call__
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     return super(Task, self).__call__(*args, **kwargs)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 107, in __call__
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     return super(PulpTask, self).__call__(*args, **kwargs)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     return self.run(*args, **kwargs)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1110, in publish
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1207, in check_publish
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1259, in _do_publish
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 737, in wrap_f
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     return f(*args, **kwargs)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     return self._publisher.process_lifecycle()
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 572, in process_lifecycle
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     super(PluginStep, self).process_lifecycle()
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 163, in process_lifecycle
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     step.process()
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 239, in process
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     self._process_block(item=item)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 301, in _process_block
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     self.process_main(item=item)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/publish.py", line 527, in process_main
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     shutil.copy2(unit._storage_path, file_path)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/usr/lib64/python2.7/shutil.py", line 144, in copy2
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     copyfile(src, dst)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/usr/lib64/python2.7/shutil.py", line 83, in copyfile
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     raise Error("`%s` and `%s` are the same file" % (src, dst))
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096) Error: `/var/lib/pulp/content/units/yum_repo_metadata_file/46/f013ec598b38b306dfd761b41a3ebf1c496f09f440679a1d7b2d4188145fda/ba86625b825e4bea5f6ab2b3e83c2cb076087507815be7e35da6d8bf697829dd-productid.gz` and `/var/cache/pulp/reserved_resource_worker-0@pulp2.dev/7669f0fa-1fc0-49a9-b834-dda548f0da0f/repodata/ba86625b825e4bea5f6ab2b3e83c2cb076087507815be7e35da6d8bf697829dd-productid.gz` are the same file


Related issues

Copied to RPM Support - Test #4730: incremental publish of yum_repo_metadata_repo fails after upgrade from 2.18/2.19 to 2-master NEW Actions

Associated revisions

Revision 8a770299 View on GitHub
Added by dkliban@redhat.com 8 months ago

Problem: YumMetadataFile attributes are not preserved

Solution: use shutil.copy2() to copy YumMetadataFile content type

re: #4550
https://pulp.plan.io/issues/4550

Revision 24e86fd9 View on GitHub
Added by ttereshc 7 months ago

Fix incremental publish for yum_repo_metadata_file content

Old way of publishing yum_repo_metadata_file content was to create a symlink.
Recently yum_repo_metadata_file content started to be copied at publish time.
This fix affects upgrade cases, when yum_repo_metadata_file was published
the old way, but incremental publish happened in a new way.

closes #4661
https://pulp.plan.io/issues/4661

Revision 21d4e80a View on GitHub
Added by ttereshc 6 months ago

Fix incremental publish for yum_repo_metadata_file content

Old way of publishing yum_repo_metadata_file content was to create a symlink.
Recently yum_repo_metadata_file content started to be copied at publish time.
This fix affects upgrade cases, when yum_repo_metadata_file was published
the old way, but incremental publish happened in a new way.

closes #4661
https://pulp.plan.io/issues/4661

(cherry picked from commit 24e86fd9ff63dda285cc2faf44bec8c8811b116c)

History

#1 Updated by ttereshc 7 months ago

  • Version set to Master

#2 Updated by dkliban@redhat.com 7 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc
  • Triaged changed from No to Yes
  • Sprint set to Sprint 51

#3 Updated by dkliban@redhat.com 7 months ago

  • Status changed from ASSIGNED to POST

#4 Updated by ttereshc 7 months ago

  • Status changed from POST to MODIFIED

#5 Updated by bmbouter 7 months ago

  • Tags Pulp 2 added

#6 Updated by bherring 7 months ago

  • Copied to Test #4730: incremental publish of yum_repo_metadata_repo fails after upgrade from 2.18/2.19 to 2-master added

#7 Updated by dkliban@redhat.com 7 months ago

  • Platform Release set to 2.19.1

#8 Updated by dkliban@redhat.com 7 months ago

  • Sprint/Milestone set to 2.19.1

#9 Updated by ttereshc 6 months ago

#10 Updated by dkliban@redhat.com 6 months ago

  • Status changed from MODIFIED to ON_QA

#11 Updated by dkliban@redhat.com 6 months ago

  • Status changed from ON_QA to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF