Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #4661

closed

incremental publish of yum_repo_metadata_repo fails after upgrade from 2.18/2.19 to 2-master

Added by ttereshc over 5 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
ttereshc
Sprint/Milestone:
Pulp - 2.19.1
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Master
Platform Release:
2.19.1
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 51
Quarter:

Description

Steps to reproduce:
1. use Pulp 2.18
2. sync repo with yum_repo_metadata_file (e.g any RHEL repo contains productid which is of the type we need)
3. publish it (in the publish directory there should be a symlink to /var/lib/pulp/published/../../../productid)
4. upgrade to the 2-master (commit c86c4339b9c1b4f158af1e961e8d68492dd2a760)
5. upload any rpm to the repo (to make publish incremental)
6. publish the repo, see the error below:

Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096) Traceback (most recent call last):
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     R = retval = fun(*args, **kwargs)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 529, in __call__
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     return super(Task, self).__call__(*args, **kwargs)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 107, in __call__
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     return super(PulpTask, self).__call__(*args, **kwargs)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     return self.run(*args, **kwargs)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1110, in publish
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1207, in check_publish
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py", line 1259, in _do_publish
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py", line 737, in wrap_f
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     return f(*args, **kwargs)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     return self._publisher.process_lifecycle()
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 572, in process_lifecycle
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     super(PluginStep, self).process_lifecycle()
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 163, in process_lifecycle
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     step.process()
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 239, in process
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     self._process_block(item=item)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 301, in _process_block
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     self.process_main(item=item)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/publish.py", line 527, in process_main
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     shutil.copy2(unit._storage_path, file_path)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/usr/lib64/python2.7/shutil.py", line 144, in copy2
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     copyfile(src, dst)
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)   File "/usr/lib64/python2.7/shutil.py", line 83, in copyfile
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096)     raise Error("`%s` and `%s` are the same file" % (src, dst))
Apr 10 14:45:22 pulp2.dev pulp[4045]: celery.app.trace:ERROR: [7669f0fa] (4045-84096) Error: `/var/lib/pulp/content/units/yum_repo_metadata_file/46/f013ec598b38b306dfd761b41a3ebf1c496f09f440679a1d7b2d4188145fda/ba86625b825e4bea5f6ab2b3e83c2cb076087507815be7e35da6d8bf697829dd-productid.gz` and `/var/cache/pulp/reserved_resource_worker-0@pulp2.dev/7669f0fa-1fc0-49a9-b834-dda548f0da0f/repodata/ba86625b825e4bea5f6ab2b3e83c2cb076087507815be7e35da6d8bf697829dd-productid.gz` are the same file

Related issues

Copied to RPM Support - Test #4730: incremental publish of yum_repo_metadata_repo fails after upgrade from 2.18/2.19 to 2-masterCLOSED - WONTFIXActions

Added by dkliban@redhat.com over 5 years ago

Revision 8a770299 | View on GitHub

Problem: YumMetadataFile attributes are not preserved

Solution: use shutil.copy2() to copy YumMetadataFile content type

re: #4550 https://pulp.plan.io/issues/4550

Actions
Copy link
 #1

Updated by ttereshc over 5 years ago

  • Version set to Master
Actions
Copy link
 #2

Updated by dkliban@redhat.com over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc
  • Triaged changed from No to Yes
  • Sprint set to Sprint 51

Added by ttereshc over 5 years ago

Revision 24e86fd9 | View on GitHub

Fix incremental publish for yum_repo_metadata_file content

Old way of publishing yum_repo_metadata_file content was to create a symlink. Recently yum_repo_metadata_file content started to be copied at publish time. This fix affects upgrade cases, when yum_repo_metadata_file was published the old way, but incremental publish happened in a new way.

closes #4661 https://pulp.plan.io/issues/4661

Actions
Copy link
 #3

Updated by dkliban@redhat.com over 5 years ago

  • Status changed from ASSIGNED to POST
Actions
Copy link
 #4

Updated by ttereshc over 5 years ago

  • Status changed from POST to MODIFIED
Actions
Copy link
 #5

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added
Actions
Copy link
 #6

Updated by bherring over 5 years ago

  • Copied to Test #4730: incremental publish of yum_repo_metadata_repo fails after upgrade from 2.18/2.19 to 2-master added
Actions
Copy link
 #7

Updated by dkliban@redhat.com over 5 years ago

  • Platform Release set to 2.19.1
Actions
Copy link
 #8

Updated by dkliban@redhat.com over 5 years ago

  • Sprint/Milestone set to 2.19.1

Added by ttereshc over 5 years ago

Revision 21d4e80a | View on GitHub

Fix incremental publish for yum_repo_metadata_file content

Old way of publishing yum_repo_metadata_file content was to create a symlink. Recently yum_repo_metadata_file content started to be copied at publish time. This fix affects upgrade cases, when yum_repo_metadata_file was published the old way, but incremental publish happened in a new way.

closes #4661 https://pulp.plan.io/issues/4661

(cherry picked from commit 24e86fd9ff63dda285cc2faf44bec8c8811b116c)

Actions
Copy link
 #9

Updated by ttereshc over 5 years ago

Actions
Copy link
 #10

Updated by dkliban@redhat.com over 5 years ago

  • Status changed from MODIFIED to 5
Actions
Copy link
 #11

Updated by dkliban@redhat.com over 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions
Send by e-mail Copy link

Also available in: Atom PDF