Actions
Issue #6327
closed
directory modulemd and subdirectories are not re-created for capsules if pulp data was lost
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:
Description
Fails at publish-errata due to missing modulemd file(s).
Reproducible in pulp-2.21 2-master:
- sync rhel8-appstream
- remove /var/lib/pulp/content/units/modulemd
- resync
- FAIL at publish-errata
Using --force-full doesn't change the results.
Relevant output:
[vagrant@pulp2 ~]$ pulp-admin rpm repo sync run --repo-id rhel8-appstream
+----------------------------------------------------------------------+
Synchronizing Repository [rhel8-appstream]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request
Downloading metadata...
[|]
... completed
Downloading repository content...
[\]
[==================================================] 100%
RPMs: 307/307 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing modules...
[/]
... completed
Importing errata...
[\]
... completed
Importing package groups/categories...
[-]
... completed
Cleaning duplicate packages...
[|]
... completed
Task Succeeded
Copying files
[\]
... completed
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[==================================================] 100%
307 of 307 items
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[=========================================== ] 87%
313 of 359 items
Task Failed
[Errno 2] No such file or directory:
u'/var/lib/pulp/content/units/modulemd/1b/71ed97689b1f4d20436633be22139a6e7f67c0
598b11a139fd0b15b96d4259'
[vagrant@pulp2 ~]$ pulp-admin tasks details --task-id e8b5c055-3afa-4ff6-ab1f-b9d0594e35e3
Operations: publish
Resources: rhel8-appstream (repository)
State: Failed
Start Time: 2020-03-11T15:55:45Z
Finish Time: 2020-03-11T15:56:51Z
Result: N/A
Task Id: e8b5c055-3afa-4ff6-ab1f-b9d0594e35e3
Worker Name: reserved_resource_worker-0@pulp2.dev
Progress Report:
...
Description: Publishing Modules
Details:
Error Details:
Error: [Errno 2] No such file or directory:
u'/var/lib/pulp/content/units/modulemd/1b/71ed97689b1f4d2043663
3be22139a6e7f67c0598b11a139fd0b15b96d4259'
Traceback:
File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 240, in process self._process_block(item=item)
File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py", line 302, in _process_block
self.process_main(item=item)
File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/publish.py", line 894, in process_main with
open(item.storage_path) as fp_in:
Items Total: 210
Num Failures: 1
Num Processed: 2
Num Success: 1
State: FAILED
Step Id: 9357c273-1ecc-4f7a-a136-875482ab4195
Step Type: modules
...
Traceback: Traceback (most recent call last):
File
"/usr/lib/python2.7/site-packages/celery/app/trace.py",
line 367, in trace_task
R = retval = fun(*args, **kwargs)
File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py",
line 686, in __call__
return super(Task, self).__call__(*args, **kwargs)
File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py",
line 108, in __call__
return super(PulpTask, self).__call__(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/celery/app/trace.py",
line 622, in __protected_call__
return self.run(*args, **kwargs)
File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py",
line 1133, in publish
result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py",
line 1230, in check_publish
result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)
File "/home/vagrant/devel/pulp/server/pulp/server/controllers/repository.py",
line 1282, in _do_publish
publish_report = publish_repo(transfer_repo, conduit, call_config)
File "/home/vagrant/devel/pulp/server/pulp/server/async/tasks.py",
line 894, in wrap_f
return f(*args, **kwargs)
File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/distributor.py",
line 174, in publish_repo
return self._publisher.process_lifecycle()
File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py",
line 573, in process_lifecycle
super(PluginStep,self).process_lifecycle()
File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py",
line 164, in process_lifecycle
step.process()
File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py",
line 240, in process
self._process_block(item=item)
File "/home/vagrant/devel/pulp/server/pulp/plugins/util/publish_step.py",
line 302, in _process_block
self.process_main(item=item)
File "/home/vagrant/devel/pulp_rpm/plugins/pulp_rpm/plugins/distributors/yum/publish.py",
line 894, in process_main
with
open(item.storage_path) as fp_in: IOError: [Errno 2] No such
file or directory:
u'/var/lib/pulp/content/units/modulemd/1b/71ed97689b1f4d204366
33be22139a6e7f67c0598b11a139fd0b15b96d4259'
Updated by ggainey about 4 years ago
Requesting a full sync with validation fixes the missing-modulemd issue.
Put the following JSON:
{
"validate":true
}
as file /etc/pulp/server/plugins.conf.d/yum_importer.json, restart pulp, and then execute pulp-admin rpm repo sync run --force-full --repo-id <affected-repo-id>. The sync runs to completion and the appropriate modulemd files are re-downloaded to the right place(s).
Updated by ggainey about 4 years ago
- Status changed from NEW to CLOSED - NOTABUG
Closing as NOTABUG. Pulp2 cannot prevent the user from inadvertently destroying /var/lib/pulp, and it already has code available that will try to fix things when that happens, as described in #c1. That code is not the default, because invoking it has a significant performance penalty on all syncs.
Actions
.