Actions
Issue #2820
closed
No such file or directory: pkgtags.sqlite.gz on repository publish
Added by jsherril@redhat.com almost 7 years ago. Updated about 5 years ago.
Status:
CLOSED - WORKSFORME
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.9.0
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:
Description
Katello user reported on pulp 2.9.3 hitting the following issue after syncing epel 7. Publishing the repo resulted in:
[Errno 2] No such file or directory: '/var/lib/pulp/content/units/yum_repo_metadata_file/72/c32e71f97dee222a4106cae699920730329ac8fb0ad7f32cc3347873ac77d0/e27de8e321f2f5ae59f788eb5290eddc5fe89f60ea46a92631360548f0c3f92c-pkgtags.sqlite.gz
He has since upgraded to 2.13.0 but it seemed that the problem still persisted. Its unclear how to get into this situation, but it is similar to those seen in the past so i figured an issue was worth filing. I will post the full sync and publish tasks.
If you feel this is resolved we can close, but have a recorded place to keep track of how often this is hit I thought was worthwhile.
Updated by jsherril@redhat.com almost 7 years ago
[{"exception"=>nil,
"task_type"=>"pulp.server.managers.repo.sync.sync",
"_href"=>"/pulp/api/v2/tasks/f9925bc8-b7bc-40d6-a0b2-722e965eb54b/",
"task_id"=>"f9925bc8-b7bc-40d6-a0b2-722e965eb54b",
"tags"=>["pulp:repository:hg-EPEL-EPEL6_x86_64", "pulp:action:sync"],
"finish_time"=>"2017-06-13T05:43:14Z",
"_ns"=>"task_status",
"start_time"=>"2017-06-13T05:42:22Z",
"traceback"=>nil,
"spawned_tasks"=>
[{"_href"=>"/pulp/api/v2/tasks/0060caa9-a574-4c9c-8dd1-cb47c598855b/",
"task_id"=>"0060caa9-a574-4c9c-8dd1-cb47c598855b"}],
"progress_report"=>
{"yum_importer"=>
{"content"=>
{"items_total"=>0,
"state"=>"FINISHED",
"error_details"=>[],
"details"=>
{"rpm_total"=>0, "rpm_done"=>0, "drpm_total"=>0, "drpm_done"=>0},
"size_total"=>0,
"size_left"=>0,
"items_left"=>0},
"comps"=>{"state"=>"FINISHED"},
"purge_duplicates"=>{"state"=>"FINISHED"},
"distribution"=>
{"items_total"=>0,
"state"=>"FINISHED",
"error_details"=>[],
"items_left"=>0},
"errata"=>{"state"=>"FINISHED"},
"metadata"=>{"state"=>"FINISHED"}}},
"queue"=>"reserved_resource_worker-1@katello.sopo.sucive.gub.uy.dq",
"state"=>"finished",
"worker_name"=>"reserved_resource_worker-1@katello.sopo.sucive.gub.uy",
"result"=>
{"result"=>"success",
"importer_id"=>"yum_importer",
"exception"=>nil,
"repo_id"=>"hg-EPEL-EPEL6_x86_64",
"traceback"=>nil,
"started"=>"2017-06-13T05:42:22Z",
"_ns"=>"repo_sync_results",
"completed"=>"2017-06-13T05:43:13Z",
"importer_type_id"=>"yum_importer",
"error_message"=>nil,
"summary"=>
{"content"=>{"state"=>"FINISHED"},
"comps"=>{"state"=>"FINISHED"},
"purge_duplicates"=>{"state"=>"FINISHED"},
"distribution"=>{"state"=>"FINISHED"},
"errata"=>{"state"=>"FINISHED"},
"metadata"=>{"state"=>"FINISHED"}},
"added_count"=>0,
"removed_count"=>0,
"updated_count"=>0,
"id"=>"593f7b727bdb3318d8ef8908",
"details"=>
{"content"=>
{"size_total"=>0,
"items_left"=>0,
"items_total"=>0,
"state"=>"FINISHED",
"size_left"=>0,
"details"=>
{"rpm_total"=>0, "rpm_done"=>0, "drpm_total"=>0, "drpm_done"=>0},
"error_details"=>[]},
"comps"=>{"state"=>"FINISHED"},
"purge_duplicates"=>{"state"=>"FINISHED"},
"distribution"=>
{"items_total"=>0,
"state"=>"FINISHED",
"error_details"=>[],
"items_left"=>0},
"errata"=>{"state"=>"FINISHED"},
"metadata"=>{"state"=>"FINISHED"}}},
"error"=>nil,
"_id"=>{"$oid"=>"593f7a0ceac8c8110c95137b"},
"id"=>"593f7a0ceac8c8110c95137b"},
{"exception"=>nil,
"task_type"=>"pulp.server.managers.repo.publish.publish",
"_href"=>"/pulp/api/v2/tasks/0060caa9-a574-4c9c-8dd1-cb47c598855b/",
"task_id"=>"0060caa9-a574-4c9c-8dd1-cb47c598855b",
"tags"=>["pulp:repository:hg-EPEL-EPEL6_x86_64", "pulp:action:publish"],
"finish_time"=>"2017-06-13T05:50:09Z",
"_ns"=>"task_status",
"start_time"=>"2017-06-13T05:43:49Z",
"traceback"=>
"Traceback (most recent call last):\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in trace_task\n R = retval = fun(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 506, in __call__\n return super(Task, self).__call__(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 107, in __call__\n return super(PulpTask, self).__call__(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 437, in __protected_call__\n return self.run(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 1106, in publish\n result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)\n File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 1198, in check_publish\n result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)\n File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 1250, in _do_publish\n publish_report = publish_repo(transfer_repo, conduit, call_config)\n File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 709, in wrap_f\n return f(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py\", line 174, in publish_repo\n return self._publisher.process_lifecycle()\n File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 572, in process_lifecycle\n super(PluginStep, self).process_lifecycle()\n File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 163, in process_lifecycle\n step.process()\n File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 239, in process\n self._process_block(item=item)\n File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 301, in _process_block\n self.process_main(item=item)\n File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py\", line 516, in process_main\n add_metadata_file_metadata(unit.data_type, link_path)\n File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py\", line 80, in add_metadata_file_metadata\n timestamp_element.text = str(int(os.path.getmtime(file_path)))\n File \"/usr/lib64/python2.7/genericpath.py\", line 54, in getmtime\n return os.stat(filename).st_mtime\nOSError: [Errno 2] No such file or directory: '/var/lib/pulp/content/units/yum_repo_metadata_file/72/c32e71f97dee222a4106cae699920730329ac8fb0ad7f32cc3347873ac77d0/e27de8e321f2f5ae59f788eb5290eddc5fe89f60ea46a92631360548f0c3f92c-pkgtags.sqlite.gz'\n",
"spawned_tasks"=>[],
"progress_report"=>
{"hg-EPEL-EPEL6_x86_64"=>
[{"num_success"=>1,
"description"=>"Copying files",
"step_type"=>"save_tar",
"items_total"=>1,
"state"=>"FINISHED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"4411bde8-6834-466e-8ed6-b069f8cf11da",
"num_processed"=>1},
{"num_success"=>1,
"description"=>"Initializing repo metadata",
"step_type"=>"initialize_repo_metadata",
"items_total"=>1,
"state"=>"FINISHED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"215e7799-55df-4723-9e50-661f9b5a4fd1",
"num_processed"=>1},
{"num_success"=>0,
"description"=>"Publishing Distribution files",
"step_type"=>"distribution",
"items_total"=>0,
"state"=>"FINISHED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"938f461d-3610-4caa-8b67-397b18c3420c",
"num_processed"=>0},
{"num_success"=>298,
"description"=>"Publishing RPMs",
"step_type"=>"rpms",
"items_total"=>298,
"state"=>"FINISHED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"62e4b857-cd8e-46ac-baac-82eafcb8f636",
"num_processed"=>298},
{"num_success"=>0,
"description"=>"Publishing Delta RPMs",
"step_type"=>"drpms",
"items_total"=>1,
"state"=>"SKIPPED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"7837a9c8-f74b-4b91-a0df-1866eef68035",
"num_processed"=>0},
{"num_success"=>6304,
"description"=>"Publishing Errata",
"step_type"=>"errata",
"items_total"=>6304,
"state"=>"FINISHED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"76ab5e99-a7cc-4b38-b111-d7934288f96a",
"num_processed"=>6304},
{"num_success"=>211,
"description"=>"Publishing Comps file",
"step_type"=>"comps",
"items_total"=>211,
"state"=>"FINISHED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"17b41947-ee17-4617-b3f1-cf2d4632addb",
"num_processed"=>211},
{"num_success"=>0,
"description"=>"Publishing Metadata.",
"step_type"=>"metadata",
"items_total"=>1,
"state"=>"FAILED",
"error_details"=>
[{"traceback"=>
" File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 239, in process\n self._process_block(item=item)\n\n File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 301, in _process_block\n self.process_main(item=item)\n\n File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py\", line 516, in process_main\n add_metadata_file_metadata(unit.data_type, link_path)\n\n File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py\", line 80, in add_metadata_file_metadata\n timestamp_element.text = str(int(os.path.getmtime(file_path)))\n\n File \"/usr/lib64/python2.7/genericpath.py\", line 54, in getmtime\n return os.stat(filename).st_mtime\n",
"error"=>
"[Errno 2] No such file or directory: '/var/lib/pulp/content/units/yum_repo_metadata_file/72/c32e71f97dee222a4106cae699920730329ac8fb0ad7f32cc3347873ac77d0/e27de8e321f2f5ae59f788eb5290eddc5fe89f60ea46a92631360548f0c3f92c-pkgtags.sqlite.gz'"}],
"details"=>"",
"num_failures"=>1,
"step_id"=>"bf2ccebd-1129-4540-b785-32c4805ccabe",
"num_processed"=>1},
{"num_success"=>0,
"description"=>"Closing repo metadata",
"step_type"=>"close_repo_metadata",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"b3c8dea7-1335-438f-8e18-602e1a3f1f15",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Generating sqlite files",
"step_type"=>"generate sqlite",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"7a7c2600-4a37-427a-b0e7-5a331650f71f",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Generating HTML files",
"step_type"=>"repoview",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"e8256072-87a2-4327-bc7c-e00245cb6f58",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Publishing files to web",
"step_type"=>"publish_directory",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"433ffc5d-814f-4117-becd-f894095da7be",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Writing Listings File",
"step_type"=>"initialize_repo_metadata",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"478de48d-ed7f-4baf-9ec2-bd47a2bc625d",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Writing Listings File",
"step_type"=>"initialize_repo_metadata",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"3c7f575b-a5a0-45a4-abfd-4ea2e8959284",
"num_processed"=>0}]},
"queue"=>"reserved_resource_worker-1@katello.sopo.sucive.gub.uy.dq",
"state"=>"error",
"worker_name"=>"reserved_resource_worker-1@katello.sopo.sucive.gub.uy",
"result"=>nil,
"error"=>
{"code"=>"PLP0000",
"data"=>{},
"description"=>
"[Errno 2] No such file or directory: '/var/lib/pulp/content/units/yum_repo_metadata_file/72/c32e71f97dee222a4106cae699920730329ac8fb0ad7f32cc3347873ac77d0/e27de8e321f2f5ae59f788eb5290eddc5fe89f60ea46a92631360548f0c3f92c-pkgtags.sqlite.gz'",
"sub_errors"=>[]},
"_id"=>{"$oid"=>"593f7b72eac8c8110c9515cf"},
"id"=>"593f7b72eac8c8110c9515cf"}]
please excuse the ruby hash syntax
Updated by ipanova@redhat.com almost 7 years ago
Justin, this report does not contain enough of information to be able to tell where the issue is or if there is still the issue.
What i noticed first of all, is time of sync execution, it took less then a minute, that means that it was not full, and even if the user upgraded to the version where the issue is fixed, without force sync nothing will work.
Updated by jsherril@redhat.com almost 7 years ago
The user in this case did do a force sync, skipping the metadata check. This did not seem to resolve the issue (and honestly given the issue I wouldn't expect it to).
I agree there isn't enough information to reproduce. Most every issue like this does not have clear reproducer steps.
Feel free to close. If more users hit the problem we can reopen it.
Updated by amacdona@redhat.com almost 7 years ago
Closing for now, please reopen if it more information is available.
Updated by amacdona@redhat.com almost 7 years ago
- Status changed from NEW to CLOSED - WORKSFORME
Actions
.