Project

Profile

Help

Issue #8292

FILES.json not found when syncing all of galaxy.ansible.com

Added by gerrod 9 days ago. Updated about 22 hours ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Feb 22 18:24:22 ip-172-32-0-61 rq[240455]: pulp [c1c01c8eb5b4411dafd848e54fe8148e]: pulp_ansible.app.tasks.utils:INFO: Reading meta/runtime.yml from artifact/c2/939ffe56fe75ffd1ae96e6f0159421aa120c286a9464ac851bb00c25568e42
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]: pulp [c1c01c8eb5b4411dafd848e54fe8148e]: pulp_ansible.app.tasks.utils:INFO: Reading MANIFEST.json from artifact/c2/939ffe56fe75ffd1ae96e6f0159421aa120c286a9464ac851bb00c25568e42
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]: pulp [c1c01c8eb5b4411dafd848e54fe8148e]: pulp_ansible.app.tasks.utils:INFO: Reading FILES.json from artifact/c2/939ffe56fe75ffd1ae96e6f0159421aa120c286a9464ac851bb00c25568e42
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]: pulp [c1c01c8eb5b4411dafd848e54fe8148e]: pulp_ansible.app.tasks.utils:INFO: Reading meta/runtime.yml from artifact/3c/906e302f64afc9d49e7c39e4749dba43dfc94c5907468da670dbb511b6292d
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]: pulp [c1c01c8eb5b4411dafd848e54fe8148e]: pulp_ansible.app.tasks.utils:INFO: Reading MANIFEST.json from artifact/3c/906e302f64afc9d49e7c39e4749dba43dfc94c5907468da670dbb511b6292d
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]: pulp [c1c01c8eb5b4411dafd848e54fe8148e]: pulp_ansible.app.tasks.utils:INFO: Reading FILES.json from artifact/3c/906e302f64afc9d49e7c39e4749dba43dfc94c5907468da670dbb511b6292d
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]: pulp [c1c01c8eb5b4411dafd848e54fe8148e]: rq.worker:ERROR: Traceback (most recent call last):
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 975, in perform_job
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     rv = job.perform()
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 696, in perform
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     self._result = self._execute()
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 719, in _execute
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     return self.func(*self.args, **self.kwargs)
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/home/ec2-user/devel/pulp_ansible/pulp_ansible/app/tasks/collections.py", line 129, in sync
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     d_version.create()
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/home/ec2-user/devel/pulpcore/pulpcore/plugin/stages/declarative_version.py", line 148, in create
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     loop.run_until_complete(pipeline)
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     return future.result()
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/home/ec2-user/devel/pulpcore/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     await asyncio.gather(*futures)
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/home/ec2-user/devel/pulpcore/pulpcore/plugin/stages/api.py", line 43, in __call__
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     await self.run()
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/home/ec2-user/devel/pulpcore/pulpcore/plugin/stages/content_stages.py", line 114, in run
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     await self._post_save(batch)
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/home/ec2-user/devel/pulp_ansible/pulp_ansible/app/tasks/collections.py", line 843, in _post_save
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     get_file_obj_from_tarball(tar, "FILES.json", artifact.file.name)
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:   File "/home/ec2-user/devel/pulp_ansible/pulp_ansible/app/tasks/utils.py", line 144, in get_file_obj_from_tarball
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]:     raise FileNotFoundError(f"{file_path} not found")
Feb 22 18:24:22 ip-172-32-0-61 rq[240455]: FileNotFoundError: FILES.json not found

History

#1 Updated by alikins 8 days ago

Would it be possible to get a little further back in the log?

Mainly looking for the name of the collection artifact that triggers this?

#2 Updated by gerrod 7 days ago

Looking through the database, there are two collection versions that don't have FILES.json. Both under namespace hs0210, fos 1.0.1 and tos 1.0.0 are missing their FILES.json.

#3 Updated by newswangerd 2 days ago

That's sort of worrying. Those versions were uploaded recently (within the last 2 months). This makes me think we're not properly validating collections at import time.

#4 Updated by awcrosby about 22 hours ago

We created an issue to validate FILES.json within galaxy-importer: https://issues.redhat.com/browse/AAH-403

Please register to edit this issue

Also available in: Atom PDF