Actions
Issue #3145
closed
strange error when feed_url is not a valid manifest
Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:
Description
Ticket moved to GitHub: "pulp/pulp_file/605":https://github.com/pulp/pulp_file/issues/605
I tried to sync with this importer config:
{
'name': 'foo',
'download_policy': 'immediate',
'sync_mode': 'mirror',
'validate': True,
'ssl_validation': True,
'feed_url': 'https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/file/',
}
Astute readers have noticed that I forgot to append PULP_MANIFEST to the feed_url. User error, so I expect a sync to fail.
BUT the sync fails with this unexpected traceback, which suggests the importer attempted to create the directory "/repos":
Nov 20 20:44:15 pulp3.dev celery[3685]: [2017-11-20 20:44:15,261: ERROR/ForkPoolWorker-1] Task failed : [ff4fdfa2-2db7-4322-be1e-d4aefa07fe51]
Nov 20 20:44:15 pulp3.dev celery[3685]: [2017-11-20 20:44:15,267: ERROR/ForkPoolWorker-1] Task pulpcore.app.tasks.importer.sync[ff4fdfa2-2db7-4322-be1e-d4aefa07fe51] raised unexpected: PermissionError(13, 'Permi
Nov 20 20:44:15 pulp3.dev celery[3685]: Traceback (most recent call last):
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/.virtualenvs/pulp/lib64/python3.5/site-packages/celery/app/trace.py", line 374, in trace_task
Nov 20 20:44:15 pulp3.dev celery[3685]: R = retval = fun(*args, **kwargs)
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/devel/pulp/pulpcore/pulpcore/tasking/tasks.py", line 272, in __call__
Nov 20 20:44:15 pulp3.dev celery[3685]: return super(UserFacingTask, self).__call__(*args, **kwargs)
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/.virtualenvs/pulp/lib64/python3.5/site-packages/celery/app/trace.py", line 629, in __protected_call__
Nov 20 20:44:15 pulp3.dev celery[3685]: return self.run(*args, **kwargs)
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/devel/pulp/pulpcore/pulpcore/app/tasks/importer.py", line 83, in sync
Nov 20 20:44:15 pulp3.dev celery[3685]: importer.sync()
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/devel/pulp_file/pulp_file/app/models.py", line 71, in sync
Nov 20 20:44:15 pulp3.dev celery[3685]: Synchronizer(self).run()
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/devel/pulp_file/pulp_file/app/models.py", line 107, in run
Nov 20 20:44:15 pulp3.dev celery[3685]: self._fetch_manifest()
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/devel/pulp_file/pulp_file/app/models.py", line 131, in _fetch_manifest
Nov 20 20:44:15 pulp3.dev celery[3685]: download()
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/download/futures/single.py", line 287, in __call__
Nov 20 20:44:15 pulp3.dev celery[3685]: self._attempt()
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/download/futures/single.py", line 207, in _attempt
Nov 20 20:44:15 pulp3.dev celery[3685]: with self.writer:
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/download/futures/writer.py", line 70, in __enter__
Nov 20 20:44:15 pulp3.dev celery[3685]: self.open()
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/download/futures/writer.py", line 103, in open
Nov 20 20:44:15 pulp3.dev celery[3685]: self._mkdir()
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/download/futures/writer.py", line 136, in _mkdir
Nov 20 20:44:15 pulp3.dev celery[3685]: os.makedirs(_dir)
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/usr/lib64/python3.5/os.py", line 231, in makedirs
Nov 20 20:44:15 pulp3.dev celery[3685]: makedirs(head, mode, exist_ok)
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/usr/lib64/python3.5/os.py", line 231, in makedirs
Nov 20 20:44:15 pulp3.dev celery[3685]: makedirs(head, mode, exist_ok)
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/usr/lib64/python3.5/os.py", line 231, in makedirs
Nov 20 20:44:15 pulp3.dev celery[3685]: makedirs(head, mode, exist_ok)
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/usr/lib64/python3.5/os.py", line 231, in makedirs
Nov 20 20:44:15 pulp3.dev celery[3685]: makedirs(head, mode, exist_ok)
Nov 20 20:44:15 pulp3.dev celery[3685]: File "/usr/lib64/python3.5/os.py", line 241, in makedirs
Nov 20 20:44:15 pulp3.dev celery[3685]: mkdir(name, mode)
Nov 20 20:44:15 pulp3.dev celery[3685]: PermissionError: [Errno 13] Permission denied: '/repos'
Updated by dalley over 6 years ago
This is still an issue and needs to be improved, but the error message is different. Using the same process:
{
"_href": "http://localhost:8000/pulp/api/v3/remotes/file/5d4bbb34-5468-4538-8210-07e46f9afb9a/",
"created": "2018-05-15T19:16:49.050469Z",
"id": "5d4bbb34-5468-4538-8210-07e46f9afb9a",
"last_synced": null,
"last_updated": "2018-05-15T19:16:49.050480Z",
"name": "bar",
"proxy_url": "",
"ssl_validation": true,
"type": "file",
"url": "https://repos.fedorapeople.org/pulp/pulp/demo_repos/test_file_repo/",
"validate": true
}
May 15 19:17:07 pulp3.dev pulp[28219]: rq.worker:INFO: Cleaning registries for queue: resource_manager
May 15 19:17:07 pulp3.dev pulp[2571]: pulp_file.app.tasks.synchronizing:INFO: Synchronizing: repository=foo remote=bar
May 15 19:17:08 pulp3.dev pulp[2571]: rq.worker:ERROR: ValueError: Error: manifest line:1: must be: <relative_path>, <digest>, <size>
Traceback (most recent call last):
File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/worker.py", line 793, in perform_job
rv = job.perform()
File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/job.py", line 599, in perform
self._result = self._execute()
File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/job.py", line 605, in _execute
return self.func(*self.args, **self.kwargs)
File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 62, in synchronize
delta = find_delta(manifest, content)
File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 129, in find_delta
Key(relative_path=e.relative_path, digest=e.digest) for e in manifest.read()
File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 129, in <listcomp>
Key(relative_path=e.relative_path, digest=e.digest) for e in manifest.read()
File "/home/vagrant/devel/pulp_file/pulp_file/manifest.py", line 101, in read
yield Entry.parse(Line(number=n, content=line))
File "/home/vagrant/devel/pulp_file/pulp_file/manifest.py", line 41, in parse
n=line.number))
ValueError: Error: manifest line:1: must be: <relative_path>, <digest>, <size>
Traceback (most recent call last):
File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/worker.py", line 793, in perform_job
rv = job.perform()
File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/job.py", line 599, in perform
self._result = self._execute()
File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/job.py", line 605, in _execute
return self.func(*self.args, **self.kwargs)
File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 62, in synchronize
delta = find_delta(manifest, content)
File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 129, in find_delta
Key(relative_path=e.relative_path, digest=e.digest) for e in manifest.read()
File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 129, in <listcomp>
Key(relative_path=e.relative_path, digest=e.digest) for e in manifest.read()
File "/home/vagrant/devel/pulp_file/pulp_file/manifest.py", line 101, in read
yield Entry.parse(Line(number=n, content=line))
File "/home/vagrant/devel/pulp_file/pulp_file/manifest.py", line 41, in parse
n=line.number))
ValueError: Error: manifest line:1: must be: <relative_path>, <digest>, <size>
May 15 19:17:08 pulp3.dev pulp[2571]: rq.worker:WARNING: Moving job to 'failed' queue
Updated by pulpbot almost 3 years ago
- Description updated (diff)
- Status changed from NEW to CLOSED - DUPLICATE
Actions
.