Project

Profile

Help

Issue #3145

strange error when feed_url is not a valid manifest

Added by mhrivnak over 2 years ago. Updated almost 2 years ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Severity:
2. Medium
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Target Release - File:
Sprint:

Description

I tried to sync with this importer config:

{
        'name': 'foo',
        'download_policy': 'immediate',
        'sync_mode': 'mirror',
        'validate': True,
        'ssl_validation': True,
        'feed_url': 'https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/file/',
}

Astute readers have noticed that I forgot to append PULP_MANIFEST to the feed_url. User error, so I expect a sync to fail.

BUT the sync fails with this unexpected traceback, which suggests the importer attempted to create the directory "/repos":

Nov 20 20:44:15 pulp3.dev celery[3685]: [2017-11-20 20:44:15,261: ERROR/ForkPoolWorker-1] Task failed : [ff4fdfa2-2db7-4322-be1e-d4aefa07fe51]
Nov 20 20:44:15 pulp3.dev celery[3685]: [2017-11-20 20:44:15,267: ERROR/ForkPoolWorker-1] Task pulpcore.app.tasks.importer.sync[ff4fdfa2-2db7-4322-be1e-d4aefa07fe51] raised unexpected: PermissionError(13, 'Permi
Nov 20 20:44:15 pulp3.dev celery[3685]: Traceback (most recent call last):
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.5/site-packages/celery/app/trace.py", line 374, in trace_task
Nov 20 20:44:15 pulp3.dev celery[3685]:     R = retval = fun(*args, **kwargs)
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/devel/pulp/pulpcore/pulpcore/tasking/tasks.py", line 272, in __call__
Nov 20 20:44:15 pulp3.dev celery[3685]:     return super(UserFacingTask, self).__call__(*args, **kwargs)
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/.virtualenvs/pulp/lib64/python3.5/site-packages/celery/app/trace.py", line 629, in __protected_call__
Nov 20 20:44:15 pulp3.dev celery[3685]:     return self.run(*args, **kwargs)
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/devel/pulp/pulpcore/pulpcore/app/tasks/importer.py", line 83, in sync
Nov 20 20:44:15 pulp3.dev celery[3685]:     importer.sync()
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/devel/pulp_file/pulp_file/app/models.py", line 71, in sync
Nov 20 20:44:15 pulp3.dev celery[3685]:     Synchronizer(self).run()
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/devel/pulp_file/pulp_file/app/models.py", line 107, in run
Nov 20 20:44:15 pulp3.dev celery[3685]:     self._fetch_manifest()
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/devel/pulp_file/pulp_file/app/models.py", line 131, in _fetch_manifest
Nov 20 20:44:15 pulp3.dev celery[3685]:     download()
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/download/futures/single.py", line 287, in __call__
Nov 20 20:44:15 pulp3.dev celery[3685]:     self._attempt()
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/download/futures/single.py", line 207, in _attempt
Nov 20 20:44:15 pulp3.dev celery[3685]:     with self.writer:
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/download/futures/writer.py", line 70, in __enter__
Nov 20 20:44:15 pulp3.dev celery[3685]:     self.open()
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/download/futures/writer.py", line 103, in open
Nov 20 20:44:15 pulp3.dev celery[3685]:     self._mkdir()
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/home/vagrant/devel/pulp/plugin/pulpcore/plugin/download/futures/writer.py", line 136, in _mkdir
Nov 20 20:44:15 pulp3.dev celery[3685]:     os.makedirs(_dir)
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/usr/lib64/python3.5/os.py", line 231, in makedirs
Nov 20 20:44:15 pulp3.dev celery[3685]:     makedirs(head, mode, exist_ok)
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/usr/lib64/python3.5/os.py", line 231, in makedirs
Nov 20 20:44:15 pulp3.dev celery[3685]:     makedirs(head, mode, exist_ok)
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/usr/lib64/python3.5/os.py", line 231, in makedirs
Nov 20 20:44:15 pulp3.dev celery[3685]:     makedirs(head, mode, exist_ok)
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/usr/lib64/python3.5/os.py", line 231, in makedirs
Nov 20 20:44:15 pulp3.dev celery[3685]:     makedirs(head, mode, exist_ok)
Nov 20 20:44:15 pulp3.dev celery[3685]:   File "/usr/lib64/python3.5/os.py", line 241, in makedirs
Nov 20 20:44:15 pulp3.dev celery[3685]:     mkdir(name, mode)
Nov 20 20:44:15 pulp3.dev celery[3685]: PermissionError: [Errno 13] Permission denied: '/repos'

History

#1 Updated by amacdona@redhat.com over 2 years ago

  • Triaged changed from No to Yes

#2 Updated by dalley almost 2 years ago

This is still an issue and needs to be improved, but the error message is different. Using the same process:

{
    "_href": "http://localhost:8000/pulp/api/v3/remotes/file/5d4bbb34-5468-4538-8210-07e46f9afb9a/",
    "created": "2018-05-15T19:16:49.050469Z",
    "id": "5d4bbb34-5468-4538-8210-07e46f9afb9a",
    "last_synced": null,
    "last_updated": "2018-05-15T19:16:49.050480Z",
    "name": "bar",
    "proxy_url": "",
    "ssl_validation": true,
    "type": "file",
    "url": "https://repos.fedorapeople.org/pulp/pulp/demo_repos/test_file_repo/",
    "validate": true
}
May 15 19:17:07 pulp3.dev pulp[28219]: rq.worker:INFO: Cleaning registries for queue: resource_manager                                     
May 15 19:17:07 pulp3.dev pulp[2571]: pulp_file.app.tasks.synchronizing:INFO: Synchronizing: repository=foo remote=bar                     
May 15 19:17:08 pulp3.dev pulp[2571]: rq.worker:ERROR: ValueError: Error: manifest line:1: must be: <relative_path>, <digest>, <size>      
                                      Traceback (most recent call last):                                                                   
                                        File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/worker.py", line 793, in perform_job
                                          rv = job.perform()
                                        File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/job.py", line 599, in perform
                                          self._result = self._execute()                                                                   
                                        File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/job.py", line 605, in _execute
                                          return self.func(*self.args, **self.kwargs)                                                      
                                        File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 62, in synchronize 
                                          delta = find_delta(manifest, content)                                                            
                                        File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 129, in find_delta 
                                          Key(relative_path=e.relative_path, digest=e.digest) for e in manifest.read()                     
                                        File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 129, in <listcomp> 
                                          Key(relative_path=e.relative_path, digest=e.digest) for e in manifest.read()                     
                                        File "/home/vagrant/devel/pulp_file/pulp_file/manifest.py", line 101, in read                      
                                          yield Entry.parse(Line(number=n, content=line))                                                  
                                        File "/home/vagrant/devel/pulp_file/pulp_file/manifest.py", line 41, in parse                      
                                          n=line.number))
                                      ValueError: Error: manifest line:1: must be: <relative_path>, <digest>, <size>                       
                                      Traceback (most recent call last):                                                                   
                                        File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/worker.py", line 793, in perform_job
                                          rv = job.perform()
                                        File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/job.py", line 599, in perform
                                          self._result = self._execute()                                                                   
                                        File "/home/vagrant/.virtualenvs/pulp/lib64/python3.6/site-packages/rq/job.py", line 605, in _execute
                                          return self.func(*self.args, **self.kwargs)                                                      
                                        File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 62, in synchronize 
                                          delta = find_delta(manifest, content)                                                            
                                        File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 129, in find_delta 
                                          Key(relative_path=e.relative_path, digest=e.digest) for e in manifest.read()                     
                                        File "/home/vagrant/devel/pulp_file/pulp_file/app/tasks/synchronizing.py", line 129, in <listcomp> 
                                          Key(relative_path=e.relative_path, digest=e.digest) for e in manifest.read()                     
                                        File "/home/vagrant/devel/pulp_file/pulp_file/manifest.py", line 101, in read                      
                                          yield Entry.parse(Line(number=n, content=line))                                                  
                                        File "/home/vagrant/devel/pulp_file/pulp_file/manifest.py", line 41, in parse                      
                                          n=line.number))
                                      ValueError: Error: manifest line:1: must be: <relative_path>, <digest>, <size>                       
May 15 19:17:08 pulp3.dev pulp[2571]: rq.worker:WARNING: Moving job to 'failed' queue                                  

Please register to edit this issue

Also available in: Atom PDF