Project

Profile

Help

Issue #2967

Updated by kersom over 6 years ago

As a user, I should be able to specify an URL to a feed, and be able to use the sync command to synchronize debian packages from an URL. 

 A guest during test day attempted to create and sync a debian repo using a debian mirror and sync fails. 

 * Pulp packages and plugins: 
 
 <pre> 
 rpm -qa | grep pulp 

 python-pulp-puppet-common-2.14.0-0.3.rc.fc25.noarch 
 pulp-puppet-plugins-2.14.0-0.3.rc.fc25.noarch 
 pulp-ostree-plugins-1.3.0-0.3.rc.fc25.noarch 
 python-pulp-streamer-2.14.0-0.3.rc.fc25.noarch 
 python-pulp-bindings-2.14.0-0.3.rc.fc25.noarch 
 pulp-deb-plugins-1.5.2-0.2.beta.fc25.noarch 
 python-pulp-docker-common-3.0.0-0.3.rc.fc25.noarch 
 python-pulp-oid_validation-2.14.0-0.3.rc.fc25.noarch 
 pulp-docker-plugins-3.0.0-0.3.rc.fc25.noarch 
 pulp-puppet-admin-extensions-2.14.0-0.3.rc.fc25.noarch 
 pulp-ostree-admin-extensions-1.3.0-0.3.rc.fc25.noarch 
 python-pulp-common-2.14.0-0.3.rc.fc25.noarch 
 pulp-docker-admin-extensions-3.0.0-0.3.rc.fc25.noarch 
 python-pulp-python-common-2.0.1-0.3.rc.fc25.noarch 
 pulp-python-plugins-2.0.1-0.3.rc.fc25.noarch 
 python-pulp-rpm-common-2.14.0-0.3.rc.fc25.noarch 
 pulp-rpm-plugins-2.14.0-0.3.rc.fc25.noarch 
 pulp-admin-client-2.14.0-0.3.rc.fc25.noarch 
 pulp-deb-admin-extensions-1.5.2-0.2.beta.fc25.noarch 
 python-pulp-repoauth-2.14.0-0.3.rc.fc25.noarch 
 pulp-rpm-admin-extensions-2.14.0-0.3.rc.fc25.noarch 
 python-pulp-client-lib-2.14.0-0.3.rc.fc25.noarch 
 python-pulp-deb-common-1.5.2-0.2.beta.fc25.noarch 
 pulp-selinux-2.14.0-0.3.rc.fc25.noarch 
 pulp-server-2.14.0-0.3.rc.fc25.noarch 
 python-pulp-ostree-common-1.3.0-0.3.rc.fc25.noarch 
 pulp-python-admin-extensions-2.0.1-0.3.rc.fc25.noarch 
 </pre> 


 <pre> 
 [test@fedora25-base214 ~]$ cat /etc/redhat-release 
 Fedora release 25 (Twenty Five) 
 </pre> 


 <pre> 
 pulp-admin status 
 +----------------------------------------------------------------------+ 
                           Status of the server 
 +----------------------------------------------------------------------+ 

 Api Version:             2 
 Database Connection:    
   Connected: True 
 Known Workers:          
   _id:              scheduler@fedora25-base214 
   _ns:              workers 
   Last Heartbeat: 2017-08-13T15:43:56Z 
   _id:              resource_manager@fedora25-base214 
   _ns:              workers 
   Last Heartbeat: 2017-08-13T15:43:57Z 
   _id:              reserved_resource_worker-0@fedora25-base214 
   _ns:              workers 
   Last Heartbeat: 2017-08-13T15:43:57Z 
   _id:              reserved_resource_worker-1@fedora25-base214 
   _ns:              workers 
   Last Heartbeat: 2017-08-13T15:43:57Z 
 Messaging Connection:   
   Connected: True 
 Versions:               
   Platform Version: 2.14rc3 

 </pre> 

 <pre> 

 [test@fedora25-base214 ~]$ [root@fedora25-test2 ansible]# pulp-admin deb repo create --repo-id debian-repo --feed http://ftp.debian.org/debian/dists/stable/ --feed=http://ftp.debian.org/debian/dists/stable/ 
 Successfully created repository [debian-repo] 

 [test@fedora25-base214 ~]$ [root@fedora25-test2 ansible]# pulp-admin deb repo sync run --repo-id=debian-repo 
 +----------------------------------------------------------------------+ 
                  Synchronizing Repository [debian-repo] 
 +----------------------------------------------------------------------+ 

 This command may be exited via ctrl+c without affecting the request. 



 Task Failed 

 [Errno 2] No such file or directory: 
 u'/var/cache/pulp/reserved_resource_worker-0@fedora25-base214/21a48ad3-6a97-4738 u'/var/cache/pulp/reserved_resource_worker-0@fedora25-test2/76fe6f18-9c1e-414c-a 
 -9294-4ed9f37b8e15/Release' 



 4d4-040f17af44a7/Release' 


 </pre> 

 <pre> 
 [test@fedora25-base214 ~]$ journalctl -f > debian.test.logs & 
 </pre> 


 <pre> 



 Aug 13 12:09:10 fedora25-base214 pulp[2584]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[19b9b8b9-01ed-4f1e-b1d3-9882b90b536a] succeeded in 0.0236528830001s: None 
 Aug 13 12:09:30 fedora25-base214 gnome-terminal-[2370]: Allocating size to GtkScrollbar 0x7caf495390 without calling gtk_widget_get_preferred_width/height(). How does the code know the size to allocate? 
 Aug 13 12:09:32 fedora25-base214 gnome-terminal-[2370]: Allocating size to GtkScrollbar 0x7caf2508e0 without calling gtk_widget_get_preferred_width/height(). How does the code know the size to allocate? 
 Aug 13 12:10:16 fedora25-base214 pulp[2577]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[043bcbd4-3f65-4d6b-9247-912e476590c8] 
 Aug 13 12:10:16 fedora25-base214 pulp[2584]: celery.worker.strategy:INFO: Received task: pulp.server.tasks.repository.delete[1e7f1ec0-bb4a-4497-877b-5afa335df288] 
 Aug 13 12:10:16 fedora25-base214 pulp[2584]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[35f9bdbd-91ba-4a86-8f66-fde9d3e17645] 
 Aug 13 12:10:16 fedora25-base214 pulp[2577]: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[043bcbd4-3f65-4d6b-9247-912e476590c8] succeeded in 0.0389802519999s: None 
 Aug 13 12:10:16 fedora25-base214 pulp[2584]: celery.worker.job:INFO: Task pulp.server.tasks.repository.delete[1e7f1ec0-bb4a-4497-877b-5afa335df288] succeeded in 0.0342449720001s: <pulp.server.async.tasks.TaskResult object at 0x7f80ec85d450> 
 Aug 13 12:10:16 fedora25-base214 pulp[2584]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[35f9bdbd-91ba-4a86-8f66-fde9d3e17645] succeeded in 0.0255510260004s: None 
 Aug 13 12:10:20 fedora25-base214 pulp[2592]: pulp.server.webservices.middleware.exception:INFO: Missing resource(s): repository=debian-repo1 
 Aug 13 12:10:43 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 Aug 13 12:10:46 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 Aug 13 12:10:49 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 Aug 13 12:10:52 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 Aug 13 12:10:55 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 Aug 13 12:10:58 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 Aug 13 12:11:01 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 Aug 13 12:11:04 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 Aug 13 12:11:07 fedora25-base214 pulp[2577]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[c54c2e95-7d7e-4c09-bf12-d9d84594d7cb] 
 Aug 13 12:11:07 fedora25-base214 pulp[2584]: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[21a48ad3-6a97-4738-9294-4ed9f37b8e15] 
 Aug 13 12:11:07 fedora25-base214 pulp[2713]: pulp_deb.plugins.importers.importer:INFO: [21a48ad3] Repo sync started. 
 Aug 13 12:11:07 fedora25-base214 pulp[2713]: pulp_deb.plugins.importers.sync:INFO: [21a48ad3] Downloading http://ftp.debian.org/debian/dists/stable/dists/stable/Release 
 Aug 13 12:11:07 fedora25-base214 pulp[2584]: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[9207f966-2c78-4200-a530-e7de56f2f5b2] 
 Aug 13 12:11:07 fedora25-base214 pulp[2713]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.debian.org 
 Aug 13 12:11:07 fedora25-base214 pulp[2577]: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[c54c2e95-7d7e-4c09-bf12-d9d84594d7cb] succeeded in 0.0389599970003s: None 
 Aug 13 12:11:07 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 Aug 13 12:11:08 fedora25-base214 pulp[2713]: nectar.downloaders.threaded:INFO: Download failed: Download of http://ftp.debian.org/debian/dists/stable/dists/stable/Release failed with code 404: Not Found 
 Aug 13 12:11:08 fedora25-base214 pulp[2713]: pulp.server.async.tasks:INFO: [21a48ad3] Task failed : [21a48ad3-6a97-4738-9294-4ed9f37b8e15] 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800) Task pulp.server.managers.repo.sync.sync[21a48ad3-6a97-4738-9294-4ed9f37b8e15] raised unexpected: IOError(2, 'No such file or directory') 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800) Traceback (most recent call last): 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       R = retval = fun(*args, **kwargs) 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 522, in __call__ 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       return super(Task, self).__call__(*args, **kwargs) 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__ 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       return super(PulpTask, self).__call__(*args, **kwargs) 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__ 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       return self.run(*args, **kwargs) 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 769, in sync 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       sync_report = sync_repo(transfer_repo, conduit, call_config) 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 725, in wrap_f 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       return f(*args, **kwargs) 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/pulp_deb/plugins/importers/importer.py", line 135, in sync_repo 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       report = self._current_sync.process_lifecycle() 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 572, in process_lifecycle 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       super(PluginStep, self).process_lifecycle() 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 163, in process_lifecycle 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       step.process() 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 256, in process 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       self._process_block() 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 303, in _process_block 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       self.process_main() 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)     File "/usr/lib/python2.7/site-packages/pulp_deb/plugins/importers/sync.py", line 95, in process_main 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800)       release=open(self.parent.release_file, "rb"), 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:ERROR: (2584-56800) IOError: [Errno 2] No such file or directory: u'/var/cache/pulp/reserved_resource_worker-0@fedora25-base214/21a48ad3-6a97-4738-9294-4ed9f37b8e15/Release' 
 Aug 13 12:11:08 fedora25-base214 pulp[2584]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[9207f966-2c78-4200-a530-e7de56f2f5b2] succeeded in 0.00361970600034s: None 
 Aug 13 12:11:10 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 Aug 13 12:11:16 fedora25-base214 tracker-extract[2140]: unable to create file '/run/user/1000/dconf/user': Permission denied.    dconf will not work properly. 
 </pre> 


 Related story: https://pulp.plan.io/issues/2763#change-21246 




Back