Project

Profile

Help

Issue #7239

Updated by ttereshc over 3 years ago

When I run my Pulp 2 to 3 migration, I see the stack trace below.      I've tried flushing the rq queue and resetting the pulpcore DB, but I still see the error.    The result is that my migration task is stuck "waiting" forever and doesn't receive a worker. 

 Related versions: 

 pulp-2to3-migration (0.2.0b7.dev0) 
 pulp-2to3-migration-client (0.2.0b5) 
 pulp-rpm (3.5.0.dev0) 
 pulpcore (3.4.0) 

 Stack trace: 

 <pre> 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: pulp: rq.worker:ERROR: Worker rq:worker:resource-manager: found an unhandled exception, quitting... 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: Traceback (most recent call last): 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib/python3.6/site-packages/rq/utils.py", line 169, in utcparse 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: return datetime.datetime.strptime(string, _TIMESTAMP_FORMAT) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib64/python3.6/_strptime.py", line 565, in _strptime_datetime 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: tt, fraction = _strptime(data_string, format) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib64/python3.6/_strptime.py", line 362, in _strptime 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: (data_string, format)) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: ValueError: time data '' does not match format '%Y-%m-%dT%H:%M:%S.%fZ' 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: During handling of the above exception, another exception occurred: 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: Traceback (most recent call last): 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 510, in work 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: result = self.dequeue_job_and_maintain_ttl(timeout) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 578, in dequeue_job_and_maintain_ttl 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: job_class=self.job_class) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib/python3.6/site-packages/rq/queue.py", line 550, in dequeue_any 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: job = job_class.fetch(job_id, connection=connection) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 303, in fetch 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: job.refresh() 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 515, in refresh 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: self.restore(data) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 478, in restore 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: self.started_at = str_to_date(obj.get('started_at')) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib/python3.6/site-packages/rq/utils.py", line 256, in str_to_date 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: return utcparse(as_text(date_str)) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib/python3.6/site-packages/rq/utils.py", line 172, in utcparse 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: return datetime.datetime.strptime(string, '%Y-%m-%dT%H:%M:%SZ') 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib64/python3.6/_strptime.py", line 565, in _strptime_datetime 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: tt, fraction = _strptime(data_string, format) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: File "/usr/lib64/python3.6/_strptime.py", line 362, in _strptime 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: (data_string, format)) 
     Jul 28 22:24:35 centos7-katello-devel-stable.example.com pulpcore-resource-manager[20338]: ValueError![](: ValueError: time data '' does not match format '%Y-%m-%dT%H:%M:%SZ') '%Y-%m-%dT%H:%M:%SZ' 
 </pre>

Back