Actions
Issue #2588
closed
Pulp Gets Stuck On A Particular Mirror
Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:
Description
This is being filed on behalf of Jason Nance who originally opened an issue against the Katello project (http://projects.theforeman.org/issues/18506) and we have since moved it here.
Pulp 2.9.3, Squid 3.5.20
Product repository is configured as on demand, and points to a mirror list:
http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=updates
Client is trying to install a package which has not previously been downloaded, but Pulp is failing to download the package. Pulp says "Skipping requests to mirror.clarkson.edu due to repeated connection failures", but then retries the same mirror again.
Specific to this instance, http://mirror.clarkson.edu/centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm is unavailable (mirror.clarkson.edu is down in my area at this time).
If I run wget 'http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=updates' I get a different mirror list (nearly every time), but Pulp/Squid seems to not be refreshing its mirror list
Client (yum install MySQL-python) shows the following (10 times):
MySQL-python-1.2.5-1.el7.x86_6 FAILED
https://sl2mmgplsat0001.ipa.lab.gen.zone/pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm: [Errno 14] HTTPS Error 503 - Service Unavailable
Trying other mirror.
foreman-tail shows the following:
==> /var/log/messages <==
Feb 14 13:42:09 sl2mmgplsat0001 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (21): mirror.clarkson.edu
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:09 -0600] "GET /pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/messages <==
Feb 14 13:42:15 sl2mmgplsat0001 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.clarkson.edu due to repeated connection failures: HTTPConnectionPool(host='mirror.clarkson.edu', port=80): Max retries exceeded with url: /centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x4690890>, 'Connection to mirror.clarkson.edu timed out. (connect timeout=6.05)'))
Feb 14 13:42:15 sl2mmgplsat0001 pulp_streamer: [-] 127.0.0.1 - - [14/Feb/2017:19:42:15 +0000] "GET /var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
Feb 14 13:42:15 sl2mmgplsat0001 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (22): mirror.clarkson.edu
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:09 -0600] "GET /streamer/var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4zMyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS9lNS9mYmJlODFiZTdlYzhhNWM4NjMyNzIwMGIyMzNlOTc1MzNlYzc1NDQwNzBjZWM1NzhiM2VhZTA1MDIzN2E0Yy9NeVNRTC1weXRob24tMS4yLjUtMS5lbDcueDg2XzY0LnJwbSIsICJleHBpcmF0aW9uIjogMTQ4NzEwMTQxOX0%3D;signature=rjPNJ2oWHsEOD7h0WvE1zaCLnkg0Fd8ZLRpmZjBosv7VWgxfCVZSbBLHv_Khvgq9TDPdAYH6GLXu_ljj9vpkc_FFEtMO6PczQT9zGzMWu8Fm21vhGMDwejbVw34Z1K1ERcC53V25_WFczMwikfCYrpDnU83DMor5H60p53l1gC3zpb3pIPQI79WfccbbKveYx31W3mqBXXVSc0WoUS7HOmE3Zg-_zzX53Q24ERrwEJlkHBd0cFCqpYinok-yjniHObhfAEUNaH6BfZiJfAa1VkkIje4Jflyoh6Eta73u_VihZmGfN669svBmMP4VCiACltnSFvieQK6uAY8NwSObDQ%3D%3D HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:15 -0600] "GET /pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/messages <==
Feb 14 13:42:21 sl2mmgplsat0001 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.clarkson.edu due to repeated connection failures: HTTPConnectionPool(host='mirror.clarkson.edu', port=80): Max retries exceeded with url: /centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x4690890>, 'Connection to mirror.clarkson.edu timed out. (connect timeout=6.05)'))
Feb 14 13:42:21 sl2mmgplsat0001 pulp_streamer: [-] 127.0.0.1 - - [14/Feb/2017:19:42:21 +0000] "GET /var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
Feb 14 13:42:21 sl2mmgplsat0001 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (23): mirror.clarkson.edu
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:15 -0600] "GET /streamer/var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4zMyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS9lNS9mYmJlODFiZTdlYzhhNWM4NjMyNzIwMGIyMzNlOTc1MzNlYzc1NDQwNzBjZWM1NzhiM2VhZTA1MDIzN2E0Yy9NeVNRTC1weXRob24tMS4yLjUtMS5lbDcueDg2XzY0LnJwbSIsICJleHBpcmF0aW9uIjogMTQ4NzEwMTQyNX0%3D;signature=bzUdGI6ZutyR95p8NrTVxYJQrSMz9eqYz4Z-FCuBO-PYlkx3sYksikaygltJV-VE6_ZdQNEeg8ttsVR8EtDktX3VSNU-5X6map6eWRDN2gcj_Wkz0SuQX9Q7kLO-GsWHcf8-mzL6YUzPmOKr9YZYSlnO_VYS8sPqx_FqtTeh2Ry17Jg452Y3D0ggfWxBU-O5xo1ggbM10aLaXjWXuRDDAtqg7PH-eMpbzdnFhyWnGYIpa-6VYXu9Rz2u3J4ZBj4Cejmn1zVvXyv4fcn6x3vauwcNJMan4SZt8cJhNN6Hx2ye_ySUkJ12mnSEkaA4SWUXEeG8Hf0OCOLgsK3gPbYC2w%3D%3D HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:21 -0600] "GET /pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/messages <==
Feb 14 13:42:27 sl2mmgplsat0001 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.clarkson.edu due to repeated connection failures: HTTPConnectionPool(host='mirror.clarkson.edu', port=80): Max retries exceeded with url: /centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x4690e10>, 'Connection to mirror.clarkson.edu timed out. (connect timeout=6.05)'))
Feb 14 13:42:27 sl2mmgplsat0001 pulp_streamer: [-] 127.0.0.1 - - [14/Feb/2017:19:42:27 +0000] "GET /var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
Feb 14 13:42:27 sl2mmgplsat0001 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (24): mirror.clarkson.edu
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:21 -0600] "GET /streamer/var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4zMyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS9lNS9mYmJlODFiZTdlYzhhNWM4NjMyNzIwMGIyMzNlOTc1MzNlYzc1NDQwNzBjZWM1NzhiM2VhZTA1MDIzN2E0Yy9NeVNRTC1weXRob24tMS4yLjUtMS5lbDcueDg2XzY0LnJwbSIsICJleHBpcmF0aW9uIjogMTQ4NzEwMTQzMX0%3D;signature=tdkNdk4ZivUJ80DkFV0c0hiZodMhNmdQVe8-gbVJBVcoCRvlixFGb2DqhgTKNixnrTv1aV8bOVpJ4NWkkH15jLy6LZaQ9eXuc0GjczSNMpWUIJ_aYHnWbOnC6PN78_Xt3TwWzzD7evBQdMHTXKLCPrKZ2mhSRvQUflbm_4pe08Lf6IIW50-2owndJcfy0lUsMvGIO_pAdvyGkwi2FjlGSFJb_Tdl2qhlqrgS6uIQiQu_f7zbOt_mfZQmxFGi_sGoiCQ4k2_JEL1Xkf9qmCTItIJhkyX6tNcjnTX2hZ0HAiT7riooqFDZzZTw0zmHL1FuccjaDRlzBSShFRxmjlfUbw%3D%3D HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:27 -0600] "GET /pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/messages <==
Feb 14 13:42:33 sl2mmgplsat0001 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.clarkson.edu due to repeated connection failures: HTTPConnectionPool(host='mirror.clarkson.edu', port=80): Max retries exceeded with url: /centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x4690cd0>, 'Connection to mirror.clarkson.edu timed out. (connect timeout=6.05)'))
Feb 14 13:42:33 sl2mmgplsat0001 pulp_streamer: [-] 127.0.0.1 - - [14/Feb/2017:19:42:33 +0000] "GET /var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
Feb 14 13:42:34 sl2mmgplsat0001 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (25): mirror.clarkson.edu
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:27 -0600] "GET /streamer/var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4zMyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS9lNS9mYmJlODFiZTdlYzhhNWM4NjMyNzIwMGIyMzNlOTc1MzNlYzc1NDQwNzBjZWM1NzhiM2VhZTA1MDIzN2E0Yy9NeVNRTC1weXRob24tMS4yLjUtMS5lbDcueDg2XzY0LnJwbSIsICJleHBpcmF0aW9uIjogMTQ4NzEwMTQzN30%3D;signature=Af0pcqiwj0XDRQrfcjRh6o4h4opZ8hrbn7IsyEoBHMQis2QyX8TqzM79eg_PVPcGxf1h4eY0RsxxeFqQOrTrw7DMyTXd6R1AS9fVKX0kI7GlJlnxqs7BX5QtemFhA1U4Q7ZdO_A16DcxUQW-swlkKzVQffYFMrARIbC4DcrNebJRJYyXKep4XSQ79u8CMtwNGoO_fvnOf9JeclLdguwZI5Zuzbm-CykEGJLDNjIu3fvZQRi6fpI-B6rv3X9ercFT5kSx5xp9_oRCiGkAAa95Y9YIua2iBRgefQCnxk0tp0_4StPKgEUNyagYkED6SRl7ucqbXbEM42ka7MLQaB6EAQ%3D%3D HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:33 -0600] "GET /pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/messages <==
Feb 14 13:42:40 sl2mmgplsat0001 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.clarkson.edu due to repeated connection failures: HTTPConnectionPool(host='mirror.clarkson.edu', port=80): Max retries exceeded with url: /centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x4690cd0>, 'Connection to mirror.clarkson.edu timed out. (connect timeout=6.05)'))
Feb 14 13:42:40 sl2mmgplsat0001 pulp_streamer: [-] 127.0.0.1 - - [14/Feb/2017:19:42:39 +0000] "GET /var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
Feb 14 13:42:40 sl2mmgplsat0001 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (26): mirror.clarkson.edu
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:33 -0600] "GET /streamer/var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4zMyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS9lNS9mYmJlODFiZTdlYzhhNWM4NjMyNzIwMGIyMzNlOTc1MzNlYzc1NDQwNzBjZWM1NzhiM2VhZTA1MDIzN2E0Yy9NeVNRTC1weXRob24tMS4yLjUtMS5lbDcueDg2XzY0LnJwbSIsICJleHBpcmF0aW9uIjogMTQ4NzEwMTQ0M30%3D;signature=Ux8s1cuKeyzUVpa1spxeEVftblI9n1xC11aInsXzCeCcx9k276MM8KWMKRQUZP5XesAOQuy8PYmfGXVg0YoBFrHdcHEive6O2Ox_lD3ZkA8CcwMeBES-rkxwgQLKYCszm2_avzHJRHoVcSXp9e8Ng0__hhSDyYO6HDX3kRJe4N2BIwrFFnWn8K8fBW_v3Mx6EsiErJq8CqauGES3sRLZCrQxdsUwah_fCKe2nfSvhT__ES2P9jfAdQeUenHDdnwOVOI5AMrld6V9yujGtnRnYZbSnp0xwEngl0xfLHqRyxc9jwiVJ7eD0nSrGXEZ3wVLJ9FGqmbpPv_lBUE6zj538Q%3D%3D HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:40 -0600] "GET /pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/messages <==
Feb 14 13:42:46 sl2mmgplsat0001 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.clarkson.edu due to repeated connection failures: HTTPConnectionPool(host='mirror.clarkson.edu', port=80): Max retries exceeded with url: /centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x4690cd0>, 'Connection to mirror.clarkson.edu timed out. (connect timeout=6.05)'))
Feb 14 13:42:46 sl2mmgplsat0001 pulp_streamer: [-] 127.0.0.1 - - [14/Feb/2017:19:42:45 +0000] "GET /var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
Feb 14 13:42:46 sl2mmgplsat0001 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (27): mirror.clarkson.edu
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:40 -0600] "GET /streamer/var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4zMyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS9lNS9mYmJlODFiZTdlYzhhNWM4NjMyNzIwMGIyMzNlOTc1MzNlYzc1NDQwNzBjZWM1NzhiM2VhZTA1MDIzN2E0Yy9NeVNRTC1weXRob24tMS4yLjUtMS5lbDcueDg2XzY0LnJwbSIsICJleHBpcmF0aW9uIjogMTQ4NzEwMTQ1MH0%3D;signature=S4A8_oVGDjM1hSBdKlmXFUICm-QOhytN4meGjWI5teMxSNGxuUF99hr__uy_PINxqFpHt5Xu055SkkUHQ5zjBRnmjK1L1A54hge0eXGCSFED9LwGw9X2CRVT7LaETj79F14pWXu5C108gvXDKhfpjOiNcXx0p8piM-w5niuKXsdwQi7zTLpZ41PmamI4qwpiDZvi8TArhfQN2hGuPtCsT1au5o-cgiXda01pk04q59ivt59CjLLboM-cHPzLqZkEEgw1JM9xL3HtRRr3Bzy2Soo8W6QIipbjVjoqak3nB0QDD---Op3o98W4LzuJuPbUJH1bNVd-ovJB1mEEnZiMcw%3D%3D HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:46 -0600] "GET /pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/messages <==
Feb 14 13:42:52 sl2mmgplsat0001 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.clarkson.edu due to repeated connection failures: HTTPConnectionPool(host='mirror.clarkson.edu', port=80): Max retries exceeded with url: /centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x4690cd0>, 'Connection to mirror.clarkson.edu timed out. (connect timeout=6.05)'))
Feb 14 13:42:52 sl2mmgplsat0001 pulp_streamer: [-] 127.0.0.1 - - [14/Feb/2017:19:42:52 +0000] "GET /var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
Feb 14 13:42:52 sl2mmgplsat0001 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (28): mirror.clarkson.edu
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:46 -0600] "GET /streamer/var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4zMyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS9lNS9mYmJlODFiZTdlYzhhNWM4NjMyNzIwMGIyMzNlOTc1MzNlYzc1NDQwNzBjZWM1NzhiM2VhZTA1MDIzN2E0Yy9NeVNRTC1weXRob24tMS4yLjUtMS5lbDcueDg2XzY0LnJwbSIsICJleHBpcmF0aW9uIjogMTQ4NzEwMTQ1Nn0%3D;signature=JKLf9y6ohBUnUxcVAKZqNUxiyPyhaIpsm2aHdem2Dp9xwmhyRJbBRuhDkSedF6Ae9rMXawKTOSWofX-Fzyk9h3oOZwPwZdpvhnbmb5MLnYP45M-eE8n-FhsI5GFkZcv9TpauTJL_gzpUzKostL8fNuarL9wa_GQdbYVfTIu9SETHMFIRYxKV4Xr0P6DvDaA2v2wjaraXQja2dX8Vh9AkkJ3X0CSHvDdXpeuGe70rknuAj1d9quPpGHncewCbBXqEus2_X1TL7xpGhbZNf60R8wSMZ2Oj7162ZKG_xQJRg4l-8Am3LMPhSxJF7TCdwTxe8plHrvqisI8ofgbnpsEwlw%3D%3D HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:52 -0600] "GET /pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/messages <==
Feb 14 13:42:58 sl2mmgplsat0001 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.clarkson.edu due to repeated connection failures: HTTPConnectionPool(host='mirror.clarkson.edu', port=80): Max retries exceeded with url: /centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x4690cd0>, 'Connection to mirror.clarkson.edu timed out. (connect timeout=6.05)'))
Feb 14 13:42:58 sl2mmgplsat0001 pulp_streamer: [-] 127.0.0.1 - - [14/Feb/2017:19:42:58 +0000] "GET /var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
Feb 14 13:42:58 sl2mmgplsat0001 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (29): mirror.clarkson.edu
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:52 -0600] "GET /streamer/var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4zMyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS9lNS9mYmJlODFiZTdlYzhhNWM4NjMyNzIwMGIyMzNlOTc1MzNlYzc1NDQwNzBjZWM1NzhiM2VhZTA1MDIzN2E0Yy9NeVNRTC1weXRob24tMS4yLjUtMS5lbDcueDg2XzY0LnJwbSIsICJleHBpcmF0aW9uIjogMTQ4NzEwMTQ2Mn0%3D;signature=pbkwa8nSLs9NEcgFkkgPbLoml54upZ4tZlyCV1wFzkZXHxOmNUjfMpV_Q4FUdWkp7K20i1kmZ4ZYPsxZsWy70qNPTARZREP8lTF1KVq0epsJn0e-fmw7lgz4McVkc_1vRREDhEOP4v6rGBYDw5-t37bIIfQQpjrGaFLe6Zw8VtLxkr_SQDKhKusDqUCyqxsVmkDI_qYN6eFQRw8JtYT8nfTTNpwAhL4kO6prYXs8wQcXmUrFLEuGTV-V1hCbRKEUTaL5xtIp3hdX9c0N5E4TKITXwisYjVZlYmt1owgXKUpKvIHh4j5mFopWjFokv1RAAylSOz8oHlAn9fDasTcPyw%3D%3D HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:58 -0600] "GET /pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/messages <==
Feb 14 13:43:04 sl2mmgplsat0001 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.clarkson.edu due to repeated connection failures: HTTPConnectionPool(host='mirror.clarkson.edu', port=80): Max retries exceeded with url: /centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x4690cd0>, 'Connection to mirror.clarkson.edu timed out. (connect timeout=6.05)'))
Feb 14 13:43:04 sl2mmgplsat0001 pulp_streamer: [-] 127.0.0.1 - - [14/Feb/2017:19:43:04 +0000] "GET /var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
Feb 14 13:43:04 sl2mmgplsat0001 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (30): mirror.clarkson.edu
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:42:58 -0600] "GET /streamer/var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4zMyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS9lNS9mYmJlODFiZTdlYzhhNWM4NjMyNzIwMGIyMzNlOTc1MzNlYzc1NDQwNzBjZWM1NzhiM2VhZTA1MDIzN2E0Yy9NeVNRTC1weXRob24tMS4yLjUtMS5lbDcueDg2XzY0LnJwbSIsICJleHBpcmF0aW9uIjogMTQ4NzEwMTQ2OH0%3D;signature=LTIdtE_1BOI8k4lfB6YmUjuqQ-invKsucjBRFrc2xdMTKLBWf4zDyteI-P22DJFk6aj9tzOLY5-FYFnpWefNAW34L31ZXzMaxEIS345zl6tjiE-y36WnsMOSyI1D7lnIvOPyd8QmKUDS-gT4BI2PNCd_CwNFWH4o1eYd0KZGwiU-v8qp_CScsZSJSz_A7GCEAO7JvWS99OyQoukvm7Q3cOY_QtXnbEmyqs8_mofzJ14FN2GZUvaaAJO_u4NMyDK4iAJoHnx939vQNBy1Fa31bDeElu5mONQdJ1RjpqH8L_9qdgVt_sgRVhhjgmE30skL7YQXclUgt_vnFmVF5bhSng%3D%3D HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:43:04 -0600] "GET /pulp/repos/tkc-holdings/pnp-production/c7stable/custom/centos/c7/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 302 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/messages <==
Feb 14 13:43:10 sl2mmgplsat0001 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.clarkson.edu due to repeated connection failures: HTTPConnectionPool(host='mirror.clarkson.edu', port=80): Max retries exceeded with url: /centos/7.3.1611/os/x86_64/Packages/MySQL-python-1.2.5-1.el7.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x4690cd0>, 'Connection to mirror.clarkson.edu timed out. (connect timeout=6.05)'))
Feb 14 13:43:10 sl2mmgplsat0001 pulp_streamer: [-] 127.0.0.1 - - [14/Feb/2017:19:43:10 +0000] "GET /var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
172.16.246.33 - 2c9076bc5a25cd69015a3dc17f910176 [14/Feb/2017:13:43:04 -0600] "GET /streamer/var/lib/pulp/content/units/rpm/e5/fbbe81be7ec8a5c86327200b233e97533ec7544070cec578b3eae050237a4c/MySQL-python-1.2.5-1.el7.x86_64.rpm?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTcyLjE2LjI0Ni4zMyJ9LCAicmVzb3VyY2UiOiAiL3N0cmVhbWVyL3Zhci9saWIvcHVscC9jb250ZW50L3VuaXRzL3JwbS9lNS9mYmJlODFiZTdlYzhhNWM4NjMyNzIwMGIyMzNlOTc1MzNlYzc1NDQwNzBjZWM1NzhiM2VhZTA1MDIzN2E0Yy9NeVNRTC1weXRob24tMS4yLjUtMS5lbDcueDg2XzY0LnJwbSIsICJleHBpcmF0aW9uIjogMTQ4NzEwMTQ3NH0%3D;signature=mtpT_N2hwt4gwIgzs51gVBhuikaEDXyoG3MDUPmQOs8NIeh004k1V5OWffJvec-MjKMKi2F8TC7nIiBj3KByqAgCiHiPs1t6JR2M0PYje8RxRFfAIncYVSI1SIRJsFOCNFEcwqd_d8AvWTeIDENPXeiueDRiA0u4HRCmq_ZTs5Ht_fk0s3X0uPRQdclbCLncmdPzjMzt7MItk4T-c0uSpN5-fr04qmj-6vLK6fHGOh4XkvF0WXJJLMr9jvWkSkqF0KDiAzspHrZc36AjHTOjAB6qBWWs7c-0_61fjJXZi4RkFv7lLoqnuS91s7r4Em64G9ROks_VBLttoJdJOSoUPw%3D%3D HTTP/1.1" 503 - "-" "urlgrabber/3.10 yum/3.4.3"
Updated by bizhang about 7 years ago
Jortel will investigate this before we triage.
Updated by bizhang about 7 years ago
- Triaged changed from No to Yes
- Tags Pulp 3 added
Updated by jason@tresgeek.net about 7 years ago
For those who may run into this issue one way I have found to get around it is to find another mirror which uses the same directory (URI) structure and then add an entry in /etc/hosts on your Pulp server to point the host name of the broken mirror to the IP of a working mirror. Of course, there are lots of caveats here such as "doesn't work with https" and "doesn't work if working mirror uses name-based virtual host".
Updated by dkliban@redhat.com about 7 years ago
The issue has had it's 'Triaged' flag set to False so that it will come up again when we hold bug triage tomorrow. Most likely we will not have a plan of action by then, but I hope to discuss it in more detail during our triage on Friday. Feel free to join us[0] if you want to participate in the discussion.
[0] http://docs.pulpproject.org/dev-guide/contributing/bugs.html#triage-process
Updated by ttereshc almost 7 years ago
@jortel will take a closer look and we will triage it next time.
Updated by jortel@redhat.com almost 7 years ago
The way the RPM plugin handles mirror list is as follows:
1. Fetch the list of mirrors.
2. Iterate the mirrors trying to perform the entire sync against the URL.
3. Download (immediate) from the selected mirror.
4. Create catalog entries for the selected mirror.
This approach is flawed. Especially with regard to catalog entries because it populates the catalog with the mirror selected when the sync occurs. If the mirror list changes or the selected mirror goes off line, the sync will fail (perhaps permanently). Changing this for pulp2 would require a significant change in the importer flow. The design for pulp3 is significantly different and will store the mirror list url in the catalog and resolve the mirrors at download time and properly try all the mirrors (round-robin, nearest etc).
I do not recommend we try to fix this for pulp2.
Updated by ttereshc almost 7 years ago
- Severity changed from 2. Medium to 3. High
Updated by lzap@redhat.com over 5 years ago
Guys, I am experiencing similar error when syncing repos via my transparent proxy configured as:
Updated by bmbouter about 5 years ago
- Status changed from NEW to CLOSED - WONTFIX
Updated by bmbouter about 5 years ago
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
Actions
.