Project

Profile

Help

Download (19.9 KB)

Issue #1170 ยป Issue #5864 - 2015-08-21T16_08_00Z.eml

osabina, 08/21/2015 06:08 PM

 
Return-Path: <osabina@g8r.com>
Received: from mi010.mc1.hosteurope.de ([80.237.138.245]) by wp245.webpack.hosteurope.de running ExIM with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) id 1ZSor0-0003nd-W6; Fri, 21 Aug 2015 18:07:43 +0200
Received: from mail-yk0-f176.google.com ([209.85.160.176]) by mx0.webpack.hosteurope.de (mi010.mc1.hosteurope.de) with esmtps (TLSv1.2:AES128-GCM-SHA256:128) id 1ZSoqz-0001Lu-Fr for dropbox+pulp+c71e@plan.io; Fri, 21 Aug 2015 18:07:42 +0200
Received: by ykbi184 with SMTP id i184so74845151ykb.2 for <dropbox+pulp+c71e@plan.io>; Fri, 21 Aug 2015 09:07:39 -0700
Received: from rsg4110ozsabina.rsg.lan ([198.2.191.56]) by smtp.gmail.com with ESMTPSA id r129sm7785918ywd.26.2015.08.21.09.07.38 for <dropbox+pulp+c71e@plan.io> (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 21 Aug 2015 09:07:38 -0700
Date: Fri, 21 Aug 2015 12:07:38 -0400
From: Ozzie Sabina <oz@sabina.org>
Sender: Ozzie Sabina <osabina@g8r.com>
To: Pulp <dropbox+pulp+c71e@plan.io>
Message-ID: <E5258A81-063A-486E-AA53-A38390CF22D2@sabina.org>
In-Reply-To: <redmine.journal-5855.20150821144542.23a524d342b525df@plan.io>
References: <redmine.issue-1170.20150807201735@plan.io>
<redmine.journal-5855.20150821144542.23a524d342b525df@plan.io>
Subject: Re: [Pulp - Issue #1170] rpm importer does not support switching
protocols from http to ftp mid download using a redirect- fails downloading
distribution files
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="Apple-Mail=_62BF8401-0EBD-46F8-A5E6-2B457DCFCCE7";
charset=UTF-8
Content-Transfer-Encoding: 7bit
Delivery-date: Fri, 21 Aug 2015 18:07:43 +0200
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=g8r.com; s=google;
h=sender:from:content-type:message-id:mime-version:subject:date
:references:to:in-reply-to; bh=jLNOoOLXgzYWmV9iWHGfxdxKKO1wbwUbTu8yxyewXks=;
b=WbCpLOjAK4vN0b/SwiSzdtgj5RKHeppUJgY3qWWCzE3M5VCUfg+rnVnUNul77nS9tS
uvX927aoMMwaB3HbEay354VUDbcn/Kp5Ha2iR+ryUcszoQN/eGWH3bzLq/8a+/vp5aU5
TvZHAUFpsreuIuLJg5wWR60g6SP26j5LoHTxI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net;
s=20130820;
h=x-gm-message-state:sender:from:content-type:message-id:mime-version
:subject:date:references:to:in-reply-to;
bh=jLNOoOLXgzYWmV9iWHGfxdxKKO1wbwUbTu8yxyewXks=;
b=VI9H3rgt/O+vID+2YOT5p1oW23D88i3oPnh8ODGvEGqIw3YBm726cXiFsbB4gh3ecv
8skA5uKzaobPPFKyoP2Hd8lJSZ5oJJWXRTtBHnqEqgN+iz5shQLT49UbM7UsvMbpnIng
r7aTDzcTXbb+3WtI1QUAVMPeTHvVLm7LAY8OTtXq5QFXgD/y5iJQrb+XvzP6rOnXa8hx
4xsUwukbwl8tJ6OZxkCq4E8vix7asocRJ2vd0dCykj2FIjF2px8vGyxvT6blV2+f0vi9
rY3REAGG6fZ9/fBcCXKLt8GgvVzhUtOoyQ0WbsTZOCvPWWDtIFXx1SyZv5QENsVB7cjs PFFg==
X-Gm-Message-State: ALoCoQm+3W/E2H/7uxT+/G3kvAj0lFKFAsNQOuX0gZKf2w2uFrfY9Ew7jb0hz3ZApxQJhLGlGS0l
X-Received: by 10.170.47.145 with SMTP id 139mr12359798ykp.97.1440173259252;
Fri, 21 Aug 2015 09:07:39 -0700 (PDT)
X-Mailer: Apple Mail (2.2102)
X-HE-Spam-Level: /
X-HE-Spam-Score: -0.6
X-HE-Spam-Report: Content analysis details: (-0.6 points) pts rule name
description ---- ----------------------
-------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL:
Sender listed at http://www.dnswl.org/, low trust [209.85.160.176 listed in
list.dnswl.org] 0.1 HTML_MESSAGE BODY: HTML included in message -0.1
DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED
Message has a DKIM or DK signature, not necessarily valid
X-HE-SPF: PASSED
Envelope-to: dropbox+pulp+c71e@plan.io


--Apple-Mail=_62BF8401-0EBD-46F8-A5E6-2B457DCFCCE7
Content-Type: text/plain;
charset=utf-8
Content-Transfer-Encoding: quoted-printable


From what I could tell nectar only uses Requests.py which only supports h=
ttp. I don=E2=80=99t think the switch is the problem.

Oz

> On Aug 21, 2015, at 10:45, Pulp <dropbox+pulp+c71e@plan.io> wrote:
> =

> --- Please write your response above this line ---
> =

> Issue #1170 has been updated by bcourt.
> Subject changed from rpm importer does not support ftp - fails download=
ing distribution files to rpm importer does not support switching protoco=
ls from http to ftp mid download using a redirect- fails downloading dist=
ribution files
> Triaged changed from No to Yes
> This appears to be happening because the web server is redirecting from=
an http to an ftp url and nectar is not following the protocol change.
> =

> Issue #1170: rpm importer does not support switching protocols from htt=
p to ftp mid download using a redirect- fails downloading distribution fi=
les <https://pulp.plan.io/issues/1170#change-5855>
> Author: osabina
> Status: NEW
> Priority: Normal
> Assignee: =

> Category: =

> Severity: 2. Medium
> Version: 2.6.3
> Target Release: =

> OS: CentOS 6
> Backwards Incompatible: No
> Triaged: Yes
> Tags: =

> QA Contact: =

> I was unsure whether to open this as an issue here, with Nectar or with=
RPM Support so defaulting to top level.
> =

> To reproduce:
> =

> pulp-admin rpm repo create --relative-url test --feed http://www.gtlib.=
gatech.edu/pub/centos/6.6/os/x86_64/ <http://www.gtlib.gatech.edu/pub/cen=
tos/6.6/os/x86_64/> --repo-id test
> pulp-admin rpm repo sync run --repo-id test
> [...]
> Downloading distribution files...
> [=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ] 75%
> Distributions: 3/4 items
> =

> Errors encountered during distribution sync:
> =

> File: http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.=
iso <http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso=
>
> Error
> Code: None
> Error Message: None
> =

> This appears to be because Nectar is using requests.py which does not s=
upport ftp. From my logs:
> =

> Aug 7 18:28:20 pulp: requests.packages.urllib3.connectionpool:INFO: Sta=
rting new HTTP connection (1): www.gtlib.gatech.edu <http://www.gtlib.gat=
ech.edu/>
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) N=
o connection adapters were found for 'ftp://ftp.gtlib.gatech.edu/pub/cent=
os/6.6/os/x86_64/images/boot.iso' <ftp://ftp.gtlib.gatech.edu/pub/centos/=
6.6/os/x86_64/images/boot.iso'>
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) T=
raceback (most recent call last):
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) F=
ile "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", li=
ne 190, in _fetch
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) r=
esponse =3D session.get(request.url, headers=3Dheaders)
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) F=
ile "/usr/lib/python2.6/site-packages/requests/sessions.py", line 469, in=
get
> Aug 7 18:28:20 pulp: nectar.download ers.threaded:ERROR: (25682-61856) =
return self.request('GET', url, **kwargs)
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) F=
ile "/usr/lib/python2.6/site-packages/requests/sessions.py", line 457, in=
request
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) r=
esp =3D self.send(prep, **send_kwargs)
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) F=
ile "/usr/lib/python2.6/site-packages/requests/sessions.py", line 595, in=
send
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) h=
istory =3D [resp for resp in gen] if allow_redirects else []
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) F=
ile "/usr/lib/python2.6/site-packages/requests/sessions.py", line 189, in=
resolve_redirects
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) a=
llow_redirects=3DFalse,
> Aug 7 18:28:20 pulp: nectar.downloaders.threa ded:ERROR: (25682-61856) =
File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 563, i=
n send
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) a=
dapter =3D self.get_adapter(url=3Drequest.url)
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) F=
ile "/usr/lib/python2.6/site-packages/requests/sessions.py", line 642, in=
get_adapter
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) r=
aise InvalidSchema("No connection adapters were found for '%%s'" %% url)
> Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) I=
nvalidSchema: No connection adapters were found for 'ftp://ftp.gtlib.gate=
ch.edu/pub/centos/6.6/os/x86_64/images/boot.iso' <ftp://ftp.gtlib.gatech.=
edu/pub/centos/6.6/os/x86_64/images/boot.iso'>
> Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERRO=
R: some distro file downloads failed
> Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Sync com=
plete.
> =

> This is likely a related issue - https://pulp.plan.io/issues/197 <https=
://pulp.plan.io/issues/197>
> I worked around this by finding another mirror that didn't point to the=
ir ftp server to download this iso.
> =

> You have received this notification because you have either subscribed =
to or are involved in a project on Pulp Planio.
> To change your notification preferences, please click here: https://pul=
p.plan.io/my/account?tour=3Dmail_preferences <https://pulp.plan.io/my/acc=
ount?tour=3Dmail_preferences> =

> =

> This notification was cheerfully delivered by <https://plan.io/> =

> <https://plan.io/>

--Apple-Mail=_62BF8401-0EBD-46F8-A5E6-2B457DCFCCE7
Content-Type: text/html;
charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html charse=
t=3Dutf-8"></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode=
: space; -webkit-line-break: after-white-space;" class=3D""><div class=3D=
""><br class=3D""></div><div class=3D"">From what I could tell nectar onl=
y uses Requests.py which only supports http. &nbsp;I don=E2=80=99t think =
the switch is the problem.</div><div class=3D""><br class=3D""></div><div=
class=3D"">Oz</div><br class=3D""><div><blockquote type=3D"cite" class=3D=
""><div class=3D"">On Aug 21, 2015, at 10:45, Pulp &lt;<a href=3D"mailto:=
dropbox+pulp+c71e@plan.io" class=3D"">dropbox+pulp+c71e@plan.io</a>&gt; w=
rote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><table=
width=3D"100%" style=3D"font-family: ProximaNova-Regular, Verdana, sans-=
serif; letter-spacing: normal; orphans: auto; text-indent: 0px; text-tran=
sform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><tbody class=3D""><tr class=3D""><td style=3D"font-famil=
y: MarketWeb, Verdana, sans-serif; font-size: 0.8em; text-align: center; =
width: 1099px; color: rgb(215, 215, 215);" class=3D""><p class=3D"">--- P=
lease write your response above this line ---</p></td></tr><tr class=3D""=
><td class=3D"">Issue #1170 has been updated by bcourt.<ul class=3D""><li=
class=3D""><strong class=3D"">Subject</strong><span class=3D"Apple-conve=
rted-space">&nbsp;</span>changed from<span class=3D"Apple-converted-space=
">&nbsp;</span><i class=3D"">rpm importer does not support ftp - fails do=
wnloading distribution files</i><span class=3D"Apple-converted-space">&nb=
sp;</span>to<span class=3D"Apple-converted-space">&nbsp;</span><i class=3D=
"">rpm importer does not support switching protocols from http to ftp mid=
download using a redirect- fails downloading distribution files</i></li>=
<li class=3D""><strong class=3D"">Triaged</strong><span class=3D"Apple-co=
nverted-space">&nbsp;</span>changed from<span class=3D"Apple-converted-sp=
ace">&nbsp;</span><i class=3D"">No</i><span class=3D"Apple-converted-spac=
e">&nbsp;</span>to<span class=3D"Apple-converted-space">&nbsp;</span><i c=
lass=3D"">Yes</i></li></ul><p class=3D"">This appears to be happening bec=
ause the web server is redirecting from an http to an ftp url and nectar =
is not following the protocol change.</p><hr style=3D"width: 1099px; heig=
ht: 1px; background-color: rgb(204, 204, 204); border: 0px; background-po=
sition: initial initial; background-repeat: initial initial;" class=3D"">=
<h1 style=3D"font-family: ProximaNova-Bold, 'Trebuchet MS', Verdana, sans=
-serif; margin: 0px; font-size: 1.2em;" class=3D""><a href=3D"https://pul=
p.plan.io/issues/1170#change-5855" style=3D"color: rgb(43, 122, 148);" cl=
ass=3D"">Issue #1170: rpm importer does not support switching protocols f=
rom http to ftp mid download using a redirect- fails downloading distribu=
tion files</a></h1><ul class=3D""><li class=3D"">Author: osabina</li><li =
class=3D"">Status: NEW</li><li class=3D"">Priority: Normal</li><li class=3D=
"">Assignee:<span class=3D"Apple-converted-space">&nbsp;</span></li><li c=
lass=3D"">Category:<span class=3D"Apple-converted-space">&nbsp;</span></l=
i><li class=3D"">Severity: 2. Medium</li><li class=3D"">Version: 2.6.3</l=
i><li class=3D"">Target Release:<span class=3D"Apple-converted-space">&nb=
sp;</span></li><li class=3D"">OS: CentOS 6</li><li class=3D"">Backwards I=
ncompatible: No</li><li class=3D"">Triaged: Yes</li><li class=3D"">Tags:<=
span class=3D"Apple-converted-space">&nbsp;</span></li><li class=3D"">QA =
Contact:<span class=3D"Apple-converted-space">&nbsp;</span></li></ul><p c=
lass=3D"">I was unsure whether to open this as an issue here, with Nectar=
or with RPM Support so defaulting to top level.</p><p class=3D"">To repr=
oduce:</p><p class=3D"">pulp-admin rpm repo create --relative-url test --=
feed<span class=3D"Apple-converted-space">&nbsp;</span><a class=3D"extern=
al" href=3D"http://www.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/" style=3D=
"color: rgb(43, 122, 148);">http://www.gtlib.gatech.edu/pub/centos/6.6/os=
/x86_64/</a><span class=3D"Apple-converted-space">&nbsp;</span>--repo-id =
test<br class=3D"">pulp-admin rpm repo sync run --repo-id test<br class=3D=
"">[...]<br class=3D"">Downloading distribution files...<br class=3D"">[=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ] 75%<br class=3D"">Distributions: 3/4 =
items</p><p class=3D"">Errors encountered during distribution sync:</p><p=
class=3D"">File:<span class=3D"Apple-converted-space">&nbsp;</span><a cl=
ass=3D"external" href=3D"http://www.gtlib.gatech.edu/pub/centos/6.6/os/x8=
6_64/images/boot.iso" style=3D"color: rgb(43, 122, 148);">http://www.gtli=
b.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso</a><br class=3D"">E=
rror<br class=3D"">Code: None<br class=3D"">Error Message: None</p><p cla=
ss=3D"">This appears to be because Nectar is using requests.py which does=
not support ftp. From my logs:</p><p class=3D"">Aug 7 18:28:20 pulp: req=
uests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection =
(1):<span class=3D"Apple-converted-space">&nbsp;</span><a class=3D"extern=
al" href=3D"http://www.gtlib.gatech.edu/" style=3D"color: rgb(43, 122, 14=
8);">www.gtlib.gatech.edu</a><br class=3D"">Aug 7 18:28:20 pulp: nectar.d=
ownloaders.threaded:ERROR: (25682-61856) No connection adapters were foun=
d for '<a href=3D"ftp://ftp.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/ima=
ges/boot.iso'" style=3D"color: rgb(43, 122, 148);" class=3D"">ftp://ftp.g=
tlib.gatech.edu/pub/centos/6.6/os/x86_64/images/boot.iso'</a><br class=3D=
"">Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) =
Traceback (most recent call last):<br class=3D"">Aug 7 18:28:20 pulp: nec=
tar.downloaders.threaded:ERROR: (25682-61856) File "/usr/lib/python2.6/si=
te-packages/nectar/downloaders/threaded.py", line 190, in _fetch<br class=
=3D"">Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-6185=
6) response =3D session.get(request.url, headers=3Dheaders)<br class=3D""=
>Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) Fi=
le "/usr/lib/python2.6/site-packages/requests/sessions.py", line 469, in =
get<br class=3D"">Aug 7 18:28:20 pulp: nectar.download ers.threaded:ERROR=
: (25682-61856) return self.request('GET', url, **kwargs)<br class=3D"">A=
ug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File=
"/usr/lib/python2.6/site-packages/requests/sessions.py", line 457, in re=
quest<br class=3D"">Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERRO=
R: (25682-61856) resp =3D self.send(prep, **send_kwargs)<br class=3D"">Au=
g 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (25682-61856) File =
"/usr/lib/python2.6/site-packages/requests/sessions.py", line 595, in sen=
d<br class=3D"">Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (=
25682-61856) history =3D [resp for resp in gen] if allow_redirects else [=
]<br class=3D"">Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (=
25682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py"=
, line 189, in resolve_redirects<br class=3D"">Aug 7 18:28:20 pulp: necta=
r.downloaders.threaded:ERROR: (25682-61856) allow_redirects=3DFalse,<br c=
lass=3D"">Aug 7 18:28:20 pulp: nectar.downloaders.threa ded:ERROR: (25682=
-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py", lin=
e 563, in send<br class=3D"">Aug 7 18:28:20 pulp: nectar.downloaders.thre=
aded:ERROR: (25682-61856) adapter =3D self.get_adapter(url=3Drequest.url)=
<br class=3D"">Aug 7 18:28:20 pulp: nectar.downloaders.threaded:ERROR: (2=
5682-61856) File "/usr/lib/python2.6/site-packages/requests/sessions.py",=
line 642, in get_adapter<br class=3D"">Aug 7 18:28:20 pulp: nectar.downl=
oaders.threaded:ERROR: (25682-61856) raise InvalidSchema("No connection a=
dapters were found for '%%s'" %% url)<br class=3D"">Aug 7 18:28:20 pulp: =
nectar.downloaders.threaded:ERROR: (25682-61856) InvalidSchema: No connec=
tion adapters were found for '<a href=3D"ftp://ftp.gtlib.gatech.edu/pub/c=
entos/6.6/os/x86_64/images/boot.iso'" style=3D"color: rgb(43, 122, 148);"=
class=3D"">ftp://ftp.gtlib.gatech.edu/pub/centos/6.6/os/x86_64/images/bo=
ot.iso'</a><br class=3D"">Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers=
.yum.parse.treeinfo:ERROR: some distro file downloads failed<br class=3D"=
">Aug 7 18:28:55 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Sync com=
plete.</p><p class=3D"">This is likely a related issue -<span class=3D"Ap=
ple-converted-space">&nbsp;</span><a class=3D"external" href=3D"https://p=
ulp.plan.io/issues/197" style=3D"color: rgb(43, 122, 148);">https://pulp.=
plan.io/issues/197</a></p><p class=3D"">I worked around this by finding a=
nother mirror that didn't point to their ftp server to download this iso.=
</p></td></tr><tr class=3D""><td style=3D"font-size: 0.8em; width: 1099px=
;" class=3D""><hr style=3D"width: 1099px; height: 1px; background-color: =
rgb(204, 204, 204); border: 0px; background-position: initial initial; ba=
ckground-repeat: initial initial;" class=3D""><p class=3D"">You have rece=
ived this notification because you have either subscribed to or are invol=
ved in a project on Pulp Planio.<br class=3D"">To change your notificatio=
n preferences, please click here:<span class=3D"Apple-converted-space">&n=
bsp;</span><a class=3D"external" href=3D"https://pulp.plan.io/my/account?=
tour=3Dmail_preferences" style=3D"color: rgb(43, 122, 148);">https://pulp=
.plan.io/my/account?tour=3Dmail_preferences</a></p></td><td class=3D""></=
td></tr><tr class=3D""><td style=3D"font-family: MarketWeb, Verdana, sans=
-serif; font-size: 1.2em; text-align: center; width: 1099px; color: rgb(2=
15, 215, 215);" class=3D""><br class=3D""><div class=3D""><a href=3D"http=
s://plan.io/" style=3D"color: rgb(215, 215, 215); text-decoration: none;"=
class=3D"">This notification was cheerfully delivered by</a></div></td><=
td class=3D""></td></tr><tr class=3D""><td style=3D"text-align: center; w=
idth: 1099px;" class=3D""><a href=3D"https://plan.io/" title=3D"Planio" s=
tyle=3D"color: rgb(43, 122, 148);" class=3D""><img src=3D"https://assets.=
plan.io/images/planio_logo_gray_204x50.png" height=3D"25" width=3D"102" a=
lt=3D"Planio" style=3D"vertical-align: middle;" class=3D""></a></td></tr>=
</tbody></table></div></blockquote></div><br class=3D""></body></html>=

--Apple-Mail=_62BF8401-0EBD-46F8-A5E6-2B457DCFCCE7--
    (1-1/1)