Issue #2031: possible incorrect URL param parsing by streamer - Pulp

Actions

Send by e-mail Copy link

Issue #2031

closed

possible incorrect URL param parsing by streamer

Added by cduryee over 8 years ago. Updated over 4 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

jcline@redhat.com

Category:

Sprint/Milestone:

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version:

Platform Release:

2.8.6

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Pulp 2

Sprint:

Quarter:

Description

A katello user reported that they got a 403 from pulp from the following url:

 "GET /streamer/var/lib/pulp/content/distribution/ks-CentOS--7-x86_64/images/pxeboot/vmlinuz?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTkyLjE2OC4xNC4xMzEifSwgInJlc291cmNlIjogIi9zdHJlYW1lci92YXIvbGliL3B1bHAvY29udGVudC9kaXN0cmlidXRpb24va3MtQ2VudE9TLS03LXg4Nl82NC9pbWFnZXMvcHhlYm9vdC92bWxpbnV6IiwgImV4cGlyYXRpb24iOiAxNDY1NzgzNTkxfQ==;signature=ANlO8fxrFZ3mi9J8bf64XGBOBeM3Wal49VDtZPlJvwpa2X7ezF6tl8jfC7RKrjwWuWxAotT8UMEKn4foZqodZogao4HGaQkddkcFAPrZ53OjYijF_3P4h8fETImWC2cJkY4Cq0lbNi2tQ96dLe7nEEVioXwN1jYOsm42ZBbKbq3wapTU3bAtnSSzD3AjF9G4n9KRJ-YZLmuk1DNxsKI0sMdjiGKiWqz7jJXyji7pkMP_QoGAhhudQwsdmVzB9H3BEEqFoXYmM0Zl7kqB6sB8Msn-UMPoVyOd1GaJ7Wc-FEs9QQ_CnxgF8xEkaMadM2DbbYRB8R-CUb2NbvH2WRZIPQ== HTTP/1.1" 403 287 "-" "iPXE/1.0.0 (c4bce43)"

associated stack trace is:

[Mon Jun 13 14:05:02.378815 2016] [:error] [pid 2701] [client 192.168.14.131:22683] Traceback (most recent call last):
[Mon Jun 13 14:05:02.378860 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/share/pulp/wsgi/streamer_auth.wsgi", line 36, in allow_access
[Mon Jun 13 14:05:02.378996 2016] [:error] [pid 2701] [client 192.168.14.131:22683] url.validate(key, remote_ip=remote_ip)
[Mon Jun 13 14:05:02.379032 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/lib/python2.7/site-packages/pulp/server/lazy/url.py", line 587, in validate
[Mon Jun 13 14:05:02.379288 2016] [:error] [pid 2701] [client 192.168.14.131:22683] policy, signature = self.bundle
[Mon Jun 13 14:05:02.379319 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/lib/python2.7/site-packages/pulp/server/lazy/url.py", line 565, in bundle
[Mon Jun 13 14:05:02.379364 2016] [:error] [pid 2701] [client 192.168.14.131:22683] query = Query.decode(self.content.query)
[Mon Jun 13 14:05:02.379433 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/lib/python2.7/site-packages/pulp/server/lazy/url.py", line 363, in decode
[Mon Jun 13 14:05:02.379475 2016] [:error] [pid 2701] [client 192.168.14.131:22683] k, v = pair.split('=')
[Mon Jun 13 14:05:02.379500 2016] [:error] [pid 2701] [client 192.168.14.131:22683] ValueError: too many values to unpack

It appears that pair.split() is finding more equals signs than it expects, and can't fill in (k,v).

This is related to http://projects.theforeman.org/issues/15128#note-4.

Related issues

Actions

Copy link

Updated by cduryee over 8 years ago

Description updated (diff)

Actions

Copy link

Updated by cduryee over 8 years ago

Description updated (diff)

Actions

Copy link

Updated by jcline@redhat.com over 8 years ago

Status changed from NEW to ASSIGNED
Assignee set to jcline@redhat.com

While I poked around to make sure this was a Python bug I figured out exactly why it was crashing and burning so I'm just going to fix it right now.

Actions

Copy link

Updated by jcline@redhat.com over 8 years ago

Status changed from ASSIGNED to POST

https://github.com/pulp/pulp/pull/2614

Actions

Copy link

Updated by jcline@redhat.com over 8 years ago

To reproduce this issue, you can curl the path in the issue. It will still get a 403, but there won't be a traceback:

 curl -v 'http://localhost/streamer/var/lib/pulp/content/distribution/ks-CentOS--7-x86_64/images/pxeboot/vmlinuz?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTkyLjE2OC4xNC4xMzEifSwgInJlc291cmNlIjogIi9zdHJlYW1lci92YXIvbGliL3B1bHAvY29udGVudC9kaXN0cmlidXRpb24va3MtQ2VudE9TLS03LXg4Nl82NC9pbWFnZXMvcHhlYm9vdC92bWxpbnV6IiwgImV4cGlyYXRpb24iOiAxNDY1NzgzNTkxfQ==;signature=ANlO8fxrFZ3mi9J8bf64XGBOBeM3Wal49VDtZPlJvwpa2X7ezF6tl8jfC7RKrjwWuWxAotT8UMEKn4foZqodZogao4HGaQkddkcFAPrZ53OjYijF_3P4h8fETImWC2cJkY4Cq0lbNi2tQ96dLe7nEEVioXwN1jYOsm42ZBbKbq3wapTU3bAtnSSzD3AjF9G4n9KRJ-YZLmuk1DNxsKI0sMdjiGKiWqz7jJXyji7pkMP_QoGAhhudQwsdmVzB9H3BEEqFoXYmM0Zl7kqB6sB8Msn-UMPoVyOd1GaJ7Wc-FEs9QQ_CnxgF8xEkaMadM2DbbYRB8R-CUb2NbvH2WRZIPQ=='

2. Check /var/log/httpd/error_log and see the traceback from the issue description
3. Apply the fix
4. Repeat 1
5. Check /var/log/httpd/error_log and note there's a denial for the request, but there's no traceback (the signature check is failing).

Alternately, request content that needs to be lazily fetched until you luck out and get a url that's got padding ('=' at the end of the policy or signature)

Actions

Copy link

Updated by amacdona@redhat.com over 8 years ago

Triaged changed from No to Yes

Added by Jeremy Cline over 8 years ago

Revision 9bf5a56e | View on GitHub

Handle queries containing base64 padding gracefully

The lazy URL parser does not work with base64-encoded queries containing padding ('='). It's worth investigating why the standard library parser isn't used, but this is an easy fix for the immediate future.

closes #2031