Project

Profile

Help

Issue #2031

closed

possible incorrect URL param parsing by streamer

Added by cduryee almost 8 years ago. Updated almost 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.8.6
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

A katello user reported that they got a 403 from pulp from the following url:

 "GET /streamer/var/lib/pulp/content/distribution/ks-CentOS--7-x86_64/images/pxeboot/vmlinuz?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTkyLjE2OC4xNC4xMzEifSwgInJlc291cmNlIjogIi9zdHJlYW1lci92YXIvbGliL3B1bHAvY29udGVudC9kaXN0cmlidXRpb24va3MtQ2VudE9TLS03LXg4Nl82NC9pbWFnZXMvcHhlYm9vdC92bWxpbnV6IiwgImV4cGlyYXRpb24iOiAxNDY1NzgzNTkxfQ==;signature=ANlO8fxrFZ3mi9J8bf64XGBOBeM3Wal49VDtZPlJvwpa2X7ezF6tl8jfC7RKrjwWuWxAotT8UMEKn4foZqodZogao4HGaQkddkcFAPrZ53OjYijF_3P4h8fETImWC2cJkY4Cq0lbNi2tQ96dLe7nEEVioXwN1jYOsm42ZBbKbq3wapTU3bAtnSSzD3AjF9G4n9KRJ-YZLmuk1DNxsKI0sMdjiGKiWqz7jJXyji7pkMP_QoGAhhudQwsdmVzB9H3BEEqFoXYmM0Zl7kqB6sB8Msn-UMPoVyOd1GaJ7Wc-FEs9QQ_CnxgF8xEkaMadM2DbbYRB8R-CUb2NbvH2WRZIPQ== HTTP/1.1" 403 287 "-" "iPXE/1.0.0 (c4bce43)"

associated stack trace is:

[Mon Jun 13 14:05:02.378815 2016] [:error] [pid 2701] [client 192.168.14.131:22683] Traceback (most recent call last):
[Mon Jun 13 14:05:02.378860 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/share/pulp/wsgi/streamer_auth.wsgi", line 36, in allow_access
[Mon Jun 13 14:05:02.378996 2016] [:error] [pid 2701] [client 192.168.14.131:22683] url.validate(key, remote_ip=remote_ip)
[Mon Jun 13 14:05:02.379032 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/lib/python2.7/site-packages/pulp/server/lazy/url.py", line 587, in validate
[Mon Jun 13 14:05:02.379288 2016] [:error] [pid 2701] [client 192.168.14.131:22683] policy, signature = self.bundle
[Mon Jun 13 14:05:02.379319 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/lib/python2.7/site-packages/pulp/server/lazy/url.py", line 565, in bundle
[Mon Jun 13 14:05:02.379364 2016] [:error] [pid 2701] [client 192.168.14.131:22683] query = Query.decode(self.content.query)
[Mon Jun 13 14:05:02.379433 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/lib/python2.7/site-packages/pulp/server/lazy/url.py", line 363, in decode
[Mon Jun 13 14:05:02.379475 2016] [:error] [pid 2701] [client 192.168.14.131:22683] k, v = pair.split('=')
[Mon Jun 13 14:05:02.379500 2016] [:error] [pid 2701] [client 192.168.14.131:22683] ValueError: too many values to unpack

It appears that pair.split() is finding more equals signs than it expects, and can't fill in (k,v).

This is related to http://projects.theforeman.org/issues/15128#note-4.


Related issues

Related to Packaging - Task #2083: Issues common to 2.9.1 and 2.8 streamCLOSED - NOTABUGsemyers

Actions
Actions #1

Updated by cduryee almost 8 years ago

  • Description updated (diff)
Actions #2

Updated by cduryee almost 8 years ago

  • Description updated (diff)
Actions #3

Updated by jcline@redhat.com almost 8 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jcline@redhat.com

While I poked around to make sure this was a Python bug I figured out exactly why it was crashing and burning so I'm just going to fix it right now.

Actions #4

Updated by jcline@redhat.com almost 8 years ago

  • Status changed from ASSIGNED to POST
Actions #5

Updated by jcline@redhat.com almost 8 years ago

To reproduce this issue, you can curl the path in the issue. It will still get a 403, but there won't be a traceback:

1.

 curl -v 'http://localhost/streamer/var/lib/pulp/content/distribution/ks-CentOS--7-x86_64/images/pxeboot/vmlinuz?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTkyLjE2OC4xNC4xMzEifSwgInJlc291cmNlIjogIi9zdHJlYW1lci92YXIvbGliL3B1bHAvY29udGVudC9kaXN0cmlidXRpb24va3MtQ2VudE9TLS03LXg4Nl82NC9pbWFnZXMvcHhlYm9vdC92bWxpbnV6IiwgImV4cGlyYXRpb24iOiAxNDY1NzgzNTkxfQ==;signature=ANlO8fxrFZ3mi9J8bf64XGBOBeM3Wal49VDtZPlJvwpa2X7ezF6tl8jfC7RKrjwWuWxAotT8UMEKn4foZqodZogao4HGaQkddkcFAPrZ53OjYijF_3P4h8fETImWC2cJkY4Cq0lbNi2tQ96dLe7nEEVioXwN1jYOsm42ZBbKbq3wapTU3bAtnSSzD3AjF9G4n9KRJ-YZLmuk1DNxsKI0sMdjiGKiWqz7jJXyji7pkMP_QoGAhhudQwsdmVzB9H3BEEqFoXYmM0Zl7kqB6sB8Msn-UMPoVyOd1GaJ7Wc-FEs9QQ_CnxgF8xEkaMadM2DbbYRB8R-CUb2NbvH2WRZIPQ=='

2. Check /var/log/httpd/error_log and see the traceback from the issue description
3. Apply the fix
4. Repeat 1
5. Check /var/log/httpd/error_log and note there's a denial for the request, but there's no traceback (the signature check is failing).

Alternately, request content that needs to be lazily fetched until you luck out and get a url that's got padding ('=' at the end of the policy or signature)

Actions #6

Updated by amacdona@redhat.com almost 8 years ago

  • Triaged changed from No to Yes

Added by Jeremy Cline over 7 years ago

Revision 9bf5a56e | View on GitHub

Handle queries containing base64 padding gracefully

The lazy URL parser does not work with base64-encoded queries containing padding ('='). It's worth investigating why the standard library parser isn't used, but this is an easy fix for the immediate future.

closes #2031

Added by Jeremy Cline over 7 years ago

Revision 9bf5a56e | View on GitHub

Handle queries containing base64 padding gracefully

The lazy URL parser does not work with base64-encoded queries containing padding ('='). It's worth investigating why the standard library parser isn't used, but this is an easy fix for the immediate future.

closes #2031

Actions #8

Updated by Anonymous over 7 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #9

Updated by semyers over 7 years ago

  • Platform Release set to 2.8.6
Actions #10

Updated by semyers over 7 years ago

  • Status changed from MODIFIED to 5
Actions #11

Updated by semyers over 7 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #12

Updated by semyers over 7 years ago

  • Related to Task #2083: Issues common to 2.9.1 and 2.8 stream added
Actions #14

Updated by bmbouter almost 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF