Project

Profile

Help

Issue #2031

possible incorrect URL param parsing by streamer

Added by cduryee over 4 years ago. Updated 6 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.8.6
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

A katello user reported that they got a 403 from pulp from the following url:

 "GET /streamer/var/lib/pulp/content/distribution/ks-CentOS--7-x86_64/images/pxeboot/vmlinuz?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTkyLjE2OC4xNC4xMzEifSwgInJlc291cmNlIjogIi9zdHJlYW1lci92YXIvbGliL3B1bHAvY29udGVudC9kaXN0cmlidXRpb24va3MtQ2VudE9TLS03LXg4Nl82NC9pbWFnZXMvcHhlYm9vdC92bWxpbnV6IiwgImV4cGlyYXRpb24iOiAxNDY1NzgzNTkxfQ==;signature=ANlO8fxrFZ3mi9J8bf64XGBOBeM3Wal49VDtZPlJvwpa2X7ezF6tl8jfC7RKrjwWuWxAotT8UMEKn4foZqodZogao4HGaQkddkcFAPrZ53OjYijF_3P4h8fETImWC2cJkY4Cq0lbNi2tQ96dLe7nEEVioXwN1jYOsm42ZBbKbq3wapTU3bAtnSSzD3AjF9G4n9KRJ-YZLmuk1DNxsKI0sMdjiGKiWqz7jJXyji7pkMP_QoGAhhudQwsdmVzB9H3BEEqFoXYmM0Zl7kqB6sB8Msn-UMPoVyOd1GaJ7Wc-FEs9QQ_CnxgF8xEkaMadM2DbbYRB8R-CUb2NbvH2WRZIPQ== HTTP/1.1" 403 287 "-" "iPXE/1.0.0 (c4bce43)"

associated stack trace is:

[Mon Jun 13 14:05:02.378815 2016] [:error] [pid 2701] [client 192.168.14.131:22683] Traceback (most recent call last):
[Mon Jun 13 14:05:02.378860 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/share/pulp/wsgi/streamer_auth.wsgi", line 36, in allow_access
[Mon Jun 13 14:05:02.378996 2016] [:error] [pid 2701] [client 192.168.14.131:22683] url.validate(key, remote_ip=remote_ip)
[Mon Jun 13 14:05:02.379032 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/lib/python2.7/site-packages/pulp/server/lazy/url.py", line 587, in validate
[Mon Jun 13 14:05:02.379288 2016] [:error] [pid 2701] [client 192.168.14.131:22683] policy, signature = self.bundle
[Mon Jun 13 14:05:02.379319 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/lib/python2.7/site-packages/pulp/server/lazy/url.py", line 565, in bundle
[Mon Jun 13 14:05:02.379364 2016] [:error] [pid 2701] [client 192.168.14.131:22683] query = Query.decode(self.content.query)
[Mon Jun 13 14:05:02.379433 2016] [:error] [pid 2701] [client 192.168.14.131:22683] File "/usr/lib/python2.7/site-packages/pulp/server/lazy/url.py", line 363, in decode
[Mon Jun 13 14:05:02.379475 2016] [:error] [pid 2701] [client 192.168.14.131:22683] k, v = pair.split('=')
[Mon Jun 13 14:05:02.379500 2016] [:error] [pid 2701] [client 192.168.14.131:22683] ValueError: too many values to unpack

It appears that pair.split() is finding more equals signs than it expects, and can't fill in (k,v).

This is related to http://projects.theforeman.org/issues/15128#note-4.


Related issues

Related to Packaging - Task #2083: Issues common to 2.9.1 and 2.8 streamCLOSED - NOTABUG

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 9bf5a56e View on GitHub
Added by Jeremy Cline over 4 years ago

Handle queries containing base64 padding gracefully

The lazy URL parser does not work with base64-encoded queries containing padding ('='). It's worth investigating why the standard library parser isn't used, but this is an easy fix for the immediate future.

closes #2031

Revision 9bf5a56e View on GitHub
Added by Jeremy Cline over 4 years ago

Handle queries containing base64 padding gracefully

The lazy URL parser does not work with base64-encoded queries containing padding ('='). It's worth investigating why the standard library parser isn't used, but this is an easy fix for the immediate future.

closes #2031

History

#1 Updated by cduryee over 4 years ago

  • Description updated (diff)

#2 Updated by cduryee over 4 years ago

  • Description updated (diff)

#3 Updated by jcline@redhat.com over 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to jcline@redhat.com

While I poked around to make sure this was a Python bug I figured out exactly why it was crashing and burning so I'm just going to fix it right now.

#4 Updated by jcline@redhat.com over 4 years ago

  • Status changed from ASSIGNED to POST

#5 Updated by jcline@redhat.com over 4 years ago

To reproduce this issue, you can curl the path in the issue. It will still get a 403, but there won't be a traceback:

1.

 curl -v 'http://localhost/streamer/var/lib/pulp/content/distribution/ks-CentOS--7-x86_64/images/pxeboot/vmlinuz?policy=eyJleHRlbnNpb25zIjogeyJyZW1vdGVfaXAiOiAiMTkyLjE2OC4xNC4xMzEifSwgInJlc291cmNlIjogIi9zdHJlYW1lci92YXIvbGliL3B1bHAvY29udGVudC9kaXN0cmlidXRpb24va3MtQ2VudE9TLS03LXg4Nl82NC9pbWFnZXMvcHhlYm9vdC92bWxpbnV6IiwgImV4cGlyYXRpb24iOiAxNDY1NzgzNTkxfQ==;signature=ANlO8fxrFZ3mi9J8bf64XGBOBeM3Wal49VDtZPlJvwpa2X7ezF6tl8jfC7RKrjwWuWxAotT8UMEKn4foZqodZogao4HGaQkddkcFAPrZ53OjYijF_3P4h8fETImWC2cJkY4Cq0lbNi2tQ96dLe7nEEVioXwN1jYOsm42ZBbKbq3wapTU3bAtnSSzD3AjF9G4n9KRJ-YZLmuk1DNxsKI0sMdjiGKiWqz7jJXyji7pkMP_QoGAhhudQwsdmVzB9H3BEEqFoXYmM0Zl7kqB6sB8Msn-UMPoVyOd1GaJ7Wc-FEs9QQ_CnxgF8xEkaMadM2DbbYRB8R-CUb2NbvH2WRZIPQ=='

2. Check /var/log/httpd/error_log and see the traceback from the issue description
3. Apply the fix
4. Repeat 1
5. Check /var/log/httpd/error_log and note there's a denial for the request, but there's no traceback (the signature check is failing).

Alternately, request content that needs to be lazily fetched until you luck out and get a url that's got padding ('=' at the end of the policy or signature)

#6 Updated by amacdona@redhat.com over 4 years ago

  • Triaged changed from No to Yes

#8 Updated by Anonymous over 4 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#9 Updated by semyers over 4 years ago

  • Platform Release set to 2.8.6

#10 Updated by semyers over 4 years ago

  • Status changed from MODIFIED to 5

#11 Updated by semyers over 4 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#12 Updated by semyers over 4 years ago

  • Related to Task #2083: Issues common to 2.9.1 and 2.8 stream added

#14 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF