Project

Profile

Help

Issue #1140

API call to URLs that do not exist cause 500s on RHEL 6

Added by amacdona@redhat.com about 6 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
2.7.0 Beta
Platform Release:
2.7.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

When making a call to a URL that does not exist, Pulp should respond with a 404. However, with RHEL 6 (and Django 1.4), the middleware is attempting to render a nonexistent 404 template and therefore returns a 500.

curl  -X GET -k -u admin:admin "https://localhost/pulp/api/v2/doesnotexist/"
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator,
 root@localhost and inform them of the time the error occurred,
and anything you might have done that may have
caused the error.</p>
<p>More information about this error may be available
in the server error log.</p>
<hr>
<address>Apache/2.2.15 (Red Hat) Server at localhost Port 443</address>
</body></html>

Traceback:

Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680) Internal Server Error: /pulp/api/v2/doesnotexist/
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680) Traceback (most recent call last):
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680)   File "/usr/lib/python2.6/site-packages/django/core/handlers/base.py", line 148, in get_response
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680)     response = callback(request, **param_dict)
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680)   File "/usr/lib/python2.6/site-packages/django/utils/decorators.py", line 91, in _wrapped_view
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680)     response = view_func(request, *args, **kwargs)
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680)   File "/usr/lib/python2.6/site-packages/django/views/defaults.py", line 20, in page_not_found
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680)     t = loader.get_template(template_name) # You need to create a 404.html template.
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680)   File "/usr/lib/python2.6/site-packages/django/template/loader.py", line 145, in get_template
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680)     template, origin = find_template(template_name)
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680)   File "/usr/lib/python2.6/site-packages/django/template/loader.py", line 138, in find_template
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680)     raise TemplateDoesNotExist(name)
Jul 15 10:26:44 mgmt4 pulp: django.request:ERROR: (14267-91680) TemplateDoesNotExist: 404.html

This does not affect 404s that are raised as a MissingResource.


Related issues

Related to Puppet Support - Issue #1485: TemplateDoesNotExist exception for '500.html' raised on v3 API endpointCLOSED - WONTFIX<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to Puppet Support - Issue #1486: TemplateDoesNotExist exception for '404.html' raised on v3 API endpointCLOSED - WONTFIX<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 996e4366 View on GitHub
Added by bmbouter about 6 years ago

Adds a 404 handler for Django which responds with JSON

https://pulp.plan.io/issues/1140 closes #1140

Revision 996e4366 View on GitHub
Added by bmbouter about 6 years ago

Adds a 404 handler for Django which responds with JSON

https://pulp.plan.io/issues/1140 closes #1140

History

#1 Updated by bmbouter about 6 years ago

We originally did not experience this exception because we were using DEBUG=True but when we set that to DEBUG=False by fixing issue #910 we now have to provide a 404.html file and probably a 500.html in the template root. Setting the template root is our case is a bit tricky.

There are two default template loaders in Django 1.4+, 'django.template.loaders.filesystem.Loader' and 'django.template.loaders.app_directories.Loader'. The loader 'django.template.loaders.app_directories.Loader' looks for a templates directory in each INSTALLED_APP listed in settings.py but we've avoided making one of those because we don't really need one and I don't feel this is a good enough reason to make one so that loader won't benefit us. The other loader 'django.template.loaders.filesystem.Loader' looks for absolute paths as specified by the TEMPLATE_DIRS setting and the absolute path won't be the same on all platforms.

One fix would be to create a "templates" directory in the webservices directory and have the full path used by TEMPLATE_DIRS be created dynamically as follows:

import os
DIRNAME = os.path.dirname(__file__)
TEMPLATE_DIRS = (
    os.path.join(DIRNAME, "templates"),
)

Then we could put an empty 404.html and 500.html in that directory and I think it would resolve the issue.

Another option is to catch these errors and add the correct behavior in the pulp exception handling middleware directly. I like that solution slightly better because we could set the content_type of the return to json which is more correct since it's from the API. We need to be careful here though that the following are true:

  • 404's don't turn into 500's by the time the middleware is returned
  • 500 errors are truly the stacktrace of the actual error, not just Django's inability to find the 500.html file
  • we'll probably need to give some thought to how the json structure is of a 500 error and a 404

This isn't a clear-cut implementation outline but two possible paths. This probably needs to be done with some care.

#2 Updated by bmbouter about 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bmbouter

#3 Updated by jortel@redhat.com about 6 years ago

  • Triaged changed from No to Yes

#4 Updated by bmbouter about 6 years ago

I was looking into a solution that adjusted the middleware when I learned that a 404 error does not even flow through the middleware so we can't fix it there. If the url does not match anything in urls.py it is handled by the default page_not_found handler directly. The 500 exception this bug shows is in that handler attempting to form the 404 response. The solution I'm putting together involves us

As such we can write our own 404 handler and not even make a template at all. This also will correctly set the response status code, content type, and contain an empty JSON body. :-)

We identify that Django should use our handler by setting handler404 in urls.py. I tested it and it works well.

I also investigated if 500 errors would be an issue given that we don't have a 500.html, and I don't think they are because our middleware correctly serializes them into a reasonable json representation like the following:

{
  "exception": [
    "IOError: asdf\n"
  ], 
  "traceback": [
    "  File \"/usr/lib/python2.7/site-packages/django/core/handlers/base.py\", line 109, in get_response\n    response = callback(request, *callback_args, **callback_kwargs)\n", 
    "  File \"/usr/lib/python2.7/site-packages/django/views/generic/base.py\", line 48, in view\n    return self.dispatch(request, *args, **kwargs)\n", 
    "  File \"/usr/lib/python2.7/site-packages/django/views/generic/base.py\", line 69, in dispatch\n    return handler(request, *args, **kwargs)\n", 
    "  File \"/home/bmbouter/Documents/pulp/server/pulp/server/webservices/views/decorators.py\", line 237, in _auth_decorator\n    return _verify_auth(self, operation, super_user_only, method, *args, **kwargs)\n", 
    "  File \"/home/bmbouter/Documents/pulp/server/pulp/server/webservices/views/decorators.py\", line 191, in _verify_auth\n    value = method(self, *args, **kwargs)\n", 
    "  File \"/home/bmbouter/Documents/pulp/server/pulp/server/webservices/views/util.py\", line 110, in wrapper\n    return func(*args, **kwargs)\n", 
    "  File \"/home/bmbouter/Documents/pulp/server/pulp/server/webservices/views/repositories.py\", line 171, in post\n    raise IOError('asdf')\n"
  ], 
  "_href": "/pulp/api/v2/repositories/", 
  "error_message": "asdf", 
  "http_request_method": "POST", 
  "http_status": 500
}

I need to go write some tests for this now and put in my PR.

#5 Updated by bmbouter about 6 years ago

  • Status changed from ASSIGNED to POST

#6 Updated by bmbouter about 6 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#7 Updated by dkliban@redhat.com almost 6 years ago

  • Status changed from MODIFIED to 5

#8 Updated by amacdona@redhat.com over 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#9 Updated by bmbouter over 5 years ago

  • Related to Issue #1485: TemplateDoesNotExist exception for '500.html' raised on v3 API endpoint added

#10 Updated by bmbouter over 5 years ago

  • Related to Issue #1486: TemplateDoesNotExist exception for '404.html' raised on v3 API endpoint added

#11 Updated by bmbouter over 2 years ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF