Web development with Python

Table of Content

Introduction

Primarily two kinds of tools: Those that are just called through CGI and its derivatives, and those that launch their own server which may include an HTTP server or can be called from a web server. The former are easier to set up and don't require anything else besides the web server (possibly with extra modules like mod_fastcgi to improve performance), but they seem to offer less features than application servers.

After trying out the different solutions below, I am beginning to lean on CherryPy: OOP thanks to its application server, runs on both Unix and Windows, and feature-rich while still simple to learn by newbies. Other items to check when choosing a tool should be the quality of the documentation (too many sites with no tutorials. It only takes a few examples...), and traffic in the mailing list.

TurboGears

Nevow

Django

PyWork

ClearSilver

Snakelets

Aquarium

4Suite

Spyce

SkunkWeb

Twisted

Crusador

Draco

Albatross

  1. Untar and run "python setup.py install"
  2. Create cgi-bin/simple.html :

    <html>
    <head>
    <title>My CGI application</title>
    </head>
    <body>
    Hello from my second simple CGI application!
    </body>
    </html>
  3. Create cgi-bin/simple.py :

    #!/usr/bin/python

    from albatross import SimpleContext

    ctx = SimpleContext('.')
    templ = ctx.load_template('simple.html')
    templ.to_html(ctx)
    print 'Content-Type: text/html'
    print
    ctx.flush_content()

Slither

Zope

Resources

Karigell

pyWeb

Quixote

Quixote puts all the code and HTML for a Web-based application into a Python package containing .py and .ptl files, and provides a simple framework for publishing code and objects on the Web.

There are three components to a Quixote app:

An application's code lives in a Python package that contains both .py and .ptl files.  Complicated logic should be in .py files, while .ptl files, ideally, should contain only the logic needed to render your Web interface and basic objects as HTML.  As long as your driver script calls enable_ptl(), you can import PTL modules (.ptl files) just as if they were Python modules.

Quixote's publisher will start at the root of this package, and will treat the rest of the URL as a path into the package's contents.  Here are some examples, assuming that the URL_PREFIX is "/q", your web server is setup to rewrite /q requests as calls to (eg.) /www/cgi-bin/books.cgi, and the root package for your application is 'books'. For instance, a call to http://localhost/g calls books._q_index().

PTL, Python Template Language, is used to mix HTML with Python code. An import hook is defined so that PTL files can be imported just like Python modules. The basic syntax of PTL is the same as Python's, with a few small changes, as seen in this example:

 template barebones_header(title=None,
                          description=None):
    """
    <html><head>
    <title>%s</title>
    """ % html_quote(str(title))
    if description:
        '<meta name="description" content="%s">' % html_quote(description)
    '</head><body bgcolor="#ffffff">'

Templates in PTL are analogous to Python's functions.  Templates are defined using the template keyword, obviously, and they can't have docstrings. The difference between a template and a regular Python function is that inside a template the results of expressions are accumulated, and the return value of the template is simply all those results concatenated together

Will work with any Web server that supports CGI, FastCGI, SCGI protocol (note that although the mod_python module for Apache 2 should work on Windows, the Python SCGI package will not due to Unix-specific APIs like fork(), etc.), along with Apache's mod_python and AOLserver's PyWX. Provides its own templating language called PTL (Python Template Language). Quixote can be restarted automatically by Apache. Thus, Quixote has the advantage of providing a OO-based model to develop web applications without requiring an application server if you don't want to run any (in which case, you'll use CGI or mod_python) or run one if you want to increase performance (FastCGI or SCGI). In case you wish to use ZODB to ensure persistance, take a look at Dulcinea, a collection of modules useful for developing applications with Quixote and the ZODB.

Setup

  1. Install Python. If you already have Python installed and it is older than 2.2, run cd /tmp/Python-2.1.2/Tools/compiler ; python setup.py install
  2. Download and untar the Quixote package, and run python setup.py install. FYI, stuff is installed under d:\Python22\Lib\site-packages\quixote\
  3. Open a terminal window, and run the following commands to check that Quixote was installed correctly:

    $ python
    >>> import quixote
    >>> quixote.enable_ptl()
    >>> import quixote.demo
     
  4. From the original package, copy demo/demo.cgi and demo/demo.conf into your web server's cgi-bin directory. I prefer to create a sub-directory cgi-bin/quixote
  5. Open demo.cgi, and edit the shebbang line to point to where the python interpreter is located. For instance:

    #!d:\Python22\python.exe

  6. Open demo.conf, and change the paths to fit your setup, eg.

    DEBUG_LOG = "d:\Program Files\Sambar 5.2 Web Server\log\quixote-demo-debug.log"
    ERROR_LOG = "d:\Program Files\Sambar 5.2 Web Server\log\quixote-demo-error.log" 
     
  7. Aim at http://localhost/cgi-bin/quixote/demo.cgi

The page is actually displayed through the "app = Publisher('quixote.demo')" line in demo.cgi, which calls the _q_index(request) method of the pages.ptl template located in the Quixote demo package that was installed previously (eg. d:\Python22\Lib\site-packages\quixote\demo\).

Programming

Every namespace (package or module) that Quixote uses must supply two special identifiers: _q_index() and _q_exports. _q_index is supplied in the _init_.py module in a package, eg. splat/web/__init__.py includes this line:

from splat.web.index import _q_index

_q_index is equivalent to index.html, and is a callable, ie. a function, a method, or a PTL template, which returns the default page for a namespace.

_q_exports is used to explicitely tell Quixote which routines are callable, ie. can be accessed through a URL. This is a way to make sure users don't start calling routines that should remain off-limit. Here's an example:

#splat/web/__init__.py:
_q_exports = ['about']

_q_getname() can be used as a fallback before returning a 404. This is also convenient when you wish to map a URL to a virtual object, ie. object publishing. For instance, you could use _q_getname() to map the URL http://localhost/bug/0124 to a bug entry in the SQL server:

from quixote.errors import TraversalError
from splat.web.util import get_bug_database
[...]
def _q_getname (request, name):
    try:
        bug_id = int(name)
    except ValueError:
        raise TraversalError("invalid bug ID: %r (not an integer)" % name)
        bug_db = get_bug_database()
    bug = bug_db.get_bug(bug_id)
    if bug is None:
        raise TraversalError("no such bug: %r" % bug_id)
    return header("Bug %s" % bug) + format_bug_page(bug) + footer()

A "callable" is a function that can be called through a URL, eg. http://localhost/widgets/ .  Quixote requires you to be explicit; a _q_exports variable lists all the methods that are public. Methods not listed in _q_exports are private. Quixote puts all the code and HTML for a Web-based application into a Python package containing .py and .ptl files, and provides a simple framework for publishing code and objects on the Web. For example, the MEMS Exchange interface lives in a package nmed mems.ui. Once our Web server has been configured to direct all requests to Quixote, accessing http://fab/ will run the function mems.ui._q_index. Accessing http://fab/run/ runs mems.ui.run._q_index .

PTL, Python Template Language, is used to mix HTML with Python code. An import hook is defined so that PTL files can be imported just like Python modules. Inside a PTL template, you can use all of Python's control-flow statements.

There are three components to a Quixote application:

  1. A driver script, usually a CGI or FastCGI script. This is the interface between your web server (eg., Apache) and the bulk of your application code, eg. demo.cgi. The driver script is responsible for creating a Quixote publisher customized for your application and invoking its publishing loop.
  2. A configuration file. This file specifies various features of the Publisher class, such as how errors are handled, the paths of various log files, and various other things. Read through quixote/config.py for the full list of configuration settings. The most important configuration parameters are:
    ERROR_EMAIL e-mail address to which errors will be mailed
    ERROR_LOG file to which errors will be logged

    For development/debugging, you should also set DISPLAY_EXCEPTIONS true and SECURE_ERRORS false; the defaults are the reverse, to favour security over convenience.
  3. Finally, the bulk of the code will be in a Python package or module, called the root namespace. The Quixote publisher will be set up to start traversing at the root namespace.

cgi-bin/quixote/demo.cgi

#!d:\Python22\python.exe
# Example driver script for the Quixote demo: publishes the contents of the quixote.demo package.
from quixote import enable_ptl, Publisher
# Install the import hook that enables PTL modules.
enable_ptl()
# Create a Publisher instance
app = Publisher('quixote.demo')
# (Optional step) Read a configuration file
app.read_config("demo.conf")
# Open the configured log files
app.setup_logs()
# Enter the publishing main loop
app.publish_cgi()

d:\Python22\Lib\site-packages\quixote\demo\__init__.py

_q_exports = ["simple", "error", "widgets", "form_demo"]
 
import sys
from quixote.demo.pages import _q_index
from quixote.demo.widgets import widgets
from quixote.demo.forms import form_demo
from quixote.demo.integer_ui import IntegerUI
 
def simple (request):
    # This function returns a plain text document, not HTML.
    request.response.set_content_type("text/plain")
    return "This is the Python function 'quixote.demo.simple'.\n"
 
def error (request):
    raise ValueError, "this is a Python exception"
 
def _q_getname(request, component):
    return IntegerUI(request, component)

_q_exports = ["simple", "error"]

This means that only these two names are explicitly exported by the Quixote demo. (The empty string is implicitly exported from a namespace if a _q_index() callable exists there -- thus "/qdemo/" is handled by _q_index() in the "quixote.demo" namespace. Arbitrary names may be implicitly exported using a _q_getname() function; see Lesson 4 below.)

Q&A

What does python setup.py install actually do?

How can I improve performance?

Set up your web server to handle Python scripts : CGI, FastCGI, SCGI, Apache's mod_python, AOLServer's PyWX

Does it run on Windows?

We are mainly familiar with Unix, and develop and deploy Quixote under Linux. However, we've had several reports of people using Quixote under Windows, more-or-less successfully. There are still a few Unix-isms in the code, but they are being rooted out in favour of portability.

Resources

CherryPy

Moved to its own page.

WebWare for Python

A framework available from http://webware.sourceforge.net/. The main component is WebKit, which is an application server that sits between the web server and your WebWare-based Python application. The reason for an independ server along with a web server is the same as eg. CherryPy: A smarter web server is required to build OO web applications, ie. a web server that just delivers files from a filesystem is not adapter to OO programming.

Click on the picture below to see where WebWare stands in the architecture (diagram taken from Jason Hildebrand's Introduction to Webware: Developing Web Applications in Python)

WebKit

To install WebKit on a Windows host...

  1. Check that the version of Python you installed supports thread:
    1. python
    2. import thread
    3. If you get an error message, you'll have to recompile Python with support for threads
  2. Download and unpack WebWare
  3. cd \path\to\Webware
  4. python install.py
  5. (enter password; As indicated, the password is saved in clear text in Webware 0.8/WebKit/Configs/Application.config)
  6. Install the CGI adapter
  7. Go to the WebKit sub-directory in the WebWare package, and run appserver.bat. The AppServer is a program that must be running anytime you want to access your servlets or PSP files. By default, WebKit will try to talk to the AppServer on host localhost on port 8086 (you can run "netstat -an" to check that an an application is listening on TCP 8086)
  8. http://localhost/cgi-bin/wkcgi.exe/

Testing Webware

(Not sure at this point what I'm doing... I got the following instructions here)

To build your own Webware server:

  1. Cd to the Webware package, and run "python bin/MakeAppWorkDir.py c:/tmp/webware", where c:/tmp/webware is where Webware will install its directory tree
  2. cd c:/tmp/webware
  3. Start the server with "./AppServer"
  4. Copy the CGI adapter: cp [webware package]\WebKit\Adapters\wkcgi\wkcgi.exe C:\www\cgi-bin\
  5. Aim your browser to http://localhost/cgi-bin/wkcgi.exe

You should see "Welcome to Webware!", which is actually c:\tmp\webware\MyContext\Main.py

Hello, world!

from WebKit.Servlet import Servlet
class Hello(Servlet):
   def respond(self, trans):
       trans.response().write('Content-type: text/html\n\nHello, world!\n')

Resources

Q&A

What does "python install.py" actually do?

How does running "http://localhost/cgi-bin/wkcgi.exe/" ends up displaying stuff in the Examples/ directory?

What is CGIWrapper?

So far, CGIWrapper has received little attention among the Webware community. Most people gravitate to its alternative, WebKit, which is described later.

http://webware.sourceforge.net/Papers/IntroToWebware.html

Where do I go after running "python install" in the WebWare package?

I followed the directions, and ran "python bin/MakeAppWorkDir.py c:/tmp/webware", followed by "cd c:/tmp/webware", "./AppServer" to launch the server, copied C:\tmp\webware\WebKit.cgi into my web server's cgi-bin/ directory, and aimed at http://localhost/cgi-bin/WebKit.cgi/:

ERROR Traceback (most recent call last):
  File ".\WebKit\Adapters\CGIAdapter.py", line 48, in run
IOError: [Errno 9] Bad file descriptor

Using http://localhost/cgi-bin/WebKit.cgi/MyContext/Main returns the same thing.

WebKit.cgi doesn't work under Windows. Use wkcgi.exe instead, ie. once the Webware server is running, aim at http://localhost/cgi-bin/wkcgi.exe

Cheetah

JonPy

Hello, world!

After installing the Jonpy package, just create this hello.py file in the cgi-bin/ directory of your web server and aim at http://localhost/cgi-bin/hello.py:

#!C:\Python22\python.exe
 
import jon.cgi as cgi
 
class Handler(cgi.Handler):
        def process(self, req):
                req.set_header("Content-Type", "text/plain")
                req.write("<html><body>Hello, %s!\n</body></html>" % req.params.get("greet", "world"))
 
cgi.CGIRequest(Handler).process()

Important: If copy/pasting this sample, it might not run due to invisible characters chocking the Python interpreter. Just remove and re-add leading tabs on each line, and give it another go.

CGI Module

Code to handle a request must subclass the abstract class cgi.Handler .

import jon.cgi as cgi
 
class Handler(cgi.Handler):
  def process(self, req):
    req.set_header("Content-Type", "text/plain")
    req.write("Hello, %s!\n" % req.params.get("greet", "world"))
 
cgi.CGIRequest(Handler).process()

This class uses a single method process which receives a single parameter of type cgi.Request, which is used to retrieve information about the request and to send the response. A subclass of cgi.Request is used to call the handler. Which subclass is used depends on the protocol used to communicate with the web server. This module provides cgi.CGIRequest which implements the standard CGI protocol.

Globals

Functions

Classes

Error(Exception)
SequencingError(Error)
Request
CGIRequest(Request)
Handler
DebugHandlerMixIn
DebugHandler(DebugHandlerMixIn, Handler)

modpy module

Classes

Error(Exception)
Request(cgi.Request)

fcgi module

Classes

Server
 Request(cgi.Request)

mime module

Classes

Error(Exception)
Entity(rfc822.Message)

wt module

Classes

TemplateCode
GlobalTemplate(TemplateCode)
Handler(cgi.Handler)
DebugHandler(cgi.DebugHandlerMixIn, Handler)

session module

Classes

Error(Exception)
Session(dict)
FileSession(Session)
SQLSession(Session)

dbpool module

Globals

Variables
Functions

Q&A

Temp

CGI

Once Python is installed, just create the following hello.py in your web server's cgi-bin/ directory:

#!c:\python22\python.exe
print "Content-Type: text/plain\n\n"
print "Hello, World!"

and aim at http://localhost/cgi-bin/hello.py

import cgi

ISAPI

mod_python

mod_FastCGI

mod_SCGI

Nufox

"Nufox is a python XUL toolkit written ontop of Twisted and Nevow"

Resources