Web development with Python


Primarily two kinds of tools: Those that are just called through CGI and its derivatives, and those that launch their own server which may include an HTTP server or can be called from a web server. The former are easier to set up and don't require anything else besides the web server (possibly with extra modules like mod_fastcgi to improve performance), but they seem to offer less features than application servers.

Why dump PHP and write web apps as long-running processes?

"PHP is an embarrassment, a blight upon my craft. It’s so broken, but so lauded by every empowered amateur who’s yet to learn anything else, as to be maddening. It has paltry few redeeming qualities and I would prefer to forget it exists at all. But I’ve got to get this out of my system. So here goes, one last try." http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/

From comp.lang.python: "A long running process has lots of benefits that makes design and development easier and makes your app faster.

"Plus, if you manage it right, you have a guarantee that you can never be in a half-updated state - that's somewhat tricky when you have inter-dependencies in PHP code and the file system itself manages things. What happens if a request comes in while you're half-way through uploading new code to the server? By default, you could use half old code and half new code. Keeping everything in memory makes it easier to prevent that."

Why use a web framework?

Some of the benefits of using a web framework (from http://cherrypy.org/):

Another take:

"I would recommend Django as it seems to scale up nicely and I know that it has an active community which can be a godsend for getting help as a "newbie". That being said it does have a bit of a learning curve compared to some of the lighter frameworks, but not so much of one as to counteract the advantage of scalability. Again, this is all based on opinions that I have read (and remember) and not  on any facts.

Maybe this article will help you.  The comments on /. should round up anything missing from the article (I hope).

CGI, FastCGI/SCGI, WSGI, mod_python?

In a nutshell:

CGI Sample1

Once Python is installed, here's how to configure .htaccess to let Apache run Python scripts:

Options -Indexes +ExecCGI
AddHandler cgi-script .py

Next, create the following hello.py in your web server's cgi-bin/ directory (Note: If editing on a Windows host before uploading the file on a Linux host, watch out for CRLF end characters: Run "dos2unix" to take care of this):

#!/usr/bin/env python
# enable debugging
import cgitb
print "Content-Type: text/plain"
print "Hello, World!"

Run "chmod 755" on it, and aim at http://localhost/cgi-bin/hello.py

CGI Sample2

This shows how to handle missing items in a form, and the importance of using a framework to separate code and HTML output:

#!/usr/bin/env python
# Import modules for CGI handling
import cgi, cgitb, traceback, sys
# Create instance of FieldStorage
form = cgi.FieldStorage()
print "Content-Type: text/html;charset=iso-8859-1\n\n"
        #OK name = form.getvalue('name')
        name = form['name'].value
        mail = form['mail'].value
        tel = form['tel'].value
except Exception:
        print "Error in script."
header = """
    <meta http-equiv="content-type" content="text/html; charset=iso-8859-1" />
    <title>My site</title>
    <link href="/style.css" rel="stylesheet" type="text/css" />
footer = """
#print "Name is %s" % (cgi.escape(name), )
print header
print "Hello"
print footer


For simple, test applications, you can use Python's (2.5 and later) included web server and WSGI server through a package named wsgiref. If you want to access the script from a remote host, make sure the script listens on the host's IP address instead of "localhost". As explained, simply run "python environment.py" and aim your browser at http://localhost:8051.

For real world applications, the recommended solution is running Apache and its mod_wsgi module in daemon mode : "The preferred mode of running a WSGI application under mod_wsgi if you have no idea of how to setup and tune Apache to match your specific web application, is daemon mode. Even if you reckon you do know how to setup Apache it is still safer to use daemon mode. Even if your WSGI application is running in daemon mode, if the only thing you are using Apache for is to host the WSGI application and serve static files, then it is also recommended that you use worker MPM rather than prefork MPM as worker MPM will cut down on memory use by the Apache child processes." ("Why are you using embedded mode of mod_wsgi?")

Here's a list of WSGI servers which looks pretty thorough.

Note that two mod_wsgi modules are available: The one for Apache is by Graham Dumpleton and is still active and recommended, while the one for Nginx by Manlio Perillo isn't as stable, is no longer developed and is not recommended ("Blocking requests and nginx version of mod_wsgi").

As of 0.8.40, Nginx supports the uwsgi protocol by default. As for Lighttpd...

WSGI is Python only, and is a specification (API) for web servers and application servers to communicate with web applications. Rather than a language agnostic protocol, a standard function signature is defined:

def simple_app(environ, start_response):
    """Simplest possible application object"""
    status = '200 OK'
    response_headers = [('Content-type','text/plain')]
    start_response(status, response_headers)
    return ['Hello world!\n']

That is a complete (if limited) WSGI application. A web server with WSGI support (such as Apache with mod_wsgi) can invoke this function whenever a request arrives. Note that the uwsgi server implemented by the uWSGI project is no longer Python-specific (it can also handle Perl, Ruby, Lua, etc.)

The reason this is so great is that we can avoid the messy step of converting from a HTTP GET/POST to CGI to Python, and back again on the way out. It's a much more direct, clean and efficient linkage." (source)

"WSGI is a Python calling convention. Flask and other frameworks provide a callable object that takes 'environ' and 'start_headers' parameters and return an iterable of bytes. Various WSGI servers can call these objects to make requests.


Some servers like gunicorn speak HTTP themselves on one side, and directly WSGI on the other side. Others like flup, rather than HTTP, speak FastCGI or SCGI (which are network protocols) with a front-end server like Apache, lighttpd or nginx.

Apache’s mod_wsgi is a bit special: both the Apache integration and the worker processes (assuming daemon mode) are managed by mod_wsgi. They speak an internal/private protocol together." (source)

"Note that although mod_wsgi has features similar to FASTCGI/SCGI solutions, it isn't intended to be a replacement for those hosting mechanisms in all situations for Python web hosting. Specifically, mod_wsgi is not designed for nor intended for use in over allocated shared mass virtual hosting setups for different users on a single Apache instance" (http://code.google.com/p/modwsgi/)

Here's how to install and configure Nginx and uWSGI (which is faster than gunicorn and better integrated with Nginx) on a Debian host:

  1. apt-get update
  2. apt-get install nginx

Check if the uWSGI package is available, and its version:

  1. vi /etc/apt/sources.list:

    deb http://ftp.fr.debian.org/debian testing main
    deb http://ftp.debian.org/debian/ wheezy-updates main
  2. apt-get install uwsgi uwsgi-plugin-python

If the packages are a bit old, and you need more recent features, you'll have to compile the latest source, either manually or through pip:

  1. apt-get install python python-dev
  2. pip install http://projects.unbit.it/downloads/uwsgi-lts.tar.gz

Alternatively, some users recommend running Python through virtualenv and installing uWSGI therein:

  1. apt-get install python-virtualenv
  2. virtualenv env
  3. env/bin/pip install http://projects.unbit.it/downloads/uwsgi-lts.tar.gz

"Nginx natively includes support for upstream servers speaking the uwsgi protocol since version 0.8.40. If you are unfortunate enough to use an older version (that nevertheless is 0.7.63 or newer), you can find a module in the nginx directory of the uWSGI distribution." http://uwsgi-docs.readthedocs.org/en/latest/Nginx.html

  1. Upgrade Nginx from 0.7.63 to a more recent release (at least 0.8.40)
  2. Figure out how to install uWSGI
  3. Figure out how to configure Nginx to speak to uWSGI
  4. Figure out how to write a Python application an run it through Nginx + uWSGI

FCGI and some light framework

Since mod_python is Apache-specific and hasn't been updated in a while, and CGI is slow for bigger sites, FastCGI was invented: In this case, the application is compiled and runs as a stand-alone application which is called by the web server through a Unix/TCP port. On Apache, FastCGI is available through either mod_fastcgi or mod_fcgid ("mod_fcgi is NOT a replacement for mod_fastcgi").

Note that FastCGI will keep a process in RAM either until a timeout is reached or the script has answered so many requests. Then, FastCGI will kill the process and start a new process. So if you're using a shared host, it's a better idea to test new scripts on a local server where you're free to restart scripts at will, and only upload scripts once they're good to go.

Here's an example of a basic script running on a shared host that uses mod_fcgid + Flup to run a Python script:

#!/usr/bin/env python2.6
import cgitb
# enable debugging
def myapp(environ, start_response):
        start_response('200 OK', [('Content-Type', 'text/plain')])
        return ['Done!\n']
if __name__ == '__main__':
        from flup.server.fcgi import WSGIServer


Further Reading

WSGI Frameworks


To read

DONE python & fcgi verses mod_python

DONE Plain old Python with FastGCI

DONE HOWTO Use Python in the web

DONE (uses mod_fastcgi) Apache, FastCGI and Python by Ferry Boender, version 1.0, Feb 12, 2009

DONE (doesn't deal with how to use multiple DB servers in write mode) Scaling Python for High-Load Web Sites

DONE The Web Server Benchmarking We Need

OLD Cameron Laird's personal notes on Python and the Web

DONE How to serve a WSGI application via CGI

DONE Getting Python to work (CGI)

DONE Python - CGI Programming

DONE Five minutes to a Python CGI

DONE 4.4. CGI - Dynamic Web Pages

DONE Programming CGI With Python

DONE http://stackoverflow.com/questions/219110/how-python-web-frameworks-wsgi-and-cgi-fit-together

DONE http://stackoverflow.com/questions/257481/whats-the-difference-between-scgi-and-wsgi

DONE http://stackoverflow.com/questions/1747266/is-there-a-speed-difference-between-wsgi-and-fcgi

DONE http://www.vmunix.com/mark/blog/archives/2006/01/02/fastcgi-scgi-and-apache-background-and-future/

DONE http://helpful.knobs-dials.com/index.php/CGI,_FastCGI,_SCGI,_WSGI,_servlets_and_such

DONE http://www.subspacefield.org/~travis/python_web_frameworks.html

DONE http://geekfault.org/2010/03/01/nginx-et-python-le-perfect-setup/

Which solution to run Python web apps?

Ideally, write the web application with a framework that supports WSGI, and configure the web server to support WSGI directly instead of using SCGI/FastCGI and some wrapper to WSGI.



mod_fcgid + wrapper (Flup, uWSGI, etc.)

Here's how to connect mod_fcgid to a WSGI application:


Includes support for uWSGI



FastCGI = slow

mod_wsgi for Nginx = deadware, slow

uWSGI = alternative to mod_wsgi and requires recompiling Nginx

gunicorn = basic HTTP server that supports WSGI; less features than uWSGI, but doesn't require recompiling Nginx















  1. Untar and run "python setup.py install"
  2. Create cgi-bin/simple.html :

    <title>My CGI application</title>
    Hello from my second simple CGI application!
  3. Create cgi-bin/simple.py :


    from albatross import SimpleContext

    ctx = SimpleContext('.')
    templ = ctx.load_template('simple.html')
    print 'Content-Type: text/html'







Quixote puts all the code and HTML for a Web-based application into a Python package containing .py and .ptl files, and provides a simple framework for publishing code and objects on the Web.

There are three components to a Quixote app:

An application's code lives in a Python package that contains both .py and .ptl files.  Complicated logic should be in .py files, while .ptl files, ideally, should contain only the logic needed to render your Web interface and basic objects as HTML.  As long as your driver script calls enable_ptl(), you can import PTL modules (.ptl files) just as if they were Python modules.

Quixote's publisher will start at the root of this package, and will treat the rest of the URL as a path into the package's contents.  Here are some examples, assuming that the URL_PREFIX is "/q", your web server is setup to rewrite /q requests as calls to (eg.) /www/cgi-bin/books.cgi, and the root package for your application is 'books'. For instance, a call to http://localhost/g calls books._q_index().

PTL, Python Template Language, is used to mix HTML with Python code. An import hook is defined so that PTL files can be imported just like Python modules. The basic syntax of PTL is the same as Python's, with a few small changes, as seen in this example:

 template barebones_header(title=None,
    """ % html_quote(str(title))
    if description:
        '<meta name="description" content="%s">' % html_quote(description)
    '</head><body bgcolor="#ffffff">'

Templates in PTL are analogous to Python's functions.  Templates are defined using the template keyword, obviously, and they can't have docstrings. The difference between a template and a regular Python function is that inside a template the results of expressions are accumulated, and the return value of the template is simply all those results concatenated together

Will work with any Web server that supports CGI, FastCGI, SCGI protocol (note that although the mod_python module for Apache 2 should work on Windows, the Python SCGI package will not due to Unix-specific APIs like fork(), etc.), along with Apache's mod_python and AOLserver's PyWX. Provides its own templating language called PTL (Python Template Language). Quixote can be restarted automatically by Apache. Thus, Quixote has the advantage of providing a OO-based model to develop web applications without requiring an application server if you don't want to run any (in which case, you'll use CGI or mod_python) or run one if you want to increase performance (FastCGI or SCGI). In case you wish to use ZODB to ensure persistance, take a look at Dulcinea, a collection of modules useful for developing applications with Quixote and the ZODB.


  1. Install Python. If you already have Python installed and it is older than 2.2, run cd /tmp/Python-2.1.2/Tools/compiler ; python setup.py install
  2. Download and untar the Quixote package, and run python setup.py install. FYI, stuff is installed under d:\Python22\Lib\site-packages\quixote\
  3. Open a terminal window, and run the following commands to check that Quixote was installed correctly:

    $ python
    >>> import quixote
    >>> quixote.enable_ptl()
    >>> import quixote.demo
  4. From the original package, copy demo/demo.cgi and demo/demo.conf into your web server's cgi-bin directory. I prefer to create a sub-directory cgi-bin/quixote
  5. Open demo.cgi, and edit the shebbang line to point to where the python interpreter is located. For instance:


  6. Open demo.conf, and change the paths to fit your setup, eg.

    DEBUG_LOG = "d:\Program Files\Sambar 5.2 Web Server\log\quixote-demo-debug.log"
    ERROR_LOG = "d:\Program Files\Sambar 5.2 Web Server\log\quixote-demo-error.log" 
  7. Aim at http://localhost/cgi-bin/quixote/demo.cgi

The page is actually displayed through the "app = Publisher('quixote.demo')" line in demo.cgi, which calls the _q_index(request) method of the pages.ptl template located in the Quixote demo package that was installed previously (eg. d:\Python22\Lib\site-packages\quixote\demo\).


Every namespace (package or module) that Quixote uses must supply two special identifiers: _q_index() and _q_exports. _q_index is supplied in the _init_.py module in a package, eg. splat/web/__init__.py includes this line:

from splat.web.index import _q_index

_q_index is equivalent to index.html, and is a callable, ie. a function, a method, or a PTL template, which returns the default page for a namespace.

_q_exports is used to explicitely tell Quixote which routines are callable, ie. can be accessed through a URL. This is a way to make sure users don't start calling routines that should remain off-limit. Here's an example:

_q_exports = ['about']

_q_getname() can be used as a fallback before returning a 404. This is also convenient when you wish to map a URL to a virtual object, ie. object publishing. For instance, you could use _q_getname() to map the URL http://localhost/bug/0124 to a bug entry in the SQL server:

from quixote.errors import TraversalError
from splat.web.util import get_bug_database
def _q_getname (request, name):
        bug_id = int(name)
    except ValueError:
        raise TraversalError("invalid bug ID: %r (not an integer)" % name)
        bug_db = get_bug_database()
    bug = bug_db.get_bug(bug_id)
    if bug is None:
        raise TraversalError("no such bug: %r" % bug_id)
    return header("Bug %s" % bug) + format_bug_page(bug) + footer()

A "callable" is a function that can be called through a URL, eg. http://localhost/widgets/ .  Quixote requires you to be explicit; a _q_exports variable lists all the methods that are public. Methods not listed in _q_exports are private. Quixote puts all the code and HTML for a Web-based application into a Python package containing .py and .ptl files, and provides a simple framework for publishing code and objects on the Web. For example, the MEMS Exchange interface lives in a package nmed mems.ui. Once our Web server has been configured to direct all requests to Quixote, accessing http://fab/ will run the function mems.ui._q_index. Accessing http://fab/run/ runs mems.ui.run._q_index .

PTL, Python Template Language, is used to mix HTML with Python code. An import hook is defined so that PTL files can be imported just like Python modules. Inside a PTL template, you can use all of Python's control-flow statements.

There are three components to a Quixote application:

  1. A driver script, usually a CGI or FastCGI script. This is the interface between your web server (eg., Apache) and the bulk of your application code, eg. demo.cgi. The driver script is responsible for creating a Quixote publisher customized for your application and invoking its publishing loop.
  2. A configuration file. This file specifies various features of the Publisher class, such as how errors are handled, the paths of various log files, and various other things. Read through quixote/config.py for the full list of configuration settings. The most important configuration parameters are:
    ERROR_EMAIL e-mail address to which errors will be mailed
    ERROR_LOG file to which errors will be logged

    For development/debugging, you should also set DISPLAY_EXCEPTIONS true and SECURE_ERRORS false; the defaults are the reverse, to favour security over convenience.
  3. Finally, the bulk of the code will be in a Python package or module, called the root namespace. The Quixote publisher will be set up to start traversing at the root namespace.


# Example driver script for the Quixote demo: publishes the contents of the quixote.demo package.
from quixote import enable_ptl, Publisher
# Install the import hook that enables PTL modules.
# Create a Publisher instance
app = Publisher('quixote.demo')
# (Optional step) Read a configuration file
# Open the configured log files
# Enter the publishing main loop


_q_exports = ["simple", "error", "widgets", "form_demo"]
import sys
from quixote.demo.pages import _q_index
from quixote.demo.widgets import widgets
from quixote.demo.forms import form_demo
from quixote.demo.integer_ui import IntegerUI
def simple (request):
    # This function returns a plain text document, not HTML.
    return "This is the Python function 'quixote.demo.simple'.\n"
def error (request):
    raise ValueError, "this is a Python exception"
def _q_getname(request, component):
    return IntegerUI(request, component)

_q_exports = ["simple", "error"]

This means that only these two names are explicitly exported by the Quixote demo. (The empty string is implicitly exported from a namespace if a _q_index() callable exists there -- thus "/qdemo/" is handled by _q_index() in the "quixote.demo" namespace. Arbitrary names may be implicitly exported using a _q_getname() function; see Lesson 4 below.)


What does python setup.py install actually do?

How can I improve performance?

Set up your web server to handle Python scripts : CGI, FastCGI, SCGI, Apache's mod_python, AOLServer's PyWX

Does it run on Windows?

We are mainly familiar with Unix, and develop and deploy Quixote under Linux. However, we've had several reports of people using Quixote under Windows, more-or-less successfully. There are still a few Unix-isms in the code, but they are being rooted out in favour of portability.



Moved to its own page.

WebWare for Python

A framework available from http://webware.sourceforge.net/. The main component is WebKit, which is an application server that sits between the web server and your WebWare-based Python application. The reason for an independ server along with a web server is the same as eg. CherryPy: A smarter web server is required to build OO web applications, ie. a web server that just delivers files from a filesystem is not adapter to OO programming.

Click on the picture below to see where WebWare stands in the architecture (diagram taken from Jason Hildebrand's Introduction to Webware: Developing Web Applications in Python)


To install WebKit on a Windows host...

  1. Check that the version of Python you installed supports thread:
    1. python
    2. import thread
    3. If you get an error message, you'll have to recompile Python with support for threads
  2. Download and unpack WebWare
  3. cd \path\to\Webware
  4. python install.py
  5. (enter password; As indicated, the password is saved in clear text in Webware 0.8/WebKit/Configs/Application.config)
  6. Install the CGI adapter
  7. Go to the WebKit sub-directory in the WebWare package, and run appserver.bat. The AppServer is a program that must be running anytime you want to access your servlets or PSP files. By default, WebKit will try to talk to the AppServer on host localhost on port 8086 (you can run "netstat -an" to check that an an application is listening on TCP 8086)
  8. http://localhost/cgi-bin/wkcgi.exe/

Testing Webware

(Not sure at this point what I'm doing... I got the following instructions here)

To build your own Webware server:

  1. Cd to the Webware package, and run "python bin/MakeAppWorkDir.py c:/tmp/webware", where c:/tmp/webware is where Webware will install its directory tree
  2. cd c:/tmp/webware
  3. Start the server with "./AppServer"
  4. Copy the CGI adapter: cp [webware package]\WebKit\Adapters\wkcgi\wkcgi.exe C:\www\cgi-bin\
  5. Aim your browser to http://localhost/cgi-bin/wkcgi.exe

You should see "Welcome to Webware!", which is actually c:\tmp\webware\MyContext\Main.py

Hello, world!

from WebKit.Servlet import Servlet
class Hello(Servlet):
   def respond(self, trans):
       trans.response().write('Content-type: text/html\n\nHello, world!\n')



What does "python install.py" actually do?

How does running "http://localhost/cgi-bin/wkcgi.exe/" ends up displaying stuff in the Examples/ directory?

What is CGIWrapper?

So far, CGIWrapper has received little attention among the Webware community. Most people gravitate to its alternative, WebKit, which is described later.


Where do I go after running "python install" in the WebWare package?

I followed the directions, and ran "python bin/MakeAppWorkDir.py c:/tmp/webware", followed by "cd c:/tmp/webware", "./AppServer" to launch the server, copied C:\tmp\webware\WebKit.cgi into my web server's cgi-bin/ directory, and aimed at http://localhost/cgi-bin/WebKit.cgi/:

ERROR Traceback (most recent call last):
  File ".\WebKit\Adapters\CGIAdapter.py", line 48, in run
IOError: [Errno 9] Bad file descriptor

Using http://localhost/cgi-bin/WebKit.cgi/MyContext/Main returns the same thing.

WebKit.cgi doesn't work under Windows. Use wkcgi.exe instead, ie. once the Webware server is running, aim at http://localhost/cgi-bin/wkcgi.exe



Hello, world!

After installing the Jonpy package, just create this hello.py file in the cgi-bin/ directory of your web server and aim at http://localhost/cgi-bin/hello.py:

import jon.cgi as cgi
class Handler(cgi.Handler):
        def process(self, req):
                req.set_header("Content-Type", "text/plain")
                req.write("<html><body>Hello, %s!\n</body></html>" % req.params.get("greet", "world"))

Important: If copy/pasting this sample, it might not run due to invisible characters chocking the Python interpreter. Just remove and re-add leading tabs on each line, and give it another go.

CGI Module

Code to handle a request must subclass the abstract class cgi.Handler .

import jon.cgi as cgi
class Handler(cgi.Handler):
  def process(self, req):
    req.set_header("Content-Type", "text/plain")
    req.write("Hello, %s!\n" % req.params.get("greet", "world"))

This class uses a single method process which receives a single parameter of type cgi.Request, which is used to retrieve information about the request and to send the response. A subclass of cgi.Request is used to call the handler. Which subclass is used depends on the protocol used to communicate with the web server. This module provides cgi.CGIRequest which implements the standard CGI protocol.




DebugHandler(DebugHandlerMixIn, Handler)

modpy module



fcgi module



mime module



wt module


DebugHandler(cgi.DebugHandlerMixIn, Handler)

session module



dbpool module






"Nufox is a python XUL toolkit written ontop of Twisted and Nevow"