Spreading Python applications

You have just written a fantastic and useful Python application, and you’re ready to share it with the world. Distutils, a Python module that provides a standard way of distributing and installing Python apps, can help you simplify the process of installation.

Every module distributed using Distutils contains a script called setup.py that sets up and installs the Python app. Every setup.py script contains a call to the function setup in the distutils.core module. The function takes many arguments, which lets you set the app name, version, author, description, license, and other attributes. The attributes that control which files and directories are installed are scripts, py_modules, packages, and ext_modules.

The scripts keyword argument for the setup function is a list of filenames of scripts. The py_modules keyword argument is a list indicating the names of the modules that will be installed. These files are installed in /usr/bin. Using the scripts keyword argument is useful for executable Python scripts or executable binaries. For example, a backup utility could be included in an installation.

The packages keyword argument is used along with the package_dir and package_data arguments to specify extra files that are needed by the modules. Finally, the ext_modules keyword argument specifies C/C++ extensions that need to compiled and installed.

To see how Distutils’ setup attributes work in a real application, let’s start with a look at a basic setup.py script for EventCal, a Python library module that generates HTML calendars with events. It uses only the py_modules keyword argument. (The code in this article has been tested with Python 2.5. It may also work with versions earlier than 2.5.)

### setup.py ###
from distutils.core import setup

setup (name='EventCal',
      version='0.43',
      description='EventCal generates HTML calendars in day-, week- and month-views',
      author='Rudolf Olah',
      author_email='omouse@gmail.com',
      url='http://code.google.com/p/eventcal/',
      license='MIT',
      py_modules=['eventcal']
)

To test the script, run the command python setup.py install --root=testing. The install command will attempt to build and install the module eventcal.py into ./testing/usr/lib/python2.5/site-packages/. (Without the --root argument, the command would use the default root installation directory, /.) To clean a test install, run python setup.py clean.

Python modules specified with py_modules are installed into the site-packages directory because they are considered library modules. If more than one module needed to be installed, the py_modules keyword argument would be py_modules=['module0', 'module1', ...].

While this script works and copies eventcal.py to the proper directory, EventCal also comes with a test.html demonstration file. To include it, we add it to the list of tuples for the data_files keyword argument. The first element of each tuple is the directory that the files will be copied into; the second element contains a list of strings indicating the paths of the files to copy. The directory path can be absolute or relative. If it is relative (it has no leading slash), Distutils will transform it into the absolute path: os.path.join (sys.prefix, 'your-relative-directory'). For the EventCal example, data_files is set to [('share/eventcal', ['test.html'])], and upon installation test.html will be copied into /usr/share/eventcal/.

Extra directories

Taking the example of EventCal again, we can see that it has a languages directory which contains text files with day and month names in different languages. Using the data_files keyword argument here would be tedious because you would have to specify every file in each extra directory one by one. Instead we can use the keywords packages, package_dir, and package_data. packages indicates the names of installation packages, while package_dir is a dictionary where each key is an installation package and each value is a string indicating the path of the package. package_data is a dictionary where each key is an installation package and each value is a list of paths to files and directories to install; Unix regular expressions are supported. When we use packages, all .py files will compiled into Python bytecode (.pyc) files and included in the package. Python bytecode files speed up execution of Python scripts.

### setup.py ###
from distutils.core import setup

setup (name='EventCal',
      version='0.43',
      description='EventCal generates HTML calendars in day-, week- and month-views',
      author='Rudolf Olah',
      author_email='omouse@gmail.com',
      url='http://code.google.com/p/eventcal/',
      license='MIT',
      packages=['eventcal'],
      package_dir={'eventcal': '.'},
      package_data={'eventcal': ['languages/*']},
      data_files=[('share/eventcal', ['test.html'])]
)

The above example will compile all modules in the eventcal package in the current folder, ‘.’, and copy all files in the languages directory into the installation path. For instance, if the installation path is ~/eventcal_install/, the modules will be copied into that directory, and the languages directory and the files in it will be found at ~/eventcal_install/languages.

Including C/C++ extensions

Python can call functions written in C/C++ to increase their execution speed. A collection of C/C++ functions is called an extension. For example, an app that generates prime numbers could use C/C++ functions to decrease the generation time. To include C/C++ extension code in the installation of a package, give the ext_modules keyword argument to the setup function.

The ext_modules keyword argument accepts a list of distutils.core.Extension objects. Each Extension object is created with a name and a list of filenames. For example, for a calculator program that includes two C files, calc.c and graphing.c, the keyword argument would be set to ext_modules=[Extension ('calculator', ['calc.c', 'graphing.c'])].

Dependencies

Unfortunately, Python’s documentation for adding a setup dependency is incorrect. It says that the requires keyword argument can be used with the setup function, but this does nothing. Creating a distutils.dist.Distribution object with the keyword requires fails with an error message. Hopefully, in the future, this feature will become available.

Final steps

Now that we know how to use these keyword arguments, we can create a Python distribution. There are two types of distributions: binary and source. Binary distributions compile Python scripts (.py files) into Python bytecode (.pyc files) and package it all into a .tar.gz or a .zip file for Linux, Mac OS X, or Windows (you can create Windows binary distributions on Linux or Mac OS X). Binary distributions install files into the system Python path (/usr/lib/python2.5/site-packages/). To create a binary distribution, run python setup.py bdist, and to create a Windows binary distribution run python setup.py bdist_wininst. bdist (short for binary distribution) and bdist_wininst (binary distribution Windows install) are command-line arguments to the setup.py script.

A source distribution bundles all of the files in a .tar.gz or a .zip file that may be decompressed into any folder. When it is decompressed, a new directory is created with the name and version of the package, and the package’s files are copied into the new directory. To create a source distribution, run python setup.py sdist. sdist is another command-line argument to the setup.py script; it’s short for source distribution.

Alternatives

An alternative to Distutils, setuptools, is part of the Python Enterprise Application Kit (PEAK) and is covered in an article at IBM developerWorks. Setuptools is similar to Perl’s CPAN and simplifies setup.py script creation by supplying shortcut functions such as find_packages. You can run easy_install to download and install whatever packages can be found at the Python Package Index. Currently there are more than 2,000 packages, and there is a good chance that what you need is listed there. The only drawback to setuptools is that it can create a conflict with your Linux distribution’s package manager. Below is a sample setup.py script that uses setuptools:

### setup.py ###
from setuptools import setup

setup (name='EventCal',
      version='0.43',
      description='EventCal generates HTML calendars in day-, week- and month-views',
      author='Rudolf Olah',
      author_email='omouse@gmail.com',
      url='http://code.google.com/p/eventcal/',
      license='MIT',
      packages=find_packages (),
      package_data={'': ['languages/*']},
      data_files=[('share/eventcal', ['test.html'])]
)

There are just two differences between the Distutils and setuptools versions of EventCal’s setup.py script. Setuptools looks for all packages that need to be installed, and if there are dependencies, it makes sure that they are installed.

It is important to let people install your programs easily. Distutils is simple to use for packaging up Python applications. All you need to do is create a setup.py file, import distutils.core, and call setup with the right keyword arguments, and you will have a nice package waiting to be distributed within a few minutes.