python eggs

British Atmospheric Data Centre

Python Eggs

British Atmospheric Data Centre

Overview Part I: Installers

What are Eggs? What Eggs can do for me? Package installation with Eggs

Part II: Developers Component overview Package compatibility Developing with Eggs

Part III: Demo

British Atmospheric Data Centre

What are Python Eggs?

Eggs = +

JARs CPAN

+ Backwards Compatibility

British Atmospheric Data Centre

Eggs ≈ JARs

.egg zip archive Drop into sys.path pkg_resources API

What about Dependencies? Scripts? C Extensions?

EGG-INFO

package_1

package_2

code

data

_____________.egg

British Atmospheric Data Centre

Eggs ≈ CPAN

$ easy_install MyPackage>=2.1... find MyPackage v2.1 ...

... download MyPackage v2.1 ...

... download MyPackage dependencies ...

... build all ...

... install all ...

$ python

>>> import MyPackage

>>>

British Atmospheric Data Centre

Python Cheeseshop

British Atmospheric Data Centre

British Atmospheric Data Centre

easy_install examples

$ easy_install MyPackage

$ easy_install MyPackage==3.2

$ easy_install -f http://example.com MyPackage

$ easy_install --upgrade MyPackage

$ easy_install –d temp_eggs MyPackage-prealpha

$ easy_install $SROOT/MyPackage/trunk#egg=MyPackage-dev

$ easy_install -m MyPackage==3.2a1

British Atmospheric Data Centre

Installing setuptoolsrecipe 1: globally

Ensure distutils is available Ensure you can write python installation

$ wget http://peak.telecommunity.com/dist/ez_setup.py

$ python ez_setup.py

British Atmospheric Data Centre

Installing setuptoolsrecipe 2: using PYTHONPATH Ensure distutils is available In ~/.profile

export PYTHONPATH=$HOME/lib/pythonexport PATH=$HOME/bin:$PATH

In ~/.pydistutilsrc

[install]install_lib = ~/lib/pythoninstall_scripts = ~/bin

$ wget http://peak.telecommunity.com/dist/ez_setup.py$ python ez_setup.py

British Atmospheric Data Centre

Key Advantages

Self contained Try new software easily

install / test / throw away Track development

select version download from SVN

Supports plug-in architectures

British Atmospheric Data Centre

Part IIDevelopers

British Atmospheric Data Centre

Egg components<<package>>pkg_resources

dependenciesresourcesmetadatanamespaces

<<package>>setuptools

build eggs

<<package>>distutils

<<script>>easy_install

finddownloadinstallupgrade<<egg>>

setuptools

<<script>>ez_setup.py

British Atmospheric Data Centre

Distutils

$ python setup.py install

$ python setup.py install –prefix=$HOME

$ python setup.py build

$ sudo python setup.py install

$ python setup.py build –build-opt install

British Atmospheric Data Centre

Distutils

Problems Insufficient documentation Easier to shortcut than use properly

Consequences Inconsistent setup.py interface Monolithic frameworks

British Atmospheric Data Centre

CDAT

British Atmospheric Data Centre

cdat-lite / vcs-lite

$ URL=http://home.badc.rl.ac.uk/spascoe/ndg_eggs$ easy_install –f $URL vcs-lite

British Atmospheric Data Centre

Matplotlib

British Atmospheric Data Centre

Matplotlib

easy_install numpy

easy_install matplotlib

easy_install basemap

easy_install –f $SVN basemap

British Atmospheric Data Centre

M2Crypto

$ easy_install M2Crypto

No way to use the –openssl option!

setup.py's key mistakes Parses the –openssl argument outside distutils. Sets include_dirs even if using default location.

British Atmospheric Data Centre

It will probably work if you

Stick to the distutils framework! including

argument parsing install locations

use __file__ or __name__ to find resources

British Atmospheric Data Centre

Setuptools: preliminariesimport ez_setup

ez_setup.use_setuptools()

from setuptools import setup, find_packages

setup(

name = 'MyPackage',

version = '1.0',

...

packages = find_packages('lib',

exclude=["test_*"]),

package_dir = {'' : 'lib'},

...

)

British Atmospheric Data Centre

Setuptools: dependenciessetup(

...

install_requires = [

'MyDep1>=0.3', 'MyDep2==1.2rc2',

'MyDep3==dev'

],

dependency_links = [

'http://www.example.com/mydep2',

'http://www.example.com/svn/mydep3/trunk#egg=MyDep3-dev'

],

...

)

British Atmospheric Data Centre

Setuptools: resources

setup(

...

zip_safe = False,

include_package_data = True,

exclude_package_data = {'': ['README.txt']},

...

)

British Atmospheric Data Centre

Setuptools: scripts

setup(

...

entry_points = {

'console_scripts': [

'foo = my_package.some_module:main_func',

'bar = other_module:some_func',

],

'gui_scripts': [

'baz = my_package_gui.start_func',

]},

...

)

British Atmospheric Data Centre

Setuptools: and more

namespace packages entry points extras SVN hooks unit tests

British Atmospheric Data Centre

pkg_resources API

#udunits_name=sys.prefix+'/lib/python'+version+'/site-packages/unidata/udunits.dat'

import pkg_resourcesudunits_name = pkg_resources.resource_filename('unidata', 'udunits.dat')

import pkg_resourcespkg_resources.require('ZSI==2.0rc1')

British Atmospheric Data Centre

Using Eggs in NDG

Stripping out CDAT components cdat-lite, vcs-lite

Splitting the ndg package into multiple eggs namespace packages

Managing multiple versions ZSI

Shared code in ndg_util.egg Component bundles

British Atmospheric Data Centre

Part III: Demo

A WSGI compliant OpeNDAP / WMS server in exactly 0 lines of python

British Atmospheric Data Centre

#!/bin/sh

# Log commands to stdoutset -x

# Setup staging arearm -fr eggs myserver demo.logmkdir eggsexport PYTHONPATH=$PWD/eggsexport PATH=$PYTHONPATH:$PATH

DAPSVN=http://pydap.googlecode.com/svn/trunk/

# Bootstrap setuptoolspython ez_setup.py -d eggs >>demo.log 2>&1

# Install pydap prerequisiteseasy_install -d eggs -f http://www.pythonware.com/products/pil \ numpy Imaging >>demo.log 2>&1

# Install pydap and pluginseasy_install -d eggs \ dap[server] dap.plugins.netcdf dap.responses.wms \ $DAPSVN/responses/html >>demo.log 2>&1

# Create the server projectpaster create -t dap_server myserver project=My-Server >>demo.log 2>&1

# Move data to server directorycp *.nc myserver/data >>demo.log 2>&1

# Start the serverpaster serve myserver/server.ini

Demo script

British Atmospheric Data Centre

British Atmospheric Data Centre

[server:main]use = egg:Paste#http# Change to 0.0.0.0 to make publichost = 127.0.0.1port = 8080

[filter-app:main]use = egg:Paste#httpexceptionsnext = cascade

[composit:cascade]use = egg:Paste#cascadeapp1 = staticapp2 = pydapcatch = 404

[app:static]use = egg:Paste#staticdocument_root = %(here)s/data

[app:pydap]use = egg:dapname = My-Serverroot = %(here)s/dataverbose = 0template = %(here)s/template

Demo paste.deploy configuration

British Atmospheric Data Centre

British Atmospheric Data Centre

British Atmospheric Data Centre

The End

British Atmospheric Data Centre

References

http://peak.telecommunity.com/DevCenter/PythonEggs http://peak.telecommunity.com/DevCenter/EasyInstall http://peak.telecommunity.com/DevCenter/setuptools http://peak/telecommunity.com/DevCenter/PkgResources

http://mail.python.org/mailman/listinfo/distutils-sig/