python for linux system administration
DESCRIPTION
Slides for "Python for Linux System Administration" given at Ohio Linux Fest, Columbus, OH, 09-26-2009TRANSCRIPT
Python
for Linux system administration
(yes, this is a commercial)
Vern CederFort Wayne LUGFort Wayne, IN
Instead of?
the “official” languages for sysamins
bash* (and awk and sed) *or your favorite similar shell
perl
(if you don't believe me,ask O'Reilly)
http://www.oreillyschool.com/courses/asac4/
A scripting language should
handle input & output
process text – search, replace, pattern matching, etc.
traverse filesystems
use system utilities and libraries (glue)
What's wrong with bash & perl?
Nothing, really...
bash
is a great glue language
pipes things like a champ
has a ton of utilities (the whole system, in fact)
awk and sed
handle strings brilliantly
perl
combines it all...
but...
bash is a pain when things get complex
sed is tricky
perl makes my head hurt
(I always have rather liked awk)
So what about Python?
handles strings well
is a good “glue”
“batteries included” → I/O, filesystem, etc
large collection of external libs
Python also
is very readable(and easy to maintain)
is expressive
is easy to grasp
is either OO (or not)
is fun
everyone's doing it
Redhat
Ubuntu
etc, etc, etc...
but...
(there's always a “but”)
regular expressions aren't built-in
not (quite) as common as perl
and let's talk about
the elephant in the room
indentation
yes, Python uses indentation to organize code
it makes code more readable
it's no weirder than {} or @$%
get over it
stringssome built-in string methods
splitstripjoin
replacefindcount
startswithendswith
lowerupperisdigitswapcase
expandtabscenter
encode/decodeformat
for example
to do what wc does:
#!/usr/bin/env python
import sys
data = sys.stdin.read()
chars = len(data)
words = len(data.split())
lines = len(data.split('\n'))
print ("{0} {1} {2}".format(lines, words, chars))
doc@paladin:~/work/olf$ ./wc.py < wc.py
12 22 189
or number of occurrences?
in bash (not mine):
doc@pal:~/olf$ tr " " "\n" < wc.py | grep len | wc -w
3
in Python:
#!/usr/bin/env python
import sys
data = sys.stdin.read()
print data.count(sys.argv[1])
doc@paladin:~/work/olf$ ./num_occur.py len < wc.py
3
regular expressions
re module
syntax similar to perl
import re
>>> re.findall("[Ll]en", "len is the Length")
['len', 'Len']
exception handling
y = 10
try:
x = y / 0
except ZeroDivisionError, e:
print e
integer division or modulo by zero
glue
multiple ways to call otherprograms and pipe the results
sys.stdin, sys.stdout, sys.stderr
os.system(), os.spawnl()
subprocess.call()
subprocess.Popen()
Modules: subprocess
from subprocess import *
p = Popen(["ls", "-l"], stdout=PIPE, stderr=PIPE)
out, err = p.communicate()
files, directories and morethe os and sys modules
os.environos.getcwdos.chmodos.chown
os.linkos.mkdiros.removeos.rename
sys.argvsys.stdinsys.stdoutsys.stderr
sys.platformsys.exit
Modules: os
os.walk()
import os>>> for x in os.walk('.'):... print x... ('.', ['emptydir'], [ 'chinese-python-poster.jpg', 'olf_proposal.txt', 'wc.py', 'olf.odp', 'shell.png', 'olf.txt', 'Pil.gif', 'adminscripting.png', 'num_occur.py'])('./emptydir', [], [])
Modules: os.path
existsgetmtimeisfileisdirislinkismountsamefilesplit
command line arguments
sys.argv
list of all arguments
optparse
parsing all types of arguments
returns options and parameters
automatic help
Modules: others
databases – sqlite3 and others
fork
threading
ssh – paramiko#!/usr/bin/env python
import paramiko
hostname = 'localhost'
port = 22
username = 'test'
password = 'password'
paramiko.util.log_to_file('paramiko.log')
s = paramiko.SSHClient()
s.load_system_host_keys()
s.connect(hostname, port, username, password)
stdin, stdout, stderr = s.exec_command('ifconfig')
print stdout.read()
s.close()
daemons
python-daemon
import daemon
from spam import main_program
with daemon.DaemonContext():
main_program
ctypes
load and use C librariesalso works with Windows DLL's
>>> from ctypes import *
>>> libc = CDLL("libc.so.6")
>>> libc.printf("hello %s\n", "Python")
hello Python
13
>>> print libc.time(None)
1253757776
>>> import datetime
>>> datetime.datetime.fromtimestamp(libc.time(None))
datetime.datetime(2009, 9, 23, 22, 5, 56)
A 2 line HTTP server
from http.server import HTTPServer,
SimpleHTTPRequestHandler
server = HTTPServer(("",8000),
SimpleHTTPRequestHandler)
server.serve_forever()
What about Python 3?
it's a better language than 2.x
it's not backward compatible
it's supported by the developers
it's the future
it's not here (for sysadmins) yet
ipython, the uber shell
extensive history
usable as a system shell
http://ipython.scipy.org
In [1]: print "hello"
------> print("hello")
hello
In [2]: ls
adminscripting.png olf.odp Pil.gif
Quick Python Book, 2nd ed
covering Python 3
due out late this year
http://www.manning.com/ceder
World's largest Python conference
PyCon 2010Feb. 17-25Atlanta, GA
NOW withPoster sessions!
us.pycon.org
Talks
Tutorials
LightningTalks
Keynotes
Open Space
Hands-On Lab
Exhibit Hall
Sprints
Photo: james.rintamakiLicense: Attribution-Share Alike 2.0 Generic
Resources& contact info
Python for Unix and Linux System Administration,Noah Gift, Jeremy M. Jones, O'Reilly Media 2008
Pro Python System Administration, Rytis Sileika, Apress, (not yet published)
“Python for system administrators”, James Knowlton, IBM DeveloperWorks, 2007
http://www.ibm.com/developerworks/aix/library/au-python/
Python Cookbook, Martelli, Ravenscroft & Ascher, O'Reilly Media 2005
Contact info
http://tech.canterburyschool.org/tech/VernCeder
http://www.manning.com/ceder