Practical Python Programming Spring 2013
[email protected] Page 1 of 17
Practical Python Programming
May 23rd and 24th, 2013
9:00 a.m. to 12:00 noon
BioSci B118
This is an informal introduction to the basic concepts and features of Python - an interpreted,
object-oriented, high-level programming language that is used in a wide variety of application
domains (www.python.org) - but really it is "a programming language that knows how to stay out
of your way when you write your programs. It enables you to implement the functionality you
want without any hassle, and lets you write programs that are clear and readable" (Magnus Lie
Hetland). Python is:
• powerful... and fast
• plays well with others
• runs everywhere
• is friendly... and easy to learn
• is Open
Through hands-on exercises, you will learn basic calculations, work with coordinate data, and
get a taste of just some of the many library modules (primarily in preparation for scripting with
ArcGIS software).
“If you're using a programming language named after a sketch comedy troupe,
you had better have a sense of humor.” www.python.org
Practical Python Programming Spring 2013
[email protected] Page 2 of 17
Tasks The workshop’s coding samples and scripts are shown below as plain text and screen captures
from IDLE, so you know exactly what you should be typing in yourself. Please see the
companion slides for more information.
Note: This document is named ppp_tasks_s2013.pdf and the companion slides are named
ppp_slides_s2013.pdf. The companion slides have information on the following background
information, programming concepts, and tutorial theory:
1. Who, when, what, where, why, how?
2. Easy environments
3. Programming primer
4. Coding conventions
5. Fussing with the flow
6. Script samples
7. Excellent e-resources
Environments and Primer
Tip: Make note of what happens with each of the lines of code.
Click the START button > PROGRAMS > ARCGIS > PYTHON 2.6 > IDLE (PYTHON GUI)
The following text – except do NOT type the ‘>>>’ prompt – can be typed directly in to the
interactive window (also called the Python Shell window):
>>> import this
>>> help()
>>> import
>>> topics
>>> NUMBERS
>>> strings # note the lower case
>>> quit
>>> help(keywords) # now retype with quotes around 'keywords'
>>> help('print')
>>> print 'what is a blue moon?'
>>> abluemoon = 'second to last full moon in a four-moon season'
>>> print abluemoon
NOTE: We are using the ArcGIS default
installation of the Integrated DeveLopment
Environment because it is readily available
for GIS development. Many other IDEs are
available for your use, but this one makes
interacting with ArcGIS easy peasy.
Practical Python Programming Spring 2013
[email protected] Page 3 of 17
>>> for once in abluemoon:
print once # we’ll learn more about looping later
Tip: Type Alt-P a couple of times (and try Alt-N)
>>> 1 + 2 + 3
>>> _
>>> _ - 4
>>> a = 1
>>> b, c = 2, 3 # we’ll do more variables later
>>> A + b * c # then type a + b * c
>>> 'stuff'
>>> _
Note: The default result variable ‘_’ is available only in the interactive window.
>>> # calculator expressions
>>> 1 / 2
>>> 1 / 2.0
>>> 1.0 / 2
>>> -3 ** 2
>>> (-3) ** 2
>>> 'p' + 'p' + 'p'
>>> 'p' * 3
>>> 'p' + 3
>>> 'Ha, ' * 5 + 'Ha!'
Note: Try a few calculations on your own.
>>> # commenting and docstrings
>>> # dear interpreter, please ignore all comments, thank you
Practical Python Programming Spring 2013
[email protected] Page 4 of 17
>>> """ Geography is an important antidote to the "infantile"
habit of thinking the world is a laboratory in which we can
carry out all kinds of experiments, or a huge rubbish heap where
we can get rid of all our trash. """
>>> """ Type something
and type some more
and more
and then some more again
and maybe
put yourself out of your misery
and finish """
>>> print _
>>> # print statements
>>> print 'And now for something completely different!'
>>> print 'And now for something\ncompletely different!'
>>> print 'Green' + 'eggs' + 'and' + 'ham'
>>> print 'Green', 'eggs', 'and', 'ham'
>>> print 'Today is', 24, 'February', 2011
>>> print "This is my pretend long string \
that I just don't want to fit on one line"
>>> print "Let's go!"
>>> print 'Let\'s go!'
>>> print "She said, \"Hello\""
>>> print 'C:\new folder'
>>> print 'C:\temp'
>>> print 'C:\\newfolder', 'C:\\temp'
>>> # sequences (lists and strings)
>>> mylist = [123, 456, 'seven ate nine']
>>> mylist[0]
Practical Python Programming Spring 2013
[email protected] Page 5 of 17
>>> mylist[1]
>>> mylist[2]
>>> mylist[-1]
>>> 'mylist'[0]
>>> 'mylist'[2]
>>> list('mylist')
>>> 'm' in mylist
>>> 'm' in 'mylist'
Note: An entire day could be devoted to a workshop on lists, strings, other sequences such as
tuples, and all kinds of indexing, formatting, operations, slicing, and methods! The examples
here are modestly minor. An excellent quick reference for many topics is available here:
http://rgruet.free.fr/PQR26/PQR2.6.html.
First file
In the Python Shell (interactive window), click FILE >>> NEW WINDOW
Tip: It’s a good idea to type a few meaningful comments at the top of your script file. Once you
have opened a new file editor script window, type the following, including the blank lines for
readability (note: choose your preference for commenting using the ‘#’ character OR the triple-
quoted string):
# Date: yyyymmdd
# By: [email protected]
# Description: This script is helping me learn Python
OR
"""
Date: yyyymmdd
Description: This script is helping me learn Python
"""
Click FILE >>> SAVE as C:\Workspace\ppp\sequences.py (do NOT forget the .py)
Practical Python Programming Spring 2013
[email protected] Page 6 of 17
In the interactive window, type help(‘SEQUENCES’) and give the documentation a
quick look through. Similarly, type help(‘METHODS’) , help(‘string’), and help(‘list’) and
examine those topics. Return your focus to the script window and start typing:
# the following sequence commands and methods use palindromes
pal1 = "never odd or even"
pal2 = 'kayak'
pal3 = 'a man a plan a canal panama'
# fun with strings - indices point between characters
print pal1.capitalize()
print pal1[0]
print pal1[-1]
print pal1[1:4]
print pal1[0:5]
print pal1[6:]
# type the dot, press ctrl-space, and use arrow key to select
method
print pal2.upper()
print pal2[-3:]
length = len(pal2)
print '# of characters in "' + pal2 + '" =' + length
Click RUN >>> RUN MODULE. Click OK to SAVE. Can you figure out what the error
message is? Change the last ‘+’ to a ‘,’ or apply the str() function.
print '# of characters in "' + pal2 + '" =', length
Practical Python Programming Spring 2013
[email protected] Page 7 of 17
OR
print '# of characters in "' + pal2 + '" =' + str(length)
print pal2.strip('k')
Click RUN >>> RUN MODULE again (or press the F5 key). View the results in the
interactive window, cross-referencing with the code in the script window. Type:
print pal3.swapcase()
print 'last word: ' + pal3[21:len(pal3)]
print pal3.strip(' panama')
print pal3.lstrip(' panama')
print pal3.rstrip(' panama')
print pal3.replace('a', '@')
Click RUN >>> RUN MODULE again (or press the F5 key). Examine.
Now, use the mouse cursor to click and drag to highlight/select all the text from “# fun
with strings…” to the end of the file. Click FORMAT > COMMENT OUT REGION.
Click RUN again to see what happens.
Type some more, alternating between running the code and inspecting the results as
you go along. Note: Wherever you see blank lines would be a good point to RUN.
# fun with lists
list1 = list(pal3)
print list1
print list1.count('a')
list2 = pal3.split()
print list2
list3 = list2
Practical Python Programming Spring 2013
[email protected] Page 8 of 17
list3.sort()
print list3
print list2[0]
print list2[-2:]
list4 = [10, 20, 30, 40, 50]
print list4
print list4[1]
print len(list4)
list4.append(60)
print list4
# not sequences, but useful functions to know
a = 1
b = 2
c, d = 'three', '100'
print a + b
print str(a) + str(b) + c
print int(d)
print float(d)
goodbye = '\nokay, that\'s pretty good for a first script file!'
print '*' * len(goodbye)
print goodbye
Practical Python Programming Spring 2013
[email protected] Page 9 of 17
Tip: Remember the useful trick of highlighting lines of your program and then using the
FORMAT menu to COMMENT OUT REGION and UNCOMMENT REGION. The INDENT
REGION and DEDENT REGION will come in handy when you start using loops and other
constructs requiring blocks of code.
Coding conventions
Thoroughly read the Style Guide for Python Code:
http://www.python.org/dev/peps/pep-0008/
Fussing with the flow
Start a new file for each of the scripts below. Give them a meaningful file name with a .py
extension! Make notes (comments in the code will do!) if you think the instructor says something
important during this hands-on demonstration of programming with control and decision.
Extra ‘if’ example:
# test if number is even or odd
number = input("Tell me a number: ")
if number % 2 == 0:
print number, "is even."
elif number % 2 == 1:
print number, "is odd."
else:
print number, "is not an integer."
Practical Python Programming Spring 2013
[email protected] Page 11 of 17
Reminder on IDLE for Python (as installed with ArcGIS)
1. Click START > PROGRAMS > ARCGIS > PYTHON 2.6 > IDLE (PYTHON GUI)
2. Click HELP > IDLE HELP for a quick recap on the integrated development environment
application that installs with Python; close when finished reading
3. Click FILE > NEW WINDOW to open the file editor where you will type the workshop’s
coding samples and scripts as shown below as plain text and screen captures from IDLE
4. Type a few meaningful comments at the top of your script file, similar to below (choose
your preference for commenting using the ‘#’ character OR the triple-quoted string):
# Date: yyyymmdd
# By: [email protected]
# Description: This script is helping me learn Python
5. Click FILE >>> SAVE as C:\Workspace\ppp\scriptname.py (do NOT forget the .py)
6. Click RUN >>> RUN MODULE (or press the F5 key). Click OK to SAVE. View the results
in the interactive window, cross-referencing with the code in the script window.
7. You are strongly encouraged to access Python’s built-in and online HELP.
Practical Python Programming Spring 2013
[email protected] Page 12 of 17
Tip: Remember the useful trick of highlighting lines of your program and then using the
FORMAT menu to COMMENT OUT REGION and UNCOMMENT REGION. The INDENT
REGION and DEDENT REGION will come in handy when you start using loops and other
constructs requiring blocks of code.
Script samples
Start a new file for each of the scripts below. Make notes (comments in the code will do!) if you
think the instructor says something important during this hands-on demonstrations of practical
Python programming. Especially note the built-in modules highlighted from the standard library.
Various bits of the Python language are highlighted, along with the thought process of how to
end up with a simple program that does what you want. Regions of code have been commented
out. The instructor will guide you through typing via trail an error, along with tips and tricks, to
help you realize the complete program, plus with occasional variations. The samples alone are
not complete lessons. Tip: Zoom in on the text to see it better (small font optimizes page-fitting).
day_of_year.py
# day of year from date string – pseudo-Julian date - online help is highly recommended
# http://www.doughellmann.com/PyMOTW/time, http://docs.python.org/release/2.6.5/library/time.html
import time
date = '2011-03-19'
(year, month, day) = date.split('-')
year = int(year)
month = int(month)
day = int(day)
t = time.mktime((year, month, day, 0, 0, 0, 0, 0, 0))
print t
print time.gmtime(t)
print time.gmtime(t)[7]
list_rename_files.py
# list files of specific type in an input folder and rename
# http://docs.python.org/release/2.6.5/library/glob.html, http://www.doughellmann.com/PyMOTW/glob
import glob
import os
# variables
input_folder = 'C:\\Workspace\\ppp\\files'
remove_text = '20'
# Get listing of only TIFF file types
file_type = "\\*.tif"
data_folder = input_folder + file_type
data_list = glob.glob(data_folder)
for each in data_list:
print each
new_file = each.replace(remove_text,'')
print new_file
os.rename(each, new_file)
NOTE: The code samples
may not work as is! They are
meant to be used in guided
hands-on instruction.
Practical Python Programming Spring 2013
[email protected] Page 13 of 17
descriptive_stats.py
# calculate descriptive statistics
##from math import sqrt
# list of values
numbers = [235, 689, 568, 654, 639, 779, 301, 655, 299, 443]
# calculations using built-in functions
minval = min(numbers)
maxval = max(numbers)
n = len(numbers)
summed = sum(numbers)
mean = sum(numbers) / n
### convert to float
##mean = sum(numbers) / float(n)
### standard deviation calculation with math module
### should be 192.02343607
##stdev = 0
##for number in numbers:
## diffsq = (number - mean)**2
## # loop for sum
## stdev += diffsq
### complete calculation
##stdev = sqrt(stdev / (n - 1))
###stdev = sqrt(stdev / float(n - 1))
##
### alternative to for loop is a list comprehension
##diffsq2 = [(number - mean)**2 for number in numbers]
##stdev2 = sqrt(sum(diffsq2) / (n - 1))
##
### another alternative to for loop is map with lambda
### http://diveintopython.org/power_of_introspection/lambda_functions.html
##diffsq3 = map(lambda number: (number - mean)**2, numbers)
##stdev3 = sqrt(sum(diffsq3) / (n - 1))
print 'Minimum:', minval
print 'Maximum:', maxval
print 'Sum:', summed
print 'Mean:', mean
##print 'Standard Deviation', stdev#, stdev2, stdev3
random_coordinates.py
# generate random coordinates and zip to tuples
# http://docs.python.org/release/2.6.5/library/random.html
# http://www.doughellmann.com/PyMOTW/random
# specify extent as top-left and bottom-right
import random
# extent x, y coordinates from TopLeft and BottomRight
TL = [170844, 6659344]
BR = [865133, 5425575]
# variables
number = 100
xmin = TL[0]
xmax = BR[0]
ymin = BR[1]
ymax = TL[1]
NOTE: The code samples
may not work as is! They are
meant to be used in guided
hands-on instruction.
Practical Python Programming Spring 2013
[email protected] Page 14 of 17
print xmin, xmax
print ymin, ymax
# initilaize lists
xlist = []
ylist = []
for i in xrange(1,number):
# demo this without random. module prefix
x = randint(xmin, xmax)
y = randint(ymin, ymax)
## x = random.randint(xmin, xmax)
## y = random.randint(ymin, ymax)
xlist.append(x)
ylist.append(y)
print xlist
print ylist
##coords = zip(xlist, ylist)
##
##print coords
##
### loop through tuple and print
##for each in coords:
## print 'X:' + str(each[0])
## print 'Y:' + str(each[1])
##
##print len(coords)
### why aren't there 100?
locations2dd.py
# convert DMS to decimal degrees while reading and writing text files
# http://docs.python.org/release/2.6.5/library/os.html
# http://docs.python.org/release/2.6.5/library/os.path.html
import os
# assign table file to a variable
table = 'C:\\Workspace\\ppp\\locations.txt'
# create new file for writing calculation results to
out_dir = os.path.split(table)[0]
file_name = os.path.split(table)[1]
out_file = out_dir + os.sep + 'dd_' + file_name
new_file = open(out_file, 'w')
print out_file
# open file in read mode
f = open(table, 'r')
# loop through each line
for line in f:
# print each line as stored in the table
print 'Original line:', line
# print each line items as stored in the table
print 'Original line items:', line.split(',')
# print each line replacing newline character
print 'Remove newline:', line.replace('\n','')
# print each line as converted to a list object - comma
columns = line.replace('\n','').split(',')
# assign list items to variables
oid = columns[0]
NOTE: The code samples
may not work as is! They are
meant to be used in guided
hands-on instruction.
Practical Python Programming Spring 2013
[email protected] Page 15 of 17
longitude = columns[1]
latitude = columns[2]
# divide column list item in to new list of three values: D, M, S
long_list = longitude.split(' ')
lat_list = latitude.split(' ')
# test if first value is a number
if long_list[0].isdigit():
# with and without str() for new_file writing demo
longdd = str(float(long_list[0]) + float(long_list[1]) / 60 + float(long_list[2]) / 3600)
latdd = str(float(lat_list[0]) + float(lat_list[1]) / 60 + float(lat_list[2]) / 3600)
else:
# not a number, so no calculations
longdd = long_list[0]
latdd = lat_list[0]
print longdd
print latdd
# concatenate expression to write to new file
new_line = oid + ',' + longdd + ',' + latdd # needs \n
## new_line = oid + ',' + longdd + ',' + latdd + '\n'
print new_line
new_file.write(new_line)
### demo the edit > replace command to change all instances of new_line to out_line
## # concatenate expression to write to new file
## out_line = oid + ',' + longdd + ',' + latdd # needs \n
## out_line = oid + ',' + longdd + ',' + latdd + '\n'
## new_file.write(out_line)
# demo the 'w' versus 'a' mode
# close both files
f.close()
new_file.close()
Debugging, errors, and exceptions
Debugging is what is done to fix mistakes that break your code (or cause it to misbehave). The
two types of errors that may need to be debugged are syntax errors and exceptions.
Syntax errors, also known as parsing errors, repeat the offending line and display a little ‘arrow’
pointing at the earliest point in the line where the error was detected. In the File window, you
can choose RUN > CHECK MODULE (Alt+X) to try and view any of these before actually
running the script.
Exceptions are errors detected during execution and result in various messages. As with all
things, and in Python, it’s Easier to Ask Forgiveness than Permission: use the try/except
statements to attempt to perform an operation. If all goes well, great; if not, ask for forgiveness.
A debugger is a program that lets you step through your code one line at a time as Python
executes them, showing you how each affects your program.
For excellent introductions on debugging in IDLE, using either the GUI or module, see:
http://inventwithpython.com/chapter7.html
http://www.dreamincode.net/forums/topic/210537-python-debugging-part-1/
http://pythonconquerstheuniverse.wordpress.com/2009/09/10/debugging-in-python/
Tip: Other IDEs have more user-friendly debugging tools than IDLE (see the Resources below).
Practical Python Programming Spring 2013
[email protected] Page 16 of 17
circle_area.py
# calculate area of a circle from user input radius – try out various debugging/exceptions
from math import pi
radius = 10
##radius = raw_input('Type a number: ')
##radius = float(raw_input('Type a number: '))
### flow control test input data type
##if radius.isdigit():
## print radius
## area = pi * float(radius)**2
area = pi * radius**2
##area = pi * pow(radius,2)
print 'Area of a circle with radius', radius, 'is', area
User-defined functions
A function is a series of statements which returns some value to a caller (it can also be passed
zero or more arguments for its use/execution). Built-in functions are objects that can be used by
all Python code without the need of an import statement. A user-defined function is a block of
organized, reusable code that is used to perform a single related action that you create. They
are similar to procedures, subroutines, and functions in other programming languages, but may
or may not return a value. The generic syntax requires the keyword def to define the function:
def functionname( parameters ):
"""function_docstring"""
function_suite
return [expression]
constant_value_field.py
# preliminary coding for adding a constant value field to a database table
# functions
def is_float(s): # Test if number with decimals
try:
float(s)
return True
except ValueError:
return False
def is_integer(s): # Test if number with no decimals
try:
int(s)
return True
except ValueError:
return False
# get user input
Field_Name = raw_input('Type the new field name: ')
Constant_Value = raw_input('Type a constant value: ')
NOTE: The code samples
may not work as is! They are
meant to be used in guided
hands-on instruction.
Practical Python Programming Spring 2013
[email protected] Page 17 of 17
# test user input
if is_integer(Constant_Value):
Constant_Field_Type = "LONG"
elif is_float(Constant_Value):
Constant_Field_Type = "DOUBLE"
else:
# assign defaults
Constant_Field_Type = "TEXT"
# this is where you would add and calculate the new field
print ‘The new field %s has the data type %s’ % (Field_Name, Constant_Field_Type)
# insert code above to test that the Field_Name
# starts with a letter and only contains alphanumeric characters
For more practice with functions, start at chapter 18 of http://learnpythonthehardway.org.
Really useful resources http://www.python.org/dev/peps/pep-0008/
http://docs.python.org/faq/index.html
http://docs.python.org/release/2.6.5/library
http://docs.python.org/release/2.6.5/library/functions.html
http://docs.python.org/release/2.6.5/reference/compound_stmts.html
http://docs.python.org/release/2.6.5/tutorial/errors.html
http://docs.python.org/release/2.6.5/library/pdb.html?highlight=pdb
http://wiki.python.org/moin/IntegratedDevelopmentEnvironments
Be sure to see the companion slides
for excellent e-resources to help you learn even more!