6. Modules

 

 

 

 

 

Why use modules?

 

 

♦     Code reuse

♦     System name-space partitioning

♦     Implementing shared services or data

 

 

 

 

Module topics

 

♦     The basics

♦     Import variations

♦     Reloading modules

♦     Design concepts

♦     Modules are objects

♦     Package imports

♦     Odds and ends

♦     Module gotchas

 

 


 

Module basics

 

 

♦     Creating modules: ������ Python files, C extensions; Java classes (Jython)

♦     Using modules:����������� import, from, reload(), 3.X: imp.reload()

♦     Module search path: ��� $PYTHONPATH

 

 

 

 

file: module1.py

 

def printer(x):���������� # module attribute

��� print x

 

 

 

Module usage

 

% python

>>> import module1������������������� # get module

>>> module1.printer('Hello world!')

Hello world!

 

>>> from module1 import printer������ # get an export

>>> printer('Hello world!')

 

>>> from module1 import *������������ # get all exports

>>> printer('Hello world!')

 

 

 

 


 

 

�from *� can obscure variables� meaning

 

>>> from module1 import *���� # may overwrite my names

>>> from module2 import *���� # no way to tell what we get

>>> from module3 import *

>>> . . .

>>> func()���� # ← ?!

 

# Advice: use �from *� with at most 1 module per file

 

 

 

 

 

 

�from� does not play well with �reload�

 

>>> from module1 import func������ # copy variable out

>>> func()������������������������ # test it

 

�change module1.py�

 

>>> from imp import reload�������� # required in 3.X

>>> reload(module1)��������������� # ← FAILS: unbound name!

 

>>> import module1���������������� # must bind name here

>>> reload(module1)��������������� # ok: loads new code

>>> func()������������������������ # ← FAILS: old object!

 

>>> module1.func()���������������� # this works now

 

>>> from module1 import func������ # so does this

>>> func()

 

# Advice: don�t do that--run scripts other ways

 

 


 

 

Module files are a namespace

 

 

♦     A single scope: local==global

♦     Module statements run on first import

♦     Top-level assignments create module attributes

♦     Module namespace: attribute �__dict__�, or dir()

 

 

 

file: module2.py

print 'starting to load...'

 

import sys

name = 42

 

def func(): pass

 

class klass: pass

 

print 'done loading.'

 

 

Usage

>>> import module2

starting to load...

done loading.

 

>>> module2.sys

<module 'sys'>

>>> module2.name

42

>>> module2.func, module2.klass

(<function func at 765f20>, <class klass at 76df60>)

 

>>> module2.__dict__.keys()�� # add list() in 3.X

['__file__', 'name', '__name__', 'sys', '__doc__', '__builtins__', 'klass', 'func']


 

 

 

 

Name qualification

 

 

♦     Simple variables

●      �X� searches for name �X� in current scopes

♦     Qualification

●      �X.Y� searches for attribute �Y� in object �X�

♦     Paths

●      �X.Y.Z� gives a path of objects to be searched

♦     Generality

●      Qualification works on all objects with attributes: modules, classes, built-in types, etc.

 

 

 

 

 

 

 

 


 

Import variants

 

 

 

♦     Module import model

●      module loaded and run on first import or from

●      running a module�s code creates its top-level names

●      later import/from fetches already-loaded module

 

 

♦     import and from are assignments

●      import assigns an entire module object to a name

●      from assigns selected module attributes to names

 

 

 

 

Operation

Interpretation

import mod

fetch a module as a whole

from mod import name

fetch a specific name from a module

from mod import *

fetch all top-level names from a module

imp.reload(mod)

force a reload of module�s code

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Reloading modules

 

 

♦     Imports only load/run module code first time

♦     Later imports use already-loaded module

♦     reload function forces module code reload/rerun

♦     Allows programs to be changed without stopping

♦     3.X: must first �from imp import reload� to use!

 

 

General form

 

import module��������������� # initial import

[use module.attributes]

...������������������������� # change module file

...

from imp import reload������ # required in 3.X

reload(module)�������������� # get updated exports

[use module.attributes]

 

 

 

Usage details

 

♦     A function, not a statement

♦     Requires a module object, not a name

♦     Changes a module object in-place:

●      runs module file�s new code in current namespace

●      assignments replace top-level names with new values

●      impacts all clients that use �import� to fetch module

●      impacts future �from� clients (see earlier example)

 


 

Reload example (optional section)

 

♦     Changes and reload file without stopping Python

♦     Other common uses: GUI callbacks, embedded code, etc.

 

 

% cat changer.py

message = "First version"

 

def printer():

��� print message

 

 

% python

>>> import changer

>>> changer.printer()

First version

>>> 

 

 

[modify changer.py without stopping python]

 

% vi changer.py

% cat changer.py

message = "After editing"

 

def printer():

��� print 'reloaded:', message

 

 

[back to the python interpreter/program]

 

>>> import changer

>>> changer.printer()����� # no effect: uses loaded module

First version

 

>>> reload(changer)������� # forces new code to load/run

<module 'changer'>

>>> changer.printer()

reloaded: After editing

 

 

 


 

 

Package imports

 

Module package imports name directory paths:

♦      Module name → �dir.dir.dir�� in import statements and reloads

●      �import dir1.dir2.mod���� →�� loads dir1\dir2\mod.py

●      �from dir1.dir2.mod import name�

♦      dir1 must be contained by a directory on sys.path (�.�, PYTHONPATH, etc.)

♦      Each dir must have �__init__.py�� file, possibly empty (till 3.3: optional)

♦      __init__.py gives directory�s namespace, can use __all__ for �from*�

♦      Simplifies path, disambiguates same-named modules files

 

Example

 

For:

dir0\dir1\dir2\mod.py

And:

import dir1.dir2.mod

 

●      dir0 (container) must be listed on the module search path

●      dir1 and dir2 both must contain an __init__.py file

●      dir0 does not require an __init__.py

 

dir0\

��� dir1\

������� __init__.py

������� dir2\

����������� __init__.py

����������� mod.py

 

 

 

Why packages?

 

root\

��� sys1\

������� __init__.py������ (__init__ needed if dir in import)

������� util.py

������� main.py���������� (�import util� finds here)

������� other.py

��� sys2\

������� __init__.py

������� util.py

������� main.py

������� other.py

��� sys3\���������������� (here or elsewhere)

������� __init__.py������ (your new code here)

������� myfile.py�������� (�import util� depends on path)

������������������������� (�import sys1.util� doesn�t)

 

 

 

Advanced: Relative import syntax (2.5+)

 

To enable in 2.X (standard in 3.X):

 

��� from __future__ import absolute_import�� # till 2.7, �from� stmt only

 

In code located in package folder pkg:

 

��� import string��� # skips pkg: finds the standard library's version

 

��� from .string import name1, name2�� # import names from pkg.string only

��� from . import string�������������� # import pkg.string

 

 

 

Advanced: Namespace packages (3.3+)

 

 

Extension to usual import algorithm: directories without __init__.py, located anywhere on path, checked for last, and used only if no normal module or package found at level: concatenation of all found becomes a virtual package for deeper imports.� Not yet used much in practice.

 

See Learning Python 5th Edition for more advanced package details.

 

 

 

 

Odds and ends

 

♦     Python 2.0+: �import module as name�

●      Like �import module� + �name = module�

●      Also good for packages: �import sys1.util as util�

 

♦     Loading modules by name string

●      exec(�import � + name)

●      __import__(name)

 

♦     Modules are compiled to byte code on first import

●      �.pyc� files serve as recompile dependency

●      compilation is automatic and hidden

 

♦     Data hiding is a convention

●      Exports all names defined at the top-level of a module

●      Special case: __all__ list gives names exported by �from *�

●      Special case: �_X� names aren�t imported by a �from*�

 

♦     The __name__ == �__main__� trick

●      __name__ auto set to �__main__� only when run as script

●      allows modules to be imported and/or run

●      Simplest unit test protocol, dual usage modes for code

 

file: runme.py

 

def tester():

��� print "It's Christmas in Heaven"

 

if __name__ == '__main__':������ # only when run

��� tester()�������������������� # not when imported

 

Usage modes

 

% python

>>> import runme

>>> runme.tester()

It's Christmas in Heaven

 

% python runme.py

It's Christmas in Heaven


 

 

 

 

Module design concepts

 

 

♦     Always in a module: interactive = module __main__

♦     Minimize module coupling: global variables

♦     Maximize module cohesion: unified purpose

♦     Modules should rarely change other module�s variables

 

mod.py

X = 99����������������� # reader sees only this

 

import mod

print mod.X ** 2������� # always okay to use

 

import mod

mod.X = 88���� ���������# almost always a Bad Idea!

 

import mod

result = mod.func(88)�� # better: isolates coupling

 

 

 

 

 

 

 

 


 

 

Modules are objects: metaprograms

 

 

♦     A module which lists namespaces of other modules

♦     Special attributes: module.__name__, __file__, __dict__

♦     getattr(object, name) fetches attributes by string name

♦     Add to $PYTHONSTARTUP to preload automatically

 

 

 

File mydir.py

verbose = 1

 

def listing(module):

��� if verbose:

������� print "-"*30

������� print ("name: %s file: %s" %

���������������� (module.__name__, module.__file__))

������� print "-"*30

 

��� count = 0

��� for attr in module.__dict__.keys():����� # scan names

������� print "%02d) %s" % (count, attr),

������� if attr[0:2] == "__":

����������� print "<built-in name>"��������� # skip specials

������� else:

����������� print getattr(module, attr)����� #__dict__[attr]

������� count = count+1

 

��� if verbose:

������� print "-"*30

������� print module.__name__, "has %d names" % count

������� print "-"*30

 

if __name__ == "__main__":

��� import mydir

��� listing(mydir)����� # self-test code: list myself

 


 

 

♦     Running the module on itself

 

 

 

C:\python> python mydir.py

------------------------------

name: mydir file: mydir.py

------------------------------

00) __file__ <built-in name>

01) __name__ <built-in name>

02) listing <function listing at 885450>

03) __doc__ <built-in name>

04) __builtins__ <built-in name>

05) verbose 1

------------------------------

mydir has 6 names

------------------------------

 

 

 


 

Another program about programs

 

 

♦     �exec� runs strings of Python code

♦     �os.system� runs a system shell command

♦     __dict__ attribute is module namespace dictionary

♦     �sys.modules� is the loaded-module dictionary

 

 

 

file: fixer.py

editor = 'vi'� # your editor's name

 

def python(cmd):

��� import __main__

��� namespace = __main__.__dict__

��� exec cmd in namespace, namespace��

��� # 3.X: exec(cmd, namespace, namespace)

 

def edit(filename):

��� import os

��� os.system(editor + ' ' + filename)

 

def fix(modname):

��� import sys��������������������� # edit,(re)load

��� edit(modname + '.py')

��� if modname in sys.modules.keys():

������� python('reload(' + modname + ')')

��� else:

������� python('import ' + modname)

 

 

 

% python

>>> from fixer import fix

>>> fix("spam")������������ # browse/edit, import by name

 

>>> spam.function()�������� # spam was imported in __main__

>>> fix("spam")������������ # edit and reload() by name

 

>>> spam.function()�������� # test new version of function


 

 

 

 

Module gotchas

 

 

 

�from� copies names but doesn�t link

 

nested1.py

X = 99

def printer(): print X

 

 

nested2.py

from nested1 import X, printer��� # copy names out

X = 88��������������������������� # changes my "X" only!

printer()

 

 

nested3.py

import nested1������������ # get module as a whole

nested1.X = 88������������ # change nested1's X

nested1.printer()

 

 

% python nested2.py

99

% python nested3.py

88

 

 

 

Statement order matters at top-level

 

♦     Solution: put most immediate code at bottom of file

file: order1.py

func1()�������������� # error: "func1" not yet assigned

 

def func1():

��� print func2()���� # okay:� "func2" looked up later

 

func1()�������������� # error: "func2" not yet assihned

 

def func2():

��� return "Hello"

 

func1()�������������� # okay:� "func1" and "func2" assigned


 

Recursive �from� import gotchas

 

♦     Solution: use �import�, or �from� inside functions

file: recur1.py

X = 1

import recur2���������� # run recur2 now if doesn't exist

Y = 2

 

file recur2.py

from recur1 import X��� # okay: "X" already assigned

from recur1 import Y��� # error: "Y" not yet assigned

 

 

>>> import recur1

Traceback (innermost last):

� File "<stdin>", line 1, in ?

� File "recur1.py", line 2, in ?

��� import recur2

� File "recur2.py", line 2, in ?

��� from recur1 import Y��� # error: "Y" not yet assigned

ImportError: cannot import name Y

 

 

 

 

�reload� may not impact �from� imports

 

♦     �reload� overwrites existing module object

♦     But �from� names have no link back to module

♦     Use �import� to make reloads more effective

import module������� � module.X reflects module reloads

from module import X � X may not reflect module reloads!

 

 

 

�reload� isn�t applied transitively

 

♦     Use multiple �reloads� to update subcomponents

♦     Or use recursion to traverse import dependencies

 

See CD�s Extras\Code\Misc\reloadall.py


 

 

 

 

Optional reading: a shared stack module

 

 

♦     Manages a local stack, initialized on first import

♦     All importers share the same stack: single instance

♦     Stack accessed through exported functions

♦     Stack can hold any kind of object (heterogeneous)

 

 

file: stack1.py

 

stack = []�������������������������� # on first import

error = 'stack1.error'�������������� # local exceptions

 

def push(obj):

� ��global stack�������������������� # 'global' to change

��� stack = [obj] + stack����������� # add item to front

 

def pop():����

��� global stack

��� if not stack:

������� raise error, 'stack underflow'� # raise local error

��� top, stack = stack[0], stack[1:]��� # remove front item

��� return top

 

def top():

��� if not stack:���������������������� # raise local error

������� raise error, 'stack underflow'� # or let IndexError

��� return stack[0]

 

def empty():����� return not stack����� # is the stack []?

def member(obj):� return obj in stack�� # item in stack?

def item(offset): return stack[offset]� # index the stack

def length():���� return len(stack)���� # number entries

def dump():������ print '<Stack:%s>' % stack

 

 

 


 

Using the stack module

 

% python

>>> import stack1

>>> for i in range(5): stack1.push(i)

...

>>> stack1.dump()

<Stack:[4, 3, 2, 1, 0]>

 

 

Sequence-like tools

>>> stack1.item(0), stack1.item(-1), stack1.length()

(4, 0, 5)

>>> stack1.pop(), stack1.top()

(4, 3)

>>> stack1.member(4), stack1.member(3)

(0, 1)

>>> for i in range(stack1.length()): print stack1.item(i),

...

3 2 1 0

 

 

Exceptions

>>> while not stack1.empty(): x = stack1.pop(),

...

>>> try:

...���� stack1.pop()

... except stack1.error, message:

...���� print message

...

stack underflow

 

 

Module clients

file: client1.py

from stack1 import *

push(123)����������������� # module-name not needed

result = pop()

 

file: client2.py

import stack1

if not stack1.empty():���� # qualify by module name

��� x = stack1.pop()

stack1.push(1.23)��������� # both clients share same stack


 

 

 

Lab Session 5

 

Click here to go to lab exercises

Click here to go to exercise solutions

Click here to go to solution source files

Click here to go to lecture example files