Python High Level Functions_Ch 11.ppt

Higher Order Functions
Chapter – 11
Author : Afshan Banu

Functions as parameters
• Have you ever wanted to pass an entire function as a
parameter
• Python has functions as first-class citizens, so you can do
this
• You simply pass the functions by name

Higher-Order Functions
• A higher-order function is a function that takes another
function as a parameter
• They are “higher-order” because it’s a function of a function
• Examples
– Lambda
– Map
– Reduce
– Filter
• Lambda works great as a parameter to higher-order
functions if you can deal with its limitations

Anonymous functions
• Anonymous functions also called lambda functions
• The difference between lambda functions and
regular Python functions is that lambda functions
evaluate a single expression and return a function.
• This means that lambda functions cannot use
statements such as conditions or even the return
keyword.

Map
map(function, iterable, ...)
• Map applies function to each element of iterable
and creates a list of the results
• You can optionally provide more iterables as
parameters to map and it will place tuples in the
result list
• Map returns an iterator which can be cast to list

Map Example
Example
1
2
3
4
5
6
7
nums = [0, 4, 7, 2, 1, 0 , 9 , 3, 5, 6, 8, 0, 3]
nums = list(map(lambda x : x % 5, nums))
print(nums)
#[0, 4, 2, 2, 1, 0, 4, 3, 0, 1, 3, 0, 3]

Map Problem
Goal: given a list of three dimensional points in the
form of tuples, create a new list consisting of the
distances of each point from the origin
Loop Method:
- distance(x, y, z) = sqrt(x**2 + y**2 + z**2)
- loop through the list and add results to a new list

Map Problem
Solution
1
2
3
4
5
6
7
8
9
from math import sqrt
points = [(2, 1, 3), (5, 7, -3), (2, 4, 0), (9, 6,
8)]
def distance(point) :
x, y, z = point
return sqrt(x**2 + y**2 + z**2)
distances = list(map(distance, points))

Filter
filter(function, iterable)
• The filter runs through each element of iterable (any
iterable object such as a List or another collection)
• It applies function to each element of iterable
• If function returns True for that element then the
element is put into a List
• This list is returned from filter in versions of python under
3
• In python 3, filter returns an iterator which must be cast
to type list with list()

Filter Example
Example
1
2
3
4
5
6
nums = [0, 4, 7, 2, 1, 0 , 9 , 3, 5, 6, 8, 0, 3]
nums = list(filter(lambda x : x != 0, nums))
print(nums) #[4, 7, 2, 1, 9, 3, 5, 6, 8, 3]

Filter Problem
NaN = float("nan")
scores = [[NaN, 12, .5, 78, math.pi],
[2, 13, .5, .7, math.pi / 2],
[2, NaN, .5, 78, math.pi],
[2, 14, .5, 39, 1 - math.pi]]
Goal: given a list of lists containing answers to an
algebra exam, filter out those that did not submit a
response for one of the questions, denoted by NaN

Filter Problem
Solution
1
2
3
4
5
6
7
8
9
0
1
2
3
4
NaN = float("nan")
scores = [[NaN, 12, .5, 78, pi],[2, 13, .5, .7, pi / 2],
[2,NaN, .5, 78, pi],[2, 14, .5, 39, 1 - pi]]
#solution 1 - intuitive
def has_NaN(answers) :
for num in answers :
if isnan(float(num)) :
return False
return True
valid = list(filter(has_NaN, scores))
print(valid)
#Solution 2 – sick python solution
valid = list(filter(lambda x : NaN not in x, scores))
print(valid)

Reduce
reduce(function, iterable[,initializer])
• Reduce will apply function to each element in iterable
along with the sum so far and create a cumulative sum of the
results
• function must take two parameters
• If initializer is provided, initializer will stand as the first
argument in the sum
• Unfortunately in python 3 reduce() requires an import
statement
• from functools import reduce

Reduce Example
Example
1
2
3
4
5
6
7
nums = [1, 2, 3, 4, 5, 6, 7, 8]
nums = list(reduce(lambda x, y : (x, y), nums))
Print(nums) #(((((((1, 2), 3), 4), 5), 6), 7),
8)

Reduce Problem
Goal: given a list of numbers I want to find the
average of those numbers in a few lines using
reduce()
For Loop Method:
- sum up every element of the list
- divide the sum by the length of the list

Reduce Problem
Solution
1
2
3
4
nums = [92, 27, 63, 43, 88, 8, 38, 91, 47, 74, 18,
16,
29, 21, 60, 27, 62, 59, 86, 56]
sum = reduce(lambda x, y : x + y, nums) / len(nums)

MapReduce
A framework for processing huge datasets on certain kinds of distributable
problems.
MapReduce is a Hadoop framework used for writing applications that can
process vast amounts of data on large clusters. It can also be called a
programming model in which we can process large datasets across
computer clusters.
Hadoop
MapReduce
Works

MapReduce
There are two primary tasks in MapReduce: map and reduce.
Map Step:
- master node takes the input, chops it up into smaller sub-
problems, and distributes those to worker nodes.
- worker node may chop its work into yet small pieces and
redistribute again
Reduce Step:
- master node then takes the answers to all the sub-
problems and combines them in a way to get the output

MapReduce
Problem: Given an email how do you tell if it is spam?
- Count occurrences of certain words. If they
occur too frequently the email is spam.

MapReduce
map_reduce.py
1
2
3
4
5
6
7
8
9
1
0
email = ['the', 'this', 'annoy', 'the', 'the', 'annoy']
def inEmail (x):
if (x == "the"):
return 1;
else:
return 0;
map(inEmail, email) #[1, 0, 0, 0, 1, 1,
0]
reduce((lambda x, xs: x + xs), map(inEmail, email)) #3

Regular Expressions
Specialized programming language embedded in Python
Based on rules definition, and applying to a string
Useful for complex pattern matching
For simple pattern matching better to use string find/search
Also called regex, and is universal to many languages

Regular Expressions : Steps
1. Submit a regular expression string
2. Compile this string to get a regex byte code
3. Apply the compiled regex byte code to target string
4. Obtain results of the match
5. Use the results

Regular Expressions : Steps
import re
pattern= re.compile( <regex expression> )
matchObject=pattern.match(“target String” )
if( matchObject) :
# use matchObject for various operations

Regular Expressions
The match object has following methods
group( ) # will return the matching substring
start ( ) # will return the starting position of the match
end( ) # will return the position where the match ended
span( ) # will return a tuple of start and end positions
import re
pattern= re.compile( “A..T” ) # . matches any one character
matchObject=pattern.match( “ACGTAAT”)
if( matchObject) : # if match was found
print(matchObject.group( ) ) # displays ‘ACGT’
print(matchObject.start(), matchObject.end() ) # 0 4
start and end are positions relative to the string provided

Regular Expressions
pattern= re.compile( “A..T” )
Compiles regular expression “A..T” to a regex object - pattern
This object has methods to apply the byte code to any string
pattern.match( ) pattern.search( ) pattern.findall( )
 The regex object methods are detailed below
match( “in this string”) # matches from beginning
search( “in this string”) # searches through the string
These function return match objects. A match object has methods to use the search results
findall ( “ in this string “) # returns a list of matching strings

Regular Expressions : Metacharacters
. matches any one character except n
w matches any alphanumeric character
W matches any non-alphanumeric character
d matches any digit
D matches any non digit
s matches any white space character
S matches any non white space character

[ ] any one character from this set
[ AT ] means either A or T is matched
[ A-G ] matches any character between A to G ( inclusive )
[^ AT ] means any character except from this set
( ) used for grouping expressions
| for using or condition
for negating meaning of metacharacter

^ match in the beginning of the string
$ match at end of the string
* Zero or more occurrence of previous character
+ One or more occurrence of previous character
? Zero or one occurrence of previous character
{m,n} minimum m occurrences and maximum n occurrences

Metacharacters : Examples
“d[A-Z]..”
This would match the following:
9G99 5DD0 4B23 and so on
Will not match
9g99

Problem Solving with REGEX
Write a function that would validate an enrolment number. valid=validateEnrolment(eno )
Example enrolment number U101113FCS498
U-1011-13-F-CS-498
13 – is the year of registration, can be between 00 to 99
CS, EC or BT
Last 3 are digits
Example 2:
“AA..T...A”
Means the pattern should have “AA” followed by any two characters followed by “T” followed by any
two characters and then followed by “A”
3: String contains "af" followed by 0 or more “s" characters:
4:String contains "af" followed by 1 or more “s" characters:

Metacharacters in Detail
regex=re.compile(“sddds”)
m=regex.search(“Hello 12 cats ran after 432 dogs”)
if m:
print(m.group())
print(m.span())
NOTE: We have used search here
GROUPING OF EXPRESSIONS
regex=re.compile( “(dd)-(dd)-(dddd)” )
match=regex.match(“12-03-2016”)
if match:
print(“Day: “,match.group(1) )
GROUPING OF EXPRESSIONS
match.group(0) # gives the entire pattern found
match.group(1) # gives the first group
match.groups(default=None) # returns all groups as List
If default is set to a value then that value is returned in
case a particular group is not existing

Metacharacters in Detail
regex=re.compile(“swwws”)
mList=regex.findall(“A cat ran after 432 dogs”)
for m in mList:
print(m )

Regular Expressions
Additional methods in regex:
Syntax: re.sub(pattern, repl, string, count=0, flags=0)
regex.sub(pattern, repl, string, count=0) # replaces
regex.subn(pattern, repl, string, count=0) # replaces and returns count
>>> import re
>>> text = 'Learn to sing because singing is fun.'
>>> re.sub('sing', 'program', text)
'Learn to program because programing is fun.'

Python High Level Functions_Ch 11.ppt

More Related Content

Similar to Python High Level Functions_Ch 11.ppt (20)

Recently uploaded (20)

Python High Level Functions_Ch 11.ppt