Data Structures and Algorithm Analysis

Algorithm Analysis
Chapter 2
Mary Margarat Valentine

Objectives
This Chapter introduces students to the analysis
and design of computer algorithms. Upon
completion of this chapter, students will be able to
do the following:
 Analyze the asymptotic performance of algorithms.
 Demonstrate a familiarity with major algorithms and data
structures.
 Apply important algorithmic design paradigms and
methods of analysis.
 Synthesize efficient algorithms in common engineering
design situations.
2

What is an Algorithm?
 Algorithms are the ideas behind computer
programs.
 Algorithm is any well-defined computational
procedure that takes some value, or set of
values, as input and produces some value,
or set of values, as output.
 Algorithm is thus a sequence of
computational steps that transform the
input into the output. 4

What is an Algorithm?
Example:
 Directions to somebody’s house is an
algorithm.
 A recipe for cooking a cake is an algorithm.
5

What is a Program?
 A program is a set of instructions which the
computer will follow to solve a problem
6

Introduction
 Why we need algorithm analysis ?
 Writing a working program is not good enough.
 The program may be inefficient!
 If the program is run on a large data set, then
the running time becomes an issue.
7

What to Analyze ?
 Correctness
 Does the input/output relation match algorithm
requirement?
 Amount of work done
 Basic operations to do task finite amount of time
 Amount of space used
 Memory used
8

What to Analyze ?
 Simplicity, clarity
 Verification and implementation.
 Optimality
 Is it impossible to do better?
9

What to Analyze ?
 Time Complexity
 Amount of computer time it needs to
execute the program to get the intended
result.
 Space Complexity
 Memory requirements based on the
problem size.
10

11
Properties of algorithms
 Input: what the algorithm takes in as input
 Output: what the algorithm produces as output
 Definiteness: the steps are defined precisely
 Correctness: should produce the correct output
 Finiteness: the steps required should be finite
 Effectiveness: each step must be able to be
performed in a finite amount of time.

What is DS & Algorithm ?
 Data Structure is a systematic way of
organizing and accessing data.
 Algorithm is a step-by-step procedure for
performing some task in a finite amount of
time.

Types of DS
 Linear data structures:
 Array
 Linked list
 Stack
 Queue
 Non-linear data structures:
 Graphs
 Trees

Linear & nonlinear Data Structures
 Main difference between linear & nonlinear
data structures lie in the way they organize
data elements.
 In linear data structures, data elements are
organized sequentially and therefore they
are easy to implement in the computer’s
memory.
 In nonlinear data structures, a data element
can be attached to several other data
elements to represent specific relationships
that exist among them. 18

Linear & nonlinear Data Structures
 Due to this nonlinear structure, they might
be difficult to be implemented in computer’s
linear memory compared to implementing
linear data structures.
 Selecting one data structure type over the
other should be done carefully by
considering the relationship among the
data elements that needs to be stored.
19

Array
 An array is a collection of homogeneous
type of data elements.
An array is consisting of a collection of
elements .
20

Stack
A Stack is a list of elements in which an
element may be inserted or deleted at one
end which is known as TOP of the stack.
21

STACKS
 A stack is a restricted linear list in which all additions
and deletions are made at one end, the top.
 If we insert a series of data items into a stack and then
remove them, the order of the data is reversed.
 This reversing attribute is why stacks are known as
last in, first out (LIFO) data structures.
Figure:-Three representations of stacks

Applications
Data Structures are applied extensively in
 Operating System,
 Database Management System,
 Statistical analysis package,
 Numerical Analysis
 Graphics,
 Artificial Intelligence,
 Simulation

Asymptotic Notation (O, ,  )
26
 Asymptotic notations are the terminology
that enables meaningful statements about
time & space complexity.
 To Calculate Running Time of an algorithm

27
The time required by the given algorithm
falls under three types
1. Best-case time or the minimum time required
in executing the program.
2. Average case time or the average time
required in executing program.
3. Worst-case time or the maximum time
required in executing program.

28
 Big “oh” (O)
 upper bound
 Omega ()
 lower bound
 Theta ()
 upper and lower bound

Time Complexity ---- For Loop
Statement
for(i=0; i <n; i++)
Total time=time(stmt1)+time(stmt2)+….+time(stmt n)
Note: For declarations, time count is 0
29
n+11 n
Total : 1+n+1+n = 2n+2 O(n)

Time Complexity ---- If-Else
Statement
if(condition)
{
stmt 1;
}
else
{
stmt2;
}
30
Either Stmt1 or Stmt2 will execute.
So, Worst case is
Max(time(stmt1, stmt2)
Max (O(n), O(1))  O(n)

Nested For loops
for(i=0; i <n; i++)
{
for(i=0; i <n; i++)
{
Stmts;
}
}
Note: For all the “for loop” statement multiplied
by the product.
31
Total : n *n O(n2)

Tabular Method
32
Iterative function to sum a list of numbers
Statement Frequency
float sum(float list[ ], int n)
{
float tempsum = 0;
int i;
for(i=0; i <n; i++)
tempsum += list[i];
return tempsum;
}
0
0
1
0
n+1
n
1
0
Total 2n+3
steps/execution

33
Statement s/e Frequency Total steps
float rsum(float list[ ], int n)
{
if (n)
return rsum(list, n-1)+list[n-1];
return list[0];
}
0 0 0
0 0 0
1 n+1 n+1
1 n n
1 1 1
0 0 0
Total 2n+2
Step count table for recursive summing function

Examples
Examples of Big Oh (O) Notation
34
O(1) Constant
O(n) Linear
O(n2) Quadratic
O(n3) Cubic
O(2n) Exponential
O(log n) Logarithm

35
Analysis of Algorithms
An algorithm is a finite set of precise instructions
for performing a computation or for solving a
problem.
What is the goal of analysis of algorithms?
 To compare algorithms mainly in terms of
running time but also in terms of other factors
(e.g., memory requirements, programmer's
effort etc.)
What do we mean by running time analysis?
 Determine how running time increases as the
size of the problem increases.

36
How do we compare algorithms?
We need to define a number of objective
measures.
(1) Compare execution times?
Not good: times are specific to a particular
computer !!
(2) Count the number of statements executed?
Not good: number of statements vary with
the programming language as well as the
style of the individual programmer.

37
Ideal Solution
Express running time as a function of the
input size n (i.e., f(n)).
Compare different functions corresponding
to running times.
Such an analysis is independent of machine
time, programming style, etc.

39
 Big “oh” (O)
 upper bound
 Omega ()
 lower bound
 Theta ()
 upper and lower bound

40
Asymptotic Notation
 O notation: asymptotic “less than”:
 f(n)=O(g(n)) implies: f(n) “≤” g(n)
  notation: asymptotic “greater than”:
 f(n)=  (g(n)) implies: f(n) “≥” g(n)
  notation: asymptotic “equality”:
 f(n)=  (g(n)) implies: f(n) “=” g(n)

Big Oh- Notation
 Denoted by “O”.
 Using this we can compute the max possible
amount of time that an algorithm will take for
its application.
 Consider f(n) and g(n) to be two positive
function of “n” where the “n” is the input size.
f(n) = Og(n) iff f(n)<=c.g(n)
C>0 ; n0<n
41

42
Asymptotic notations
O-notation :

Big Oh- Notation
Example:
Derive Big-Oh Notation if f(n)=8n+7 & g(n)=n
Solution:
Assume if n=1
f(1) = 8+7 = 15 g(1) = 1
f(n) = Og(n) iff f(n)<=c.g(n)
Assume if c=15
f(n)<=c.g(n)
f(1)<=15.g(1)
15 <= 15
So f(n) = Og(n) 43

Big Oh- Notation
Step- II
Assume if n=2
f(2) = 16+7 = 23 g(2) = 2
f(2)<=15.g(2)
23 <= 30
So f(n) = Og(n)
44

Big Oh- Notation
Step- III
Assume if n=3
f(3) = 24+7 = 31 g(3) = 3
f(2)<=15.g(2)
31 <= 45
So f(n) = Og(n)
Conclusion: f(n)<=c.g(n); for c=15 & n0 = 1
45

Omega Notation
 Denoted by “”.
 Using this we can compute the minimum
its computation.
 Consider f(n) and g(n) to be two positive
function of “n” where the “n” is the input size.
f(n) =  g(n) iff f(n)>=c.g(n)
C>0 ; n0<n
46

47
Asymptotic notations (cont.)
 - notation

Theta Notation
 Denoted by “”.
 Using this we can compute the average
its computation.
f(n) =  g(n)
iff c1.g(n) <= f(n) <=c2.g(n)
48

49
Asymptotic notations (cont.)
-notation

University Questions – Ch 1 & 2
Distinguish between datatype and data structure.
Define O notation.
What is recursive function? State its advantages
What are linear and non-linear Data Structures
What are Asymptotic Notation
Why is it necessary to analyze an algorithm
What is Data Structure and Abstract Data Type?
Explain the Asymptotic notation to measure the
time complexity of an algorithm?
50

University Questions – Ch 1 & 2
What is Recursion? Give disadvantages of recursion.
Write a program to implement Towers of Hanoi .10 M
Explain Asymptotic Notations and write the properties
of asymptotic notations 10 M
51

Data Structures and Algorithm Analysis

More Related Content

What's hot (20)

Similar to Data Structures and Algorithm Analysis (20)

Recently uploaded (20)

Data Structures and Algorithm Analysis