7_Introduction to Monitoring and Evaluation.pptx

1
Introduction to Monitoring and
Evaluation

2
Chapter objectives
 After completion of this chapter, students will be
able to:
 Discuss the basics of monitoring and evaluation
 Develop good health indicators
 Develop monitoring plan
 Describe the M&E activities

3
Monitoring and Evaluation
 Monitoring and evaluation are key management
functions of an organization
 Monitoring is used to regularly track changes in
indicators – measurable markers of change over time
 Monitoring measures progress toward results by
collecting information on inputs, activities, outputs,
and sometimes short-term outcomes
 Common procedures for program monitoring
include tracking service statistics and reviewing
records and training reports
 Regular, systematic monitoring provides
information for planning purposes and a reliable
basis for an evaluation

4
Monitoring and Evaluation…
 Evaluation, on the other hand, is used to
assess the effectiveness of efforts to improve
services and to prevent and manage priority
health problems
 Evaluation measures outcomes and impact
 It assesses the extent to which your
organization achieves its desired results and
helps you understand why the results were
or were not achieved
 Evaluation also provides an opportunity for
continuous learning from experience

5
 The differences between monitoring and evaluation lie
in their purposes, time frames, and sources of
information
 difference in purpose
 Monitoring is driven by a management need,
whereas evaluation is driven by the need to
document outcomes of an intervention and report to
a donor or other stakeholder
 Monitoring thus focuses on operational
implementation, while evaluation focuses on the
effects of the activities on the health of the target
population
 Evaluation attributes for results, explain cause and
effect

6
 Difference in time frame when each is used
 Monitoring is an ongoing, routine process used
throughout an intervention
 Evaluation requires the collection of baseline and
post-intervention data that allow you to compare
changes during the period of the intervention and,
sometimes, after a suitable follow-up period – it is
episodic
 Difference in information sources
 Monitoring data usually come from what is readily
available: the HIS or routine service records
 Outcome and impact indicators require
measurements at the beneficiary or population
level

7
Levels in the Results Chain
 The results of health services and
programmatic interventions can be measured
at different levels
 Many M&E guidelines are based on a chain of
five levels of results: inputs, activities,
outputs, outcomes, and impact
 These result chains are the backbone of
measurements in M&E
 The table below summarizes the results levels
that can be monitored and evaluated

Levels in the Results Chain
Results chain Definition Examples
Inputs
The materials and resources
needed to carry out your team
or unit’s implementation plan
and achieve the desired result.
 human resources
 financial resources
 equipment
 facilities
 policies
Process
(Activities)
The activities carried out
through your implementation
plan
 meetings
 training
 supervision
 services
Outputs
The immediate product of an
activity.
 number of staff trained
 number of clients
 number of products
 quantity of products
distributed
Outcomes
A short-term change in a
population group as a result of
a set of activities.
 change in knowledge
 change in behaviour
 change in practices
 improved services
Impact
Long-term changes within a
beneficiary of population
 change in disease rates
 change in death rates
 change in birth rates 8

9
Steps in developing an M&E plan
 M&E plan help you determine whether you have
achieved your desired results and to track progress
toward those results during implementation
 A well-designed M&E plan answers five questions:
1. is your expected result measurable?
2. what indicators will you use to monitor your
outputs and evaluate your outcomes?
3. what are your data sources and how will you
gather data from these sources?
4. what are the time frames for each indicator?
5. who will collect the data?
 These questions are important, without a valid M&E
plan, you may misinterpret the effects of your
intervention

10
Step 1: Define the expected results
 determine an appropriate level for results.
 together, the type and scope of the intervention,
available resources, and time frame for
implementation determine a feasible result
 determine at which level you should monitor and
evaluate your results, i.e. inputs processes
→ →
outputs outcomes impact
→ →
1. To hire 20 additional midwifes for our hospital
(input)
2. To provide 24 hr delivery services (process)
3. To provide delivery service for 200 mothers (output)
4. To increase skilled delivery attendance by 50%
(outcome)

11
Define the expected results…
 make sure that the expected results meet SMART
criteria
 Specific (S): Clearly written and understood
 Measurable (M): It is possible to monitor progress and
evaluate results
 Appropriate (A): The desired result is appropriate to
the scope of your work & the organization mission
 Realistic (R): realistically achieved using the resources
available and within the time frame
 Time bound (T): There is a specific time period for
achieving the desired result.
 example of a measurable result:
 The proportion of deliveries attended by skilled health
attendant in the catchment area will increase to 70% at the

Step 2: Identify the indicators and their
definitions
 Indicators in healthcare are variables or summaries of
variables that show or indicate how a health system is
functioning
 Indicators are markers of change over time
 Every indicator needs a detailed definition.
 If it is a percentage, what are the numerator and
denominator?
 The definition should be detailed enough to ensure
that different people at different times can collect
identical types of data for the indicator
 The proportion of deliveries attended by skilled birth
attendant
12
100
deliveries
expected
of
number
Total
atttendant
skilled
with
deliveries
of
Number
rate
coverage
delivery
Skilled x


Indicators….
Indicators should conform to the qualities of a good
indicator
For priority health problems and essential services,
use standard or national core indicators
Select a minimum number of key indicators, making
sure that selected indicators are really necessary to
measure your desired results
 Indicator cannot:
 answer questions about why results are or are not
achieved
 explain linkages between interventions and results
 prescribe actions that should be taken to improve 13

14
Qualities of a Good Indicator
 Valid: The indicator measures what it is intended
to measure
 Reliable: Measurement of the indicator would be
the same no matter how many times
measurement is carried out
 Precise: The indicator is defined in clear, specific
terms so that it can be measured.
 Easily understood: Both experts and non experts
can grasp the meaning of the indicator.
 Discrete: The indicator captures a single
component or aspect of a more complex result. It
measures only one thing, not a set of things.

15
Qualities of a Good Indicator…
 Timely: The indicator can be measured at
appropriate time intervals according to the
availability of data.
 Comparable: When possible, the indicator
avoids narrow or unique definitions whose
values would be difficult to compare with
other results.
 Feasible to use: Data for the indicator are
easy to obtain from a credible source and
relatively inexpensive – affordable within your
organization’s resources

16
Guiding principles of indicator selection in
Ethiopia
 Standardization – Common definitions of
indicators, data collection instruments, and data
processing and analysis procedures
 Integration - A single M&E plan, shared by all
partners, a single channel from which all derive
their information
 Simplification - Collecting, analyzing, and
interpreting only the information that is
immediately relevant to performance
improvement
 makes best use of scarce resources, especially
human resources


17
Step 3: Identify the data sources
 identify a data source for each indicator in the M&E
plan, data that are from a credible source and that
your organization can afford.
 choose data that are already available through the
organization rather than a new data collection
strategy, which could be costly and time-consuming
 always consider the advantages and disadvantages
of each data source
 you should be as specific as possible about the data
source, so the same source can be used consistently
throughout your intervention
 changing data sources for the same indicator can
lead to inconsistencies and misinterpretations

18
Step 4: Determine data collection
methods
 define the methods or tools that you will use to
collect data for each indicator.
 For indicators based on primary data (data that you
collect yourself), you should describe the type of
instrument needed to gather the data
 Examples might be structured questionnaires, direct
observation checklists, or scales to weigh infants
 For secondary data (data collected by others that is
available for your use), you should explain the
method of calculating the indicator and the source of
data, providing enough detail on the calculation
method so that others can replicate it

19
Determine data collection methods…
 Remember, while it is easier and less expensive to use
secondary data, its quality is often less reliable than
that of primary data
 It is also important to note the frequency of data
collection for each indicator
 Depending on the type of indicator, you may need to
collect data monthly, quarterly, annually, or even less
frequently
 The information from Steps 1–4 will provide the
content of your monitoring and evaluation plan

20
Step 5: Collect baseline and follow-up
data
 You will need to collect baseline data on each
indicator before your activities begin
 Then, at different points during implementation, you
will collect follow-up data on each indicator for
comparison with baseline levels and anticipated
results
 Allow to assess the progress of each intervention or
service and make needed changes along the way
 At the end of the implementation period, you will
need to collect data on your indicators in order to
compare final levels to your baseline and to your
anticipated results

21
Step 6: share and use your data
 To make sure that data will be used-not just collected-
think about how you and your team will disseminate
the monitoring and evaluation information and
obtain feedback from different stakeholders
 monitoring and evaluation should be undertaken
with the purpose of immediately using the results to
identify gaps in performance and take action to
reduce or fill those gaps
 you should always plan ways to use monitoring and
evaluation results for learning and action

22
share and use your data…
 A few basic questions will guide you:
 Who needs what kind of information and when do
they need it?
 What type of setting should you use to
communicate results to staff, senior management,
and other key stakeholders? report, organize a
meeting, workshop?
 Should you organize community meetings to solicit
feedback from your beneficiaries on the initial M&E
findings?
 How should you present information so it will be
useful to different decision-makers? - visually, in
charts, graphs, or maps?

Example of a Monitoring Plan
23
Activity Indicator Data
Source
Frequency
of data
collection
Responsible
person
Timeline
Physicians at
five regional
hospitals
receive one-on-
one mentoring
in ART
• Number of sites
receiving clinical
mentoring
• Number of
physicians receiving
one-on-one
mentoring
• Number of days of
clinical mentoring
provided
Mentor
monthly
reports
Monthly Mentors March
2009
Objective: Health care workers who have received
classroom training improve their ability to provide
antiretroviral therapy (ART) according to national
guidelines

Features of a good monitoring tool
 a monitoring tool that can deliver sufficient
information in time to identify and correct
problems as soon as they emerge
 A monitoring tool should be easy to construct
and use
 Tables, checklists, and simple graphs are
particularly useful methods of collecting,
organizing, and presenting monitoring
information
24

monitoring tool …
 your monitoring tool should enable you to
track:
 the status of activity implementation;
 the delivery of outputs;
 the status of key outcome indicators vis-a-
vis anticipated results
 budget expenditures;
 the availability of human and financial
resources;
 obstacles to implementation and what is
being done to overcome them
25

Purpose of Evaluation
 There are two reasons for carrying out an
evaluation.
1. Evaluation provides information about the
success of your team, unit, or organization in
meeting its objectives
 This information helps determine which
activities to expand, modify, or eliminate.
 It can also reveal ways to improve the design
and management of future activities.
2. Evaluation can demonstrate accountability to
your donor and other stakeholders, including
your government and the beneficiaries of your26

 Managers can carry out two broad types of
evaluations: formative and summative
 Formative evaluations:
 are evaluations conducted with the primary
purpose of furnishing information that will
guide program improvement
 is a type of evaluation performed during the
entire planning process and program
execution to answer evaluation questions
important to modify an intervention
27
Formative and summative
Evaluations

Formative Evaluation…
 Formative evaluation includes a needs
assessment to find out the desires and
requirements of a population group and
determine how best to meet them
 Such evaluations provide solutions for program
improvement by answering questions such as:
 how can the intervention be modified to
achieve its outputs and outcomes?
 are there better solutions compared to those
proposed by the program?
28

Summative Evaluation
 Evaluations that permit conclusions to be drawn
about impact, outcome or benefits of the
program or method
 This type of evaluation is conducted after the
completion of a set of activities or intervention to
assess the quality of the intervention and its key
results
 Summative evaluations are evaluations
undertaken to render a summary judgment on
certain critical aspects of a program
 Answers questions such as:
 is the program effective/effective/relevant?
29

Formative and Summative Evaluations
in a Program’s life
30
Relative
Emphasis
Program Life
Formative Evaluation
Summative Evaluation

7_Introduction to Monitoring and Evaluation.pptx

More Related Content

Similar to 7_Introduction to Monitoring and Evaluation.pptx (20)

More from birara4 (7)

Recently uploaded (20)

7_Introduction to Monitoring and Evaluation.pptx

Editor's Notes