Project benefits and impacts measurement

Project benefits and impacts measurement

• 1.
  Project benefits /impacts and their measurement Paul Mayes 16 th April 2010 http:// betterprojects.jiscinvolve.org /
• 2.
  Working definition isthat a benefit is a positive type of impact There can be negative impacts from a project, some predicted in a risk analysis (but offset against greater positive impacts) but some unexpected
• 3.
  What types ofbenefit are there? ‘ Hard’ benefits: H1 Production of material, software, services, processes, etc H2 Improved collaboration H3 Improved awareness of the ideas, concepts, materials, etc within the institution, the sector and other wider areas eg government or the general public H4 Improved skills and attitudes of users, project team, institution staff, stakeholders and staff in sector
• 4.
  ‘ Soft’ /indirect benefits: S1 Catalyst for change both in institution and wider S2 Impact on professional or governmental views, actions or policies S3 Improvement of image, standing or effectiveness of the funding body of the project S4 Improvement of image, standing or effectiveness of the institution S5 Improvement of image, standing or effectiveness of the project team
• 5.
  This approach assumesa large commitment by the project team to benefits / impact and their measurement This will also be over a long period of time: When a project idea originally occurs (this of course assumes that the project bid is not a kneejerk response to a funding call) During the bid writing process During the project implementation itself For at least 5 years after the project finishes
• 6.
  Why are benefits(and their tracking or measurement) important? In advance of the project: List of possible benefits strengthens first draft of user engagement plan and helps form an evaluation/measurement approach List of possible benefits strengthens the bid document by giving the funding body more assurance about the value to them of the project List of possible benefits may strengthen buy-in within institution and encourage more/ better cooperation with partners/stakeholders outside the institution During the project: Part of good project management practice to achieve targets, milestones, etc At end of project: Clear approach to benefits/impacts strengthens final report (being slightly cynical here) Underpins better identification / ‘measurement of value’ as a framework for further areas of research and development Provides a long-term and binding structure for sustaining the project outcomes Will help develop an understanding of how the balance of investment can be adjusted to improve impact
• 7.
  Some practical stepsfor ‘benefits realisation’ Ensure that the project team is really aware of the commitment that will be needed before, during and after the project Develop a community of practice inside and outside of the institution Foster a climate of sustainability of the project outcomes Really adopt a user engagement approach
• 8.
  Some examples ofmethods & tools for tracking or measurement For H1 User satisfaction measures Usage data H2 Before and after focus groups or other user engagement tools H3 Awareness surveys Tracking the use of project outcomes or mention of the project in conventional media (eg journals, newspapers), newer media (eg Twitter, blogs), conference mentions, etc H4 Skills and attitude audits of users, staff, etc throughout the project and afterwards using a learning impact measurement framework such as that of Bersin & Associates (see article at http:// www.cedma - europe.org/newsletter%20articles/Clomedia/The%20Learning%20Impact%20M easurement%20Framework%20%28Nov%2006%29.pdf )
• 9.
  For S1 Achange assessment timeline started right from when the original idea of the for the project arose S2 Tracking of evidence of awareness, use or mention of the project ideas Soft surveying of professional or government staff S3 Some quantitative data to give to funding body about the return on their investment in terms of mention of their name in the media, benefits to their other projects and evidence of change resulting from the project S4 Some quantitative data to give to the institution in terms of enhancement of reputation, benefits to learning & teaching/research/BCE resulting from the project, etc S5 Soft investigation of change of image / standing of project team Hard measures such as increases in conference paper and publication acceptance, invitations of participate in activities or further funding obtained