Our Impact: Evaluation Methods and ROI
We use a multi-layered approach, based on Kirkpatrick's learning and training evaluation theory.
As a vital starting point, we agree at the outset:
- The outcomes and measures to be achieved.
- The criteria for evaluation and success of the intervention.
- The partnership roles in the evaluation process.
Our intervention approach is holistic and is based on achieving the overall outcomes. With regular reviews to enable actions, taken in a timely manner, to build on success, amend the programme or address blockages if they exist.
Example of our Multi-Layered Approach
Level 1, Short term (Reaction):
Purpose: to ensure the intervention is hitting the ‘right mark’ for participants.
Methods: impact questionnaires, two-way dialogue, temperature checks.
Level 2, Medium term:
Purpose: to ensure participants are assimilating the learning.
Methods: data and stories gleaned through workshops, coaching, and action learning conversations.
Level 3, Medium Term (Behaviour change and capability improvements):
Purpose: to measure positive behavioural change and capability improvements
Methods: surveys, random sampling, focus groups - including Managers, with industry associations, citizens, commissioners and customers.
Level 4, Long term:
Purpose: to measure the overall impact the intervention has had on organisational performance.
Methods: Progress from level 3, 360° or 180° reports; stories from coaching and action conversations; agreed organisational measures, both quantitative and qualitative.