There are a number of purposes of programme evaluation and these include:
• To assess its impact on the problems being addressed;
• To suggest ways of managing the scheme more effectively;
• To identify good practice for wider dissemination;
• To report back accurately to partners, local people, and relevant organisations on the schemes success/failures
Evaluation Framework:
NSFRITS is a complex programme involving a range of organisations, and a number of work packages, each using a range of methodologies, research techniques, strategies and developments, including 3 pilot test routes. In order to evaluate this project the ACC is currently developing an evaluation framework which will enable this in a systematic way. More details of this will be available soon.
Evaluation Methodology:
Due to the range of approaches used by the organisations in this project this evaluation will involve both process (formative) and impact (summative) evaluation to examine both the process of NSFRITS and the overall impact. These will be related back specifically to the project goals. Some of the questions asked by these two types of evaluation methods are detailed below.
• Formative Evaluation (Process Evaluation)
• How, why, and under what conditions does the programme work, or fail to work?
• This examines the implementing and delivery of the programme and the mechanisms through which it may succeed or fail
• Summative (Impact Evaluation)
• What are specific outcomes of a programme for different groups of people?
• What was expected of it at the outset?
• Estimate effectiveness of scheme
• Compare with other policies
• Compare with if did nothing
Collaborative Evaluator
The ACC will act as a collaborative evaluator rather than an independent evaluator during this project, due to the funding procedures. The advantages of this are that the evaluator can act as a ‘critical friend' contributing the research knowledge and expertise of the ACC to the delivery of the programme, and also questioning partners in the project to reflect on strategies they have adopted, the knowledge they have learned, and how this can be adopted into the delivery of the project. It also enables the evaluator to evaluate the project empirically using both impact and process evaluation methodologies. However, a limitation is that the evaluator is not truly independent of the project and has a say on the direction of the project (as 1 of 10 partners with a vote at the International Management Group (IMG) Meeting which adds steer to the project.
