Evaluation of IPHU is both summative (is this initiative worth continuing?) and formative (how can we do it better?). The evaluation spans short, medium and long term time frames.
The Evaluation Framework (2013) provides a general overview of PHM's approach to the evaluation of IPHU. It reviews the various elements of the IPHU program to be considered in the evaluation with a focus on the in course evaluation, the immediate post course evaluation and the longer term evaluation.
In-course evaluation involves class discussion and nightly meetings. The Smiley Template has proved useful in generating immediate feedback on particular activities. Download the Word version here for editing and posting.
Immediate post course evaluation includes an evaluation meeting at the end of the course and a course experience questionnaire. One questionnaire which has been used for several courses (here) is available on line for customisation and can be completed online (here). Please avoid using hard copy if possible as it adds considerable data entry to the work of the organising team.
We have a short long range follow up questionnaire, six months and five years, (here) which is designed to be distributed and completed by email or on line.
Some evaluation data from previous courses is linked below. However, we have experienced significant difficulties in several respects in implementing our evaluation framework. These difficulties include:
- documenting the unstructured post course face to face evaluation and filing the report appropriately so it can be retrieved by other course planners;
- ensuring prompt analysis of the raw data, reviewing, considering and documenting the analysis and lessons learned and sharing those lessons across the variety of course planning groups; this is particularly challenging where the data are collected in hard copy;
- developing an integrated long range follow up system including the systematic scheduling of different courses for six months and five year follow up.
Addressing these weaknesses will be helped by:
- creating a flexible, multilingual contact relations management (CRM) database system which can be used for managing participant data across all courses and for subsequent questionnaires and other purposes;
- keeping contact details on the CRM uptodate;
- maintaining a single integrated calendar and schedule for follow evaluations;
- funding for a small secretariat function.
Comment, suggestions and further examples are now sought:
- feedback on the Evaluation Framework;
- feedback on the two post course questionnaires (immediate and longer term) and examples of different approaches to post-course evaluation;
- evaluation questionnaires in other languages;
- proposal for acquiring a proper CRM database system;
- suggestions for ensuring more systematic collection and analysis of evaluations and dissemination / filing of lessons learned.
English
