Impact Level Evaluation
The Capability Building Unit of DSWD’s Pantawid Program sought to understand the most effective means to determine the learning needs of individuals. Their objective was to enable members of the organization to be capable in helping reduce the occurrence and curb the cycle of poverty in the Philippines by investing on the human capital of children from families in extreme and chronic poverty.
Starting 2015, I conducted workshops with the Capability Building Unit on Learning Measurement and Evaluation, Competency Assessment, and Intervention Design and Delivery. In 2019, in partnership with the Civil Service Commission, we wanted to see how these interventions helped the Capability Building Unit and, ultimately, DSWD’s Pantawid Program using an Impact Level Effectiveness Evaluation.
To evaluate how the course objectives of the previous interventions were met, and to gauge the extent by which the learnings were applied by participants after the training session, results from the evaluation administered after the conduct of the workshop were reviewed and compared to a follow up survey. This comparison provided data on course effectiveness and a glimpse of its impact on participants’ skills post-workshop.
A Focus Group Discussion then conducted to validate and determine the extent of applicability and effectiveness of the course. Answers were sought for the following questions – How did you apply your learning? What did you achieve when you applied your learning? What are your plans relative to Learning Measurement and Evaluation moving forward? While the course specifically focused on the creation, use and analysis of assessment tools and its results, it was discovered that participants also achieved learning in other areas related to learning measurement and evaluation – specifically, design of learning interventions, technical assistance provision, and even policy enhancements.
The extent by which the application of learnings yielded results come in, as gleaned from the survey analysis, three forms, that of (1) individual value, where the application of learnings improved one’s manner of doing work, (2) divisional value, where the application of learnings improved one’s division’s work, and (3) organizational value, where the application of learnings improved the organization.
Firstly, application was immediate because the interest of participants in the concepts and approaches was very significant. The team had wanted to understand Learning Measurement and Evaluation early on, alongside Instructional Systems Design and Development. Secondly, application was immediate because the context of the work of the participants demanded the use of the tools, concepts and practices covered in the workshop.
Lastly, application was immediate and possible because DSWD-Pantawid had the support of its then leader up until now. Budgets were also allocated strategically for internal capability building activities which allowed for the possibility of attendees to share the knowledge acquired from the course.
The application of lessons are self-sustaining for the same reasons that application was immediate – the team continues to be interested in Learning Measurement and Evaluation as well as Instructional Systems Design and Development and want both to be a formally established practice in DSWD-Pantawid; the work continues to demand that they organize their work processes based on best practices covered by the course; and, the culture is conducive for them to continually apply and practice the lessons learned from the courses.
