Instructor Information

Instructor             : Professor dr. Siswanto Agus Wilopo, S.U, M.Sc., Sc.D.

Office                    : Gedung IKM Room 001, Jl. Farmaco 1, Bulaksumur, Yogyakarta
Office Hours        : 08.0-10.00 & Monday and Wednesday
Office Telephone: 62-274-548-156 or 5656076
E-mail                    : sawilopo@ugm.ac.id

Teaching Assistant

Coordinator          : dr. Rosalia Kurniawati Harisaputra, MPH

Office                     : Gedung IKM Room 008, Jl. Farmaco 1, Bulaksumur, Yogyakarta
Office Hours         : 10.00-12.00 & Tuesday and Thursday
Office Telephone : 62-274-548-156 or 5656076
E-mail                     : (rosaliakurnia@ugm.ac.id)

Course Description

This introductory course aims to help students understand how to design, monitor and evaluate health development projects. The health development project focus on the intervention which will increase health status of particulars population. The health development project evaluation will cover on the family planning and reproductive health program, maternal and child health program, tuberculosis, HIV-AIDS, malaria and nutrition program. The course will focus on enabling students to apply the concepts that they learn, including by critically assessing projects and identifying the most appropriate approach to design, monitoring and evaluation of health project. Each student will choose one project to work with for the entirety of the class and, by the end of the class, produce proposal for a monitoring and evaluation assessment and design for that project. Students will prepare this proposal according to relevant assignment. The proposal for health project design will be covered on the research methodology course. This course will be divided into four parts.

In Part I of the course, we will cover some of the basic aspects of project design which focus on: the family planning and reproductive health program, maternal and child health program, tuberculosis, HIV-AIDS, malaria and nutrition program. Course will review the components of a health development project, includes aims, inputs, outputs, outcomes, and impact. Lecture will then discuss theories of change and logic models, which describe the causal process through which the project’s activities are supposed to achieve its intended impact. Student will have laboratory discussion with teaching assistances for this topic.  For the first assignment of this class will be for each student to produce a document outlining the theory of change, logic model, and logical framework for her/his project’s outcome(s).

In Part II of the course, we will discuss project monitoring and adaptive project management. Lectures will focus on the particular role that monitoring can play in helping health development improve their project performance and learn from their successes and failures from the past project. Students will discuss, in laboratory class, how to apply these approaches to their project and include a detailed monitoring and adaptive management plan in their project proposed.

In Part III of the course, we will examine different approaches to evaluating the effect of health development project. This part will begin by reviewing the basic principles of research design that underlie these evaluation approaches, focusing on two different ways of thinking about causality (deterministic and frequentist) and associated approaches to comparative case selection and respondent sampling. We will then discuss four commonly types of evaluation designs: Comparative Case Study Designs, Participatory Designs, Randomized Controlled Trials (RCTs), and Quasi-Experimental Designs. Students will assess the relevance of each of these evaluation types for their project and should discuss in class with teaching assistants.

In Part IV of the course, we will discuss different field of health project development for the family planning and reproductive health program, maternal and child health program, tuberculosis, HIV-AIDS, malaria and nutrition program. The lecture will cover data collection instruments, approaches to data analysis, and the potential ethical and political challenges faced while collecting and analyzing data. In their laboratory exercise, students will be asked to identify the appropriate data collection instruments for their evaluation design and discuss how they will address the ethical and political challenges that they may face during data collection and analysis for their proposed project.

The course is not required that students have, training in the research design, data collection, or data analysis skills necessary to apply the monitoring and evaluation approaches reviewed in the class. For the final proposal, students should discuss in the laboratory session and present goal, research design, data collection, and data analysis plan.  

Textbook & Course Materials

Required Text

1. CDC (2011) Developing an Effective Evaluation Plan Setting the course for effective program evaluation. Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; Division of Nutrition, Physical Activity, and Obesity: Atlanta, GA, USA
2. Gertler, Paul J.; Martinez, Sebastian; Premand, Patrick; Rawlings, Laura B. and Vermeersch, Christel M. J (2016). Impact Evaluation in Practice – (Second Edition). World Bank and IDB: Washington, DC, USA.
3. Newcomer, Kathryn E.; Hatry, Harry P.  and Wholey, Joseph S. (2015) Handbook of practical program evaluation. (Fourth Edition) John Wiley & Sons, Inc., Hoboken, New Jersey, USA

Suggested Text

1. Shah, Anwar (2020) Policy Program and Project Evaluation A Toolkit for Economic Analysis in a Changing World. This Palgrave Macmillan: Gewerbestrasse 11, 6330 Cham, Switzerland
2. Beatty, Paul C.; Collins, Debbie; Kaye, Lyn; Padilla, Jose-Luis; Willis, Gordon B; Wilmot, Amanda (2020) Advances in Questionnaire Design, Development, Evaluation, and Testing. John Wiley & Sons, Inc.: 111 River Street, Hoboken, NJ 07030, USA.

Course Requirements

The course requirements are intended to help students understand and apply the course material. The weekly discussion with teaching assistant will test students’ understanding of the readings. The written assignments focus on the application of the course material to specific health development projects. Students should upload each assignment to the appropriate Elok discussion thread by the deadlines posted. Student should have the following:

• Internet connection (DSL, LAN, or cable connection desirable).
• Access to Web site/Other

Course Structure

This course will be delivered in lectures and discussion during laboratory sessions using online and hybrid approach (when it is possible). The lecture will be given twice weekly or depending to our department’s schedule. In each topic, lecturer will present statistical theory or concept and will discuss them within the class. The instructor will assign the exercises (homework) at the end of each laboratory session as stated in the lecture module. Exercises provide reinforcement of quantitative skills and specific content areas. During the course, students will experience and apply their knowledge through the development of data analysis given in the exercise or laboratory module.

Enrolment

This course is open to a limited number of individuals outside of the MPH's programs. Preference is given to UGM affiliated students, including doctorate students. We regret that auditing is not permitted. To apply for this course please fill out and submit the application available at the study program. Cost and submission information are in the application form.

Lecture Topics

Lecture will cover statistical theories and its application for health research. It will be given at least twice a week (Tuesday and Thursday: 10.00-12.00). Each session is about 100 minutes. There will be 14 sessions of lectures during this class and 12 Laboratory modules.

Laboratory Exercise

Students will be given opportunity to explore further details of the lecture materials in the form of discussion and exercise in the class. The teaching assistant is assigned to lead this class discussion and exercise. Their tasks provide student’s better understanding on the lecture materials and problem sets for the previous homework.

Problem Sets for Homework

Problem sets require the use of text editor and reference manager. The homework should be submitted to teaching assistant on time. The deadline of the submission will be announced in the class or laboratory exercise.

Online Resources

All lecture and laboratory materials are distributed in Web class. These resources are available in Elok or Gamel.   

Part 2: Student Learning Outcomes

Learning Objectives

This course will explain how to:

1. Identify a project’s theory of change and develop an associated logic model and logical framework,
2. Assess the relevance of each of the three main evaluation designs (participatory, comparative case study designs, randomized controlled trials/quasi-experimental) to the project,
3. Identify how the evaluation can be managed and monitored in a way that integrates adaptive management and organizational learning, and how this may influence the effect of the evaluation,
4. Develop a preliminary monitoring and evaluation plan for the project related to family planning and reproductive health program, maternal and child health program, tuberculosis, HIV-AIDS, malaria, and nutrition program.

Learning Outcomes

By the end of this course, students should be able to:

1. Prepare design project evaluation for public health intervention,
2. Apply framework of evaluation for health project,
3. Develop the theory of change for a project or program and develop a related logical framework,
4. Identify and apply the key components of four main types of evaluation designs - participatory, comparative case study designs, randomized controlled trials, and quasi- experimental,
5. Identify opportunities for organizational learning and adaptive management in the monitoring and evaluation of a health project or program,
6. Identify and address key ethical and political challenges in the implementation of monitoring and evaluation plans,
7. Critically assess the strengths and weaknesses of the different monitoring and evaluation approaches, and
8. Design a monitoring and evaluation plan for one of the four different types of evaluations design for family planning and reproductive health program, maternal and child health program, tuberculosis, HIV-AIDS, malaria, and nutrition program.

Lecture Topics

Lecture will cover project design, monitoring and evaluation theories and its application for public health project. It will be given at least twice a week. Each session is about 100 minutes. There will be 14 sessions of lectures during this class (see next following Table Class Calendar: 2021-2022). 

Laboratory Exercise

Students will be given opportunity to explore further details of the lecture materials in the form of discussion and exercise in the class. The teaching assistants are assigned to lead this class discussion and exercise. Their tasks provide student’s better understanding on the lecture materials and problem sets for the homework/ assignment.

Problem Sets for Homework

Problem sets require the use of text editor and reference manager. The homework should be submitted to teaching assistant on time. The deadline of the submission will be announced in the class or laboratory exercise.