Program Evaluation

What is Program Evaluation?

A systematic process of collecting and analyzing data to determine effectiveness and efficiency of one or more programs, policies, or orginizations

What is the definition of a program?

Any set of related activities undertaken to achieve an intended outcome

Uses of program evaluation

to make changes in our practices, to inform decisions on whether or not we should continue to keep doing something or change it

Purpose of formative evaluation

assess whether a program is feasible, appropriate, and acceptable before it is fully implemented

Purpose of summative evaluation

To assess the effectiveness of the program and the extent of change as a result of the program

True or false, program evaluation is about evaluating your patient population’s overall health

false

true or false, the best program evaluations are done alongside the designing of the actual program you plan on implementing

true

What is quantitative data?

data in the form of numbers, counts, means/medians, ect

What is qualtitative data?

verbal/written data that comes form review of documents and abstracting major themes form them

Ways to collect quantitative data

Data collection from patients (ex: BP), surveys, direct observations that are scored, numbered rubrics/grading

Ways to collect qualtitative data

semi-structured one-on-one interviews, focus groups, photographs

how is data collected from a focus group?

captured audio recording and notes of cross-chatter b/w participants

how is data collected from a semi-structured interview?

follow script w/ questions but allow conversation to flow and take twists w/ patient, audio recorded words are transcribed and major themes are determined through analysis

Even though both research and evaluation can be published, how do they differ?

research seeks to contribute to generalizable knowledge and test a hypothesis, evaluation focuses on determining the value or worth of a program and providing info to help it improve

Community oriented, participatory model for creating successful community health promotion interventions

PRECEDE-PROCEED

is PRECEDE-PROCEED the only model for program implementation/evaluation?

no

Key concept of the PRECEDE-PROCEED model

community involvement

phases of PRECEDE

social assessment, epidemiological assessment, educational & ecological assessment, health program & policy development

What is the purpose of social assessment (phase 1)

define the QOL and priorities/problems in the priority population, involve them in their own need

What is the purpose of epidemiological assessment (phase 2)

Identify the health or other issues that most clearly influence the outcome of the community

What is the purpose of Educational and ecological assessment (phase 3)

Identify behaviors/lifestyle/environmental factors that must be changed to affect the health issues identified in phase 2

Include knowledge and many affective traits that can facilitate or hinder a person’s motivation to change

predisposing factors

Barriers or facilitators created mainly by societal forces or systems (ex: access to healthcare facilities), the resources and skills required to make desire behavioral and environmental changes

enabling factors

Involve the different types of feedback and rewards that those in the priority population receive form others after behavior change

reinforcing factors

What is the purpose of health program & policy development (phase 4)

Intervention alignment (to the factors from phase 3) and administrative/policy assessment

When does PRECED end and PROCEED begin?

between phases 4 and 5

What is a process evaluation? (phase 5)

Monitors/measures implementation to control, assure, or improve quality of the program

in the P-P model, short term and intermediate term outcomes are called _____ and long term outcomes are called _______

“Impact Evaluation”, “Outcome Evaluation”

What is impact evaluation (phases 6-7)

evaluating whether the intervention is having the intended impact on the behavioral and environmental factors its aimed at, and evaluate its efficiency and effectiveness

What is outcome evaluation (phase 8)

Evaluates wheter the interventions effects are producing the outcomes the community identified in phase 1 and adjust accordingly, assessing long term effects on QOL indicator and overall health