PUBPOL 479 First Exam Flashcards

Relationship between evaluation, program evaluation, impact evaluation

Evaluation - broadest; program evaluation - more specific; impact evaluation - most specific

Program / policy evaluation

A type of policy analysis

Examples include needs assessments, evidence synthesis/literature review, process evaluation, value for money/Cost benefit/Cost-effectiveness evaluations, impact/outcome evaluation

Needs assessment

Identify and prioritize the needs of a project; understand problem you want to address; create description of problem & existing solutions; focuses attention on an overlooked problem (challenges on the ground)

Examples of a needs assessment

What is the level of learning, who is falling behind?

What does the community of Washtenaw County need most in terms of services from United Way?

Who do people currently borrow from, at what interest rates?

What is the absenteeism rate of clinic staff?

Why are needs assessments are often done before other evaluations?

Determine the right target for the intervention (what geographic location, what age range, what income group)

Improve the design of the intervention

Help understand if the problem is not what we thought (ie: might be lack of teachers, not sexism stopping girls from going, so we have to create a teacher program)

Evidence synthesis

Summary of results from related descriptive studies, process, and impact evaluations. A synthesis of the prior research and evaluation evidence.

(more simply, taking all literature we know and put them in one document to summarize your results, shows us what we already know, sufficient when it suggests a clear path forward for what’s left to research)

Process evaluation

Documents/understands what programs or policies look like “on the ground”

Usually tracking paperwork/finances

Done with other types of assessments

Implementation evaluation

Focused on program fidelity (does each site that implements a program do it the way they are “supposed” to?)

At risk mothers nursing program: A process evaluation might visit several states/counties to see how they were doing their programs, were they adhering to the standards set out in the original program etc. (program fidelity).

Formative evaluation

Examine the quality of different programs to see how things are going/improve the programs. Monitoring/gathering data.

Cost benefit analysis

Value for money, cost-benefit, benefit-cost or cost-effectiveness evaluations.

Do the benefits outweigh the costs?

Hard to weigh social things (ie: discrimination, spill-over benefits, long-term benefits)

Done in combination with other evaluations

Benefits - cost = net benefits

Impact / outcome analysis

Evaluate the effect of a program or policy

Impact evaluations are asking causal questions: Does A cause B?

Descriptive questions

Describes the population (ie: % of children who live in multigenerational households), asks about or describes the world, may highlight differences/disparities, trends/change over time, can be quantitative or qualitative, describes the problem

Causal questions

A causes B

Asks does the program bring the desired outcome?

Examples:

Did expansions to the EITC improve the housing situations of low-income families?

Does the EITC cause better housing?

(Answer: yes for some outcomes, no for others)

Does something improve / worsen?

Outcome

The desired change the program or policy wants to bring about

Four types of program evaluations

Descriptive implementation, descriptive output/outcomes, casual implementation, causal output/outcomes

Why does the distinction between causal and descriptive matter?

Type of evaluation that you need to do is different depending on the type of questions

you want to answer

Belmont Report

Principles to follow when conducting research

Respect for persons - informed consent

Beneficence - do no harm; maximize possible benefits and minimize harm

Justice - the distribution of benefits and burdens of research should not be unfair

Setting Up for an Evaluation

1. What are the questions you want to answer? (Learn about programs and policies from stakeholders; understanding what the expected outcomes are of a program; the way in which they come about (logic models); choosing questions when there might be many

2. How do you go about answering those questions (Choosing the right approach (method of evaluation); Gathering the right data – measuring, collecting or finding data)