Chapter 11: Evaluation Research

What is evaluation research?

The systematic application of social research methods to assess the design, implementation, and effectiveness of programs or policies.

Which of the following sequences BEST represents the correct process of conducting a qualitative content analysis?

Data collection → Decontextualization → Recontextualization → Categorization → Compilation

How is evaluation research used by policymakers?

It helps policymakers determine whether programs are effective, need improvement, or should be continued, modified, or terminated.

How does evaluation research differ from basic research?

Evaluation research is applied and focused on solving specific problems, while basic research is conducted for knowledge and theory development.

-Evaluation research differs from basic research because its applied in nature*

Why is evaluation research conducted?

• To assess program efficiency and effectiveness

• To replace assumptions with data

• To provide accountability

• To inform decision-making

What are the 5 purposes of evaluation research?

1. Assess program performance

2. Determine if goals are met

3. Evaluate effectiveness

4. Provide accountability

5. Educate stakeholders/community

What is the first step in evaluation research?

Identify and engage stakeholders.

What are the 7 Steps of Evaluation research?

1. Engage stakeholders

2. Develop research question

3. Design methodology

4. Gather data

5. Analyze data

6. Develop findings/conclusions

7. Communicate results

What are the Main Types of Evaluation Research?

• Needs assessment

• Process (policy) evaluation

• Logic models

• Outcome evaluation

• Impact evaluation

What is a needs assessment?

A formative evaluation that identifies the needs of a target population and gaps in services

-Understanding the needs of a target population.

What is process evaluation?

Examines whether a program is being implemented as intended.

What is a logic model?

A visual representation linking program inputs, activities, and expected outcomes.

What is outcome evaluation?

Measures the immediate effectiveness of a program on a target population (often using quantitative data).

What is impact evaluation?

Assesses whether a program achieves its broader, long-term goals and social impact.

2 main types of Evaulation

Formative and Summative

What is formative evaluation?

Evaluation conducted during program development to improve design and implementation.

What are examples of formative evaluation?

• Needs assessment

• Process evaluation

• Policy evaluations

What is summative evaluation?

Evaluation conducted after implementation to determine overall effectiveness and whether the program should continue.

What are examples of summative evaluation?

• Outcome evaluation

• Impact evaluation

A researcher evaluates whether a new program is needed before implementation. What type is this?

Formative evaluation (needs assessment)

A researcher examines whether a program reduced recidivism rates. What type is this?

Outcome evaluation (summative)

A researcher evaluates whether a program achieved its overall societal goal. What type is this?

Impact evaluation (summative)

A researcher studies whether a program is being implemented correctly. What type is this?

Process evaluation (formative)

What are the 4 Characteristics of Effective Evaluation?

• Utility ( evaluation is useful and meets the needs of stakeholders.)

• Feasibility (evaluation is practical, realistic, and cost-effective)

• Propriety (evaluation is ethical, fair, and respects participants.)

• Accuracy (evaluation produces valid, reliable, and correct findings.)

What is utility in evaluation research?

The evaluation is useful and meets the needs of stakeholders.

What is feasibility?

The evaluation is practical, realistic, and cost-effective.

• Is it doable?

What is propriety?

The evaluation is ethical and fair.

What is accuracy?

The evaluation produces valid and reliable findings.

What are the 5 Stages of the Policy Process?

1. Problem identification

2. Policy formulation

3. Policy adoption

4. Policy implementation

5. Policy evaluation

At what stage is evaluation research used in the policy process?

Policy evaluation stage (to assess effectiveness and recommend changes)

Why are ethics important in evaluation research?

Because findings can directly impact programs, funding, and people’s lives.

What are ethical concerns in evaluation research?

• Maintaining confidentiality

• Avoiding bias

• Preventing harm

• Protecting participants

What are consequences of unethical evaluation research?

• Harm to participants

• Incorrect conclusions

• Poor policy decisions

• Reinforcement of inequality

What are common challenges in evaluation research?

• Political pressure

• Loss of objectivity

• Stakeholder influence

• Real-world constraints

What does “The purpose of evaluation is to improve, not prove” mean?

Evaluation research aims to improve programs rather than simply prove they work.

Which statement BEST distinguishes evaluation research from basic research?

Evaluation research applies research methods to assess programs, while basic research generates knowledge for its own sake

A researcher is hired to determine whether a newly proposed reentry program is needed in a community and how it should be structured. What type of evaluation is this? Formative of Summative?

Formative evaluation

Which of the following is the BEST example of an impact evaluation?

Assessing whether a program contributed to long-term reductions in crime at the community level

Which of the following occurs FIRST in the evaluation research process?

Identify and engage stakeholders

An evaluation is designed to ensure findings are useful to policymakers and program administrators. Which characteristic is being emphasized?

Utility

Which of the following is MOST likely to be an ethical violation in evaluation research?

Excluding key stakeholders from the evaluation process

A researcher evaluates whether a rehabilitation program is being delivered as originally designed. What type of evaluation is this? Process or Outcome Confusion?

Process evaluation