Study Notes on Program Evaluation and Evaluation Research

Program Evaluation and Evaluation Research

Purpose of Evaluation

  • Determine the effectiveness of a program, class, or treatment regarding meeting established goals and objectives.

  • Improve the structure and content of specific programs or treatments.

  • Justify the continued support of a specific program or treatment.

  • Assess which components of a program or event should be continued, discontinued, or adapted.

Nature of Evaluations

  • Evaluations are almost always a form of applied research.

  • Evaluations should be based on a comprehensive needs assessment that considers:

    • The needs of the agency implementing the program.

    • The needs of patients or participants involved.

    • The desired benefits from the program or service.

  • Evaluations can include both formative and summative evaluation components.

Getting Started with Evaluation

  • Key questions to ask regarding the goals and objectives of the event:

    • What is the purpose of the event?

      • Is it meant to bring in revenue ($) to your community?

      • Is it intended to provide an enjoyable experience for residents?

      • Will it improve the external image of an area?

      • Is it designed to provide exposure to sponsors?

      • Should it familiarize visitors with your town?

      • Will it improve the quality of life for patients?

      • Should it enhance specific skills (e.g., cognition, dexterity, strength, flexibility)?

      • Is it aimed at educating visitors?

      • Other purposes may also exist.

  • Understanding the purpose(s) of the event or intervention is the most crucial step in both developing and evaluating it.

Steps in the Evaluation Process

  1. Identify the program you wish to evaluate.

  2. Identify potential needs, benefits, and positive outcomes derived from the program.

  3. Determine how the identified needs and outcomes will best be measured.

  4. Establish goals and objectives relating to needs.

  5. Create instruments or tools to collect data regarding the evaluation.

  6. Determine an appropriate sampling strategy for data collection.

  7. Collect and analyze the data gathered from the evaluation.

  8. Make recommendations and changes as appropriate based on the findings.

  9. Comparisons can sometimes be made with previous evaluations to assess progress.

Types of Evaluation Questions

  • Perceptual Questions:

    • Assess impressions, satisfaction, images, beliefs, etc.

  • Behavioral Questions:

    • Focus on what people did and how much they spent, among other actions.

  • Quizzes or Tests:

    • Evaluate what participants learned about various topics.

  • Advancement Questions:

    • Measure the improvement of conditions or abilities as a result of the program.

  • Visitor Information Questions:

    • Gather demographic information, education level, origin of participants, etc.