DATA-COLLECTION

DATA COLLECTION INSTRUMENTS

INTRODUCTION

• Data Collection Strategies

• Characteristics of Good Measures

• Quantitative and Qualitative Data

• Tools for Collecting Data

DATA COLLECTION STRATEGIES

• There is no single best method; decision depends on:

  • What you need to know: Numbers or stories?

  • Where the data reside: Environment, files, people

  • Available resources and time

  • Complexity of data to be collected

  • Frequency of data collection

  • Intended forms of data analysis

RULES FOR COLLECTING DATA

• Use multiple data collection methods.

• Utilize existing data with knowledge of:

  • How measures were defined

  • Data collection and cleaning processes

  • Extent of missing data

  • Accuracy assurance of data

Collecting Original Data

• Be sensitive to burdens on participants

• Importance of pre-testing the instrument

• Establish and maintain protocols and accurate records

• Verify coding and data entry accuracy

STRUCTURED APPROACH

• Collect all data in the same way.

• Crucial for:

  • Multi-site and cluster evaluations for comparison.

  • Essential when comparisons between alternate interventions are needed.

When to Use Structured Approach

• Address quantitative extent questions

• Work with large populations or samples

• Clearly define what needs to be measured

• Show results numerically

• Make comparisons across different sites or interventions

SEMI-STRUCTURED APPROACH

• General procedures followed, but not uniformly:

  • More open and fluid

  • Not following a rigid script

  • May solicit more detail from participants

  • Allows participants to express thoughts in their own words

When to Use Semi-Structured Approach

• Conduct exploratory work

• Seek understanding, themes, or issues

• Gather narratives or stories

• Aim for in-depth information

• Understand unexpected data results

CHARACTERISTICS OF GOOD MEASURES

• Relevance: Is the measure meaningful?

• Credibility: Is it believable and appropriate?

• Validity: Does it measure what it is supposed to?

• Reliability: Is it consistent upon repeated trials?

Relevance

• Ensure the measure captures what truly matters.

• Avoid measuring for convenience instead of necessity.

Credibility

• Measure must be seen as a reasonable way to capture information.

Internal Validity

• Assess how well the measure reflects the intended purpose.

• Example: Are waiting lists a valid measure of demand?

Reliability

• Precision and stability of the measure.

• Example assessments: Birth weights, timing with stopwatches.

QUANTITATIVE APPROACH

• Data in numerical form; precise measurements required.

  • Examples: Age, cost, length, etc.

• More challenging to develop but easier to analyze.

QUALITATIVE APPROACH

• Deal with descriptions; observed or self-reported data.

• Less structured, easier to develop.

• Produces rich, detailed data; harder to analyze.

Which Data to Choose

• Qualitative: When narrative or in-depth information is needed.

• Quantitative: When precise measurements or statistical analysis are desired.

OBTRUSIVE VS. UNOBTRUSIVE METHODS

• Obtrusive: Directly obtain data from subjects.

  • Examples: Interviews, surveys, focus groups.

• Unobtrusive: Collect data without direct interaction.

  • Examples: Document analysis, observational methods.

TRIANGULATION TO INCREASE ACCURACY OF DATA

• Triangulation of Methods: Using various methods to collect the same information.

• Triangulation of Sources: Gathering data from diverse sources.

• Triangulation of Evaluators: Collecting data from multiple evaluators.

DATA COLLECTION TOOLS

• Participatory Methods

• Records and Secondary Data

• Observation

• Surveys and Interviews

• Focus Groups

• Diaries, Journals, Self-reported Checklists

• Expert Judgment

• Delphi Technique

• Other Tools

TOOL 1: PARTICIPATORY METHODS

• Engage communities in data collection.

• Examples:

  • Community meetings

  • Mapping

  • Transect walks

COMMUNITY MEETINGS

• Organize effectively with defined purpose & rules.

• Manage speaker times and format for Q&A sessions.

MAPPING

• Utilize maps to engage stakeholders in understanding.

• Generate discussions and verify secondary data.

• Types include natural resources, demographics, etc.

TRANSECT WALKS

• Observe community behaviors and resources by walking a transect line.

• Good observational skills are necessary.

TOOL 2: RECORDS AND SECONDARY DATA

• Sources may include:

  • Existing records, census data, reports, etc.

TOOL 3: OBSERVATION

• Observe real-time behaviors and conditions.

ADEQUATE OBSERVATION GUIDELINES

• Use multi-observer setups if possible.

• Train observers for consistency and reliability.

• Develop a pilot test for the observation instrument.

TOOL 4: SURVEYS AND INTERVIEWS

• Effective in gathering opinions and perceptions.

• Attention needed on response rates and sample representation.

STRUCTURED VS. SEMI-STRUCTURED SURVEYS

• Structured: Exact wording with predetermined responses; easier to analyze.

• Semi-Structured: General questions with open-ended responses; provide richer data but harder to analyze.

MODES OF SURVEY ADMINISTRATION

• Various methods include telephone surveys, self-administration, and face-to-face questionnaires.

• Consider language or translation issues in development contexts.

TOOL 5: FOCUS GROUPS

• Qualitative approach with small groups discussing specific topics.

• Inappropriateness: Language barriers or lack of participant trust can hinder effectiveness.

FOCUS GROUP PROCESS

• Phase 1 - Opening: Ice-breakers and establishing ground rules.

• Phase 2 - Warm-up: Stimulate interactive experiences.

• Phase 3 - Main Body: Engage in deeper discussions.

• Phase 4 - Closure: Summarize findings and invite comments.

TOOL 6: DIARIES AND SELF-REPORTED CHECKLISTS

• Real-time data on daily events; useful for sensitive data.

TOOL 7: EXPERT JUDGMENT

• Utilize experts for structured or unstructured data collection.

SELECTING EXPERTS

• Develop criteria beyond recognition to include diverse perspectives and expertise.