Lecture 9: Surveys and Interviews in Crime Research

Frequent Topics in Crime Surveys

  • Victimization

  • Self-reports of offending

  • Distinction between frequency and prevalence

  • Perceptions and attitudes

I. Designing the Data Collection Instrument

Guidelines for Asking Questions

1. Format

  • Ensure the instrument is easy to read and uncluttered.

  • Utilize short items as they are best for respondents.

  • Consider length/time needed to complete the questionnaire.

  • Prepare for data entry after completion.

2. Wording

  • Utilize clear wording throughout the survey.

  • Avoid negative items; they can lead to confusion.

  • Refuse biased terms to ensure neutrality.

  • Use euphemisms appropriately to make the questions relatable.

  • Avoid double-barreled questions to eliminate confusion over multiple issues.

  • Conduct pilot tests if possible to refine questions and format.

3. Social Desirability

  • Understand that respondents may want to “look good” when providing answers.

  • To mitigate this:

    • State privacy provisions clearly to protect respondents.

    • Frame questions as representative of “common behavior” to encourage honest responses.

4. Use Existing Measures

  • Where possible, use validated questionnaires to ensure reliability.

  • Additional questions can be integrated as necessary.

  • Benefits:

    • Saves time and costs associated with questionnaire design.

    • Helps ensure the validity of the measures used in the study.

Response Formats

Structured/Closed-Ended Responses:

  • Examples include:

    • Multiple choice questions

    • Fill-in-the-blank responses

  • Advantages: Easy to process and analyze; reduced likelihood of non-response.

  • Disadvantages: Limited response choices; scales may bias respondent choice (e.g., when asked to rate happiness).

Unstructured/Open-Ended Responses:

  • Allows respondents to provide answers in their own words.

  • Advantages: No limits imposed by content; reflects the respondent's true feelings.

  • Disadvantages: Potential for misunderstanding and bias; higher likelihood of non-response.

Dichotomous Questions

  • Examples include:

    • Support for Covid lockdowns:

    • Yes

    • No

    • Nationality question:

    • Canadian ___

    • Not Canadian ___

Multi-Option Checklist

  • Example question: “Please check if you have ever tried the following substances:”

    • Marijuana

    • Psilocybin (magic mushrooms)

    • Ecstasy

    • OxyContin

    • Cocaine

    • Other:

  • Considerations:

    • Are all alternatives covered?

    • Is the list a reasonable length?

    • Is the wording impartial?

Levels of Measurement

  • Nominal: Questions about categories without a specific order.

    • Example: What is your occupation?

    • 1. Plumber

    • 2. Teacher

    • 3. Doctor

    • 4. Flight attendant

  • Ordinal: Questions requiring ranking in a specific order.

    • Example: Rank your preference for the following TV programs:

    • 1. The Summer I Turned Pretty

    • 2. Seinfeld

    • 3. Euphoria

    • 4. Real Housewives of Beverly Hills

Interval Questions: Likert Response Scale

  • Example statement:
    “Most homeless people are just too lazy to get a job.”

  • Scale:

    1. Strongly Disagree

    2. Disagree

    3. Neutral

    4. Agree

    5. Strongly Agree

Questions vs. Statements

  • Question Example:
    “What do you think about Vancouver’s Safe Consumption sites?”

  • Statement Example:
    “The Safe Consumption Sites promote the use of illegal drugs by sending a pro-drug message.”

  • Response options:

    • Strongly Agree…

    • Strongly Disagree

Contingency Questions

  • Questions dependent on previous answers to provide clarity.

Matrix Questions

  • An arrangement of related statements to gauge agreement or disagreement.

  • Disadvantages may include rigidity in fitting questions into format and tendency to create a response set among respondents.

Issues of Question Content

  • Assess whether the question is necessary or useful.

  • Consider if respondents can understand the question without confusion.

  • Evaluate if several questions are needed instead of combining into one complex question.

  • Confirm if respondents possess the necessary information to respond accurately.

  • Determine if respondents need to look up additional records to answer comprehensively.

  • Analyze whether the question needs further specification or if it is too vague.

  • Avoid bias in phrasing to ensure truthful responses.

  • Specify the timeframe for questions when necessary.

  • Keep wording direct but sensitive to respondents.

Order of Questions

  • Recognize potential order effects on responses:

    • Start with interesting questions to engage respondents.

    • Follow with provocative questions.

    • Place boring or difficult questions at the end.

  • For self-report surveys, use easy, straightforward questions first, then move to more complex or sensitive topics.

  • For face-to-face interviews, maintain similar ordering for quality responses.

II. Collecting Survey Data

Self-Report

a) Group-Administered Surveys

  • Example context: Academic settings (classroom), correctional facilities (prison).

  • Advantages:

    • More cost-effective and quicker data collection compared to mail.

    • High response rates due to engagement.

  • Benefits of self-reporting include privacy and direct contact with the researcher.

  • Major concerns involve issues of anonymity and ensuring voluntary participation.

b) Mail/E-mail Surveys

  • Advantages include:

    • Enhanced privacy and low cost, especially for emails.

    • Response rate classifications:

    • 50% adequate

    • 60% good

    • 70% very good

  • Disadvantages include:

    • Low response rates and lack of control over question context or order.

    • Challenges due to potential literacy barriers and inability to clarify responses.

    • Require administrative support and database management for effective analysis.

  • Maximizing response rates strategies:

    • Simplify return process (e.g., prepaid envelopes).

    • Keep survey length reasonable.

    • Consider timing of distribution.

    • Implement warning mailings.

    • Offer incentives for participation.

    • Include cover letters and reminders for participation.

c) Online Surveys

  • Cost-effective, private, and convenient method for data collection.

  • Efficient uploading of information and control over question order and formatting.

  • Disadvantages:

    • Open online surveys raise doubts about generalizability of results.

    • Challenges in ensuring unique respondents when incentivized.

    • No control over context and potential issues with literacy or lack of clarifying questions.

In-Person Interviews

  • Conducted face-to-face with an interviewer.

  • Useful for:

    • Circumstances where mail or phone surveys are impractical.

    • Situations requiring visual aids or detailed explanations.

    • Establishing rapport with interviewees which can enhance data quality.

  • Response rates typically hover between 80-85%.

Increased Data Quality Benefits:

  • Established rapport increases comfort and willingness to share.

  • Use of visual aids enhances understanding of questions.

  • Probing techniques: silent, encouragement, elaboration, clarification, and repetition can enrich responses.

  • Interviewers can make independent observations to supplement self-reported data.

Problems with In-Person Interviews:

  • Risks of social desirability bias affecting response authenticity.

  • Potential interviewer biases influencing responses.

  • Interview effects tend to be minimized in video interviews but persist in personal interactions.

  • The format can be expensive and time-consuming due to travel and safety concerns, although video interviews may alleviate some issues.

Consider a Multi-Method Approach

  • Combining various data collection methods can enhance overall data quality.

  • Avoid employing inconsistent methods across different sampling segments, as this can lead to data incongruities.

III. Response Rates

  • Example calculation for response rates:

    • Sample size: 250; Completed surveys: 180

    • Response rate = 180/250 = 72%

    • Response rate definitions:

    • Calculated as the number of complete surveys divided by the total number of eligible cases.

  • Adjusting denominators based on misaddressed contacts:

    • Example:

    • 25 individuals had incorrect emails, 5 deceased, and 5 incarcerated.

    • Revised denominator = 250 - 35 = 215.

    • Revised response rate = 180/215 = 84%.

Impact of Changing the Denominator

  • It's crucial to clarify adjustments made to the denominator and their rationale, as they significantly affect the perceived response rate.

Categories of Nonresponse

  1. Individuals not included in sampling frame (e.g., unreachable by phone).

  2. Individuals within the sampling frame who were not contacted (e.g., unanswered calls).

  3. Individuals who were contacted but refused to respond.

  4. Individuals who were contacted but were unable to answer (e.g., due to illness or being on vacation).

  • It is permissible to adjust denominators based on these categories to provide a clearer representation of data.

  • There are no strict rules governing these adjustments, yet transparency in methodology is essential for credibility.