Surveys

Surveys: An Introduction

Importance of Surveys

  • Surveys are a ubiquitous tool for gathering opinions and impressions.

  • A fundamental tool for understanding people's thoughts.

  • It's important to understand the possibilities and pitfalls of using surveys.

Surveys as a Research Tool

  • Surveys are easy to conduct compared to experimental tasks or complex measurements.

  • However, constructing a good survey can be challenging.

  • The key question is, how much can we learn from asking questions compared to experimental methods?

  • The answer depends on how the questions are asked.

  • Well-crafted questions can lead to reliable insights into thoughts, beliefs, and opinions.

  • Poorly written questions may not accurately capture true feelings.

Terminology: Open-Ended vs. Forced-Choice Questions

Open-Ended Questions

  • Allow respondents to answer in any way they choose.

  • Example: "How do you think Instagram usage affects one's self-esteem?"

  • Responses are unconstrained.

Forced-Choice Questions

  • Provide a limited number of response options.

  • Respondents must choose one or more of the provided options.

  • Example: "How do you think Instagram usage affects one's self-esteem?" (Scale of 1 to 5, from very negatively to very positively).

Advantages and Disadvantages

Forced-Choice Questions

  • Advantage: Responses are easy to analyze.

  • Disadvantage: Participants cannot provide nuance or details; limited in what they can say.

  • Example: "Using Instagram has negatively affected my self-esteem" (Scale of 1 to 10).

  • Data from two participants:

    • Participant 1: 8/10

    • Participant 2: 4/10

  • Average: 8+42=6\frac{8 + 4}{2} = 6

  • Data analysis is simple but lacks depth.

Open-Ended Questions

  • Advantage: Rich, detailed answers that are not limited by the questioner.

  • Disadvantage: Challenging to convert free-flowing answers into numerical data (coding).

  • Qualitative methods involve collecting free-form answers and coding/categorizing them.

  • This process requires raters and is more work than analyzing forced-choice questions.

  • Example answer: "Instagram, I guess I have a love hate relationship with it. Like it's really good for sharing and connecting with people but sometimes people can be annoying. I've heard that Instagram is connected to lower self esteem and sometimes it does feel that way. If I'm already having a bad day and then I get on social media and someone's like, hey, my life is so great."

  • The challenge is classifying this answer (neutral, slightly negative, slightly positive).

Trade-off

  • Forced-choice: Efficiency of analysis vs. simplicity of information.

  • Open-ended: Richness and information content vs. complexity of analysis.

Likert Scale

  • A specific type of scale where respondents indicate their level of agreement with a statement.

  • Example: "Using Instagram negatively affects my self-esteem" (Strongly disagree to strongly agree).

  • Includes a neutral middle option (neither agree nor disagree).

  • The labels on the options need to match exactly, otherwise it's a Likert-type scale but not a genuine Likert scale.

Semantic Differential Format

  • Uses adjectives to anchor the scale.

  • Adjectives guide respondents on how to use the scale.

  • Labels are essential on the highest and lowest points.

  • Example: "How does spending time on Instagram make you feel about yourself?" (Scale from anxious to confident).

Construct Validity of Surveys

  • Surveys are judged on construct validity: How well does the survey measure what it intends to measure?

  • Key question: Are the questions measuring the intended attitude, opinion, feeling, or value?

  • Another consideration: Are people answering honestly and accurately?

  • People may lie, give inaccurate information, or not know the correct answer.

  • Careful construction of surveys is crucial to avoid issues with construct validity.