Nov 22 - Survey Research

Survey Research Overview

  • Course: PSYC 2301: Research Methods

  • Instructor: Dr. Stelian Medianu

Descriptive research

  • To describe a single variable or to obtain separate descriptions for each variable when several are involved

  • Focus on common survey research

Introduction to Survey Research

  • Purpose of Survey Research: To describe a variable or obtain separate descriptions for each variable involved.

  • Type of Research: Focuses on descriptive research, which is a common methodology in understanding trends and characteristics in various populations.

Examples of Survey Research

  • Opinion Poll: Assessing political attitudes.

  • Behavioral Survey: Percentage of Americans who skip breakfast (31 million).

Researcher must consider the following

  1. Survey questions must be developed

  2. Survey questions must be assembled and organized

  3. Identification of the sample

  4. It must be decided how the survey will be administered

Designing a Survey

Key Considerations

  • Objective: Gather information on:

    • Participation in educationally purposeful activities

    • Institutional requirements and challenges

    • Perceptions of the college environment

    • Estimates of educational and personal growth

    • Background and demographic information

Development of Survey Questions

Types of Questions:

  1. Open-ended Questions: Allow for qualitative answers.

    Pro:

    • participants have more flexibility to answer

    • Useful to explore new topic

    Con:

    • Difficult to code and analyze

    • Participants may vary in detail depending in their ability to express their thoughts

  2. Restricted Questions: Provide list of choices for participants to choose from

  3. Rating-scale Questions: Allow respondents to express their opinions on a scale. Ex. 1-5 how happy you feel daily etc.

    Pro:

    • Produce numerical values on interval scale- good for statistical analysis

    • Easy to understand and answer

    Con:

    • Participants tend to respond to questions in a systematic way, not paying attention to content (ex. choosing ‘strongly agree’ for all)

    Solution:

    • Word the same question in a diff way/ include a mix of questions/statements

Researchers should Avoid

◦Unnecessary Complexity

  • question needs to be simplified

◦Double-barreled questions

  • Questions ask two things in 1 question

◦Loaded questions

  • framing the question in a way that the participant would be inclined to choose one over the other (Ex. Do you favour eliminating the wasteful excesses in the school cafeteria budget? vs. Do you favour or oppose a reduction in the school cafeteria budget?)

    ◦Negative wording

  • Framing questions negatively can confuse respondents.

◦Yea-saying and nay-saying

  • A response set to agree or disagree with all the questions

Question Types Explained

Open-ended Questions

  • Advantages: Flexibility in responses and exploration of new topics.

  • Disadvantages: Complexity in coding and analysis; varied detail in responses.

Restricted Questions

  • Example: "What is your major?"

Rating-scale Questions

  • Example: Rate interactions with faculty.

  • Advantages: Produces numerical data helpful for analysis.

  • Disadvantages: Risks of response sets, where participants may answer in a systematic manner regardless of true beliefs.

Considerations for Survey Design

Issues to Avoid

  • Unnecessary Complexity: Keep questions straightforward.

  • Double-barreled Questions: Avoid asking multiple things at once (e.g., combining two issues).

  • Loaded Questions: Questions that may lead respondents.

  • Negative Wording: Framing questions negatively can confuse respondents.

  • Response Sets: Prevent tendencies to uniformly agree or disagree with questions.

Organizational Strategies

  • Placement of Questions: Use a thoughtful organization.

    • Sensitive questions in the middle. (how often do you cheat on exams)

    • Demographic questions at the end. (age, gender, ethnicity)

  • Grouping: Similar-topic questions should be grouped together.

    • Questions in the same format should be grouped together

  • Clarity: Ensure vocabulary is easy to understand for participants; avoid cluttered formats.

Identifying the Sample

  • Choose participants relevant to the survey's focus (e.g., college students).

  • Survey participants must not be too restricted

  • Aim for a sample representative of the wider population to ensure validity.

Administration of the Survey

Types of Administration Methods

  1. Internet Surveys:

    Pro: Efficient, access to a large # of people with common characteristics

    Con: may lead to sample bias, cannot control the composition of the sample

  2. Mail Surveys:

    Pro: Convenient, anonymous, non-threatening to participants

    Con: may suffer from low response rates, non-response bias

  3. Telephone Surveys:

    Pro: Allow direct interaction, conducted from home or office

    Con: time-consuming, interviewer bias

  4. In-person Surveys:

    Pro: 100% response rates; flexibility, efficient to administer

    Con: time-intensive, interviewer bias,

Strengths and Weaknesses

  • Strengths: Flexibility, gaining info about a broad range of variables, ease of data collection.

  • Weaknesses:

    • Low response rates potentially skew results.

    • Nonresponse bias where non-responders differ systematically from respondents.

  • Social Desirability Bias: Respondents may give inaccurate responses to align with social norms.

  • Non-response Bias: People who complete the survey are a self-selected group that may not be representative the population

Study Terms

  • Define and generate examples for:

    • Close-ended questions

    • Open-ended questions

    • Rating scales

    • Restricted questions

    • Response bias

    • Response rate

    • Response set

    • “Yea-saying” or “nay-saying” response set

Review Questions

  • What constitutes a survey?

  • Key features in constructing survey questions?

  • Understanding social desirability in responses and ways to minimize it.

  • Advantages and disadvantages of different survey administration methods.

Next Steps for Students

  1. Finalize data analyses by November 22.

  2. Prepare PowerPoint presentation (10-13 mins, max 10 slides) due November 28.

  3. Submit individual project papers by December 2.

  4. Prepare for the final exam on December 13 covering materials from weeks 7 to 13, accounting for 28% of the course grade.