InQuizitive Ch. 6 - Surveys and Observations: Describing What People Do

What type of validity is most important to consider when creating a poll or survey

construct validity

When observers' knowledge of the research hypothesis influences how they record their observations, ______ occurs. When participants change their behavior based on perceived expectations from the observer, it is called ______. When people change their behavior based on the presence of someone else, this is known as ______.

1. observer bias

2. observer effects

3. reactivity

Which of the following scenarios are examples of observational research?

Observational Research:

- recording how often students use the word literally in conversations with each other

- counting the number of people who walk through a busy convenience store

Not Observational Research:

- asking people whom they voted for as they leave a local polling station

- observing the blood pressure of 50 patients via the records of a local health clinic

Respondents are asked to choose a number corresponding to how much they agree or disagree in ______ questions. Respondents can provide as much or as little information as they would like in ______ questions, while ______ questions ask respondents to choose between two or more options.

1. Likert scale

2. open-ended

3. forced-choice

When the wording of a survey or poll question influences respondents' answers, its construct validity is weakened. For instance, a question with wording that encourages a particular response is called a ______ question. A ______ question using phrases such as "should never not" can be cognitively difficult and confusing to respondents. And researchers cannot be sure which part of the question people are responding to in a ______ question—a question that poses two questions.

1. leading

2. negatively worded

3. double-barreled

Identify the descriptors as applicable or not applicable to this survey question.

What mental health problems are you currently experiencing?

Applicable:

- leading

- open-ended

Not Applicable:

- negatively worded

- forced-choice

- Likert scale

Match each problem that can occur in observational studies to the potential solution that addresses it.

observer bias:

- Use a clear coding system

- Use a masked or blind study design

reactivity:

- Wait for participants to become used to the observer

- Use a one-way mirror

- Use unobtrusive data.

Match each term to the correct example.

observer effects: Based on the researcher's body language, a participant guesses that the study is about politeness in intergroup interactions, so he acts politely to align with the researcher's expectations

observer bias: Researchers interpret prisoners' behavior as more aggressive than college students' behavior even though the behavior is actually identical

reactivity: Small children stop and stare at the experimenter who is in the room taking notes on their behavior

If respondents become fatigued or are unengaged in a survey, they may answer questions in a general pattern without really considering their honest responses. This is known as using a(n) ______. If respondents continually select "yes," they are participating in ______. If respondents select all middle options, they are participating in ______. With sensitive topics, sometimes respondents answer in a way that aligns with what they believe everyone else thinks, in a way that makes them look good, or in a way they think the researcher would like them to answer. This is called ______.

1. response set

2. acquiescence

3. fence sitting

4. socially desirable responding

This question appears on a survey: "Please indicate your level of agreement with the following statement on a scale of 1 (strongly disagree) to 5 (strongly agree): My smartphone is new and I am satisfied with its performance."

In what way could this question be changed to improve its construct validity?

- Respond to the following questions on a scale 1 (strongly disagree) to 5 (strongly agree)

1. my smartphone is new

2. satisfied with my smartphone

Strongly disagree - 1

Strongly agree - 5

Identify each research question as best explored with either a self-report or an observational study.

self-report study:

- What are people's beliefs about face mask-wearing for protection in urban areas versus rural areas?

- What are some experiences of discrimination that female academics encounter?

- How much money do parents spend on baby toys each month?

observation study:

- Are women less likely than men to be addressed as "Dr." at an academic conference?

- Do more people wear face masks for protection in urban areas than in rural areas?

- Do babies spend more time with new items than with familiar ones?

Identify the true and false statements about the ethics of observational studies.

True Statement(s):

- If using hidden video recording, researchers should tell participants at the end of the study that they were filmed.

- It is ethical to observe people in a public place because people expect their behavior to be public, not private.

- When reporting observations from a public setting, researchers must not specifically identify the people they observed.

False Statement(s):

- Researchers can watch and retain any video recordings, even if the participant has not given permission.

Match each question type to the correct example.

negatively worded:

There is never a reason to be late for work

1. Disagree | 2 | 3 | 4 | 5. Agree

double-barreled:

How often are you late to or absent from work?

Never | Not very often | Sometimes | Very often

leading

In your opinion how critical is the problem with background checks for firearms in this country?

- There is a major problem with people not getting background checks.

- While there are some problems with background checks, the system is good enough.

Seo-yeon and her research assistants are examining sharing behavior in children. Match each problematic scenario from Seo-yeon's study to the correct potential solution.

Use a clear codebook: Seo-yeon and her research assistants are told that the children participating in their study are particularly lively and friendly.

Make unobtrusive observations: Seo-yeon and her research assistants notice the children are sitting awkwardly throughout the study.

Use a masked design: One of Seo-yeon's research assistants unintentionally cues the children to share as he passes out cookies.

Identify the true and false statements about the accuracy of self-reporting memories.

True Statement(s):

- Memories of negative events, such as adverse childhood experiences (ACEs), can be accurate

- The vividness of flashbulb memories tends to last over time, even many years later.

False Statement(s):

- Flashbulb memories of where people were, and what they were doing, during a significant event tend to be accurate over time.

- The level of certainty people feel about their memories is directly related to the accuracy of the memories.

Identify the true and false statements about survey and poll questions.

True Statement(s):

- Questions that come earlier in a survey may affect how later questions are answered.

False Statement(s):

- How a question is worded has little to do with how it is answered.

- Even if respondents try to be completely honest, it is impossible for them to respond to a survey accurately enough to provide meaningful data.

- The main drawback of open-ended questions is that people often write things that are illegible.

Inwoo conducts a study examining the association between loneliness and comfort food consumption. She asks participants to write about the last time they felt lonely, and then tells them they are welcome to eat as many cookies as they want. Inwoo's research assistant marks down the number of cookies the participants ate while recalling their lonely experience. Inwoo observes that all participants ate most of the cookies available to them and becomes concerned that they understood the purpose of her study.

Which of the following strategies would be helpful in addressing Inwoo's concern that participants guessed the purpose of her study?

Helpful:

- use a masked sign

Not Helpful:

- Sit in the room long enough before observing

- Use multiple observers

- Observe participants from behind a one-way mirror

Identify the true and false statements about the reliability and validity of observational studies.

True Statement(s):

- ICC is a correlation that quantifies the level of agreement between observers.

- Using multiple observers does not eliminate observer bias.

False Statement(s):

- Codebooks increase reliability because they prevent reactivity in participants.

- Good interrater reliability indicates that the observations are valid.

Sequoyah is surveying parents about the health of their children. He is worried that participants will want to look like excellent parents and will claim that their children exercise and eat healthy all the time.

Identify each strategy as helpful or not helpful for addressing Sequoyah's concern.

Helpful:

- Add items to identify those who answer in socially desirable ways

- Remind the participants that their responses are anonymous

Not Helpful:

- Add reverse

- worded items

- Eliminate the neutral option

Match each research goal to the question format that would best achieve it.

forced-choice: You want respondents to select the professor whose teaching method they enjoy the most

open-ended: You want respondents to explain their thoughts on their professor's teaching methods

semantic differential: You want to know the degree to which respondents enjoy their professor's teaching methods

Likert scale: You want respondents to indicate how much they agree with their professor's teaching methods