PSY 250
Validity: Appropriateness or accuracy of a conclusio of decision
Is this study reasonable? Accurate? Justifiable?
These are issues of validity and allow us to evaluate research claims
Construct validity: Appropriate construct measurment/manipulation
How well does my operationalization align with my conceptual variable?
Statistical validity: Precice, reasonable, and replicable statistical conclusions
How well do the results support the claim/conclusion?
External validity: Accurate represenation of the results beyond the study
How well do my results generalize to other contexts
Internal validity: Clearly established cause/effect relationship; no other explanations
How much can I conclude that the IV caused the changed in the DV
Self Report: People record their responses to a questionnaire or interview
Observational: Record behavior or physical traces of behaviors via observation
Physicological: Record biological or physiological data via specialized equipment
The tool you should use always depends on your research question
Longitudinal designs: provide evidence for temporal precedence by measuring the same variables in the same people at several points in time points or ‘waves’
Cross-sectional correlations test whether two variables, measured at the same point in time, are correlated.
Autocorrelation evaluates the correlation of each variable with itself across time
Cross-lag correlations are of more interest to researchers because they show whether an earlier measure of one variable is associated with the later measure of another variable
How do we identify high-quality science?
It is reproducible: can do the same methods from start to finish
It is replicable: can get similar results when repeating the study
It is pre-registered: hypotheses and analyses are stated upfront to avoid questionable research practices
It is open science: the materials and data are openly available for other researchers
Direct replication: reproduce the original study as closely as possible to verify that similar results are obtained
Conceptual replication: explore the same research question but using different procedures, verifying similar results
Conceptual or construct level
Replication-plus extension: reproduce the original study as closely as possible while also adding further variables or measures to test additional research questions
Meta-Analysis: As we have seen, a scientific literature, or simply literature, or field, consists of a series of related studies, conducted by various researchers, that have tested similar variables
Which of the following jobs most likely involves producer-of-research skills rather than consumer-of-research skills? (Week 1a)
University professor
As a true empiricist, one should: (Week 1a)
Base one's conclusions on direct observations
Results for question 3.
A statement, or set of statements, that describes general principles about how variables relate to one another is a(n)... (Week 1a)
Correct answer:
Theory
Results for question 4.
Why is publication an important part of the resarch process? (Week 1b)
Correct Answer:
Because scientists can verify or challenge the published work, making science self-correcting
Results for question 5.
Correct Answer:
Can 2-month-old human infants tell the difference between four objects and six objects?
Results for question 6.
6
0 / 1 point
What does it mean to say that research results are probabilistic? (Week 2a)
Correct Answer:
Research conclusions explain a certain proportion of possible cases but may not explain all
Results for question 7.
7
1 / 1 point
When is it a good idea to base conclusions on the advice of authorities? (Week 2a)
Correct answer:
When the authorities base the advice on systematic, peer-reviewed research
Results for question 8.
8
0 / 1 point
The tendency for people to notice a match between a horoscope prediction and their personal circumstances, but disregard the times when it is incorrect or too vague, is an example of... (Week 2a)
Correct Answer:
Present/present bias
uestion 9.
9
1 / 1 point
Which of the following is the most reliable source for reading the details of a psychological study? (Week 2a)
Correct answer:
Scientific journals
Results for question 10.
10
0 / 1 point
When reading an empirical journal article, which of the following represent the best two questions to ask yourself as you read? (Week 2a)
Correct Answer:
What is the argument? What is the evidence to support the argument?
How does this research relate to prior work on the topic? How can it be extended further?
Which of the following headlines reflects an association claim? (Week 2b)
Correct answer:
Workaholism is tied to psychiatric disorders
Results for question 12.
12
1 / 1 point
Which of the following headlines reflects a frequency claim? (Week 2b)
Correct answer:
Eighty percent of women feel dissatisfied with their appearance
Results for question 13.
13
0 / 1 point
Which validity would you be interrogating by asking: How well did the researchers measure sensitivity to tastes in this study? (Week 2c)
Correct Answer:
Construct validity
Results for question 14.
14
0 / 1 point
Which validity would you be interrogating by asking: How did the researchers get their sample of people for this survey? (Week 2c)
Correct Answer:
External validity
Results for question 15.
15
1 / 1 point
In a study of a new drug for asthma, a researcher finds that the group receiving the drug is doing much better than the control group, whose members are receiving a placebo. Which principle of the Belmont Report requires the researcher to also give the control group the opportunity to receive the new drug? (Week 4c)
Correct answer:
Beneficence
Results for question 16.
16
0 / 1 point
A researcher recruits participants with autism to understand the best practices to make a workplace more accommodating of neurodivergence. Which Belmont Report principle does this reflect? (Week 4c)
Correct Answer:
Justice
Results for question 17.
17
0 / 1 point
Following a study using deception, how does the researcher attempt to restore an honest relationship with the participant? (Week 4c)
Correct Answer:
By debriefing each participant in a structured conversation
Results for question 18.
18
0 / 1 point
Imagine that a researcher tests the efficacy of a new drug using funding from a pharmaceutical company that developed the drug. What is a potential ethical problem that needs to be addressed? (Week 4c)
Correct Answer:
Conflict of interest
Results for question 19.
19
1 / 1 point
A regression threat applies especially... (Week 5a)
Correct answer:
When a sample has an extremely high or extremely low pretest average
Results for question 20.
20
0 / 1 point
Which of the following internal validity threats can occur even when a control group is used? (Week 5a)
Correct Answer:
Observer bias
A study that has a large number of participants and a low amount of measurement error and situation noise is likely to have a high level of... (Week 5b)
Correct Answer:
Power and precision
Results for question 22.
22
0 / 1 point
Which of the following would NOT be helpful to increasing the power of the study to detect a significant effect? (Week 5b)
Correct Answer:
Adding manipulation checks
Results for question 23.
23
0 / 1 point
Which of the following is NOT an advantage of within-groups designs over independent-groups designs? (Week 5b)
Correct Answer:
Fewer practice/fatigue effects
Results for question 24.
24
0 / 1 point
Which of the following does NOT contribute to calculating a point estimate's margin of error? (Week 7a)
Correct Answer:
Average
Results for question 25.
25
1 / 1 point
Cohen's d tells us... (Week 7b)
Correct answer:
How strong of a difference there is between two groups
Results for question 26.
26
0 / 1 point
Which of the following 95% confidence intervals is most precise? (Week 7b)
Correct Answer:
[-8, -6]
Results for question 27.
27
0 / 1 point
A Type I error occurs when... (Week 7c)
Correct Answer:
We call something significant when it actually isn't in reality
Results for question 28.
28
0 / 1 point
When is something called "statistically significant"? (Week 7c)
Correct Answer:
When p < alpha
Results for question 29.
29
0 / 1 point
How many total design cells are there in this study? (Week 10a)
Correct Answer:
8
Results for question 30.
30
1 / 1 point
Does the plot show a two-way interaction? (Week 10a)
No, there is no interaction
Which of the following values reflect the marginal means of language group that are reported in the table? (Week 10b)
Correct Answer:
4.5 and 2.5
The design of this study is definition NOT... (Week 10b)
Correct Answer:
Fully within-subjects
Assume for the sake of the example that any difference is a significant difference. Which of the following is an accurate conclusion given the pattern in the plot? (Week 10b)
Correct Answer:
It is likely that the main effects of both independent variables are significant
Assume for the sake of the example that any difference is a significant difference. Which of the following is an accurate conclusion given the pattern in the plot? (Week 10b)
Correct Answer:
Bilinguals are faster than monolinguals overall, and both groups are slower as distraction increases
This plot shows a ____ design. (Week 10c)
Correct answer:
2 x 2 x 2
The two plots together show a... (Week 10c)
Correct answer:
Significant three-way interaction
How many F-ratios will there be for a three-way design? (Week 10c)
Correct Answer:
7
A psychologist working with four siblings who exhibit violent behavior toward each other has asked their parents to record the number of violent behaviors each child exhibits in the hour before dinner for 1 week. Each week thereafter, the psychologist has the parents introduce a positive reinforcement technique to shape the behavior of each child, one after the other. By the end of 6 weeks, there is a significant decrease in violent behaviors for each of the children. What type of design did the psychologist use in this example? (Week 11b)
Correct answer:
Multiple-baseline design
Which of these is NOT a method of addressing external validity of the conclusions of a small-N design? (Week 11b)
Correct answer:
Randomly assign participants to the treatment and control conditions
After Paris restricted car traffic and increased the number of bike lanes, the city showed a large drop in air pollution compared to nearby London that did not show any change to its air pollution. What kind of design does this example reflect? (Week 11a)
Correct Answer:
Quasi-experimental design
Which of the following distinguishes quasi-experimental variables from independent variables in a true experiment? (Week 11a)
Correct Answer:
Quasi-experimental variables cannot be randomly assigned
Which of the following four terms is not synonymous with the others? (Week 12b)
Correct answer:
Unrepresentative sample
Why do studies that use probability samples have excellent validity? (Week 12b)
Correct Answer:
All members of the population have an equal chance to be in the sample
Which of these statements is true about external validity? (Week 12c)
Correct answer:
External validity is about how the sample is obtained rather than sample size
What is the biggest problem with the following survey item: "Was your cell phone purchased within the last two years and have you downloaded the most recent updates?" (Week 13b)
It is a double-barreled question
In which of the following situations do people most accurately answer survey questions? (Week 13b)
Correct Answer:
When they are describing how they personally feel about something
When people are using an acquiescent response set, they are... (Week 13b)
Correct Answer:
Tending to agree with every item, no matter what it says
Which of the following is a means of mitigating observer bias? (Week 13c)
Making sure that the observer does not know the study hypotheses
Which of the following is a way of preventing reactivity? (Week 13c)
Correct Answer:
Using unobtrusive observations
Which of the following makes it more likely that behavioral observations will have good interrater reliability? (Week 13c)
Correct answer:
Using clear rating criteria
Longitudinal designs allow researchers to establish ____ by considering ____. (Week 15a)
Correct Answer:
Temporal precedence; cross-lag correlations
Which of the following is a benefit of meta-analysis compared to other types of research? (Week 16a)
Correct Answer:
Meta-analysis allows for calculation of effect sizes across multiple studies
Which of the following is different in a direct replication? (Week 16a)
Correct Answer:
The participants
Which of the following reflects a field setting? (Week 16b)
Correct answer:
A preschool playground with video cameras
Researchers who are in theory-testing mode prioritize ___, whereas researchers who are in generalization mode prioritize ___.
Correct Answer:
External validity; internal validity
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
With whom did Dr. LoTemplio and her team share their research? (Week 1b)
Correct answer:
With an academic audience
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
Which open science practice did Dr. LoTemplio and her team exhibit? (Week 16b)
Correct answer:
Open data
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
Which sampling method did Dr. LoTemplio and her team use to recruit participants? (Week 12c)
Correct answer:
Purposive
Snowball
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
___ is the independent variable of the study, and ___ is a dependent variable. (Week 2b)
Correct Answer:
Nature group; memory performance
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
How did Dr. LoTemplio operationalize "reduced stress"? (Week 2b)
Correct Answer:
Ratings on the Perceived Stress Scale
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
The age of the participants represents a ___ variable in this study. (Week 2b)
Control
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
What kind of measure is the memory test? (Week 3a)
Correct answer:
Observational
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
What kind of measure is the stress rating? (Week 3a)
Correct Answer:
Self-report
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
Performance on the memory test reflects a ___ scale of measurement. (Week 3a)
Correct Answer:
Ratio
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
What type of design is Dr. LoTemplio's study? (Week 4b)
Correct Answer:
Posttest-only
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
Which analysis should Dr. LoTemplio conduct for this design? (Week 10c)
One-way ANOVA
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
How many F-ratios will there be when Dr. LoTemplio conducts the analysis? (Week 10c)
Correct Answer:
1
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
Dr. LoTemplio did not consider factors like gender, race/ethnicity, years of education, personality, or socioeconomic status of her participants. Is this a problem for making a causal claim about the benefits of nature immersion? (Week 4a)
No, because these factors represent unsystematic variability in the sample
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
The fact that Dr. LoTemplio randomly assigned participants to groups allows her to make a causal claim about the benefits of nature immersion because it helps to establish... (Week 4a)
Correct answer:
Internal validity
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
Why does Dr. LoTemplio's study satisfy the causal criterion of temporal precedence? (Week 4a)
Correct Answer:
Because participants completed the measures after the manipulation
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
Imagine if participants were allowed to choose which of the three nature groups to join. Which internal validity threat would this introduce? (Week 4a)
Correct Answer:
Selection effects
Dr. LoTemplio and her team are conducting a study to test her hypothesis that spending time in nature helps to improve older adults’ memory performance and reduce their stress. The study recruited 120 adults aged 65-75 by posting flyers in local retirement communities as well as clubs and societies that have high membership of older people. The participants were randomly assigned to one of three groups: real-world nature, virtual reality nature, and a windowless indoor control condition. Each group of participants spent 10 minutes sitting in their assigned context with no instruction except to sit quietly and observe their surroundings. Then all the participants were brought back to the lab to rate their stress levels using the Perceived Stress Scale (PSS) and to take a memory test that presented 20 arbitrary word pairs (e.g., dog – table) that they were later tested on. Dr. LoTemplio uploaded all the study results to the Open Science Framework and published the findings in a peer-reviewed journal.
A potential internal validity issue for Dr. LoTemplio's study is... (Week 5a)
Correct Answer:
Demand characteristics
Which kind of plot did the researchers use? (Week 3c)
Correct answer:
Boxplot
Which group has an outlier? (Week 3c)
Correct answer:
Virtual reality nature
What do the boxes in the figure reflect? (Week 3c)
Correct answer:
Interquartile range
Does the result seem to support the hypothesis? (Week 7c)
Correct Answer:
Yes, because there is very little overlap between the real-world nature group and the other two groups
Given the pattern in the plot, what is a potential probelm that could affect the interpretation of the results? (Week 5b)
Correct answer:
Floor effect in the virtual reality and the indoor control groups could obscure a true difference between them
If Dr. LoTemplio decided to recruit 1200 participants rather than 120, this would improve the study's... (Week 2c)
Correct Answer:
Statistical validity
Dr. LoTemplio found that her measures had good internal reliability, which is ___ to ensure that they have good ___ validity. (Week 3b)
Correct answer:
Necessary but not sufficient; construct
Dr. O’Connor is conducting a study to get to the bottom of the association between social media use and loneliness. Some researchers find that increased social media use is associated with increased loneliness, whereas others find the opposite, such that increased social media use is associated with decreased loneliness. Dr. O’Connor published these two possible hypotheses and the study design following peer-review but prior to starting data collection. Dr. O’Connor recruited 1,000 participants by circulating a link to the online study on multiple social media platforms. Participants responded to a survey that included a question about loneliness (“How often do you feel lonely?” rated on a scale of 1 (never) to 6 (always)) and a question about their social media use (“How many hours do you spend on social media per day?”). The participants also reported their gender, whether they have ever been diagnosed with anxiety or depression, whether they identify as straight or part of the LGBTQ+ community, as well as separate measures of the participants’ self-esteem and the quality of their online and offline friendships.
The null hypothesis of this study is... (Week 7c)
Correct Answer:
There is no association between social media use and loneliness
Dr. O’Connor is conducting a study to get to the bottom of the association between social media use and loneliness. Some researchers find that increased social media use is associated with increased loneliness, whereas others find the opposite, such that increased social media use is associated with decreased loneliness. Dr. O’Connor published these two possible hypotheses and the study design following peer-review but prior to starting data collection. Dr. O’Connor recruited 1,000 participants by circulating a link to the online study on multiple social media platforms. Participants responded to a survey that included a question about loneliness (“How often do you feel lonely?” rated on a scale of 1 (never) to 6 (always)) and a question about their social media use (“How many hours do you spend on social media per day?”). The participants also reported their gender, whether they have ever been diagnosed with anxiety or depression, whether they identify as straight or part of the LGBTQ+ community, as well as separate measures of the participants’ self-esteem and the quality of their online and offline friendships.
Which analysis will Dr. O'Connor use to test the research question? (Week 14a)
Correct Answer:
Bivariate correlation
Dr. O’Connor is conducting a study to get to the bottom of the association between social media use and loneliness. Some researchers find that increased social media use is associated with increased loneliness, whereas others find the opposite, such that increased social media use is associated with decreased loneliness. Dr. O’Connor published these two possible hypotheses and the study design following peer-review but prior to starting data collection. Dr. O’Connor recruited 1,000 participants by circulating a link to the online study on multiple social media platforms. Participants responded to a survey that included a question about loneliness (“How often do you feel lonely?” rated on a scale of 1 (never) to 6 (always)) and a question about their social media use (“How many hours do you spend on social media per day?”). The participants also reported their gender, whether they have ever been diagnosed with anxiety or depression, whether they identify as straight or part of the LGBTQ+ community, as well as separate measures of the participants’ self-esteem and the quality of their online and offline friendships.
Which open science practice is reflected in Dr. O'Connor's study? (Week 16b)
Correct Answer:
Registered report
Dr. O’Connor is conducting a study to get to the bottom of the association between social media use and loneliness. Some researchers find that increased social media use is associated with increased loneliness, whereas others find the opposite, such that increased social media use is associated with decreased loneliness. Dr. O’Connor published these two possible hypotheses and the study design following peer-review but prior to starting data collection. Dr. O’Connor recruited 1,000 participants by circulating a link to the online study on multiple social media platforms. Participants responded to a survey that included a question about loneliness (“How often do you feel lonely?” rated on a scale of 1 (never) to 6 (always)) and a question about their social media use (“How many hours do you spend on social media per day?”). The participants also reported their gender, whether they have ever been diagnosed with anxiety or depression, whether they identify as straight or part of the LGBTQ+ community, as well as separate measures of the participants’ self-esteem and the quality of their online and offline friendships.
Which questionable research practice is least likely in Dr. O'Connor's study? (Week 16b)
Correct Answer:
HARKing
Dr. O’Connor is conducting a study to get to the bottom of the association between social media use and loneliness. Some researchers find that increased social media use is associated with increased loneliness, whereas others find the opposite, such that increased social media use is associated with decreased loneliness. Dr. O’Connor published these two possible hypotheses and the study design following peer-review but prior to starting data collection. Dr. O’Connor recruited 1,000 participants by circulating a link to the online study on multiple social media platforms. Participants responded to a survey that included a question about loneliness (“How often do you feel lonely?” rated on a scale of 1 (never) to 6 (always)) and a question about their social media use (“How many hours do you spend on social media per day?”). The participants also reported their gender, whether they have ever been diagnosed with anxiety or depression, whether they identify as straight or part of the LGBTQ+ community, as well as separate measures of the participants’ self-esteem and the quality of their online and offline friendships.
Dr. O'Connor used ___ sampling for his study. (Week 12c)
Correct Answer:
Convenience
Dr. O’Connor is conducting a study to get to the bottom of the association between social media use and loneliness. Some researchers find that increased social media use is associated with increased loneliness, whereas others find the opposite, such that increased social media use is associated with decreased loneliness. Dr. O’Connor published these two possible hypotheses and the study design following peer-review but prior to starting data collection. Dr. O’Connor recruited 1,000 participants by circulating a link to the online study on multiple social media platforms. Participants responded to a survey that included a question about loneliness (“How often do you feel lonely?” rated on a scale of 1 (never) to 6 (always)) and a question about their social media use (“How many hours do you spend on social media per day?”). The participants also reported their gender, whether they have ever been diagnosed with anxiety or depression, whether they identify as straight or part of the LGBTQ+ community, as well as separate measures of the participants’ self-esteem and the quality of their online and offline friendships.
What type of item is the survey question about loneliness? (Week 13b)
Correct Answer:
Semantic differential
Dr. O’Connor is conducting a study to get to the bottom of the association between social media use and loneliness. Some researchers find that increased social media use is associated with increased loneliness, whereas others find the opposite, such that increased social media use is associated with decreased loneliness. Dr. O’Connor published these two possible hypotheses and the study design following peer-review but prior to starting data collection. Dr. O’Connor recruited 1,000 participants by circulating a link to the online study on multiple social media platforms. Participants responded to a survey that included a question about loneliness (“How often do you feel lonely?” rated on a scale of 1 (never) to 6 (always)) and a question about their social media use (“How many hours do you spend on social media per day?”). The participants also reported their gender, whether they have ever been diagnosed with anxiety or depression, whether they identify as straight or part of the LGBTQ+ community, as well as separate measures of the participants’ self-esteem and the quality of their online and offline friendships.
The scale of measurement for social media use is... (Week 3a)
Correct Answer:
Ratio
Dr. O’Connor is conducting a study to get to the bottom of the association between social media use and loneliness. Some researchers find that increased social media use is associated with increased loneliness, whereas others find the opposite, such that increased social media use is associated with decreased loneliness. Dr. O’Connor published these two possible hypotheses and the study design following peer-review but prior to starting data collection. Dr. O’Connor recruited 1,000 participants by circulating a link to the online study on multiple social media platforms. Participants responded to a survey that included a question about loneliness (“How often do you feel lonely?” rated on a scale of 1 (never) to 6 (always)) and a question about their social media use (“How many hours do you spend on social media per day?”). The participants also reported their gender, whether they have ever been diagnosed with anxiety or depression, whether they identify as straight or part of the LGBTQ+ community, as well as separate measures of the participants’ self-esteem and the quality of their online and offline friendships.
Which problem does the lack of a "middle" option of the loneliness survey question help to address? (Week 13b)
Correct Answer:
Fence-sitting
Dr. O’Connor is conducting a study to get to the bottom of the association between social media use and loneliness. Some researchers find that increased social media use is associated with increased loneliness, whereas others find the opposite, such that increased social media use is associated with decreased loneliness. Dr. O’Connor published these two possible hypotheses and the study design following peer-review but prior to starting data collection. Dr. O’Connor recruited 1,000 participants by circulating a link to the online study on multiple social media platforms. Participants responded to a survey that included a question about loneliness (“How often do you feel lonely?” rated on a scale of 1 (never) to 6 (always)) and a question about their social media use (“How many hours do you spend on social media per day?”). The participants also reported their gender, whether they have ever been diagnosed with anxiety or depression, whether they identify as straight or part of the LGBTQ+ community, as well as separate measures of the participants’ self-esteem and the quality of their online and offline friendships.
When Dr. O'Connor followed up with the participants and asked them again how many hours per day they use social media, they reported similar results to the first survey. This shows good ___ reliability. (Week 3b)
Correct answer:
Test-retest
Dr. O’Connor is conducting a study to get to the bottom of the association between social media use and loneliness. Some researchers find that increased social media use is associated with increased loneliness, whereas others find the opposite, such that increased social media use is associated with decreased loneliness. Dr. O’Connor published these two possible hypotheses and the study design following peer-review but prior to starting data collection. Dr. O’Connor recruited 1,000 participants by circulating a link to the online study on multiple social media platforms. Participants responded to a survey that included a question about loneliness (“How often do you feel lonely?” rated on a scale of 1 (never) to 6 (always)) and a question about their social media use (“How many hours do you spend on social media per day?”). The participants also reported their gender, whether they have ever been diagnosed with anxiety or depression, whether they identify as straight or part of the LGBTQ+ community, as well as separate measures of the participants’ self-esteem and the quality of their online and offline friendships.
Dr. O'Connor was able to access the number of visits to social media websites for each participant. This is useful to ensure good construct validity of the social media use measure, particularly when it coems to... (Week 3b)
Correct Answer:
Convergent validity
Social media use is the ___ variable and rated loneliness is the ___ variable. (Week 14c)
Correct Answer:
Predictor; criterion
The association appears to be... (Week 14b)
Correct Answer:
Curvilinear
What is the "plain English" / "grandma" method to describe the results in the plot? (Week 14a)
Correct Answer:
Using social media either a little or a lot links to lower loneliness
The lonliness measure was transformed to a z-score. This means that 0 is the mean and a person with a score of -2 is 2 ___ below it. (Week 3c)
Correct answer:
Standard deviations
Which criteria does Dr. O'Connor's study meet to make a causal claim? (Week 2c)
Covariance
Dr. O'Connor wanted to ensure that the pattern of results remained the same even when considering participants' gender and mental health. To accomplish this, he would conducat an additional ___ analysis includign these ___ variables. (Week 14c)
Correct Answer:
Multiple regression; control
Predictor | Beta [95% CI] | Significance |
Social media use (linear effect) | 2.03 [1.94, 2.11] | * |
Social media use (quadratic effect) | -0.20 [-0.21, -0.19] | * |
Gender | -0.02 [-0.14, 0.11] | ns |
Mental health diagnosis | 0.03 [-0.09, 0.15] | ns |
Note. CI = confidence interval. | ||
The results of this analysis are shown in the table above. Which variable has the strongest beta? (Week 14c)
Correct Answer:
Social media use
What can we conclude from this additional analysis that includes other variables of gender and health? (Week 14c)
Correct Answer:
The effect of social media use is significant even after accounting for gender and mental health
Dr. O'Connor conducts a further analysis considering the LGBTQ+ variable, showing that the social media-loneliness association was not significant for straight participatns but significantly negative for participants belonging to the LGBTQ+ community. This means that LGBTQ+ is a... (Week 14b)
Correct Answer:
Moderating variable
Dr. O'Connor conducts one more analysis to understand whether self-esteem and quality of online and offline friendships may contribute to the social media-loneliness association. In this case, self-esteem and friendship quality are... (Week 14c)
Correct Answer:
Mediating variables
Dr. O'Connor plans a follow-up longitudinal study investigating whether social media use at earlier time points correlates with later rated loneliness. Inspecting the ___ will allow him to avoid ___ in such a study. (Week 15a)
Cross-lag correlations; directionality problems