Research methods and data analysis I (psy:2811) - Exam 1 (lecture slides & book materials) questions and answers 2026-2027(A+ GRADED)

lecture 1.2 - Psychological science is a way of thinking

lecture 1.2

Which of the following does not involve the "producer" role of research?

A data scientist at a non-profit who analyzes internet web search patterns to produce insights about preferences for candidates running for office

A wellness/lifestyle coach who reads scientific articles and summarizes their main findings for his/her class and on her blog

A research assistant who administers questionnaires to children for a study about learning to read

A faculty member who develops new methods for imaging the human brain

Three sources of information

Authority, Rationalism, & Empiricism

Authority

Someone in a position of authority tells you something is true, thus it must be true

Rationalism

deriving knowledge from logical reasoning; if the premise and statements are true and the logic is sound, then the conclusion is correct

Empiricism

Deriving knowledge from observation and experimentation

Authority and rationalism play secondary roles

theory-data cycle

Theory

statement or set of statements that describes general principles about how variables relate to one another

Theories can't be proven, but data can suggest whether theories should be changed/replaced

A prediction, or hypothesis

is a way of stating the specific outcome that the researcher expects to observe if the theory is accurate.

Data are

a set of observations representing the values of some variable, collected from one ore more research studies

Data are the backbone of a theory for why things happen

Which of the following is true of the relationship between hypotheses and theories?

Hypotheses are steps taken to determine if the theory is accurate

Theories are used to determine if hypotheses are accurate

Multiple theories are needed to test if a hypothesis is accurate

Hypotheses and theories are synonymous terms

Hypotheses are steps taken to determine if the theory is accurate

Explain how theories and data interact to form empirical science

The theory-data cycle

Explain characteristics of a good theory

Supported by data, Falsifiable, Parsimony

Parsimony

When two theories both explain data equally well, the simpler theory is preferred

Psychologists are empirical scientists; therefore, _______ .

A. They use logic to prove that their theories are right

B. Data is used to prove a theory is right

C. Data is used to test whether a theory makes the correct predictions

D. Intuition can be used to derive knowledge for how the mind works

Lecture 1.3

Sources of information: the biases we try to avoid by using this scientific approach

Lecture 1.3

Sources of information: Experience

Basing decisions on past experiences as a sole source of "knowing"

Small set of possibilities with weak or no comparison groups

confound

is when more than one thing changes at a time that may have caused an outcome. Because they happened together and either one alone or both could have caused the outcome...you cannot know the cause!

Sources of information: Intuition

a sense of knowing without direct evidence or experience, such that the information feels like it is known instinctively

Availability heuristic

Mistaking the ease and frequency of recall with the probability or likelihood of the phenomenon.

Confirmation bias

Tendency to look only at information that agrees with what we already believe.

Bias blind spot

the tendency to think that other people are more susceptible to attributional biases in their thinking than we are

Self-serving

Self-interested

Halo effect

Etc...

The local news ran a story about a baby who suffered a deadly spider attack while playing outside. The neighborhood becomes quiet as families keep their kids inside so their they won't be bitten by deadly spiders.

A) Confounds

B) The good story

C) Availability heuristic

D) Confirmation bias

Barry is studying the effects of a major natural disaster on people living nearby. He finds that many of the victims are depressed and show stress-related symptoms. Why can't Barry conclude that the natural disaster caused the depression and stress-related symptoms?

A) No comparison group

B) Depression and stress are confounds

C) His explanation is story with no data

D) All of the above

A) No comparison group

Vanessa claims that she sleeps better when she falls asleep to music. She has a comparison condition, because she has noticed that she does not listen to music every night, only when she remembers to plug in her iPod. She typically remembers to plug in her iPod on nights when she is able to finish studying earlier. What problem do you see in Vanessa's reasoning about sleeping better to music?

A) Vanessa's belief that she sleeps better with music is not falsifiable.

B) Vanessa may be sleeping better because she is less distracted by studying/going to bed sooner.

C) Vanessa is biased because she sleeps in the same bed every night

D) There is no problem with her reasoning

This i believe...based on experience

no comparison group

has confounds

(^ all bad)

This i believe...based on intuition

Good story

Availability

Present/present bias

Confirmation bias

Bias blind spot

(^ all bad)

This i believe...based on authority

- Could be the authority's personal experience

- Could be the authority's intuition

(^ all bad)

- Could be based on authorities research (good)

This i believe...based on research

scientific sources (good)

- journal articles (empirical articles, review articles)

- Chapters in edited books

- Full length books

Other sources (typically for popular audience) (bad)

- Trade books

- Magazines and newspaper articles

- wikis

1.4 Variables

1.4

Variables

document the values for the data we collect

is a characteristic or property that can take at least 2 values

Variables can be either measured or manipulated

Types of variables: nominal variable

A nominal variable classifies values in discrete categories

Data categories are mutually exclusive (each observation has only 1 category)

Data categories have no logical order (only qualitative differences)

examples:

eye color - blue (=1), brown (=2), green (=3)

has visited France - yes, no

Averages are meaningless for nominal variables

Types of variables: Ordinal

Ordinal variables are a rank order

The levels of ranking are discrete, but have a meaningful order

- meaningful values but unequal intervals between units

Example:

Finish order in race

Types of variables: Interval variables

are numeric scales with equal intervals and no true zero

- equal intervals between units but no meaningful zero

Examples:

temperature

Types of variables: ratio variable

has a numeric scale with equal intervals between steps and true zero value

-equal intervals and a meaningful zero

examples:

height

weight

A researcher is interested in whether children that sleep less exhibit more behavioral problems in school. They survey parents about their children's sleep habits, and survey teachers about the behavioral conduct of the children. They find that children who sleep less have greater behavioral problems.

What kind of variable is the sleep?

Measured

Manipulated

Constant

A researcher is interested in whether children that sleep less exhibit more behavioral problems in school. They survey parents about their children's sleep habits, and survey teachers about the behavioral conduct of those children. They find that children who sleep less have greater behavioral problems.

The sleep survey asks parents to indicate number of hours the child sleeps on a typical school night. What measurement scale is this?

Nominal

Ordinal

Interval

Ratio

A researcher is interested in whether children that sleep less exhibit more behavioral problems in school. They survey parents about their children's sleep habits, and survey teachers about the behavioral conduct of those children. They find that children who sleep less have greater behavioral problems.

What kind of variable is the behavioral problems?

Measured

Manipulated

Constant

A researcher is interested in whether children that sleep less exhibit more behavioral problems in school. They survey parents about their children's sleep habits, and survey teachers about the behavioral conduct of those children. They find that children who sleep less have greater behavioral problems.

The behavioral problems survey asks teachers to pick whether the child is disruptive in class "never", "sometimes", or "always." What measurement scale is this?

Nominal

Ordinal

Interval

Ratio

A researcher is interested in whether children that sleep less exhibit more behavioral problems in school. They survey parents about their children's sleep habits, and survey teachers about the behavioral conduct of those children. They find that children who sleep less have greater behavioral problems.

As a potential confound, the researcher decides to measure medications and classifies them as "depressants", "stimulants", "hallucinogen", or "other." What measurement scale is this?

Nominal

Ordinal

Interval

Ratio

Understand what a variable is and why you should care about their qualities

Distinguish between different scales of measurement for variables

Distinguish between measured and manipulated variables

Has visited France: yes, no

Nominal

Has visited France: never, sometimes, often

Ordinal

Has visited France: enter number of visits___

Ratio

Lecture 1.5

Claims in psychological science

claim

A claim is an argument someone is trying to make

operational definition

An operational definition defines how a concept is explicitly measured

There are multiple operational definitions that are possible for any one conceptual definition

True

False

Which of the following is the worst operational definition of self-control?

A) The choice in response to: "Do you choose a candy bar worth 50 cents now or one worth $2 next week?"

B) The number of examples a participant provides when asked if they have good self-control

C) The number of times a participant reports that giving into temptation interferes with their life

D) The number of marshmallows someone eats in a day

Three types of claims:

Frequency, Association, Casual

Frequency claims

Frequency claims are about a single variable

No intent to say what causes that level of variable; merely a statement of how often something occurs

Association claims

Association claims assert that two variables are related to each other

Association claims cannot assert very strongly why the relationship exists or which variable comes first

The two variables may have a third variable in common (Ice cream sales - accidental drownings, temperature)

Causal claims

Causal claims argue that two variables are related, and that one variable causes another variable

Manipulating a variable enables us to make causal claims

manipulated independent variable(s)

measured dependent variable(s)

independent variables

In an experiment independent variables are manipulated

dependent variables

In an experiment dependent variables are measured

Pre-school children with more self-control are more likely to have higher grades, score higher on the SAT a decade later, and are more able to cope with social and personal problems.

Type of claim:

(A) Frequency (B) Association (C) Causal

"Music improves social communication in autistic children"

(A) Frequency (B) Association (C) Causal

"children who start school a year yearly are more likely to be diagnosed with ADHD"

(A) Frequency (B) Association (C) Causal

Lasting impact of concussions on young adults

(A) Frequency (B) Association (C) Causal

Learning disabilities affect up to 10% of children

(A) Frequency (B) Association (C) Causal

Digital media use linked to behavioral problems in kids

(A) Frequency (B) Association (C) Causal

Lecture 1.6 The four validates

1.6

validity

The validity of a claim refers to how accurate it is given the data at hand

Every study balances four main types of validity:

Construct validity

External validity

Internal validity

Statistical validity

Construct validity

refers to how well the variables in a study measure what they intended to measure

External validity

refers to the degree to which the results of a study (and its conclusions) generalize to a larger population or to other situations

Internal validity

refers to the extent to which the effect arises because of the experimental treatment, and not some alternative variable

Internal vs. External validity tug-of-war

Many attempts to improve internal validity hurt external validity (and vice versa)

example:

Controlling participant variables by making them constant

=

High internal validity (few possible confounds)

Low external validity (very restricted population)

Statistical validity

Statistical validity refers to the degree to which the statistical results support the claim

Types of statistical errors

Type I errors, Type II errors

Type I errors

Say an effect exists when it does not

False positive

Worst kind of error

Type II errors

Say no effect exists when one is there

False negative, or "miss"

Not as damaging (conservative approach)

Which validity is appropriate to interrogate with rigor for every study?

External validity

Internal validity

Construct validity

Only statistical validity

A researcher is conducting a study on how the wording of questions affects people's responses. In her study, she only includes native speakers of English, and excludes people under 40 to ensure hearing is adequate. These decisions about her sample hurt the study's:

External validity

Internal validity

Construct validity

Statistical validity

When examining the statistical validity of a frequency claim, one should look for the:

Strength of the association

How representative the sample was

Margin of error of the estimate

How well the research measured the concept they hoped to measure

The probability that the researcher missed a true relationship

When examining the construct validity of an association claim, one should look for:

A) How well the measure of each variable measures the concepts they're intended to measure

B) Whether the association is positive

C) Whether the sample was randomly chosen from the population

D) How much the measure of one variable is tied to the measure of the other variable in the association

E) Whether there is a third variable that could explain the association

1.7 lecture

A researcher is interested in whether children that sleep less exhibit more behavioral problems in school. They survey parents about their children's sleep habits, and survey teachers about the behavioral conduct of those children. They find that children who sleep less have greater behavioral problems.

Which variable would we want to manipulate to turn this association claim into a causal claim?

Sleep

Behavioral problems

Instead of a survey about sleep quality, a researcher decides to manipulate sleep by having children sleep in the lab for one week. This drastically improves the ability to control variables affecting sleep like ambient noise and light, and to measure sleep disruption directly. She then finds out that only 5% of the children's families are willing to participate. The change in study design reflects a decrease of _______ validity, and an increase in _______ validity.

External; Statistical

External; Internal

Internal; External

Statistical; Internal

After a study reported that taking notes by hand improved memory retention for video materials in a laboratory experiment, a research decided to see whether the effect would generalize to psychology course lectures. In this study he surveyed students in introductory psychology classes about how they take notes, and examined the relationship between their note-taking method and final exam score. The change in study design reflects a decrease of _______ validity, and an increase in _______ validity.

External; Statistical

External; Internal

Internal; External

Statistical; Internal

A measure must be reliable to be valid, But

A measure must be reliable to be valid, But a reliable measure is not necessarily valid!

test-retest reliability

a method for determining the reliability of a test by comparing a test taker's scores on the same test taken on separate occasions

Internal reliability

in a measure that contains several items, the consistency in a pattern of answers, no matter how a question is phrased

measured with Cronbach's alpha

Survey example:

different questions that measure the same thing but with different phrasings should get similar responses

Interrater reliability

the degree to which different observers agree on their observations

important when turning observations of behavior into a variable

Inter-rater = similarity between raters

You develop a new 10-question survey of the trait of narcissism that can be used online. You first test this on your friends by sending them the survey three times in a month to test the consistency of their summary test score. Your are testing the ________ reliability of your measure.

(A) Test-retest

(B) Interrater

(C) Internal

Which other type of reliability would be important to assess for your 10-question survey?

(A) Test-retest

(B) Interrater

(C) Internal

1.8 lecture

Face validity

extent to which respondents can tell what the items are measuring

the extent to which a measure seems plausible at face value

Content validity

the extent to which a test samples the behavior that is of interest

the extent to which a measure captures all parts of a defined construct, as assessed subjectively by an expert in the field

Criterion validity

the extent to which a measure is related to an outcome

asks whether the measure correlates with a real world outcome of the concept

Convergent validity

the measure should correlate more strongly with other measures of the same constructs

how strongly scores on your measure are related to other measures of the same concept

Similar to criterion validity, but other measure does not need to be a real-world or concrete behavioral outcome

Good: correlation of r > .70

Discriminant validity

how dissimilar the scores are between your measure and a measure of a different concept

Discriminant validity reflects whether the measure is only measuring what it intends to measure

Relative comparison to tests of convergent validity

You are developing a test of driving intelligence that could be delivered on the internet to screen for drivers who have a high risk of accidents. An example of criterion validity for your test of driving intelligence would be:

A) Endorsement of your test by an expert in the field

B) Strong correlation with number of accidents in the two years after taking the questionnaire

C) Strong relationship with a questionnaire commonly used to measure driving aptitude

D) Weak relationship with technical knowledge of how automobiles work

Why is it critical that a measure has good discriminant validity?

- We want to make sure we are measuring the entirety of the construct

- We want to make sure we are only measuring one construct

- Because constructs aren't real if they're hypothetical

"In a 2015 report, 92% of over 1,000 teens surveyed said they went online daily, according to Lenhart's previous research at the Pew Research Center."

- Frequency

- Association

Causal

Frequency

"In a 2015 report, 92% of over 1,000 teens surveyed said they went online daily, according to Lenhart's previous research at the Pew Research Center."

How would we prioritize evaluating different validities?

A) Construct, Statistical, External

B) External, Statistical, Construct

C) Construct, Internal, External

D) Internal, Construct, Statistical

A) Construct, Statistical, External

"The study includes data on more than 40,000 children aged two to 17 in the US, provided by parents for a nationwide health survey in 2016."

Which validities is this most relevant for?

A) Construct and Internal

B) External and Statistical

C) Internal and Statistical

D) Construct and External

B) External and Statistical

"Associations between screen time and lower psychological well-being among children and adolescents: Evidence from a population-based study."

How would we prioritize evaluating different validities?

A) Statistical, Construct, External, Internal

B) Construct, Statistical, External, Internal

C) Construct, Internal, External, Statistical

D) External, Construct, Statistical, Internal

B) Construct, Statistical, External, Internal

You design a study to manipulate screen time in the lab. You recruit families with 2nd and 3rd graders. You will have half of the children watch 2 hours of youtube videos, and half will read books for 2 hours. Your outcome is distractibility on a subsequent math test.

What principle will be very important in choosing which groups the kids are assigned to?

A) Voluntary assignment

B) Random recruitment

C) Random assignment

D) Operational definition

C) Random assignment

After submitting your paper for publication, a peer-reviewer asks you for evidence of construct validity of your distractibility measure. Which of the following would be a measure of its criterion validity?

A) Strong positive relationship with a self-report questionnaire measuring distractibility

B) Lack of relation to self-report measure of curiosity

C) Strong negative relationship with their teacher's assessment of their ability to "stay on task"

D) Strong endorsement from another expert in the field about the quality of the measure

C) Strong negative relationship with their teacher's assessment of their ability to "stay on task"