Research Methods exam 1

Research Producers

Informs the public

Help others make informed choices by generating information

Informs policy therapy and education and business decisions

Research consumers

Good consumers are able to differentiate good science from weaker science

Make more informed decisions

For your future career

Empiricism

basing one's conclusions on systematic observations.

Harlow

Contact-comfort theory

One mother wire with food

One mother terry cloth and warmth but no food

Data showed that the monkey stayed with the warm one until hungry then after eating immediately went back

True or False: We never prove or disprove a theory rather support or refute a theory

True

Theory

set of statements as simple as possible that describes general principles about how variables relate to one another

Hypothesis

prediction, is stated in terms of the study design. It's the specific outcome the researcher will observe in a study if the theory is accurate.

Theory data cycle

1. theory

2. research questions

3. create research design to test hypothesis

4. pre-registar hypothesis

5. collect and analyze data

Supporting data strengthens the theory

non-supporting data leads to revised theories or improved research design

Falsifiability

possible to collect data that will indicate the theory is wrong

Universalism

science is evaluated based on merit not the researchers credentials. All scientists adhere to the same criteria

Communality

knowledge is created in the community and shared in the community

disinterestedness

scientist should be unbiased and uninvested in their finding

organized skepticism

scientist question everything

Basic research

goal is to enhance the general body of knowledge about a particular topic

Applied research

confide to solve practical problems the findings will be directly applied to finding a solution for a real world problem

Sources of Information

experience, intuition, authority, and empirical research

Experience

no comparison group

confounded

research is better than experience

Probabilistic

Research conclusions are meant to explain a certain proportion of possible cases but not all

Intuition

can be bias

• swayed by a good story

• swayed by what easily comes to mind (availability heuristic)

• focusing on the evidence we expect (confirmation bias)

availability heuristic

what easily comes to mind

confirmation bias

The tendency to focus on information that supports what you already believe.

Authority

people often accept claims baed on ones background credentials and power

empirical research

consulting scientific sources

ex. journal articles, chapters in edited books, full length books

Variables

something that changes or varies

needs to have at least 2 levels

Measured variable

variable that is observed and recorded

ex. height and depression

manipulated variable

variable the researcher changes or controls in a study

ex. taking a test alone or in a room with others

Construct

the name of the concept being studied

conceptual definition

a careful theoretical definition of the construct

ex. the state of being happy or comfortable

Operational definition

how the construct is measured or manipulated in the study

ex. diner life satisfaction scale (answer to 5 Qs about how satisfied you are with life)

Conceptual variable

The general idea or concept being studied (what?)

ex. school achievement

operational variables

How the concept is measured or tested in a study (how?)

ex. self reported questionnaire, checking records, teachers observation

Frequency Claim

Describes a particular level or degree of a single variable

Involve only one measurement

E.g. 1 in 4 teens are online almost constantly

Association claims

Argues that one level of a variable is likely to be associated with a particular level of another variable

Supported by studies that have at least 2 measured variables

Variables are are associated are said to be correlated

Causal Claims

Argues that one variable causes change in the level of another variable

Supported by experiments

Experiments have one manipulated and one measured variable

Must satisfy 3 criteria:

• Covariance

• Temporal precedence

• Internal validity

Validity

Indication of whether the operationalization is measuring what its supposed to measure

Construct Validity

How well the variables in a study are measured or manipulated

Statistical Validity

How well the numbers support the claim; how strong the effect is, how precise the estimate is, whether it has been replicated

Internal Validity

Applied when deciding whether a study supports a causal claim; when variable A is related to variable B the extent to which A rather than some other variable C is responsible for changes in B

External Validity

The extent to which the results of a study generalized to some larger population other times or situations

interrogating frequency claims

Construct Validity

• E.g. 39% of teens text while driving

• How well was texting defined? How did they measure texting while driving?

Statistical Validity

• E.g. Is there a confidence interval for the estimate? Margin of error?

External validity

• E.g. 73% of the world laughed today

• What people did the survey? How did they recruit participants? Urban? College students?

interrogating association claims

Construct Validity

• 2 conceptual variables need to be defined appropriately

Statistical Validity

• Consider strength and statistical significance; can also look at confidence interval

External Validity

• Generalize to other populations, contexts, times or places?

interrogating causal claims

Covariance: there is a significant association between the variables

Temporal precedence: study method ensures that variable A comes first in time before variable B

Internal Validity: study's method ensures that there was no possible alternative explanations for the change in B; A is the only thing that could have caused the change

Construct Validity 

• How well have they measured or manipulated the variables

Statistical Validity

• What's the effect size? How large is the difference?

External Validity

• To what populations seething and times can we generalize 

Tuskegee syphilis study

40 year study starting in 1932

Followed 600 black men to examine “effects of untreated syphilis over the long term”

Three majors ethical violations

• Respect for persons (participants not treated respectfully)

• Beneficence (participants were harmed)

• Justice (were a targeted and disadvantaged social group)

Belmont principle core

• Respect for persons

• Beneficence 

• Justice

Respect for persons

Informed consent

• Information about the risks and benefits of research in order to decide if they want to participate

Special protections for people with less autonomy

• Children

• People with intellectual disabilities

• Incarcerated individuals

Beneficence

Treatments must be offered if they are known to be helpful

• Even if treatment is under investigation

Anonymity:no one (including the researcher) knows a participant's identity

Confidentiality:the researcher knows who the participant is but keeps that information private.

Justice

Looks at the balance between those who participate in the research and those who benefit from it

Sample research participants form the same population that will benefit from the research

Are study participants carrying an undue burden of risk?

APA Ethical Principles (Belmont +2)

Fidelity and responsibility

• Establish relationships of trust accept responsibility for professional behavior

Integrity

• Strive to be accurate, truthful, and honest in one's role as researcher, teacher or practitioner

deception through omission

withholding of some details of the study from participants

deception through commission

act of actively lying to participants

Data Fabrication: Diederik stapel

Claim: chaotic environments promote discrimination

Problem: he didn’t conduct the research; he simply made up the results

Outcome:

• Loss of his job

• Retraction of his research

• Potential criminal proceedings

• Other researchers wasted time in related research

Data Falsification: Wansinka Saga

Misreported research data

Problematic statistical techniques

Failure to properly document and preserve research results

Animal care guidelines and the three Rs

Replacement: use alternatives when feasible

Refinement: modify procedures to minimize or eliminate distress to animals

Reduction: use the smallest number of animals necessary to achieve valid results

Animal Moral Frameworks

Social Benefit – researchers must provide significant and likely benefits (i.e., advancing knowledge or improving human and animal health)

Animal welfare – animals must not experience unnecessary harm

self report

Measuring a variable by asking people questions about themselves.

observational measure

Measuring a variable by watching and recording people’s behavior.

Physiological measure

Measuring a variable using biological data from the body.

Categorical variable

levels are qualitatively distinct categories

nominal scale

Nominal scale (catigorical)

classifies data into distinct categories based on name

Quantitative Variable

variables coded with meaningful numbers

ordinal scale, interval scale, ratio scale

ordinal scale (quantitative)

• ranked order

• distance b/w numerals not necessarily the same

• ex. 1st,2nd.3rd gold,silver,bronze

Interval scale (quantitative)

• numerals represent equal distances b/w levels

• no true zero

• ex. scale of 1-5 1=strongly disagree 5=strongly agree

Ratio Scale ( quantitative)

• numerals represent equal intervals

• true zero

• ex. height, # of correct answers

Reliability

how consistent the result of a measure is

test-retest

interrater

internal

Test-retest reliability

consistent score every time the measure is used

r= 0.5 or above

should see a strong correlation

Interrater reliability

consistent scores no matter who is measuring

Compare 2 ppl- consistency with multiple people

r= 0.7 or above

Kappa for categorical measures

closer ratings to the line of best fit= high

ratings farther from the line of best fit=low

Internal reliability

a participant provides a consistent pattern of responses across items regardless of how the researcher has phrased the question

Average inter-item correlation (AIC): average of all correlations between the different items

• Values between .15 and .50 are reasonable

Cronbach’s alpha: combines AIC with number of items in scale

• .80 or above (.7 = acceptable)

Measures within the context only within their answers

Face validity (subjective)

the measure looks like it measures what it is supposed to measure

Content validity (subjective)

the measure includes all important parts of the concept being measured

Criterion validity (objective)

Empirical assessment of validity

Whether the measure is related to a concrete outcome that it should be related to

can use known-group paradigm

known-group paradigm

Examine whether scores on them measure can distinguish among a set of groups whose behavior is already well understood

Convergent Validity (objective)

Measure should correlate strongly with a measure of the same construct

r should be strong

Discriminant Validity (objective)

Measure should correlate less strongly with measures of different constructs

r should be weak

Reliability vs validity

Validity ≠ Reliability

NOT RELIABLE NOT VALID

A measure can be less valid than it is reliable, but it cannot be more valid than it is reliable

parts of an empirical research article

Abstract

• Brief summary of the study.

Introduction

• Why the topic matters

• What we already know

• What we don't know

• Research question/hypothesis

Method

• What researchers did

• Participants

• Stimuli/materials

• Procedure

• Measurements

Results

• What researchers found

Discussion

• What the findings mean

• Whether they support the hypothesis

• How they relate to previous research

• Implications

• Limitations

References

• Sources cited in the paper

publication process

Manuscript → journal editor → peer review → rejection, revision, or acceptance → journal article

Debriefing

Explaining the deception and purpose of the research to participants after the study. Main goal: Participants should feel good about their research experience.

open science movement

Idea that researchers should make their research more open and transparent so that other scientists can check, reproduce, and build on their work.

IACUC

Institutional Animal Care and Use Committee

• Protects the animal welfare.

• Reviews animal research.

• Evaluates animal care and housing.

Data fabrication vs Data falsification

fabrication- Making up research data/results that never actually existed

falsification- Manipulating research

IRB

Institutional Review board

• Reviews research involving human participants.

• Protects the rights and welfare of participants.

• Evaluates risks and benefits.

• Makes sure ethical requirements are followed.