Experimental Design Ch. 1,2,4

Rule-of-thumb method

method typically used to generate personality descriptions from tests

test dimensions, dynamics, patient history, research, all combine to make conclusion

(pre- cookbook method)

Cookbook method

automatic clerical task like rules in cookbook

3 Goals of Science

describe → observational research

predict → correlational research

explain → experimental research

5 methods of knowing

1. intuition

2. authority

3. rationalism (logic and reasoning)

4. empiricism (observation and experience)

5. The Scientific Method

Scientific Method

only addresses empirical questions

Science

general approach to understanding the natural world

Systematic empiricism

learning based on observation though careful planning, making, recording, and analyzing

Empirical questions

questions about the way the world actually is and can be answered through systemic observation

Pseudoscience

• activities and beliefs that claim to be science but aren’t

• Implies its scientific but lacks one or more of the three features of science 

• Does not address empirical questions

Basic research

achieving more detailed and accurate understanding of human behavior without trying to address practical problems

Applied research

conducted to address practical problem

Folk psychology

intuitive beliefs about people’s behavior, thoughts, and feelings

Heuristics

mental shortcuts in forming and maintaining beliefs 

tend to assume something is true if endorsed by experts

Empirical research reports

describe one or more empirical studies

Review articles

summarize previously published research

Theoretical article

presents new theory 

Meta-analysis

 review article provides statistical summary of all previous results 

Double blind peer review

• reviewers of research article do not know identity of researcher and vice versa 

Monograph

written by single or small group of authors, gives coherent presentation of topic like extended review article

Edited volumes

books that are written by collection of authors

Theory

coherent explanation or interpretation of one or more phenomena, includes variables, structures, functions

Hypothesis

specific prediction about a new phenomenon that should be observed if theory is accurate

Hypothetico-deductive method

• observe phenomenon

• develop or use theory to make specific prediction of what happens if theory is correct

• test prediction

• refine theory in light of findings use refined theory to develop new hypotheses

• and so on

• Hypothesis derivation - hypothesis testing - theory evaluation - theory construction/ revision

Characteristics of a Good Hypothesis

Testable and falsifiable 

Logical

Positive hypothesis- existence of relationship between variables

Variable

quantity or quality that varies across people or situations

Quantitative variable

quantity such as height, measured by assigning number

Categorical variable

chosen major, nationality, occupation

Operational definition

definition of variable in terms of precisely how it is to be measured 

independent variable

variable experimenter manipulates (presumed cause)

Dependent variable

variable experimenter measures (presumed effect)

extraneous variables

any variable other than dependent variable

Confounds

varies along with variables and provides alternative explanation for results

external validity

degree to which we can generalize the findings to other circumstances or settings, like the real-world environment

Descriptive Statistics

used to organize or summarize data sets

Standard deviation

average distance between the scores and mean in a distribution 

Variance

sd squared

correlation coefficient

strength and direction of relationship between two variables (range from -1 to +1)

inferential statistics

allow researchers to draw conclusions about population based on data from sample

Statistically significant

unlikely due to random chance (less than 5%)

Type 1 error

false positive

Type 2 error

missed opportunity

false negative

Measurement

assignment of scores to individuals

Psychometrics

psychological measurement

Constructs

personality traits, emotional states, attitudes, abilities

Big Five

openness to experience

conscientiousness

extroversion

agreeableness

neuroticism 

Operational definitions

fall into one of three categories:

self-report measures

behavioral measures

physiological measures

Nominal level of measurement

categorical variables (lowest level of measurement)

number labels, phone numbers, credit card number, count, calculate proportion

Ordinal level of measurement

assigning scores that represent rank order of individuals 

Very dissatisfied to very satisfied

count and ordinal, 1st, 2nd, 3rd, surveys, questionnaires, self-report, always, sometimes, never

Don't know the difference between numbers, space between 1st and 3rd

assigning scores using numerical scales in which intervals have same interpretation throughout

Ratio level of measurement

measurement that involves assigning scores in such a way that there is a true zero point that represents the complete absence of the quantity

nominal, ordinal, and interval rolled into one

Test-retest reliability (overtime)

consistent across time 

Same group 

Internal consistency (across items)

consistency of responses across items on a multiple item scale

Inter-rater reliability (across different researchers)

• extent to which different observers are consistent in their judgements 

• Assessed using Cronbach’s a when judgements are quantitative or Cohen's k when categorical

Split-half correlation

derived by splitting the items into two sets and examining the relationship between the two sets of scores in order to assess the internal consistency of a measure

Score of +.80 or greater is considered good internal consistency

Cronbach’s a

• mean of all possible split-half correlations for set of items

• Measures internal consistency among items in measure

• Value of +.80 or higher is good

Validity

• extent to which scores from a measure represent the variable they are intended to

Face validity

extent to which measurement method superficially appears to measure construct of interest

Content validity

extent to which a measure reflects all aspects of construct of interest 

Criterion validity

extent to which people’s scores on a measure are correlated with other variables (criteria) that one would expect them to be correlated with

Criterion

variable that theoretically should be correlated with construct being measured

concurrent validity

form of criterion validity, where criterion is measured at same time as construct

Predictive validity

criterion is measured at some point in future (after construct), to determine that construct predicts criterion

Convergent validity

form of criterion validity whereby new measures are correlated with existing established measures of the same construct

Discriminant validity

ensures extent to which scores on a measure of a construct are not correlated with measures of other, conceptually distinct, constructs and thus discriminate between them 

Scores on self-esteem and mood should not be correlated

4 Steps to Measuring Psychological Constructs

1. conceptually defining construct

2. operationally defining construct

3. implementing the measure

4. evaluating the measure

Demand characteristics

subtle cues that reveal to participants how researcher expects them to respond in experiment 

outcome variable

criteria or dependent variable

predictors

categorical or continuous

Categorical Predictors

Discrete

pass/fail

Nominal

Ordinal

Non-parametric test statistics

Non-parametric test statistics

Don't know shape of population distribution

distribution-free

flexible

Continuous Predictors

Response time, percent correct

Normal distribution

Interval

Ratio level measurement

infinite range of values between two points

age, date, time

Interval

Objective

Measures same thing but different quantitative labels- measures relative quantity, not all of something 

No true 0

 Ratio level measurement

Items correct, speed 

Count order, add and subtract, multiply, divide, ratio stays the same

4 validities

Construct

Statistic

External

Internal

3 reliabilities

Statistic

Inter-rater

Test-retest

Confirmation Bias

tendency to focus on cases that confirm our intuitive beliefs and to disregard cases that disconfirm our beliefs

Skepticism

pausing to consider alternatives and to search for evidence when there is enough at stake to justify doing so

systematically collected empirical evidence

tolerance for uncertainty

accepting there are many things that we simply do not know

clinical practice of psychology

diagnosis and treatment of psychological disorders and related problems

empirically supported treatments

treatment that has been shown through systematic observation to lead to better outcomes when compared to no-treatment or placebo control groups

• acceptance and commitment therapy (ACT)

• behavioral couples therapy

• cognitive behavioral therapy (CBT)

• exposure therapy

• exposure therapy with response prevention

• family-based treatment

acceptance and commitment therapy (ACT)

depression and anxiety

behavioral couples therapy

alcohol use disorders

cognitive behavioral therapy (CBT)

eating disorders, depression, anxiety

exposure therapy

PTSD and phobias

exposure therapy with response prevention

OCD

family-based treatment

eating disorders

Chapter 1 Key Takeaways

• knowledge acquired in many ways: intuition, authority, rationalism, empiricism, scientific method

• Sciences three fundamental features are systematic empiricism, empirical questions, and public knowledge

• Psychology takes scientific approach to understanding human behavior

• Important to distinguish scientific approach to understanding human behavior from pseudoscientific approaches

• Psychologists conduct research to describe basic phenomenon, make predictions about future behaviors, and explain causes of behavior

• Basic research is conducted to learn about behavior for its own sake, applied research conducted to solve practical problems

• Researchers in psychology cultivate certain critical-thinking attitudes such as skepticism and tolerance for uncertainty

Simple Model of Scientific Research in Psychology

internal validity

degree to which we can confidently infer a causal relationship between variables

Chapter 2 Key Takeaways

• Research in psychology can be described by a simple cyclical model. Research question based on research literature leads to an empirical study, results are published and become part of research literature

• Factors that affect interestingness are the extent to which the answer is in doubt, whether it fills a gap in research literature, whether it has important practical implications

• Experimental research involves manipulating independent variable to observe the effects on a measured dependent variable while non-experimental research involves measuring variables as they naturally occur

• The mean, median, and mode are measures of central tendency used to describe the typical, average, or center scores in a distribution. The range, standard deviation and variance are measures of how dispersed or spread apart the scores are. Measures of central tendency and dispersion are important descriptive statistics

conceptual definition

describes behaviors and internal processes that make up a psychological construct, along with how it relates to other variables

reliability

consistency of a measure

Chapter 4 Key Takeaways

• psychological constructs such as intelligence and depression are variables that are not directly observable because they represent tendencies, patterns, and internal processes. Important goal of scientific research is to conceptually define psychological constructs in ways that accurately describe them

• for any conceptual definition of a construct, there will be many operational definitions or ways of measuring. The use of multiple operational definitions, or converging operations, is a common strategy in psychological research

• There are two distinct criteria by which researchers evaluate their measures: reliability and validity.

• Validity is a judgement based on various types of evidence. Measure’s reliability, whether it covers construct of interest, whether scores it produces are correlated with other variables they are expected to be.