Psychology- Research Methods/ Assessment and Effect Size

The Psychological Triad

thought, feelings, and behaviors

Goal of personality psychology

explain the whole person in their daily movement

What does BLIS stand for?

behavioral data, life data, informant data, self reports/ self judgment data

Behavioral Data

information that is systematically recorded from direct observation

How to collect Natural B data?

• real life

• diary and experience sampling methods

• EAR: electronically activated recorder

• small wearable camera

• social media profiles

What are the two types of B data?

Natural and Laboratory

Advantages of natural B data

realistic

Disadvantages of B data

• difficult

• desired contexts may not occur

How to collect laboratory b data?

• make situations happen and record behavior- experiments

• examine reactions to situations

• represent real-life contexts that are hard to observe difficulty

Advantages of Lab B data

• large range of contexts in lab

• appearance of objectivity

Disadvantages of Lab B data

difficult and expensive

Mixed Types of Data

• data doesn’t always fit into 1 category

• wide range of possible types

Life Outcomes (L) Data

obtained from archival records or self-report

Examples of L data

GPA, health, criminal history

Advantages of L Data

• objective and verifiable

• psychological relevance ( conscientious=tidy room)

Disadvantages of L Data

can be influenced by more than personality

Informant Report Data

• no training or expertise needed

• based on observing people in whatever context they know them from

• used frequently in daily life

Examples of I Data

acquaintances, coworkers, clinical psychologists

Advantages of I Data

• based on large amount on info

• many behaviors in many situations

• based on observation of behavior in the real world

• definition truth

• based on common sense about what behaviors mean

Definitional Truth

where a person's statement about themselves is true simply because they believe it, as it directly defines an aspect of their own internal reality

Disadvantages of I Data

• limited behavioral info ( work v. Friday night)

• lack of access to private experience

• error: extreme behaviors

• bias: prejudice

S Data

• questionnaires or surveys

• most frequent data source

• high face validity

Advantages of S Data

• based on large amount of info

• Access to thoughts, feelings, and intentions

• definitional truth

Disadvantages of S Data

• maybe people won’t tell you

• maybe people can’t

• too simple and too easy

What are some Widely Used Tests Personality Tests

MMPI, CPI, 16 PF

What kind of data does Personality Tests have?

S and B data

IAT

people who implicitly know they have a certain trait will respond faster

• implicit bias

Projective Tests

• provide B data

• analysis of content of stories, letters, and speeches

Advantages of Projective Tests

• good for breaking ice

• can use to get info not converted in other tests

Disadvantages of Projective Tests

• validity evidence is scare

• expensive and time consuming

• psychologist can’t be sure about what they mean

Evidence for Project Tests

• some evidence of validity

• used by 82% of psychologists

• valid for predicting certain outcomes

Objective Tests

• validity and subjectivity of tests items

• items are still not absolutely objective

• subjectivity may be good

Principle of Aggregation/ Spearman- Brown Formula

the sum of multiple measurements (test items) provides a more stable, reliable, and representative estimator of a construct than any single measurement

Making Objective Test: What are the 4 conditions of validity?

1. items mean same thing to test taker and creator

2. capability for accurate self-assessment

3. willingness to make an accurate and undistorted report

4. items must be valid indicators of the construct

Reliability

• the number you obtain from a measurement is not just the "true" value of what you are measuring, but a combination of that true value plus the combined, chaotic noise of external factors.

• def of error variance/measurement error

Factors that Undermine Reliability

• low precision

• state of participant

• state of experimenter

• environment

How to enhance reliability

• be careful

• use protocal

• measure important things and engage participants

• aggregation

Validity

• reliable

• measures what it is supposed to

Generalizability

• generalizability over participants

Case Method

can yield explanations of particular events, lessons, and scientific principles

Advantages of Case Method

• describes whole phenomenon

• source for ideas

• sometimes necessary

Disadvantages of Case Method

• unknown generalizability

• no control

Experimental Method

casual relationship btwn an independent variable and dependent variable

Correlation Coefficient

establishes the relationship btwn 2 variables

Disadvantages of Correlational

• third variable problem

• unknown direction of cause

Overlap of Clinical Psychology with Personality Psychology

• normal versus extreme patterns of psychology

• personality disorders

• both attempt to understand the whole person

Trait Approach

how people differ psychologically

Biological Approach

understand the mind in terms of the body

Psychoanalytic Approach

primary concern is with the unconscious mind and internal mental conflict

Phenomenological Approach

focus on people’s conscious experience of the world

• humanistic: how conscious awareness produces uniquely human attributes; understanding meaning and basis of happiness

• cross-cultural: how the experience of reality changes across cultures

  • individualistic v.s. collectivstic; achievement and self esteem

Leaning and Cognitive Approach

how behavior changes as a result of rewards, punishments, and other life experiences

Classic Behaviorism: Learning and Cognitive Approach

focuses on every behavior

Social Learning: Learning and Cognitive Approach

learning through observation and self evaluation

Cognitive personality: Learning and Cognitive Approach

focuses on cognitive processes including perception, memory, and thought

Advantages of Personality Psychology

• account for psychology of whole persons and real life concerns

• inclusive, interesting, important

Disadvantages of Personality Theory

over-inclusiveness or unfocused research

Advantages of Basic Approaches

good at addressing certain topics

Disadvantages of Basic Approaches

poor at addressing other topics or ignores them

Purpose of a basic approach

• to avoid overwhelming complexity rather than analyzing everything at once, researchers intentionally focus on specific types of data

• limit inquiry to certain kinds of observations and patterns

Fish and Water Effect

where people are so used to their own traits (like a fish in water) that they don't notice them, highlighting a lack of self-awareness for deeply ingrained habits or personality aspects

Difference between S data asking about past and B data recorded in a diary

S Data

• asking you to reflect on your past using self judgment

B data

• diary is used to record behaviors for researchers to analyze

The Thematic Apperception Test and the Rorschach test elicit ________ data. 

Laboratory B data

The MMPI test was designed to

assess individuals with psychological disorders

Which of the following is a limitation of projective tests? 

Projective tests are relatively inefficient and expensive to administer.

Difference between score on math test versus SAT score made publicly available to colleges?

• since math test score is not an archived record it is B data, showing your performance

• the SAT score is data from your record, so that is L data

Factor Analysis

1. generate a list of objective items

2. administer these items to a large number of people

3. analyze with factor analysis

4. consider what the items that group together have in common and name them

Significant Testing

Null Hypothesis

Alternative Hypothesis

Effect Size

What is the best way for a researcher to judge the face validity of items on a measure?

Read and consider the content of the items

Rational Method of Test Construction

to come up with items that seem directly, obviously, and logically related to what it is you wish to measure. 

Construct Validation

the systematic process of accumulating evidence to demonstrate that a measurement tool accurately assesses the specific

Criterion Validity

measures how well a test or measure correlates with an established external standard (a "criterion") to predict or reflect an outcome

Explain statistical significance vs. effect size using a real-world example

Statistical Difference

• “Is the difference real?”

• Are the results due to chance?

Effect Size

• “How large/important is the difference?

Explain what it means when one thing “partly explains” or “helps explain” another

the factor in question is a contributing cause or reason, but not the sole, complete, or total explanation

Hill et al. (2011): The big question of the paper (no statistics)

• The researchers know from previous studies that conscientious people tend to live longer.

• They wanted to identify the mediators that explain this link.

• Physical Health: Does conscientiousness lead to fewer illnesses, which then leads to a longer life?

• Cognitive Functioning:

  • The author encourages using personality assessments in medical settings to better identify and support at-risk patients.

Hill et al. (2011): The main findings

• used a 10 year longitudinal study of older adults

• High conscientiousness significantly predicted a longer life, confirming earlier research

• Cognitive functioning was found to be a partial mediator. This means that part of the reason conscientious people live longer is that they tend to maintain better cognitive health (like stronger memory) as they age.

• Physical Health did not show strong results

Hill et al. (2011): The broader takeaway

• showing that how we approach life—our level of self-discipline and order—affects the aging process of our brains.

• The author encourages using personality assessments in medical settings to better identify and support at-risk patients.

• conscientious individuals tend to be well put together and thus engage in healthier behaviors

• Insurance rates might increase for those who are deemed less

  conscientious.

Vazire et al: The big question of the paper (no statistics)

• Who is the best judge of personality—the self or others?

• challenges the idea that "you know yourself best."

• Vazire sought to prove that neither is "better" overall; rather, each has a distinct "blind spot" based on how visible a trait is and how much our ego (evaluative pressure) gets in the way

Vazire et all: The main findings

• you are best judge of low observability/ low evaluative traits: self

  • internal feelings

• friends are best judge of high evaluative traits (traits where clearly “better” or “worse” answer)/ low observability traits: friends/others

  • the ego-blindspot

• high observability/low evaluativeness: self, friends, and strangers

  • the obvious

• high observability/ high evaluativeness: friends/others

  • being rude, physical attractiveness

Vazire et al: The broader takeaway

• self-awareness is limited

• You need to incorporate the "other" perspective

• Knowing when self vs other perceptions will be more accurate for certain traits can help to create better quality assessments of

  personality.

Confirmatory Factor Analysis

a statistical technique used to verify (confirm) if a pre-defined theoretical model accurately fits the actual data

Exploratory Factor Analysis

a statistical technique used to uncover hidden patterns (factors) within a large set of variables, reducing them into fewer, meaningful, and interpretable factors

Null Hypothesis

treatment will not change our subjects in any meaningful way

p-level

the probability of observing the data if the probability was true

• if p= 0.05, means the chance of getting the results by random if the null hypothesis was true if 5%

Alternative Hypothesis

Treatment does in fact change our subjects in a meaningful and measurable way

Problems with NHST

• “Significant” does not necessarily mean strong or important

• The logic is difficult to describe (and understand)

• The criterion for significance is an arbitrary rule of thumb (.05 v. .06?)

• Chances of significance vary with sample size

Effect Size

• More meaningful than a significance (p) level

• Can be used for prediction

• Interpreting correlations