Exam 2 - Personality Research + Constructs, Variance, Power

3 Criteria for Scientific Research

• Systematic Empiricism

• Public Verification

• Solvable Problems

Systematic Empiricism

Rely on systematically-obtained observations to draw conclusions about the world

• Can’t rely on intuition, reason, or logic alone

• Can’t always directly observe something 

Public Verification

Findings must be observed, replicated, and verified by other researchers. 

Solvable Problems 

Must study questions that are potentially answerable through systematic empiricism 

• Questions must be answerable given current knowledge and research techniques

Two reasons for public verification

• Ensures phenomena scientists study are real and observable and not one person’s fabrications

• Makes science self-correcting

5 Methods of Personality Assessment

• Self-report

• Indirect methods

• Narratives

• Diary studies

• Observation

Descriptive Studies

• Most basic form of research, describing

• Qualitative

• Doesn’t explain a particular phenomenon, just describes it

Correlational Study 

• Designed to measure relationships

• Measured by the correlation coefficient

Cross-Sectional vs. Longitudinal

• Cross-sectional is subjects of different ages observed at a single point in time

• Longitudinal is the same subjects observed at different ages

Experimental Studies

• Examination of group differences with a manipulation occurring in one group

• Best design to test causality

• Randomized Controlled Trials are a special form of an experimental study

Psychometric Instrument

A standardized test or assessment designed to measure an individual’s psychological attributes

What’s an example of an indirect method to assess personality?

Projective Test

Variance

Total variability in participants’ sores

What is Total Variance made up of? (function)

Total Variance = Systematic Variance + Error Variance 

Systematic Variance

Portion of the total variability in participants’ scores that is related in an orderly, predictable fashion to the variables the researcher is investigating. Essentially, the accuracy of prediction or explanation.

Error Variance

The portion of total variance in participants’ scores that’s unrelated to the variables under investigation in the study; variance that remains unaccounted for.

T/F: Error variance means the researcher definitely made a mistake. 

False; measurement error means that doesn’t went wrong with the measure, but there are other reasons for error variance. 

T/F: Research usually has the ability to control for large amount of measurement error.

True

How do we determine if the % of systematic variance explained is meaningful?

Through statistical significance testing using the null hypothesis and p-value

Type I Error

False positive; Rejecting the null hypothesis when it is in fact true. 

Telling a man that they are pregnant is an example of which type of error?

Type I

Type II Error

False negative; Failing to reject the null hypothesis when it is false  

Telling a pregnant woman they’re not pregnant is an example of which type of error?

Type II

Alpha (a) is the probability of making a (Type I/Type II) error.

Type I

Beta is the probability of making a (Type I/Type II) error.

Type II

What is the formula for Power?

Power = 1 - Beta

What is the typical minimum value for power?

.8 → 20% chance of Type II Error

Definition of Power

The probability that it will reject a false null hypothesis. Essentially, the likelihood that a study will detect an effect when there is an effect to be detected.

Statistical power is (directly/inversely) related to (alpha/beta)

Inversely; beta

If statistical power is high, the probability of making a Type II error (increases/decreases)

Decreases

What affects statistical power?

Size of the effect and the size of the sample

Large samples have (greater/lower) test sensitivity than small samples. 

Greater

Effect Size

The strength of a relationship between two variables

Two common indices of effect size

• Correlation Coefficient r

• Cohen’s d

What are small, medium, and large values of r (correlation coefficient)?

• Small = .10

• Medium = .30 

• Large = .50

What are small, medium, and large values of Cohen’s d?

• Small = .20

• Medium = .50 

• Large = .80

T/F: Most psych research has large size effects?

False; most psych research has small or at medium size effects

How to calculate sample size for a power analysis?

• Previous published studies

• Pilot studies

• Theory

What is a good value fro power that minimizes both Type I and Type II error?

.8

If the Power = .8, what is the chance of making a Type II error?

20%

Power Analysis

Tells us what sample size will give us our needed power. This is done before conducting a study

What is the interpretation of study results if the p-value is small and the effect size is high?

IV had a strong and reliable effect on DV

What is the interpretation of study results if the p-value is small and the effect size is low

IV had a weak effect on DV inflated by large n

What is the interpretation of study results if the p-value is large (fail to reject null) and the effect size is high?

IV had strong effect on DV, but too low N to detect OR IV had strong effect of unknown reliability

What is the interpretation of study results if the p-value is large (fail to reject null) and the effect size is low?

IV had weak effect on DV