Psychometrics Exam Content 2

Raw Score

Original score without any transformation applied.

Raw Score Problems

No frame of reference, must obtain info about distribution in order for scores to be meaningful, need to compute the mean & SD to interpret the score

Raw Score Solutions

Transform the raw score to identify & describe exact location of every score

Linear Transformation

Preserves the linear relationship between data points in terms of order & spacing but can shift & scale the data (%, z scores/SD, t scores, scaled scores)

Percentages

Useful for combing scores on a common scale

Z Score

Tells the exact location of the og raw score value w/in the distribution; Mean of 0 & SD of 1

T Score

Standardized score with mean of 50 and SD of 10.

Scaled Scores

Typically used as subtest; Mean of 10 & SD of 3; Used for distribution that may be skewed

Standardized Scores

Mean of 100 & SD of 15 (WAIS)

Area Transformation

Change characteristics of raw data; Change distribution based on comparison

Percentile Rank

Score indicating percentage of scores below it; Relative to others

Stanines

Division of distribution into nine equal parts; Each part representing a specific range of scores

Norms

A group of tests achieved by some group of individuals based on normal distribution (Age, grade, education based, ex-SAT)

Positive Correlation

Both variables increase or decrease together; Direct relation

Negative Correlation

One variable increases while the other decreases; Inverse relation

Covariance

Measure of joint variability between 2 random variables; Natural; Hard to interpret

Correlation

Measure of relationship between two variables.

Pearson's Product Moment Correlation

Measures linear relationship between two interval or ratio variables.

Point Biserial

1 continuous variable, 1 true binary (ex-yes/no)

Biserial Correlation

1 continuous, 1 ordinal; Binary assumed to be latent

Phi Coefficient

Measures association between two dichotomous variables.

Kendall's Tau

2 ordinal variables

Valence

Positive or negative; Direction of association

Magnitude

0 to 1; Strength of association

Perfect Correlation EXs (rare)

Unit conversion in same system; Variable correlated w/ itself

Corr. Limitations

Invisible 3rd variables (correlation doesn't = causation), restricted range, effect of an outlier

Monotonic Relationship

One variable consistently increases or decreases with another.

Cohen Interpreting Correlational Strength

.10=small, .30=moderate, .50=large; Squared correlation shows % of shared variance

Spearman's Rank Correlation

Measures strength of association between ranked variables.

Reliability

Consistency of a measure across different instances.

Test-Retest Reliability

Stability of scores over time.

Time Variant

eg. mood, state dependent processes

Time Invariant

eg. personality traits

Reliability Coefficients

Statistical eval. of test scores; Reliability must be interpreted

Measurement Error

Variations in measurement

Internal Consistency

Unit of measurement is consistent

Reliable tests.....

Have minimal measurement error, Measurement is accurate despite testing circumstances, Scores are consistent across persons, Scores are consistent across time

Practice Effects

When the test taker could learn something by taking the test, influencing scores on subsequent administration

Simplex

Closer time points exhibit higher correlations with each other; vice versa

Alternate/Parallel Forms

Administer the two forms to the same people; Correlate the scores

Order Effects

Changes in score based on the order of forms presented

Homogeneous Items

Assess a similar underlying construct

Split-Half Reliability

Old method of estimate; How well do some scores from 1/2 of the test correlate with the other 1/2; Attenuates reliability but is corrected

Interscorer Reliability

Consistent scoring between persons

Intrarater Reliability

Consistent scoring within person

True Score

Average score obtained if an individual took a test an infinite number of times

Random Error

Difference between the true & observed score; Explanation for why observed score is higher/lower than true score; Over an infinite number of testing occasions, random error will be 0

Classical Test Theory

Observed score = true score + error; Since we can't calculate reliability we must estimate

Systematic Error

Single source of error which always increases or decreases the true score by the same amount

Absolute Agreement

Raters provide the exact same scores for each individual

Consistency

Raters do not provide the same exact score, but their scores increases or decreases at same rate

Cohen's Kappa

Metric of interrater reliability with 2 raters responsible for measuring a variable on a categorical scale

Interrater Agreement

Cohen's Kappa

Intrarater Agreement

Intraclass correlation coefficient (ICC)

Interpreting Test Retest

Negative reliability correlation indicates inconsistency and systematic shift

Confidence Interval (CI)

Describe the degree of uncertainty for an estimate of a parameter derived from a sample of participants who belong to a population (point estimate)

Standard Error of the Mean (SE(M))

We calculate a mean from our sample that can be used to estimate the value of the population mean that lies within 95% CI of our sample mean

Standard Error of the Measurement

Sources of Error

Test itself, test administration, test scoring, test takers

Test itself

Reading comprehension requirements; poorly written; ambiguous (content)

Test Administration

Failure to follow administration instructions; failure to adhere to prescribed testing conditions; failure to answer questions correctly; failure to monitor own behavior

Test Scoring

Hand scoring less accurate than computerized

Test Takers

Fatigue, health, exposure to item content, social desirability & impression management

Social Desireability

Presenting oneself in the most favorable way

Impression Management

Intentional social desirability; deliberate deception; faking good/bad

Test Length

Longer test will have better reliability; needs homogeneous items

Strategies to Combat Test Scoring Error

Double scoring & tie breakers for discrepancies

Combat Impression Management

Validity scales; reverse scoring

Test Retest

Longer intervals between tests = lower reliability