Ch 3: Measurement of Variables

variable

• something we study

• some property of an object, person, phenomenon, or event whose measurement can take on two or more values

construct

• “psychological” variable

• cannot be directly observed - involve internal processes and behavioral tendencies

  • what about discrete behaviors (ex. punching a doll)?

  • punching a doll can be an indicator of set of complex behavioral tendencies and internal processes

construct

label for a theoretical dimension on which people are thought to differ

conceptual definition

abstract or general meaning of the construct

operational definition

how the construct is measured or observed

• more objective and specific the better

measurment

the assignment of numbers to events/objects/people according to rules that permit properties of the events/objects/people to be represented by properties of the number system

continuous vs. discrete/categorical variables

• discrete/categorical - finite number of categories

• naturally and artificially categorical variables

• ordered and non-ordered categorical variables

quantitative vs/ qualitative variables

• quantitative → amount of something

• qualitative -< description of something

  • categorical by nature

true or false. variables must have at minimum 2 levels

true

5 basic levels of measurement

1. labels

2. nominal

3. ordinal

4. interval

5. ratio

4 characteristics of good measurement

1. reliable

2. valid

3. objective

4. standardized

reliability (r_xx)

extent to which scores are consistent across different occasions of testing, different editions of the test, or different raters scoring the test taker’s responses

validity (r_xy)

extent to which test is measuring what it purports to measure

• really concerns the interpretation and use of test scores

• the same test scores can be valid for one purpose but totally invalid for another purpose

• “the test scores are valid indicators of high school students’ college readiness”

reliability and validity are based on correlations

classical testing theory

X = T + E

• X = observed score

• T = true score

• E = measurement error (CTT does not take bias into account)

• reliability: extent to which X is made up of T

  • also, the extent to which X is free from E

  • average of several repeated measurements is more reliable than single measurement

    • true score is average of infinite observed core

• validity: extent to which T reflect one’s actual standing on the psychological construct

test-retest reliability

temporal consistency; repeated administration of same test to same sample

alternate-form reliability

temporal consistency and inter-form consistency; extent to which 2 separate forms of same test are equivalent

internal consistency

inter-item consistency; extent to which items in test are correlated

• split-half, odd-even

• coefficient alpha (Cronbach’s alpha) - equivalent to average of all possible split halves

inter-rater reliability/agreement

extent to which 2 or more raters are consistent (reliability) or agree (agreement); also extent to which same rater is consistent over time (intra-rater reliability/agreement)

summary of reliability

• property of scores and not tests

• choice of method for assessing reliability depends on construct of interest—issues w/ some methods?

• good level of reliability?

  • > .80 for applied decision making

  • > .70 (0r .65) for research purposes

• reliability is a necessary but not sufficient condition for validity

  • if test scores cannot correlate to themselves, then they cannot be expected to correlate w/ anything else

methods for assessing validity of test scores

criterion-related:

• predictive

• concurrent

• postdictive

content-related:

• coverage of construct domain

• contamination

construct-related

• convergent validity

• discriminant validity

face validity

summary of reliability and validity

both are estimated using correlations

• negative reliabilities are uninterpretable; range from 0 to 1

• magnitude of validity more important than direction; stronger the better

test scores can be reliable but not valid

• test scores cannot be valid but unreliable

reliability sets the upper limit for validity

• validity cannot exceed the square root of reliability

• r_xx=.80 → r_xy cannot exceed .90

test

• measurement tool → “measure” as a noun

• statistical test

• “good” test → reliability and test/measurement validity

study

• a test is not a study

• a study is not a “test of variables”

• study - investigation or assessment of relationships between variables

• not all studies are experiments, but all experiments are studies

• “good” study → research validity

• replicability is like reliability for studies