PSY 203 Research Methods Exam #3 Terms

Construct validity

Successful operationalization and measurement of constructs of interest

• Asks the question are we measuring what we think we are measuring?

• Want to only measure this and minimize the error

Random error

This is what you get when you measure any variable with actual values and the response measure results in a different amount and in different directions

• More variability than in a distribution with no random error

• Smaller and larger scores

• 
  

Observed score

Score that we get for the dependent variable in an experiment for each participant. It is a quantitative label produced by measuring an event and something we do not know ahead of time

Observed score= True score + systematic error + random error

• We are only interested in the true score, but there is going to be some measurement error and bias involved

True score

Actual measurement we obtain and should NEVER change and is what we want to assess

Systematic error

Every time these variable acts on the response or influences the response measure, it’s in the same direction and by the same amount

• There is an increase of 5 units every time

Measurement bias

Nonrandom error in a quantitative label due to systematic and/or random error

Accuracy

Degree to which the observed value matches the true value (Getting the right value in scores)

• Observed score is somewhat equal to the true score

• There is no way to measure the true score without error

Validity

Measure the construct that you think you are measuring

Face validity

Whether the content of the test is appropriate for the measured construct

• Does the test adequately sample the ‘content’ of course instruction

• Must appear legitimate to experts

Criterion-related validity

When test scores predict future behavior and test scores are correlated with current behavior

Predictive validity: Test scores are correlated with behavior obtained at some later time (Test scores compared from senior year in high school to senior year in college)

Concurrent validity: Test scores are correlated with currently obtained behavior

• Example: Hiring manager wants to know how performance on a job interview test correlates to actual on the job performance

Content validity

Convergent: Extent to which the scale correlates with measures of the same or related concepts (two measures give similar scores for the same concept)

Discriminant: Extent to which the scale does not correlate with measures of unrelated or distinct concepts (these two measures give different scores for different concepts)

Statistical conclusion validity

Degree to which the conclusions drawn from statistical analyses of data are accurate and appropriate

Construct validity

Successful operationalization and measurement of constructs of interest (measuring the same thing repeatedly)

Measurement scales for dependent variables

Categorical:

1. Nominal scales- category information for the dependent measure

2. Ordinal scales- category and rank order information for the dependant measure

Numerical:

• Interval scales- category and rank order information as well as equidistance between values

• Ratio scales- category and rank order information, equidistance between values, and a true zero

Internal validity

Is the study methodologically sound! Was it done right/well? Did the independent variable and only the independent variable cause a change in the dependent variable (validity inside the study)

• No uncontrolled variables had an effect on the dependent variable

External validity

Do the results have meaning outside the study? Does the same thing happen in other settings? (validity external)

Replication: Perform a second experiment and compare results from both experiments

Generalizability: Change in behavior, across time is maintained for a period of time after the treatment is removed

• Example: Experimenters wanted to increase the distance that participants ran with goal setting and performance feedback

Ecological validity/ setting representativeness

Degree to which the research setting reproduces the real-world variables

• Need to have internal validity before you can have eternal validity

Mundane realism

Degree of similarity between the laboratory setting to real-world settings

• Good ecological validity

Experimental realism

Degree to which situations are real and impactful to participants

• Good external validity in which you can generalize the independent-dependent variable relationships from the study to other people in other situations