Learn: Reliability vs. Validity

reliability

The ability of a test to give the same results under similar conditions.

inter-rater reliability

If you have 2 observers watching the same behavior, their scores should agree with each other.

intra-rater reliability

This refers to the consistency of a researcher's behaviour. A researcher should produce similar test results, or make similar observations or carry out interviews in the same way on more than one occasion.

validity

Accurate. The degree to which a study accurately reflects or assesses the specific concepts that the researcher is attempting to measure. Does it measure what its suppose to.

internal validity

The conclusions drawn from the experimental results may not accurately reflect what has gone on with the experiment itself

external validity

Extent to which we can generalize findings to real-world settings

generalizability

How well a study's findings can be applied to other people and circumstances.

content validity

How well test or interview items measure a variable (math data: cannot just use addition)

construct validity

Degree to which a test actually measures what it claims to measure. is research design able to say that something causes something else

face validity

Form of validity that is inferred by subjectively assessing whether the predictor "looks as if" it is related to job performance & has clearly similar content

threats to internal validity

maturation: people a changing and growing

selection bias: the systematic bias in selecting participants for your experiment

history: anything that happened in data collection that could effect your dependent variable

casual time order: unaware how previously collected databases are collected

triangulation

Combining different research methods in a study in order to collect richer data

conceptualization

the process by which we specify precisely what we mean when we use a particular term

operationalization

the process of assigning a precise method for measuring a term being examined for use in a particular study

nominal

simplest level of measurement

offer simple labels

dummy variable: 0 and 1

ex. gender --> male 1, female 0

ordinal

variables with attributes that we can rank order

distances makes no difference

ex. education, the higher education the higher number you have

interval

variables are rank ordered and maintain equal distances between attributes

ex. IQ test: difference between the intervals are the same

ratio

variables which include the characteristics of the other three but have a true zero point.

ex. age

index

constructed by accumulating scores assigned to individual attributes

ex. circle all that apply and then adding the number of things measured

scale

contructed by assigning scores to patterns of responses and is more precise

ex. circle agree to disagree and number each 1-5

typology

A systematic arrangement of the variation in the attributes of material culture that groups artifacts into similar types.

composite measures

A measure designed to provide a comprehensive assessment of an object or phenomenon, with items to assess all relevant aspects or dimensions.