Class 4 - Measurement and Scaling

primary scales

• nominal data

• ordinal data

• interval data

• ratio data

each model allows more inferences to be made as you go down the list

nominal and ordinal are categorical data, interval and ratio are continuous data

nominal scale

scale whose numbers serve only as labels or tags for identifying and classifying objects. strict one to one correspondence between the numbers and objects

• description only

• numbers assigned to a category

• each object = single number

• categories are mutually exclusive

ordinal scale

ranking scale in which numbers are assigned to objects to indicate the relative extent to indicate the relative extent to which the objects possess some characteristic

• numbers are meaningful, one is better than the other, but you don’t know how much better

• more or less of a characteristic than some another object

• relative position, no magnitude of difference between the objects

interval scale

scale in which the numbers are used to rate objects such that numerically equal distances on the scale represent equal distances in the characteristic being measured

• equal distance

• permits comparison

• no fixed zero

ratio scale

the highest scale. allows researchers to identify or classify objects, and compare intervals or differences. meaningful to compute ratios of scale value.

• possess all the properties of the nominal ,ordinal, and interval scales, and in addition, an absolute zero point

• highest level of data

• meaningful zero - zero levels of how something was recorded gives you information. ie. WIP for an iphone starting at $0 is meaningful.

X and Y scales

X is often nominal for an experiment. (treatment and control)

Y outcome is most often interval or ratio

scaling techniques

comparative scales

• paired comparison

• rank order

• constant sum

• Q-sort and other procedures

noncomparative scales

• continuous rating scales

• itemized rating scales

  • likert

  • semantic differential

  • stapel

paired comparison scaling

respondent is presented with two objects at a time and asked to select one object in the pair according to some criterion

• data obtained are ordinal in nature

rank order scaling

comparative scaling technique in which respondents are presented with several objects simultaneously and asked to order or rank them according to some criteria

• comparative in nature, only thing we know is the order (but not how much better one is compared to the other

constant sum scaling

respondents are required to allocate a constant sum of units such as points, dollars, chips, etc. among a set of stimulus objects with respect to some criterion

• eg. allocate 100 points across these 6 beverages. can only allocate 100 points, no more

• reflects the importance they attach to each attribute - if an attribute is unimportant, they will assign it zero points

• absolute zero

• fine discrimination between qualities

• disadvantages - respondents may allocate more or less units than specified, also preferences will differ between participants

• ratio level data

continuous rating scale

respondents rate objects by placing a mark at the appropriate position on a line that runs from one extreme of the criterion variable to the other

• respondents are not restricted to selecting marks previously set by the researcher

• con: people can enter extremes, are these outliers meaningful or should they be removed from the data set?

itemized rating scales

measurement scale having numbers and'/or brief descriptions associated with each category, the categories are ordered in terms of scale position

• likert scales

• semantic differential scales

• stapel scales

likert scale

measurement scale with five response categories ranging from “strongly disagree” to “strongly agree” which requires the respondents to indicate a degree of agreement or disagreement with eas of the statements related to stimulus objects

• usually a 5-7 pt scale. smaller scale if participants are less knowledgeable about what they are ranking. can’t go below 5, then it is no longer continuous data and becomes categorical, can’t conduct means

semantic differential scale

7 point rating scale with endpoints associated with bipolar labels that have semantic meaning

• ie. one side says hot, one says cold

stapel scale

a scale for measuring attitudes that consists of a single adjective in the middle of an even-numbered range of values, from -5 to +5, without a neutral point (0)

itemized rating scales decisions

• number of scale categories

• balanced vs unbalanced

  • equal number of positive and negative statements?

• odd or even categories

  • is there a true middle point (odd), or are respondents forced to choose a side

• forced versus non forced

  • if participants may have no opinion, unforced is better

• verbal description

  • labels located as close to numbers as possible

• physical form

  • strength of what which you write anchors makes a difference, usually extreme anchors are better (ie. extremely disagree instead of generally disagree)

multi-item scales

consists of multiple questions or statements to be evaluated

• multiple likert or semantic differential items

• measuring same construct

• construct = abstract concept of interest

scale development

1. identification of domain and item generalization

2. content validity

3. pre-testing questions

4. sampling and survey administration

5. item reduction

6. extraction of factor

7. tests of dimensionality

8. tests of reliability

9. tests of validity

measurement error - systematic error

affects the measurement in a constant way and represents stable factors that affect the observed score in the same way each time the measurement is made

• same impact for everyone. ie. question 9 of a survey is broken for everybody

measurement error - random error

arises from random changes or differences in respondents or measurement situations

• error that is introduced haphazardly

• ie. emotion, influences individuals differently, is subjective

reliability

the extent to which a scale produces consistent results if repeated measurements are made on the same characteristics

• same or similar results each time a scale is captured

test-retest reliability

an approach for assessing reliability in which respondents are administered identical sets of scale items at two different times under as nearly equivalent conditions as possible

• same participants get same test some time later, should get similar results

validity

the extent to which differences in observed scale scores reflect true differences among objects on the characteristic being measured, rather than systematic or random errors

• reflection of true differences in values in population?

content / face validity

subjective but systematic evaluation of the representativeness of the content of a scale for the measuring task at hand

• are the questions on the scale reasonable, do the scale items adequately cover the entire domain of the construct being measured?

criterion validity

examines whether the measurement scale performs as expected in relation to other variables selected as meaningful criteria

construct validity

addresses the question of what construct or characteristic the scale is measuring. an attempt is made to answer theoretical questions of why a scale works and what deductions can be made concerning the theory underlying the scale

• convergent

• discriminant

• nomological

convergent validity

the extent to which the scale correlates positively with other measures of the same construct

discriminant validity

the extent to which a measure does not correlate with other constructs from which it is supposed to differ

nomological validity

the extent to which the scale correlates in theoretically predicted ways with measures of different by related constructs. seeks to confirm significant correlations between the constructs as predicted by theory

• network of related concepts we can statistically know or predict

generalizability

the degree to which a study based on a sample applied to a universe of generalizations