Measurement

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/34

flashcard set

Earn XP

Description and Tags

Ways of measuring variables, reliability and validity of measurement

35 Terms

1
New cards

What 2 definitions are used in measurement?

  • Conceptual definition/construct = the definition of the variable in question at a theoretical level 

    • Start by stating the definition of their construct (conceptual variable) -> create operational definition 

  • Operational definition = reps a researcher's specific decision about how to measure/manipulate the conceptual variable for their study 


Psych research needs both definitions

2
New cards

What are 3 common types of measurement?

  • Self-reports

  • Observational/behavioural measures

  • Physiological measures


3
New cards

What are self-report measures?

  • Operationalise a variable by recording people’s answers to questions about themselves in a questionnaire or interview 

  • In research on children, self-reports may be replaced with parent reports or teacher reports 


4
New cards

What are observational/behavioural measurement?

  • Operationalizes a variable by recording observable behaviours or physical traces of behaviours 

  • May record physical traces of behaviour (e.g. stress behaviours could be measured by counting the number of tooth marks left on a person’s pencil) 


5
New cards

What are physiological measures?

  • Operationalizes a variable by recording biological data, such as brain activity or volume, hormone levels, eye movements, or heart rate 

  • Require use of special equipment -> e.g. facial EMG used to detect happy facial expression 

  • FMRIs 

  • E.g. stress

    • Measure the amount of cortisol released in saliva because people under stress show higher levels of cortisol (Carlson, 2009)

    • Skin conductance → electronic recording of the activity in the sweat glands of the hands or feet People under more stress have more activity in these glands

    • Detecting electrical patterns in different brain regions near the scalp using EEG

  • Must be validated by using other measures 

    • E.g. using fMRI to study brain activity relative to intelligence BUT PPTs were given an IQ test (OM) to know their intelligence (Dubois et al., 2018) 


6
New cards

How are operational variables primarily classified?

  • Categorical (nominal)

  • Continuous (quantitative)


7
New cards

How can nominal data be described?

Using the mode

8
New cards

What are the 3 types of quantitative variables?

  • Ordinal scale

  • Interval scale

  • Ratio scale


<ul><li><p>Ordinal scale</p></li><li><p>Interval scale</p></li><li><p>Ratio scale</p></li></ul><p></p>
9
New cards

What is an ordinal scale?

Measurement applies when the numerals of a quantitative variable represent a ranked order

  • May or may not be numeric

  • Can be described using the median, mode or range


10
New cards

What is an interval scale?

Requires two conditions:

  • Unlike ordinal scales, the numerals represent equal intervals (distances) between levels

  • There is no “true zero” → a person can get a score of 0, but the 0 does not literally mean “nothing”


Can be described using the mean, median, mode, range, or SD


11
New cards

What is a ratio scale?

Measurement applies when the numerals of a quantitative variable have equal intervals and when the value of 0 truly means “none” of the variable being measured 

  • Can be described using the mean, median, mode, range, or SD


12
New cards

Scales of measurement diagram

knowt flashcard image
13
New cards

What is reliability?

How consistent the results of a measure are 

14
New cards

What are the ways of measuring reliability?

  • Test-retest

  • Interrater

  • Internal reliability/consistency


15
New cards

What is test-retest reliability?

  • A PPT will get pretty much the same score each time they are tested with it 

  • Needs to be present when researchers are measuring constructs (such as intelligence or personality traits) that are theoretically stable over time 

  • Can apply whether the operationalization is self-report, observational, or physiological 

  • Depends on whether the variable is supposed to stay the same over time (e.g. intelligence vs allergies/stress) 


16
New cards

What is interrater reliability?

  • Consistent scores are obtained no matter who is rating the variable 

  • Important for observational measures 

  • If r is positive and strong (according to many researchers, r = .70 or higher), we would have very good interrater reliability 

  • When we’re assessing reliability, a negative correlation is rare and undesirable 

  • r can be used to evaluate interrater reliability when the observers are rating a quantitative variable BUT Cohen’s kappa is used when the observers are rating a categorical variable 

    • Measures the extent to which two raters place participants into the same categories 

    • A kappa close to 1.0 (ideally, .80 or higher) means that the two raters agreed 


17
New cards

What in internal reliability/connsistency?

  • A PPT gives a consistent pattern of answers, no matter how the researchers phrase the question 

  • Relevant for measures that use multiple items or observations to get at the same construct 

  • Applies when researchers combine multiple items or observations into one score 

  • Steps:

    • Researchers ask a large sample of PPTs to answer all of the items

    • Compute the correlations between every item + every other item

    • Compute the average inter-item correlation (AIC - average of all these correlations)

      • An AIC between .15 and .50 means that the items go reasonably well together (Clark & Watson, 2019)

    • Might compute Cronbach’s alpha (coefficient α - mathematically combines the AIC + the number of items in the scale)

      • The closer Cronbach’s alpha is to 1.0, the better the internal reliability

      • For self-report measures, researchers are looking for α = .80 or higher (Clark & Watson, 2019) 

    • If the AIC or Cronbach’s alpha is acceptable, the researchers can combine all the items together 

    • If the AIC or Cronbach’s alpha is unacceptable, the researchers look carefully at the items → may revise or omit rogue items from combined score


18
New cards

What is used to visualise reliability?

Scatterplots used to visualise → spread of dots corresponds to the strength of the relationship 

19
New cards

What is the correlation coefficient?

  • Single number that indicates the slope and spread of the dots on a scatterplot (-1-1) 

  • Slope direction can be + - or 0 


20
New cards

What does validity question?

Whether it’s actually measuring the construct it’s supposed to measure 

21
New cards

What are the 3 types of validity?

  • Face validity

  • Content validity

  • Criterion validity


22
New cards

What is face validity?

  • A measure has face validity if it is subjectively considered to be a plausible operationalization of the conceptual variable in question 

  • Researchers might check face validity by consulting experts 


23
New cards

What is content validity?

Requires knowledge of the conceptual definition; a measure must capture all parts of a defined construct 

24
New cards

What do face and content validity establish?

  • That the operationalizations look consistent with the conceptual definition 

  • BUT the other forms of validity require data → we collect data to test that the measurement is associated with something it theoretically should be associated with


Both are subjective assessments of validity


25
New cards

What is criterion validity?

  • Evaluates whether the measure under consideration is associated with a concrete behavioural outcome that it should be associated with, according to the conceptual definition 

  • Especially important for self-report measures because the correlation can indicate how well people’s self-reports predict their actual behaviour 

  • Provides strong evidence for a measure’s construct validity 


26
New cards

What is a testing example of criterion validity?

  • To assess criterion validity, your company would give both sales tests to all the current sales representatives and then find out each person’s sales figures as a measure of their selling performance. You would then compute two correlations: one between Sales Aptitude Test A and sales figures, and the other between Sales Aptitude Test B and sales figures 

  • Future sales performance is correlated more strongly with scores on Test A than with scores on Test B. If the data looked like this, the company would conclude that Test A has better criterion validity as a measure of selling ability, so it would be better for hiring salespeople. In contrast, the other data show that scores on Test B do not predict future sales performance; it has poor criterion validity as a measure of sales aptitude 


27
New cards

What is a way of gathering evidence for criterion validity?

Correlations

28
New cards

What is the known-groups paradigm and what can it be used for?

researchers see whether scores on the measure can discriminate among two or more groups whose behaviour is already confirmed 

  • Way of gathering evidence for criterion validity 

  • Can also be used to validate self-report measures

    • e.g. BDI → the average BDI score of the known group of depressed people was higher than the average score of the known group who were not depressed


29
New cards

What is convergent validity?

The patterns of correlations with measures of theoretically similar constructs

  • Convergent validity evidence also includes similar constructs, not just the same one 

  • E.g. Segal (2008) 

    • BDI scores were positively and strongly correlated with CES-D scores -> If the BDI really quantifies depression, the researchers reasoned, it should be correlated with (should converge with) other self-report measures of depression 

    • This correlation between similar self-report measures of the same construct (depression) provided evidence for the convergent validity of the BDI 


30
New cards

What is divergent (discriminant) validity?

The patterns of correlations with measures of theoretically dissimilar constructs

  • E.g. consider that many developmental disorders have similar symptoms, but diagnoses can vary. It might be important to specify, for instance, whether a child has autism or only a language delay. Therefore, a screening instrument for identifying autism should have discriminant validity; it should not accidentally diagnose that same child as having a language delay. Similarly, a scale that is supposed to diagnose learning disabilities should not be correlated with IQ because learning disabilities are not related to general intelligence. 


31
New cards

What types of validity are usually evaluated together as a patter of correlations among self-report measures?

Convergent + divergent validity

32
New cards

How do convergent and divergent validity relate to measurement?

A measurement should have higher correlations (higher r values) with similar traits (convergent validity) than it does with dissimilar traits (discriminant validity) 

33
New cards

What can convergent and divergent validity provide key evidence for?

Construct validity → the overall pattern of convergent and discriminant validity correlations helps researchers decide BUT it’s often even more important for a self-report measure to predict behaviour, as in criterion validity

34
New cards

Describe the relationship between reliability and validity

  • A measure can be less valid than it is reliable BUT can’t be more valid than it is reliable

    • Reliability has to do with how well a measure correlates with itself

    • Validity has to do with how well a measure is associated with something else

    • If a measure does not even correlate with itself, then how can it be more strongly associated with some other variable?) 

  • Reliability is necessary (but not sufficient) for validity 


35
New cards

Validity and reliability concept map

knowt flashcard image