Chapter 1: Psychometrics and the importance of psychological measurement

retardation

general intellectual functioning that is "significantly sub-average" (an IQ less than 70).

2 reasons to conduct behavioural measurement

- because they are interested in the specific behaviour on its own right. (I.e. some psychologists have studied the way facial expressions affect the perception of emotions)

- observe human behaviour as a way of assessing unobservable psychological attributes such as intelligence, depression, knowledge, aptitude, extroversion, or ability (I.e. they identify some type of observable behaviour that they think represents the particular unobservable psychological attribute, state, or process)

validity

• the extent to which a test measures or predicts what it is supposed to measure

• is a matter of degree, if the scores from a measure seem to be actually measuring the mental state or mental process that we think that are measuring, we say that our interpreration of scores on the measure is valid.

Measuring working memory

• made an inference from an observable behavior to an unobservable psychological attribute

• the recall task had to be theoretically linked to working memory

• Theoretical concepts explain people’s differences

hypothetical constructs or latent variables

entities that cannot be directly observed but are inferred on the basis of empirical evidence

- They are theoretical psychological characteristics, attributes, processes, or states that cannot be directly observed, and they include things such as learning, intelligence, self-esteem, dreams, attitudes, and feelings.

operational definitions

The operations or procedures used to measure these hypothetical constructs, or that matter to measure anything

psychological test

- is a systematic procedure for comparing the behavior of two or more people

three important components of psychological tests

1) tests involve behavioral samples of some kind

2) the behavioral samples must be collected in some systematic way

3) the purpose of the tests is to compare the behaviors of two or more people (include a comparison of performance by the same individuals at different points in time)

tests that produce quantitative data

tests produce numbers that represent the amount of some psychological attribute possessed by a person.

tests that produce qualitative data

people who take the test can be sorted into group> based on their responses to test items.

interindividual differences

• test must be capable of comparing the behavior of different people

intra-individual differences

• behavior of the same individuals at different points in time or under different circumstances

tests vary in content

- achievement tests,

- aptitude tests,

- intelligence tests

- personality tests

- attitude surveys, and so on.

tests vary in response requires

- open-ended tests

- closed-ended tests

open-ended tests

in which people can answer test questions by saying anything they want in response to the questions on the test

closed-ended tests

which require people to answer questions by choosing among alternative answers provided in the test.

tests vary in methods used to administer them

- individually administered tests

- tests administered to groups of people

tests vary in intended purpose of test scores

- criterion referenced (domain referenced)

- norm referenced

criterion referenced (domain referenced)

most often seen in settings in which a decision must be made about a person's skill level

(i.e already a fixed number that determines a pass or not)

norm referenced

usually used to compare a person's test score with scores from a reference sample or a normative sample in order to understand how the person compares with other people.

(i.e. people are selected based on who is the best relative to all the candidates)

cons of norm referenced test

Scores on________are of little value if the reference sample:

- is not representative of some population of people

- if the relevant population is not well-defined

- if there is doubt that the person being tested is a member of the relevant population.

tests vary in

- content

- type of response required

- methods used to administer them

- intended purpose of test scores

- speeded tests vs power tests

speeded tests

• time-limited tests

• are not expected to complete the entire test in the allotted time.

• are scored by counting the number of questions answered in the allotted time period

power tests

- are not time-limited, in that examinees are expected to answer all the test questions.

other synonyms for test

- measure

- instrument

- scale

- inventory

- battery (instruments intended to be administered together )

- schedule

- assessment

bundled tests

_________ are instruments intended to be administered together but are not necessarily designed to measure a single psychological attribute

psychometrics

science concerned with evaluating the attributes of psychological tests.

- the study of' the operations and procedures used to measure variability in behaviour and to connect those measurements to psychological phenomena

three attributes studied in psychometrics

1) the type of information (in most cases, scores) generated by the use of psychological tests

2) the reliability of data from psychological tests

3) issues concerning the validity of data obtained from psychological tests

Francis Galton (1822-1911)

- anthropometrics: the measurement of human features, such as head size, arm length, and physical strength

- psychometry: measurement of mental features. the art of imposing measurement and number upon operations of the mind'

- founding father of modern psychometrics

- utility of using the normal distribution to model many human characteristics

- the idea of the correlation coefficient

- pioneered the use of sampling for the purpose of identifying and treating measurement error

- focused on the variability of human characteristics

- differential psychology

anthropometrics

the measurement of human features, such as head size, arm length, and physical strength

psychometry

measurement of mental features. the art of imposing measurement and number upon operations of the mind'

differential psychology

the study of individual differences

- contrasted with experimental psychology

experimental psychology

(>cused mainly on the average person instead of the differences among people.

Challenges to measurement in psychology

1. The complexity of psychologic1l phenomena; notions such as intelligence, self-esteem, anxiety, depression, and so on, have many different aspects to them. one of our challenges is to try to identil)r and capture the important aspects of these types of human psychological

participant reactivity

participants act differently or unnaturally because they know someone is watching them.

- demand characteristics

- social desirability

- malingering

demand characteristics

some participants may try to figure out the researcher's purpose for a study, changing their behaviour to accommodate the researcher

social desirability

some people might become apprehensive, others might change their behavior to try to impress the person doing the measurement

malingering

others might even change their behavior to convey poor impression to the person doing the measurement

observer or scorer bias

people collecting the behavioral data (observing the behavior, scoring a test, interpreting a verbal response, etc.) can bring biases and expectations to their task.

composite scores

involve a series of questions, all of which are intended to measure some aspect of a particular psychological attribute or process.

sensitivity

refers to the ability of a measure to discriminate adequately between meaningful amounts or units of the dimension that is being measured.

(i.e. consider someone trying to measure the width of a hair with a stan dard yardstick. Yardstick units are simply too large to be of any use in this situation.)

- (having a 2 scale score vs 5 scale score in measuring depression. good or bad is hard to detect changes with)

- a psychologist may find that a procedure for measuring a psychological attribute or process may not be sensitive enough to discriminate between the real

differences that exist in the attribute or process.

- the sensitivity problem is exacerbated because we might not anticipate the magnitude of meaningful differences associated with the mental attributes being measured.

apparent lack of awareness of important psychometric

information

- psychological measurement is often a social or cultural activity

- Whether it provides information from a client to a therapist regarding psychiatric symptoms, from a student to a teacher regarding the student's level of knowledge, or from a job applicant to a potential employer regarding the applicant's personal ity traits and skill, applied psychological measurement often is used to L1cilitatc the flow of information among people. Unf(Jrtunalcly, such mcasuremmt often seems to be conducted with little or no regard for the psychometric quality of the tests.

- poorly constructed tests vs good quality exams

validation process

The process by which the psychologist accumulates evidence suggesting that there is an association between the operationalization scores on the test and the true levels of the theoretical concept.