Scott Braithwaite Psych 309 BYU Exam 2

Define and be able to apply the broad definition of validity.

agreement between test score or measure and the quality it is believed to measure
DOES YOUR TEST MEASURE WHAT IT SAYS ITS SUPPOSED TO MEASURE?

What are the three main types of validity evidence?

Construct (Does the instrument measure the appropriate construct?)
Criterion (How is the measure related to other tests measuring similar variables?)
Content (Does the test measure every aspect of the construct?)

What prerequisites exist for validity?

reliability

What is Face Validity and how does it differ from other aspects of validity?

the appearance that a measure has validity
It doesn't offer evidence to support conclusions drawn from test scores, but it is a good starting point to establishing 'real' validity

What is Content Validity and how is it measured?

the adequacy of representation of the conceptual domain the test is designed to cover

How do construct under-representation and construct-irrelevant variance relate to content validity?

1. failure to capture important components of a construct
2. scores are influenced by factors irrelevant to the construct

What is the content validity ratio and how is it calculated?

uses a panel of judges to rate items on a 3-point scale (Essential, Useful but Not Essential, and Not Necessary) to determine if items are necessary to measure the construct

What is a criterion and what is criterion-related validity?

1. the standard against which a test is compared
2. how well test scores correspond with a particular criterion

Name and define the three sub-types of criterion-related validity.

Predictive: accuracy with which test scores predict a criterion obtained at a later time
Concurrent: degree to which the test scores are related to the criterion and can be measured at the same time
Postdictive: accuracy with which a test score predicts a previously obtained criterion

What is the validity coefficient?

Correlation between a test and a criterion

What is the meaning of a squared validity coefficient?

The percentage of the correlation in the criterion we can expect to know in advance because of our knowledge of the test scores

What is incremental validity?

how much does your measure add to the literature? (new information on the client, construct, price, etc.)

What is a construct?

an idea or phenomenon a test is measuring for

What is construct-related validity?

how well does your test measure your construct?

What are the two types of evidence in construct validity?

convergent: measures of the same construct converge or narrow in on the same thing
discriminant: low correlations with measure of unrelated constructs, or evidence for what the test does not measure

What is the relationship between reliability and validity?

a test must be reliable to be valid, but reliability does not equal validity

Which two types of validity are logistical and not statistical? Why?

Content Validity and Face Validity; neither depend on statistical formulas but are more due to simply looking at the test

Which type of validity has been referred to as the "mother of all validities", or "the big daddy" and why?

Construct validity; borrows from the actual definition of validity

What are the different types of validity?

Content: Items relate to construct
Criterion: SAT/ACT, class exams
Construct: does the test measure what it should?

What role does the relationship between examiner and test taker play?

The better your relationship, the better you score (Generally)

What is the relationship between test examiner race and intelligence scores?

non-significant effects, race does not play a role in highly structured IQ tests

Why would examiner race effects be smaller on IQ tests than on other psychological tests?

They are often highly standardized and administered by highly trained individuals

What is the standard for test takers who are fluent in two languages?

administer in the first language, but make sure that the test in the new language is reliable and valid

Define expectancy effects and know whose name is associated with these effects.

ROSENTHAL
When an examiner has certain expectations for the outcome of a test, it introduces bias to the scores.

A review of many studies showed that expectancy effects exist in ________ situations

some, but not all

What types of situations might require the examiner to deviate from standardized testing procedures?

when those being tested are not part of a neurotypical or "normal" group

What advantages and disadvantages were mentioned in lecture and the text regarding computer administered tests?

Advantages: interaction is more pleasant, can attribute to more honest responses, excellence of standardization, individually tailored administration, timing response precision, control of bias, cost effective, etc.
Disadvantages: cannot adjust for randomness, no personal interaction

What subject variables impact testing?

non-intellectual factors (sleep, illness, hunger, mood, etc.)

What are the three major problems in behavioral observation studies?

Reactivity
Drift
Experimenter Expectancies

What is reactivity?

when a subject reacts to an observer checking on them

What is drift? How does it relate to contrast effect? How can drift be addressed?

tendency to drift from strict rules/training and adopt idiosyncratic definitions of behavior
the more you see behavior, the less it will stand out to you

What are experimenter expectancies and how do they introduce bias?

the experimenter expects certain things and begin to introduce bias in the way that they administer/code the test

How well do people do in detecting deception/lies?

we suck. we are worse than chance.

What is the Halo Effect?

ascribing attributes to someone on the basis of something other than the trait
example: people who are attractive are perceived as being kind

what does a good interviewer know how to do?

provide a relaxed and safe atmosphere through social facilitation

How are interviews similar to tests?

both gather data on a subject, both can be tested for reliability and validity

what is interpersonal influence?

the degree to which one person can influence another

what is interpersonal attraction?

the degree to which people share a feeling of understanding, mutual respect, and similarity

what types of statements should be avoided to elicit as much information as possible?

judgmental statements
evaluative statements
probing statements
hostile responses
false reassurance

what is the main goal in interviewing?

keep the interaction flowing

what is a transitional phrase?

words or noises that are meant to get the subject to continue speaking. if they do not work, use one of the following:
verbatim playback
paraphrasing/restatement
summarizing
clarification response
empathy/understanding

When should direct questions be used in an interview?

when you need specific information or time is running out

what are the advantages and disadvantages of using structured clinical interviews?

Advantages: everyone gets the same questions in the same order, uses specified rules for probing, offer reliability but sacrifice flexibility, frequently used in research
Disadvantages: requires cooperation (super difficult with psychiatry and forensics), relies exclusively on respondent making the assumptions questionable

what is the purpose of a mental status examination?

used to evaluate and screen for psychosis, brain damage, and other major psychiatric and neurological difficulties

what is general standoutishness?

a single quality that you just can't get out of your head, distracts from the interaction

How much higher is interview reliability for structured interviews?

twice as much

what is a major criticism of structured interview?

you get a lot of good information but it is very unflexible and relies on self-report data

what is social facilitation?

acting in such a way as to encourage dialogue
examples: open body language, eye contact, responses to statements being made

what is the largest source of error in interviews?

judgement

what were the three independent research traditions identified by Taylor to study human intelligence?

Psychometric: oldest approach, examines elemental structure of test and test properties through evaluation of correlates and underlying dimensions
Information-Processing: examines process that underlie how we learn and solve problems
Cognitive approach: focuses on how humans adapt to real-world demands

Through what three facilities did Binet believe intelligence expressed itself?

judgement
attention
reasoning

What two major concepts guided Binet?

Age Differentiation
General Mental Ability

What is age differentiation?

differentiating older from younger children by the former's greater capacities

What is mental age?

equivalent age capabilities of a child regardless of their chronological age, obtained through age differentiation

What is general mental ability?

a term that we now call IQ or intelligence

What is positive manifold?

as 'g' increases, performance on all intelligence test and subtests will increase

Binet searched for tasks that could be completed by what percentage of children in a particular age group?

66-75%

What concept did Spearman introduce?

Spearman's 'g', or general intelligence factor

What statistical method did Spearman develop to support his notion of 'g'?

factor analyses

According to the gf-gc theory, what are the two basic types of intelligence?

Fluid: abilities that allow us to reason, think, learn, and acquire new knowledge or information
Crystallized: knowledge/understanding that we have already acquired

How do we calculate IQ using mental age and chronological age?

(mental age/chronological age) X 100

What is a deviation IQ? and how was it used in the Stanford-Binet scale?

2 SDs below the mean = impaired

What is basal?

The minimum amount you can get right before you move up

What is ceiling?

The maximum you will do wrong before you drop a level

What factors did Wechsler focus on that those before him had not?

Nonverbal intelligence and non-intellective factors

What we some criticisms of the Binet Scale by Wechsler?

single score, no consideration of non-intellective factors, no separate test for adults

What is the age range of the Wechsler Scales?

WPPSI-III (2.5-7.5 years)
WAIS-IV (16-90 years)

Why is the inclusion of a point scale a significant improvement? What did a performance scale add?

it made it easier to group items by content
included tasks doing something which takes in more factors than intellectual performance

what are the subtests of the WAIS-IV?

Verbal Comprehension
Perceptual Reasoning
Working Memory
Processing Speed

What are the mean, standard deviation, and range for scaled scores, standard scores, and index scores?

scaled: X=10, SD=3, R=1-19
Standard: X=100, SD=15, R=50-150
Index: X=100, SD=15, R=50-150

How are the IQ scores calculated?

(mental age/chronological age) X 100
OR
sum of subtests

What is verbal comprehension?

crystallized intelligence (vocab, similarities, information)

What is perceptual reasoning?

fluid thinking (block design, matrix reasoning, visual puzzles)

What is working memory?

also fluid, what we hold in our brain (arithmetic, digit span, letter/number sequencing)

What is processing speed?

how quick our brain process information (digit/symbol coding, symbol search)

What is pattern analysis? What are the concerns when using such a method?

evaluates large differences between subtest scaled scores
inconclusive and contradictory, doesn't take individual variability into account very well
DON'T USE

What is a hold subtest?

a test that is not sensitive to cerebral damage

Which subtests are most sensitive to cerebral dysfunction? Which are considered hold subtests?

Those that are sensitive: matrix reasoning, similiarities, and block design
hold subtest: vocab

How would you differentiate the WAIS-IV subtests that measure crystallized intelligence from those that measure fluid intelligence?

The degree of abstractness (crystallized intelligence is not incredibly abstract)

Where do traditional intelligence tests fail in the study "normal" abilities?

they break down at the extremes (when you look at MR, learning disabilities, etc.)

What are the advantages of alternative intelligence tests when compared to Binet and Wechsler?

Can be used for specific populations and special purposes: Sensory limitations Physical limitations Language limitations Culturally deprived people Foreign-born individuals Non-English-speaking people
Not as reliant on verbal responses
Not as dependent on complex visual-motor integration
Useful for screening, supplementing, and reevaluations
Can be administered nonverbally
Less variability because of scholastic achievement

What are the disadvantages of alternative intelligence tests when compared to Binet and Wechsler?

Weaker standardization sample
Less stable
Less documentation on validity
Limitations in test manual
Not as psychometrically sound
IQ scores not interchangeable with Binet or Wechsler

What theme in relation to future intelligence do you notice about infant development tests?

they DO NOT predict future IQ

What is surveillance?

administration of brief standardized tool to aid in developmental disorder identification. (needs additional evaluation, needs surveillance, and needs monitoring)

What is screening?

recognizing risks of developmental delay-
eliciting and attending to concerns
documenting and maintaining developmental history
make accurate observations
identifying risk and protective factors
maintain accurate records and documenting process and findings

Which two infant development tests were discussed in class? What are some disadvantages of each of these tests?

Brazelton:
no norms
bad test-retest reliability
does not predict later IQ
Bayley:
Psychometrics break down at younger ages
Does NOT predict later IQ (but lower scores DO predict MR later in life)

What is sensitivity?

accuracy of test in identifying delayed development (true positive)

What is specificity?

accuracy in identifying individuals who are not delayed (true negative, or avoiding false positives)

What are acceptable sensitivity and specificity levels for developmental screening tests?

70-80%

How is a learning disability currently defined in the school systems? Is this a good method? Why?

Two SDs below the mean. NO because just because you meet that requirement does NOT mean that you have a learning disability. Inversely, it could mean that you never learned math or how to read/write, etc.

For what was the Woodcock-Johnson-III designed?

To aid in the diagnosis of learning disabilities based on discrepancy from IQ

Should test scores be used alone to define developmental or learning disabilities?

Unfortunately, yes, because otherwise there is no standard for diagnosis