construct validity - class 7

When we use criterion validity (3)

Criterion validity involves: Comparing a test to a direct, objective outcome (criterion).

ex. Correlating an academic aptitude test with GPA to show the test predicts academic success.

This works when the construct has a clear, observable indicator.

problem with criterion validity (3)

problem: some constructs lack direct indicators (not all constructs have obvious or objective criteria.

Ex. platonic love, self-esteem or resilience

You can’t point to a single measurable outcome (*criterion) to validate the test.

solution to problem with criterion validity/when to use solution (3)

When there is no direct criterion:

Solution: Use Construct Validity Instead

• You cannot rely on one correlation/*direct criterion to demonstrate validity.

• Instead, you accumulate multiple sources of evidence.

process of construct validation (3 main ideas)

process of construct validation: accumulating information indicating that the test acts as its supposed to act given the construct definition

*Construct validation is the process of gathering evidence to show that a test behaves as the construct theory predicts

when to use criterion VS construct validity (2)

Criterion validity works when the construct has a clear outcome *criterion (e.g., GPA, job performance).

Construct validity is used when there's no direct criterion—you build evidence over time to show the test measures what it claims to measure.

what are psychoglical constructs? (4 characteristics)

1)abstract concepts used to refer to psychological attributes (e.g. intelligence, love, beauty)

2)exist in theory; generally latent; not directly observable

3)important in describing or understanding human behaviour

4)can observe & measure the behaviours that show evidence of these constructs *LAMB

1)psychological constructs as abstract (3)

1. Abstract, Theoretical Concepts

Constructs are ideas used to describe psychological attributes (e.g., intelligence, love, beauty, extroversion).

They exist in theory, not as physical or directly observable things.

2)psyc constructs as generally latent (4)

2. Latent and Unobservable

Most psychological constructs are latent → they cannot be seen or measured directly.

We infer their existence because:

• Certain behaviors tend to occur together, or

• Behaviors can be explained by a common underlying attribute.

Ex. People who are outgoing, talkative, and sociable are thought to have the latent trait of extroversion.

3)psyc constructs as important (3)

3. Why Constructs Matter

Constructs are important because they:

• Explain human behavior

• Help us organize and make sense of psychological traits

• Allow researchers and clinicians to predict behavior and develop theories

4)observe & measure the behaviours that show evidence of these constructs (2)

4. Role of Psychological Tests

Psychological tests are designed to measure these unobservable constructs.

Since the constructs can’t be observed directly *they’re latent, tests measure the behaviors, responses, or patterns that reflect them.

Ex. We can’t see neuroticism, but we can observe behaviors like anxiety, emotional instability, or worry and use those as indirect measures.

identifying latent constructs (4)

1. Observable Evidence of Constructs: For constructs, we rely on observable signs/behaviors that indicate the presence of the construct.

2. Latent Constructs: Latent constructs = not directly observable.

Require extra steps to assess and validate tests that aim to measure them.

 3. Analogy: Gravity. Gravity itself is not directly observable. We infer its existence because it explains observable events (e.g., apples falling).

 4. Application to Psychology. Same logic applies to psychological constructs (e.g., intelligence, love, motivation).

We can’t “see” them directly, but we observe behaviors or responses that point to their existence

construct explication (3)

construct explication: how a particular construct is manifested and how such manifestations can be measured

*To gather evidence for a construct validity for our test (construct validitation) construct explication:

def: explain/articulate construct as thoroughly as possible, what our constuct is, how it is manifested and how this manifestation is related to manifestations of other constructs

‘3 steps of construct explication’

3 steps of construct explication

step 1: identify behaviours that relate to construct

step 2: identify other related constructs

step 3: identify behaviours related to other constructs

3 steps in process of construct explication (ex. explicating dominance using extroversion) (6 ideas)

Step 1: Identify Behaviours That Represent the Construct. To define a latent construct like dominance, we look for observable behaviours that tend to co-occur in the same person. (Ex. Taking the lead, giving orders, speaking first)

When these behaviours consistently appear together in someone, we infer that the person has high levels of dominance—this is how we explicate what the construct means.

Step 2: Identify Related Constructs. Next, we consider other constructs that dominance is theoretically related to. In this case: Dominance is theorized to be related to extraversion.

 Step 3: Identify Behaviours for the Related Construct. To define and assess extraversion, we again look at observable behavioural indicators, such as: Seeking social interactions, having a large social network

When these behaviours consistently co-occur within an individual over time, we infer high levels of extraversion. Because these behaviours are believed to reflect the same trait, we expect these behaviours to be correlated with one another.

 Step 4: Formulate the Hypothesis. If: Dominance is measured through its behavioural indicators (Step 1), and Extraversion is measured through its own behavioural indicators (Step 3), then we should expect scores on those two measures to be correlated, based on the theoretical relationship between the constructs.

gathering evidence of construct validity (2)

2 types of evidence we can gather

Step 1: Theoretical Evidence Stage

Define the construct.

Map out its relations to other constructs (nomological net).

Specify expected patterns. *hypothesis

Step 2: Evidence-Gathering Stage

Collect various types of empirical evidence (above) to support the construct validity of the test.

nomological net (def)

Definition:
A nomological net is a network of psychological constructs whose inter/relationships are described by theoretical laws or rules.

Step 1 of process of gathering theoretical evidence (3)

Step 1: Establish the Nomological Net.

Identify all constructs theoretically related to the target construct.

This forms a nomological net, which is like a conceptual map showing: which constructs relate to the target construct, how they are related and the laws or rules (e.g., correlations, directional predictions) that describe these relations.

‘law’ in nomological net (ex. dominance and extraversion) (3)

Ex. You theorize that extraversion is positively related to dominance.

This becomes a "law" in your nomological net: As extraversion increases, dominance increases (positive correlation).

In the nomological net, the constructs extraversion and dominance are linked by that positive relationship.)

Step 2 of process of gathering theoretical evidence (3)

Step 2: Propose Experimental Hypotheses. Once the nomological net is established, you move into the theoretical-to-empirical phase.

You generate testable experimental hypotheses based on the relationships described in the nomological net.

These hypotheses are used to design studies that produce evidence for construct validity.

purpose of 2 steps to gather evidence of construct validity (2)

The theoretical foundation (nomological net) guides what relationships should exist. *btwn our constructs

The hypotheses allow you to collect evidence to support or refute whether your test truly measures the construct (i.e., construct validity). *allows us to collect evidence of whether there is construct validity

3 pts of nomological net (gather theoretical evidence)  (4 ideas)

A nomological net is made up of three components:

1)Constructs. These are the theoretical psychological attributes (e.g., extraversion, dominance).

2)Observable Manifestations: These are the measurable behaviours or indicators that *indirectly represent each construct.

3)Relations. The relationships:

Within constructs and their behaviours (e.g., how the behaviours represent the construct *within itself/the construct).

Between different constructs and their behaviours (e.g., how behaviours representing different constructs are related).

3 pts of nomological net (ex. job satisfaction & workload) *4 ideas

1)constructs: we hypothesize are related are job satisfaction and workload

2)their observable manifestations: job satisfaction can be observied via smiles, productivity and postive feedback VS workload can be observied via number of tasks, deadlines and overtime hours 

3)the relations within and between (1) constructs and (2) their observable manifestations:

within: postive feeback related to productivity, number of tasks is related to overtime hours

between: workload is negatively related to job satisfaction—we expect that as workload goes down, job satisfaction goes up.

is there one true form of validity? (2)

Some researchers view construct validity as the umbrella or overarching form of validity, meaning it includes other types of validity within it.

Under this perspective, construct validity is the only true form of validity, and all other forms (e.g. content) are simply different types of evidence that support it *construct validity

gathering psychometric evidence (3)

Once the theoretical stage is complete (i.e., you've defined the construct and mapped its relations to other constructs in the nomological net, proposed hypothesis):

you move on to the psychometric evidence-gathering stage to support the validity of the test.

6 Types of Psychometric Evidence/ Collected Under Construct Validity *methods for establishing quantitative evidence that a test has construct validity

6 Types of Psychometric Evidence Collected Under Construct Validity (gathering psychometric evidence)

Under the all-encompassing view of validity, you collect different kinds of evidence to show your test accurately measures the construct: *test has construct validity

1)Content Validity *evidence of validity based on test content

2)Criterion Validity *evidence of validity based on relationships with external criteri

3)Reliability *evidence of validity based on reliabilty

4)Experimental Interventions

5)Convergent and Discriminant Evidence

6)Multitrait-Multimethod (MTMM) Designs

*methods for establishing quantitative evidence that a test has construct validity

1)evidence of validity based on content (3)

*Do the items fully and appropriately represent the construct?

no construct underrepresentation: does *the content on* the test sample adequately from the construct domain?

no irrelevant construct representation: does the *the content on* test properly exclude content that is unrelated to the construct?

2)evidence of validity based on criteria (2)

are the relations of the test with the external criteria *the criterion* as would expected based on theory? 

*RE: Includes predictive and concurrent validity.

3)evidence of validity based on reliabilty (3)

e.g. test-retest/internal consistency not too low or too high given the construct

*Consistency of test scores over time, across items, or across raters.

*Considered part of validity because a test can't be valid if it's not reliable.

4)evidence of validity based on experiemental interventions (3)

situational changes should influence test scores based on theory (e.g. education influencing scores on an achievement test, medication influencing scores on an anxiety test)

*You manipulate the construct the test is supposed to measure.

*If the test is valid, scores should change when the construct is increased or decreased.

5)convergent validity (2)

test scores correlate with:

other measures of the same construct

OR measures of constructs to which the test should be related based on theory (think nomological net)

Measures of different constructs that, according to theory, should be related to it our construct of interest (i.e., based on the nomological network). ex. dominance and extroversion should have high correlation btwn scores

5)discriminant valdity

test scores are uncorrelated with measures of construct to which the construct should not be related based on theory (think nomological net)

*a test should have low correlations with measures of unrelated constructs or evidence for what the test does NOT measure 

6)MTMM (2)

*Used to assess both convergent and discriminant validity.

*Examines multiple traits across multiple measurement methods.