W2 L2 & L3: The Process of Scale Construction Part 1; Rational/ Deductive Approach

look more into the difference between dimensionality and breadth, conceptually, and equationally

what are the steps of scale construction (part 1?)

1. 1.Examine the literature

2. 2.Think about the purpose

   1. steps one and two looks at how its important, how does it add

3. 3.Define your construct

4. 4.Specify the domain of your construct

   1. steps 3 and four look at how should it relate/not relate to other constructs; what is the content and how does the construct manifest

when you are specifying the domain of your construct, what are additional considerations at these steps?

•Scale Breadth

•Scale Dimensionality

•Inter-item correlations (review correlation)

STAR what does reliability look at and what does validity look at?

• reliability: consistency of test scores

  • means that the scores are repeatable

• validity: meaning of test scores

  • means that test scores assess the construct we intended

conceptually, what does a scale with high reliability and high validity do?

• high reliability: tells us the scale measures something

  • can say scale is precise in assessing something

• high validity: tells us what the scale is assessing

STAR what are different ways to estimate reliability?

• Test-rest

• Parallel (equivalent) forms

• Internal consistency

  • Split half

  • Cronbach’s alpha (α)

• Interrater reliability

STAR what are different forms of validity evidence?

• •Face

• •Content

• •Criterion-related

• –Concurrent & predictive

• •Construct Validity

• –Convergent and discriminant

what does a completed scale (measure) have?

• conceptual definition of the construct

• operational definition:

  • items

  • response categories to answer the items

  • instruction set

    • sometimes has interpretation aids, scoring criteria, norms, standards for interpreting test scores

what is the current best approach to scale construction?

construct driven approach/ rational approach/ deductive approach

what is the construct driven approach/ rational approach/ deductive approach?

• construct driven approach = rational approach = deductive approach

  • theory - construct definition - items

what is step one in the rational approach? and things to consider?

examine literature and past scales

WHY AND HOW THE CONSTRUCT SHOULD MATTER

what already exists? what is known about the construct? how does it relate to and differ from other constructs, how has it been measures, how is the measure different from and better than past conceptualizations

UNIMPORTANT what is a research example of step one in the construct- drive approach?

• organizational commitment

• Historically defined as: An emotional attachment/identification

  
  

  Later scholars argued that it there is more to it

  1. Emotional attachment (want to stay)

  2. Recognition of costs of leaving (can’t leave)

  3. Felt obligation (ought to stay)

  ●

  Developed a theory about how these components would predict work attitudes and behaviours differently

what is step 2 of the construct driven approach, and things to consider?

identify primary purposes

–What diagnostic decisions can be made?

–What kind of research is enabled by the measure? (experimental, correlational)

•What age group?  What educational group? Other important characteristics?

what is step 3 of the construct driven approach, and things to consider?

construct definition

and nomological network- constructs that should be positively, negatively, or not correlated

what high and low levels of the construct mean

focus on what it is positively, negatively related and unrelated to, incase i have to deduce this from a similar question on the exam

what is a nomological network, and why is it important for?

helps to ensure construct validity

what is step 4 of the construct driven approach, and things to consider?

define the domain of the construct

how should the construct manifest itself? (ie. thoughts, behaviours, feelings, etc.)

create a test specifications table for content validity

what is a test specifications table, and what does it help ensure?

help ensure scale adequately taps the construct

helps ensure content validity

elaborate on an example of a test specifications table in the realm of quality of marital relations

elaborate on an example of an achievement test testing to assess knowledge of psych 492

make sense of these two slides

what are some additional considerations for step 4 of the construct driven approach?

if you’re measuring the construct within specific contexts or generally

• Context-specific (domain specific)

  •relationships (e.g., I feel useless in relationships)

  •university (e.g., I feel useless as a student)

• General (nondomain specific)

  •items do not reference a context (e.g., I generally feel useless)

  
  

what are two key issues to keep in mind for step 3 & 4 of construct driven approach?

• scale breadth

• scale dimenstionality

• inter-item correlations

what is scale breadth?

how narrow or broad should your scope be

broad scale means things are heterogeneous (all ask about different things); narrow means homogeneous (all ask about the same thing)w

what is an example of broad scales?

what are example of narrow scales?

what are the statistical implications for inter-item correlations?

• the narrower the scale, the higher the inter-item correlations should be (positively correlated)

  • inter-item correlation matrix is used to assess

• narrower scales usually have higher internal consistency (alpha), but broad scales may predict more attitude/ behaviours - correlate with more

• Homogenous items (narrow measure)

  • –Responses will be more highly positively correlated

  • –Higher “inter-item” correlations

• Heterogeneous items (broad measure)

  • –Responses will be less highly positively correlated

  • –Lower “inter-item” correlations

regardless of scale breadth, what reduces the potential influence of measurement error?

having multiple items

what is correlation?

• measure of association (relation) between two variables (direction, magnitude)

• ranges from 0 to 1

  • can be + or -

what is the equation for correlation?

what does positive, negative, and zero correlation look like on a graph?

what is an example of narrow and broad inter-item correlation? what is the scripture (yk what im trying to say) for inter-item correlations?

rij

is narrow or broad inter-item correlation better?

• •Both have benefits

  • –one is not better than the other

• •Depends on the purpose of your scale

  • –If interested in understanding behavior in a particular context, then develop a narrow measure that is relevant to that context

  • –Alpha reliability will be higher all else equal

what are the pros of narrow scales?

• •Usually has higher internal consistency estimate of reliability (Cronbach’s alpha)

• •Similar items, so we can expect

  • –Higher correlations among the items

  • –So, scale is more “internally consistent”

what is cronbach’s (coefficient) alpha?

it is a function of:

• –Average inter-item correlation &

• –Number of items

what is the formula for cronbach’s (coefficent) alpha?

what is the pros of broad?

• •May predict more attitudes and behaviours

• •Construct is assessed broadly, so we can expect the scale to

  • –Correlate with more variables in later research

STAR- LOOK AT TEXTBOOK FOR CLARIFICATION what is the bandwidth-fidelity trade off?

what are we looking at when we look at scale dimensionality?

• •Multidimensional (vs. Unidimensional)

  • multidimensional Designed to tap multiple dimensions (dimensions/factors/components)

  • multidimensional Will have “subscales,” each with own definition

  • –Homogeneous items within these sets STAR- REVIEW AND MAKE SENSE OF THIS!

STAR- REVIEW AND MAKE SENSE OF THIS!what does scale dimensionality implications for inter-item correlations?

there should be clusters of inter-item correlations (items for one dimension should be strongly positively correlated)

•Should find “clusters” of correlated items

–Some items highly related with each other,

–But less related to other groups of items

what does the inter-item correlation look like for two dimensions?

STAR- MAKE SENSE OF THIS, AND ALL THE OTHER ILLUSTRATION TABLES FROM THIS LECTURE

STAR- REVIEW AND MAKE SENSE OF THIS! What are implications for correlations among the dimensions?

a broad unidimensional scale CAN have lower reliably alpha than a multidimensional scale (but narrow will be higher)

ideally low correlations

true or false: broad =multidimensional

FALSE

REVIEW? JUST MAKE SENSE OF WHAT THE EXAMPLE IS POINTING OUTWhat is an example of dimensions and scales in the three component model of organizational commitment?