Scale Development

Overview of Psychological Constructs and Measurements * Psychological study involves examining the relationships between psychological experiences, such as whether a materialistic orientation predicts lower subjective well-being or if early life cognitive resources are negatively related to the age of dementia onset. * Direct testing of relationships between psychological constructs (A and B) is impossible; instead, researchers test the relationship between measures of A and B, assuming this reflects the true relationship between the constructs. * Measures act as proxies for constructs that cannot be directly assessed. * For psychological claims to be valid, measures must correspond accurately to their intended constructs. * The Bible, Proverbs $11:1$, notes: "Dishonest scales are an abomination to the Lord […]". # The Nature of Psychological Constructs * Psychological constructs are abstract and often do not permit direct observation. * As noted by Flake and Fried ($2020$, p. $3$), while we can observe physical traits like height, we have limited insight into an individual's psychological processes, states, and attributes. * This abstraction makes establishing a clear correspondence between a construct and its measurement particularly challenging. # Latent and Observed Variables * Constructs are defined as latent variables because they are not directly observed. * Measures used to capture these constructs are known as observed variables. * It is assumed that the latent variable causes the scale scores; for example, a participant will highly agree with items on the Rosenberg self-esteem scale because they possess high underlying self-esteem. * Sample items from Rosenberg's ($1965$) Self-Esteem Scale include: - $1.$ On the whole, I am satisfied with myself. - $2.$ I am able to do things as well as most other people. - $3.$ I feel that I'm a person of worth. - $4.$ I take a positive attitude toward myself. * Common types of psychological measures include: - Behavioural observation. - Cognitive tasks. - Physiological measures. - Brain scans. - Self-report scales. # Advantages and Disadvantages of Self-Report Scales * Disadvantages: - Dependency on participant memory; responses may be skewed by recency effects. - Influence of self-presentation concerns; individuals are motivated to present themselves in a positive light to themselves and others. - Influence of response styles; individuals might exhibit an acquiescent response style, agreeing with items regardless of content. * Advantages: - Information richness; the participant has more access to internal information than any observer. - Motivation; people are generally interested in themselves and willing to invest time in self-reflection. - Ease of administration; these scales are inexpensive, fast, and simple to set up, code, and analyze. # Phases of Scale Development (Flake et al., 2017) * Scale development is categorized into three primary phases: - $1.$ Substantive Phase: Involves construct conceptualization, literature review, and generating items. - $2.$ Structural Phase: Involves item analysis, determining dimensionality, and assessing reliability. - $3.$ External Phase: Involves convergent and discriminant validity. # The Substantive Phase: Construct Conceptualization and Literature Review * Key activities involve defining the boundaries of the construct (what it is and what it is not). * Type of Psychological Experience: Determining if the scale captures attitudes, values, or behaviors. For example, vegetarian behavior ("I generally avoid eating meat") is distinct from vegetarian identity ("I identify as a vegetarian"). Research shows that in the UK and US, vegetarian identity is constant while meat consumption has decreased, suggesting these are distinct constructs. * Construct Specificity: Determining the level of measurement specificity. For example, "Global meaning in life" is a separate construct from "Meaningful work." According to the specificity-matching principle (Swann et al., $2007$), variables with matched levels of specificity relate more strongly. # Generating Items and Measurement Format Decisions * Generating items is an iterative process involving brainstorming, format decisions, and quality-control reviews. * Direct vs. Indirect Items: - Direct: Uses the name of the construct (e.g., "I have high self-esteem"). - Indirect: Obscures the construct name to avoid social desirability bias (e.g., a narcissism scale asking about leadership: "People see me as a natural leader"). * Positive and Reverse-Worded Items: - Reverse-phrased items express the opposite of the construct (e.g., "I tend to devalue myself" for a self-liking scale). - These help detect acquiescent response styles or careless responding. # Principles of Response Scales * Discrimination: The goal is to ensure that score differences are proportional to differences in the construct. If participant A scores $4$ and B scores $2$ on spider fear, A should be twice as afraid as B ($4\text{ cm}$ vs $2\text{ cm}$ analogy). * Number of Scale Points: More options provide more variability, which is necessary for modeling variance. However, too many options lead to fatigue and boredom. Scales typically use between $4$ and $10$ response options. * Odd versus Even Numbered Scales: - Odd scales (e.g., $5\text{-point}$, $7\text{-point}$) allow for mid-points/neutral positions. - Even scales force a commitment in one direction. - Mid-points can be misused by participants to avoid deliberation. * Symmetry and Labeling: Scales must be symmetrical around the mid-point. Response options should be exhaustive and mutually exclusive. Avoid overlapping labels like "occasionally" and "some of the time." # Item Brainstorming and Redundancy * Comprehensive coverage of the construct is essential. * It is recommended to generate as many as $3$ times the final intended number of items (DeVellis & Thorpe, $2022$). * Good Redundancy: Using different grammatical structures and vocabulary to express the same idea (e.g., "I will do almost anything to ensure my child's success" vs. "No sacrifice is too great if it helps my child succeed"). * Bad Redundancy: Items that are too similar (e.g., "A really important thing…" vs. "The really important thing…"). # Reviewing and Refining Items * Avoid Ambiguous Terminology: Avoid jargon like "spendthrift" or acronyms like "UI" (User Interface). Avoid words with multiple meanings like "sanction" (which can mean to authorize or to penalize). * Clarity Rules: - Avoid double negatives (e.g., "Are you against a ban on smoking?"). - Avoid double-barrelled items (e.g., "I play video games and board games" should be split). - Avoid direct negation using "no" or "not"; use antonymic expressions/reverse-phrasing instead (e.g., "I like staying in" instead of "I don't like going out"). # Questions & Discussion * A student note on Slide $20$ highlights a comment by Vlad: "You spend two years at Uni learning about data analysis. Then Vlad casually says it’s all just about modelling variance." # Theoretical Definitions and Scale Examples * Self-Competence: "The valuative experience of oneself as a casual agent, an intentional being that can bring about desired outcomes through exercising its will." Items: "Satisfied with myself on a whole", "At times I think I'm no good at all (R)". * Self-Esteem: "One's positive or negative attitude towards oneself." Items: "Almost always able to accomplish what I try for", "Sometimes deal poorly with challenges (R)". * Self-Concept Clarity: "Extent to which the contents of an individual's self-concept are clearly and confidently defined, internally consistent and temporally stable." Items: "Beliefs about myself are often conflict with one another (R)", "General clear sense of who I am". * Narcissism: "A clash between grandiose identity and self-concept." Items: "People see me as a natural leader", "I know I'm special because everyone keeps telling me so", "I am an average person (R)". * Scale of Physical Appearance Positivity (Set I & II): Items: "I am confident with the way my body looks", "I compare myself to others (R)", "When people take photos of me I generally like them", "I feel good about myself when I'm wearing a nice outfit". (R) denotes reverse-phrased items.