CONSTRUCTION OF QUESTIONNAIRES AND RATING SCALES

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/19

encourage image

There's no tags or description

Looks like no tags are added yet.

20 Terms

1
New cards

self-report

respondents to provide answers to a set of questions or scale items that inquire about their personal thoughts, feelings, or behaviors

-requires that we consider variations in responses among respondents as meaningful, and not attributable to mere measurement error

2
New cards

questionnaire

involves a single item or two to assess each construct, and typically is used because participants are unwilling, unable, or unlikely to take part in a longer assessment, or because the issue is one that admits to few options

3
New cards

rating scales

Involve multiple items (to be meaningful, Cronbach's alpha should have a minimum of three items) to assess each construct.

-Typically used by researchers with access to participants more willing to take part in longer assessment

4
New cards

strengths of questionnaires + rating scales

- Time efficiency: Useful when researchers do not have prolonged access to participants.

- Cost-effectiveness: Using brief questionnaires (one or two items per topic) avoids the prohibitive financial costs of adding extra items, especially in large-scale or national surveys.

- Flexibility in administration: They can be delivered in multiple formats, including written self-administered forms, online surveys, or as part of an interview study.

- Ability to cover many concepts at once: Brief questionnaires allow researchers to evaluate numerous different topics simultaneously without creating an overly long instrument.

- Reduced participant fatigue: Keeping instruments brief prevents participant burnout, which helps preserve data quality.

- Well-established best practices: Although there are no strict formal rules, 70 years of social science research has produced a large body of practical "rules of thumb" to guide their construction.

5
New cards

weaknesses of questionnaires + rating scales

- Inability to capture complex constructs deeply: Because brief questionnaires often rely on a single item (or just a couple of items) per topic, they cannot fully capture or "triangulate" a person's views on a target the way multi-item measures do.

- Risk of participant fatigue and lower data quality (when lengthy): If a questionnaire tries to measure many concepts using multiple items, it becomes overly long, leading to participant fatigue and a resulting drop in data quality.

- Lack of formal rules: There are no standardized, formal rules governing questionnaire design, meaning researchers must rely on general rules of thumb rather than rigid guidelines.

6
New cards

double-barreled questions

a single item that asks more than one question at once

7
New cards

Open-ended survey item (vs. closed-ended)

A question that does not constrain answers to a preset list.

(+) strengths: Captures answers outside limited options (e.g., an unlisted candidate) and taps views with higher fidelity/sensitivity.

(-) weaknesses: Requires recall rather than easier recognition (respondents may forget an option they would recognize on a list) and can make coding/analysis costly and complex when answers span many possibilities rather than a narrow, predictable set.

8
New cards

Question order effects in questionnaires (rapport analogue & priming)

Because self-report questionnaires often lack interviewer rapport, item sequence shapes comfort, retention, and responses:

- Threat/rapport ordering: Start with nonthreatening items to build comfort and commitment before asking sensitive/threatening questions (e.g., illicit substance use) or sensitive demographics (income, race/ethnicity, often placed near the end) so participants do not drop out.

- Priming / earlier-item influence: Early answers can prime mindsets and bias later items (e.g., affirming free speech vs. banning hate speech shifting subsequent views on expression).

- Mitigation: Keep nonthreatening items first, and counterbalance or randomize potentially reactive items across participants/blocks (rather than reordering the entire questionnaire).

9
New cards

forced choice measures

forcing respondents to take a stand (albeit a weak stand) on one side or another on the issue, as no "middle" response option is offered

10
New cards

Thurstone's method of equal-appearing intervals

An attitude-scaling approach developed by Thurstone in which respondents endorse the item(s) matching their position while rejecting items that are too extreme or too mild. Items are nonmonotone / noncumulative (endorsing one item does not raise the probability of endorsing others, so scores are averaged rather than summed).

1) item generation

2) judge ratings

3) item selection

4) administration & scoring

11
New cards

reliability (as it related to Thurnstone scales)

reliability = major common indicator of scale quality + internal consistency; is not a meaningful concept in the context of Thurstone scales that made use of the original scoring method (choose your favorite item), because such measures made use of participants' overall responses to only a few items of a scale

12
New cards

coefficient of reproducibility

If an investigator can reconstruct the specific items a respondent checked by knowing the total number of items endorsed: Coefficient of Reproducibility = 1-(Total Errors/Total Responses)

13
New cards

factor analysis as an aid to scale construction

factor analysis helps researchers build and refine scales by revealing how scale items group together around underlying dimensions (multidimensional structure), using two main approaches depending on whether you start with a specific theory:

14
New cards

exploratory factor analysis (EFA)

statistical technique to assess the multidimensional structure of a scale if the researcher does not have hypotheses regarding the number and types of underlying factors that will emerge in the solution

- identify the subset of items that are most strongly correlated with such items forming a factor and allowing for//identifying multiple factors

15
New cards

confirmatory factor analysis

Statistical technique used to assess the multidimensional structure of a scale if the researcher has hypotheses regarding the number and types of underlying factors expected to emerge in the solution

- requires reviewing the relevant prior literature to make informed decisions about how to phrase the items to represent each of the dimensions.

- the investigator must force or specify in advance the proposed number of factors and test whether they fit the respondents' data

- Only items that are hypothesized (without peeking at the collected data) to represent a factor should be allowed to load on that factor

16
New cards

response format

the structured set of answer options or rating anchors presented to participants so they can indicate their endorsement or evaluation of a scale item

17
New cards

number of response options (Categories) in rating scales

Most scales use 5 or 7 categories (with 7-point scales suggested by Malhotra et al., 2009 as offering optimal discriminability), though some use more (e.g., 10-point scales).

(+) Advantages of more options: Provides finer gradations, a more continuous measurement of the construct, and greater sensitivity to subtle differences among respondents (though these gains are relatively minor; Wakita et al., 2012).

(-) Trade-offs of more options: Increases respondent decision time and may exceed people's ability to make meaningful, fine-grained distinctions between adjacent categories.

18
New cards

bipolar vs. unipolar repsonse ratings

Whether a rating scale runs from opposite poles or from absence/low to high, which shapes both usage and recommended option counts:

- Bipolar ratings: Span opposite endpoints (negative to positive, such as strongly disagree to strongly agree on most Likert-type scales, or bad-good on semantic differential scales). Typically use 5-point or 7-point response scales.

- Unipolar ratings: Span from low/none to high (e.g., not at all to very much, or frequency estimates). Generally limited to 5-point response options.

- Flexible dimensions: Some constructs can be framed either way depending on the item—for example, rating a policy from very unjust to very just (bipolar) versus not at all just to completely just (unipolar).

19
New cards

anchor extremity

extremity of endpoint labels

-less extreme: participants tend to select each of the possible options more equally

-more extreme: endpoints are rarely selected and most responses were distributed toward the middle --> less variability in responses

20
New cards

use of revered items in rating scales

Negatively phrased or reversed items are added to counter acquiescence response bias (agreeing with positive statements regardless of content due to agreeableness or desire to please the researcher), with scores reverse-coded in the same direction before computing a composite.

(-) Negatively worded stems demand extra working memory and clash with how people naturally store beliefs, increasing the risk of misinterpretation and introducing more measurement error than they prevent

* Reverse response options instead of stems: Keeping item stems affirmative while alternating the direction of the response anchors yields better internal consistency than mixed stems without increasing cognitive load

*Semantic differential format: Using Osgood-style bipolar adjective ratings provides an effective alternative format that inherently avoids acquiescence response bias