Analysis of Questionnaires

Questionnaire

a set of printed or written questions with a choice of answers, devised for the purpose of a survey or statistical study, main data collection tools in social sciences

item

statement or question and the response alternatives

aim of items

to give an incomplete description of reality

often consist of different scales (coherent, often fixed set of items)

measure not-directly observable items (IQ)

Validity

indicates whether a study’s findings are trustworthy

Reliability

is about precision and repeatability of measurement

Content of questionnaires

Behaviour

Beliefs

Opinions/values/attitudes

Demographics

Features of Questionnaires

1. mode of administration

2. question type

mode of administration

Interactional; with interviewer of self-administered technological; telephone, paper-and-penicil or mixed!

question type

open-ended

close-ended

construct L2

Abstract summary of related characteristics, behaviors, and attitudes (latent).

heuristic approach L2

Moves from nominal definition to dimensions, then to operational definition based on empirical measures.

construct method L2

Deductive method based on theory and nomological network to derive items.

construct method step 1 L2

Start with a conceptual/theoretical framework.

construct method step 2 L2

Develop a nomological network (link construct to related variables).

construct method step 3 L2

Define variables operationally (what to measure, how).

construct method step 4 L2

Test for construct validity: convergent and divergent (via MTMM, CFA, etc).

construct method step 5 L2

Final goal: homogeneous item set covering all construct aspects.

nomological network L2

Theoretical map showing how a construct relates to other constructs and variables.

facet design method L2

Deductive method using mapping sentence and facet classification to ensure content validity.

facet design step 1 L2

Identify essential behavioral features or processes.

facet design step 2 L2

Define facets (e.g., situational, behavioral, answer-type).

facet design step 3 L2

Define elements for each facet (called structs).

facet design step 4 L2

Cross-classify all facets; each unique combo = structuple.

facet L2

A relevant dimension of the construct, like situation or behavior.

struct L2

A specific element within a facet.

structuple L2

A unique combination of struct elements, used to generate scale items.

content validity L2

Extent to which an instrument covers all relevant parts of a construct.

rational method L2

Intuitive method relying on expert or respondent judgment without formal theory.

rational method key feature L2

Relies heavily on face validity (does it "look" like it measures the construct?).

face validity L2

How much an item appears to measure what it claims to measure on the surface.

prototypical method L2

Uses act nomination to identify typical behaviors of a construct (act frequency approach).

prototypical method step 1 L2

Ask a sample to nominate typical behaviors (acts).

prototypical method step 2 L2

Large sample rates how typical each behavior is.

prototypical method step 3 L2

Select most typical behaviors as items for your construct.

prototypical method key goal L2

Represent the most prototypical expressions of a construct in measurement.

process validity L2

Whether behaviors used in measurement reflect real-world processes.

internal method L2

Data-driven method using empirical item relations (e.g., factor analysis, Cronbach's alpha).

internal method step 1 L2

Select large number of items from item banks or existing questionnaires.

internal method step 2 L2

Compute correlations between items.

internal method step 3 L2

Use factor analysis to identify homogeneous item clusters.

internal method step 4 L2

Assess internal consistency using Cronbach's alpha, item-rest correlations.

factor analytic method L2

Technique to derive latent dimensions based on inter-item correlations (EFA, PCA).

external method L2

Items are selected based on their relationship with an external criterion (e.g., job performance).

external method step 1 L2

Define relevant external criterion (must be practical or theoretically grounded).

external method step 2 L2

Collect a heterogeneous set of items related to that criterion.

external method step 3 L2

Select items that correlate highly with criterion, but low with each other.

criterion validity L2

Association between scale scores and an external criterion variable.

concurrent validity L2

Correlation with a criterion measured at the same time.

predictive validity L2

Correlation with a criterion measured in the future.

formulating questions L3

Principle of writing items clearly, precisely, and without bias or complexity.

no interpretation questions L3

Avoid items that require reading or interpretative skills above age 12.

short and simple wording L3

Use short sentences and easy words, like contingency items.

double-barreled questions L3

Avoid asking about two things in one question.

leading questions L3

Avoid questions that suggest a particular answer.

loaded questions L3

Avoid items that assume something untrue or controversial.

double negatives L3

Avoid confusing negations like "not unhappy".

indicative vs contra-indicative L3

Use a balance of positively and negatively phrased items.

same meaning for all L3

Ensure words mean the same thing to every respondent.

conditional first L3

Start questions with the conditional phrase (if necessary).

conformity bias L3

Tendency to answer in socially desirable ways.

non-response bias L3

Failure to respond to certain items or entire survey.

acquiescence L3

Tendency to agree (or disagree) regardless of item content.

nay-saying L3

Tendency to answer all items negatively regardless of meaning.

mutually exclusive L3

Answer options should not conceptually overlap.

collectively exhaustive L3

Answer options must cover all possible categories.

logical order L3

Place alternatives in a meaningful sequence (e.g. − to +).

same response options L3

Use consistent response formats across items.

avoid open-ended L3

Prefer closed-ended questions for clarity and analysis.

avoid multiple answers L3

Prefer single-option responses to avoid ambiguity.

bipolar answers L3

Measures intensity in both directions (e.g., strongly disagree ↔ strongly agree).

unipolar answers L3

Measures intensity in one direction from zero to maximum.

symmetry L3

Equal positive and negative answer options around a neutral midpoint.

balance L3

Equal intervals between answer options to allow meaningful averaging.

introductory part L3

Introduces the questionnaire, explains purpose, time, and confidentiality.

instructions section L3

Explains how to fill in sections or respond to specific types of items.

order of questions L3

Organize by topic and avoid fatigue or response bias.

layout L3

Design should be clean, readable, and use helpful formatting cues.

concluding part L3

Thanks participants and ends the questionnaire appropriately.

likert scale L3.5

A group of Likert items forming one continuous scale.

likert item L3.5

A statement the respondent evaluates using fixed response options.

covariance L3.5

Measures linear relationship strength between two quantitative variables; unstandardized.

covariance formula L3.5

cov(X,Y) = Σ(xi - x̄)(yi - ȳ) / (n - 1)

standardization L3.5

Needed because covariance depends on unit of measurement.

pearsons r L3.5

Standardized index of linear association between two continuous or dichotomous variables.

r range L3.5

Pearson’s r ranges from −1 to +1.

low correlation meaning L3.5

Low r values indicate high scatter around the regression line.

ecological fallacy L3.5

Mistakenly drawing conclusions about individuals based on group-level data.

linear combination L4

Sum of variables each multiplied by a constant (e.g., ax + by + cz).

univariate outlier L4

Extreme value on one variable (X or Y).

bivariate outlier L4

Unusual X-Y combination (e.g., rich + unhappy = Bill Gates).

restriction of range L4

Narrow value range for X or Y; lowers r.

sample selection bias L4

Biased sample causes restricted range.

group-specific correlation L4

Correlation differs within groups vs. total sample.

pearson r formula L4

r = cov(X,Y) / (sx * sy).

linear relation only L4

Pearson’s r only detects linear, not curved trends.

measurement error L4

Random error reduces reliability, thus lowers r.

low reliability L4

Leads to low observed correlation (attenuation).

dimensionality reduction L4

Reduces number of variables while preserving info.

factor analysis L4

Method to find latent structures; superordinate term.

help in minimizing information loss and dimensionality reduction and are used to examine the internal structur