Analysis of Questionnaires

Questionnaire

a set of printed or written questions with a choice of answers, devised for the purpose of a survey or statistical study, main data collection tools in social sciences

item

statement or question and the response alternatives

aim of items

to give an incomplete description of reality

often consist of different scales (coherent, often fixed set of items)

measure not-directly observable items (IQ)

Validity

indicates whether a study’s findings are trustworthy

Reliability

is about precision and repeatability of measurement

Content of questionnaires

Behaviour

Beliefs

Opinions/values/attitudes

Demographics

Features of Questionnaires

1. mode of administration

2. question type

mode of administration

Interactional; with interviewer of self-administered technological; telephone, paper-and-penicil or mixed!

question type

open-ended

close-ended

construct L2

Abstract summary of related characteristics, behaviors, and attitudes (latent).

heuristic approach L2

Moves from nominal definition to dimensions, then to operational definition based on empirical measures.

construct method L2

Deductive method based on theory and nomological network to derive items.

construct method step 1 L2

Start with a conceptual/theoretical framework.

construct method step 2 L2

Develop a nomological network (link construct to related variables).

construct method step 3 L2

Define variables operationally (what to measure, how).

construct method step 4 L2

Test for construct validity: convergent and divergent (via MTMM, CFA, etc).

construct method step 5 L2

Final goal: homogeneous item set covering all construct aspects.

nomological network L2

Theoretical map showing how a construct relates to other constructs and variables.

facet design method L2

Deductive method using mapping sentence and facet classification to ensure content validity.

facet design step 1 L2

Identify essential behavioral features or processes.

facet design step 2 L2

Define facets (e.g., situational, behavioral, answer-type).

facet design step 3 L2

Define elements for each facet (called structs).

facet design step 4 L2

Cross-classify all facets; each unique combo = structuple.

facet L2

A relevant dimension of the construct, like situation or behavior.

struct L2

A specific element within a facet.

structuple L2

A unique combination of struct elements, used to generate scale items.

content validity L2

Extent to which an instrument covers all relevant parts of a construct.

rational method L2

Intuitive method relying on expert or respondent judgment without formal theory.

rational method key feature L2

Relies heavily on face validity (does it "look" like it measures the construct?).

face validity L2

How much an item appears to measure what it claims to measure on the surface.

prototypical method L2

Uses act nomination to identify typical behaviors of a construct (act frequency approach).

prototypical method step 1 L2

Ask a sample to nominate typical behaviors (acts).

prototypical method step 2 L2

Large sample rates how typical each behavior is.

prototypical method step 3 L2

Select most typical behaviors as items for your construct.

prototypical method key goal L2

Represent the most prototypical expressions of a construct in measurement.

process validity L2

Whether behaviors used in measurement reflect real-world processes.

internal method L2

Data-driven method using empirical item relations (e.g., factor analysis, Cronbach's alpha).

internal method step 1 L2

Select large number of items from item banks or existing questionnaires.

internal method step 2 L2

Compute correlations between items.

internal method step 3 L2

Use factor analysis to identify homogeneous item clusters.

internal method step 4 L2

Assess internal consistency using Cronbach's alpha, item-rest correlations.

factor analytic method L2

Technique to derive latent dimensions based on inter-item correlations (EFA, PCA).

external method L2

Items are selected based on their relationship with an external criterion (e.g., job performance).

external method step 1 L2

Define relevant external criterion (must be practical or theoretically grounded).

external method step 2 L2

Collect a heterogeneous set of items related to that criterion.

external method step 3 L2

Select items that correlate highly with criterion, but low with each other.

criterion validity L2

Association between scale scores and an external criterion variable.

concurrent validity L2

Correlation with a criterion measured at the same time.

predictive validity L2

Correlation with a criterion measured in the future.