SRM midterm (weeks 1-3)

research design

framework for the collection + analysis of data

how to evaluate a research design? (3 core criteria)

1. RELIABILITY

2. REPLICABILITY

3. VALIDITY

research method

technique for collecting data

variables + types (3)

attributes on which cases vary

• independent (what you manipulate)

• dependent (what changes as a result)

• constant

reliability? (4)

• are measurements consistent (trustworthy)?

• stability over time

• internal reliability

• inter-observer consistency (do researchers agree)

replicability? (2)

• is the procedure explained clearly enough so it could be replicated?

• achieved by being open with methods; transparency

validity?

how accurate are the findings

types of validity (4)

1. measurement/construct (are you measuring what you think you’re measuring?)

2. internal (how unambiguously is A the cause of B?)

3. external (can results of the study be generalised to a larger population?)

4. ecological (are findings meaningful to everyday life?)

theory

system of logically coherent, in particular non-contradictory, statements about a part of reality

building blocks of theory (4)

• concept (abstract term describing phenomenon)

• law (regularity of facts; observations)

• postulate (basic assumption)

• proposition (explanatory relationships between concepts)

causality (2)

• the relationship between cause and effect

• e.g you flip a switch, the light turns on

hierarchy of theory (4)

• grand theories (abstract, very large)

• middle range theories (useful for empirical, limited space)

• background literature (focused search for theories, empirical, limited)

• collection of facts (hardly any theory)

induction (3)

• verification logic

• empirical → theoretical

• from a limited number of observations, a universal statement is made about all cases

induction cycle

deduction (3)

• falsification logic

• theoretical → empirical

• from general statements, predictions about observations are made

deduction cycle

induction problem + solutions (3)

• how many observations are needed?

• solutions

  • use falsification; much stronger

  • use probability; statistics

classical experiments (4)

• randomized controlled trial/randomized experiment

• control group + experimenal group

• random assignment

• before-after analysis

laboratory experiments (2)

• researcher has more control, less real life

• difficulty in external + ecological validity

quasi experiments (3)

• NO random assignment of participants (not possible/ethical)

• real-world settings (however manipulated)

• high ecological validity, low internal

cross-sectional design (4)

• most often surveys

• collecting data on a sample of cases at a single point in time

• examine direct patterns of association between variables

• correlation rather than causation, no manipulation, quan + qual

longitudinal design (3)

• cross-sectional but more than once over time

• hard to replicate due to scale + cost

• high internal validity

case study design (2)

• detailed + intensive analysis of a single case

• often both qualitative + quantitative research

5 types of case studies

• critical (theory + hypothesis)

• extreme/unique (common in clinical studies)

• representative/typical (exemplifying)

• revelatory (observe phenomena unable to explore before)

• longitudinal (opportunity to explore more than once)

comparative design (2)

• 2 contrasting cases using ± identical methods

• cross-cultural/national research

3 general principles for designing questions

1. always bear in mind research question

2. what exactly do you want to know?

3. how would you yourself answer it?

conditions for causality (4) !

1. cause occurs before effect

2. ‘proximity’ in time and space

3. correlation between variables

4. correlation cannot be spurious

why do quantitative research? (4)

1. show relations between variables

2. describing relations as cause + effect (causality)

3. explaining social phenomena

4. doing predictions

steps in quantitative research (9)

1. problem

2. theory

3. hypothesis

4. research design

5. operationalism

6. selecting location/method/sample

7. data collection

8. coding + analysing data

9. writing report

concepts (2)

• categories for the organisation of ideas + observations

• ‘shortcut’ for explanation (of social phenomena)

what does conceptualising mean?

making vague concepts/ideas more concrete + precise

what is an operational definition? (2)

• providing a measure of a concept

• using indicators (when the concept is less directly quantifiable)

nominal variables?

unordered categories (can’t be averaged)

ordinal variables?

ordered categories (distance between them unknown)

interval scale?

ordered + equal distances between values (LIKERT!)

ratio scale?

scale with zero point

dichotomous categories?

2-value answers (eg. yes/no)

what is operationalism?

making concepts measurable

4 forms of operationalism

1. theory-driven (RQ based on literature + theory)

2. data-driven (starts with data, find useful variables to construct ideas)

3. brainstorm-driven (what questions are we going to ask? how? to whom?)

4. combination

how to test measurement validity? (4)

1. face validity (measure apparently reflects content of concept; intuitive)

2. content validity (how well measurement covers all concepts its supposed to)

3. criterion validity (concurrent, predictive, convergent)

4. construct validity (correlation with theoretical expectations)

hypothesis (2)

• informed speculation based on existing research literature

• null hypothesis/alternative hypothesis

main methods of data collection in quantitative research (4)

• questionnaires + surveys

• content analysis (documents + texts)

• secondary data analysis → new possibilities due to big data

• structured observation (observing + recording behaviour)

likert scale (4)

• multiple-indicator measure

• part of surveys; level of agreement with statement

• each item is assigned a number; scores are aggregated

• items must relate to same object + be interrelated