Methodology and Research Design

These flashcards cover essential terms and concepts related to qualitative and quantitative research methodologies, theoretical frameworks, and ethical considerations.

Quantitative Research

measuring things that can be counted, using predetermined categories that can be treated as interval or ordinal data, and subjected to statistical analysis.

Qualitative Research

research that focuses on people's experiences and the meanings they place on events, processes and structures of their normal social setting, providing a holistic view through participants' own words and perceptions.

Positivism

A philosophical stance assuming reality is objectively observable and science is value-free (the world is a self contained reality and can be partitioned)

Interpretivism

A philosophical perspective asserting that reality is socially constructed and meaning is subjective. (reality can only be made accessible through social constructions)

Ontology

The study of the nature of being and reality; what can be known about the social world.

4 different positions of ontology

1. realism - reality is real but our views may differ

2. materialism - if its not physical its not really real

3. subtle realism - reality exists independently, but only accessible through interpretations

4. idealism - reality exists only through our mind

Epistemology

The study of knowledge. 3 main issues:

1. how knowledge can be acquired

2. what is truth

3. whether researchers can be objective.

Induction

looking for patterns and associations during observations of the world. (evidence is used to establish a conclusion)

Deduction

reaching hypotheses theoretically using a logically derived process.( evidence is used to support a conclusion)

functions of qualitative research

1. contextual - describe the form or nature of something that exists

2. explanatory - examine reasons for or associations between something that exits

3. evaluative - evaluate effectiveness of something that exists

4. generative - support development of theories

data collection approaches

1. naturally occurring data - data that exists in real world, not created by the researcher

2. generated data - data created through interaction between researcher and participants

3. secondary data - already existing data

3 ways qualitative and quantitative methods can be combined

1. qualitative research prior to quantitative research

2. qualitative research in addition to quantitative research

3. qualitative research succeeding quantitative research

Research question requirements

-       Clear intelligible and unambiguous

-       Focused, but not too narrow

-       Capable of being researched through data collection; not too abstract, or questions which require the application of philosophy rather than of data

-       Relevant and useful, whether to policy, practice or the development of (social) theory

-       Informed by and connected to existing research or theory; but with the potential to make an original contribution or to fill a gap

-       Feasible, given the resources available

-       Of at least some interest to the researcher

structured interviews

pre-formulated questions, eventually very few open questions (role of interviewer is minimal)

unstructured interviews

open ended questions, very few pre-formulated questions

• questions mainly used to jog memory

• no attempt to maintain consistency across interviews

semi-structure interviews

pre-determined open-ended questions, some pre-formulated questions

• some control over direction and content of discussion

• some consistency across interviews

focus groups

researcher led group discussion as a form of data collecting

• get collective views on a certain defined topic of interest from a defined group of people who are known to have certain experiences

observation

data generated by people without asking specific questions by a researcher, observing spontaneous phenomena, using what happens in the group as data source

4 types of observation

1. complete observer

2. observer as participant

3. participant as observer

4. complete participating

task oriented skills

a goal of the interviewer: organize the interview to reach your research goal (elicit valid and relaible data for your research)

relation-oriented skills

a task of the interviewer: create an atmosphere where the interviewee is willing to provide you with valid and relevant data

interview guide

provides the researchers with a script or protocol for the interview

topic based guide

list of areas and issues the interviewer wants to hear about. Already outlined in a format which serves as a checklist for the interviewer to make it easier to monitor which topics have already been covered

question based guide

outlines the expected content of the interview in terms of a series of questions the interviewer intends to ask

4 aspects to evaluate answers

1. valid answer - what the interviewee thinks and says is the same

2. complete answer - if the answer contains all the information there could possibly be

3. relevant answer - provides information the interviewer would like to elicit

4. clear answer - know what the interviewee means

3 stages in analyzing text

1. data management - meaning is assigned to things, data is labeled, sorted and brought together

2. descriptive accounts - involve defining elements and dimensions, refining categories, and classifying data (typologies)

3. explanatory accounts - about detecting patterns, doing associative analysis identifying clustering

2 approaches to content analysis

1. quantification of qualitative data - for the purpose of statistical inferences

2. grounded theory (constant comparative method)

2 coding strategies

1. using an a-priori list with predefined themes (deductive)

2. starting completely from scratch, no predefined concepts (grounded theory) (inductive)

probability sampling

sample elements are chosen randomly - probability of element being chosen is known (suits quantitative research)

non-probability sampling

sample elements are chosen so they represent certain characteristics - doesn’t have to be statistically representative (suits qualitative research)

statistical representation

choosing sample elements that are representative of the population

symbolic representation

a sample represents and symbolizes the relevant characteristics

the 4 types of non-probability sampling

1. purposive sampling

2. convenience & opportunistic sampling

3. theoretical sampling

4. opportunistic sampling

purposive sampling

choose participants/sample based on criteria

• criteria:

  • homogenous

  • heterogenous

  • typical case

  • extreme case

  • intensity sample

  • stratified purposive sample

  • critical case sample

= select most informative cases

convenience sampling

choosing the sample/participants based on easy access

opportunistic sampling

researcher taking advantages of opportunities that arise throughout fieldwork

theoretical sampling

special type of sampling used in grounded theory - cases are selected based on contribution to the development of theoretical constructs

• process is repeated until theoretical saturation

sampling frame

a list or structure from which you select participants

• 2 types:

1. existing sources

2. generate a new sample (snowball sampling, organizations, flow populations, household screen)

techniques to help participants express themselves

1. enabling technique - visuals or descriptions to help people talk

2. projective technique - use a stimulus, and the person projects their feelings onto it

barriers in interviewing

1. knowledge barrier - doesn’t know how to explain something

2. rationality barrier - gives logical answer instead of emotional one

3. inaccessibility barrier - doesn’t want to say something

4. politeness barrier - gives a socially desirable answer

4 enabling techniques

1. vignette - brief description of particular circumstance, person or event (paint picture/create basis for discussion)

2. card sorting - participant asked to sort through number of written or visual examples of issues

3. giving information or showing written material - when the discussion isn’t going well

4. mapping emergent issues - noticing the issues and showing them to the group

5 projective techniques

1. association techniques

2. completion procedures

3. construction techniques

4. expressive methods

5. choice ordering

Case Study

A research method that involves an in-depth examination of a contemporary phenomenon in its natural setting.

Grounded Theory

A systematic methodology in the social sciences involving the generation of theory from data systematically gathered and analyzed.

content mapping questions

opening question for a topic in the interview guide. Captures the breadth of the topic

• Types:

  • ground mapping - first questions asked to open a topic (broad)

  • dimension mapping - get respondent to focus more narrowly

  • perspective widening - allow interviewer to broaden respondents’ perspective

content mining questions

question for the subtopics. designed to discover details in each dimension (capture depth)

• Types:

  • amplification probes - encourage respondents to go deeper into something

  • exploratory probes - aimed at discovering underlying feelings and thoughts for descriptions, behaviors, events

  • explanatory probe - go deeper into the ‘why’ behind things

  • clarification probes - search for my clarity by asking for clarification of terms/language/details, testing a POV and asking more questions

question types

• open question - allows a full detail answer

• closed question - can be answered with only yes or no

• choice question - asks the respondent to choose between two or more options

• neutral question - does not suggest any answer or opionion; invites an unbiased response

• leading/suggestive question - question that pushes the respondent toward a certain answer, often showing the interviewer’s bias

• rhetorical question - does not expect an answer because the answer is obvious/implied

delphi method

structured way to gather expert opinions:

1. researcher sends a question to a group of experts

2. experts give their individual responses

3. researcher summarizes all the answers and sends that summary back to the group

4. experts then react to the summary, possible changing views

5. cycle repeats until there is consensus/no new ideas appear (saturation)

group phases Tuckman

1. forming - group comes together (uncertainty/chaos)

2. storming - roles are divided, goal is clear

3. norming - standard is set, roles are clear

4. performing - task is performed

5. adjourning/mourning - where the group breaks up

types of membership

• peripheral membership - researcher observes and interacts directly with the respondents

• active membership - researcher participates in key activities, but refrains from committing to values, goals, and attitudes

• complete membership - researcher is an active and engaged member of the group and adopts the view of the group

reliability

repeated measurement: will i find the same values (consistent with general pattern and not result of random chance)

• hard for Qualitative because it doesn’t have numbers to measure

validity

measure of the degree to which an observation actually demonstrates what it appears to demonstrate (do I measure what I want to measure)

procedural reliability

reliability criteria used in qualitative research - follow clear procedures (if someone else were to follow your steps, they should get similar results)

inter rate reliability

reliability criteria used in qualitative research - that different researchers get the same results (analyst analyze the data according to an agreed set of codes/categories) —> discuss reports and any differences are discussed and ironed out

4 types of validity

1. content validity

2. internal validity

3. external validity

4. procedural validity

triangulation

combining several qualitative methods, or several qualitative and quantitative methods (within or not within the same case)

types of triangulation

1. data triangulation

2. investor triangulation

3. theory triangulation

4. methodological triangulation (within and between method)

computer assisted qualitative methods

1. text retrievers

2. text base managers

3. code and retrieve programs

4. code based theory builders

5. conceptual network builders

2 coding procedures

1. theoretical coding

2. thematic coding

theoretical coding

process of coding and categorizing qualitative data with the goal of develop a theory in an emergent manner (has 3 steps: open coding, axial coding, selective coding)

thematic coding

identifying themes within cases and then comparing them across groups or cases (multi-stage procedure)

types of case study research

1. descriptive case study - describe incidence or prevalence of a phenomenon of interest

2. explanatory case study - trace causal linkages among actions, decisions and events overt time

3. exploratory case study - to develop pertinent hypothesis and proposition for further inquiry

single case design

study one case in depth - use for:

• critical case

• unique/extreme case

• representative/typical case

• revelatory case

• longitudinal case

multiple case design

study several cases. Gives direct replication and contrasting situations (helps understand why differences occur and under what conditions things change)

5 elements of case study research

1. designing the case study

2. preparing for data collection

3. collecting data in the field

4. evaluating and analyzing data

5. sharing the findings/writing the report

flexible design

research can evolve during the process

closed design

a research design that does not allow for changes or modifications once the study begins.

skills a good investigator should have

• ask good questions

• be a good listener

• be adaptive and flexible

• have a good understanding of the issues that are being investigated

• be unbiased by prior knowledge and ideas; sensitive and open to new information

case study protocol

written document to be used as a guide during the case study (keeps researcher focused and forces anticipation of certain problems)

• elements:

  • overview of the project

  • field procedures

  • case study questions

  • guide for the final report

pilot case

a test run before the real study (easy to access, informaiton-rich,convenient)

• goals

  • clarify concepts

  • refine research design

  • narrow focus

  • solve methodological issues

main sources of data

• documents

• archives

• interviews

• observation

• physical artifacts

principles of data collection

1. employ multiple sources of data (triangulation)

2. create a case study data base

3. maintain a chain of evidence

4 main strategies to analyze the data

1. rely on theoretical propositions

2. develop a case description

3. combine qualitative and quantitative data

4. examine rival explanations

5 techniques to analyze the data

1. pattern matching

2. explanation building

3. time-series analysis

4. logic models

5. cross-case synthesis

characteristics of case study research

-       Phenomenon of interest examined in natural setting

-       Data collected via multiple means

-       One of few entities (e.g., person, group or organization) is examined

-       Complexity of the unit is studied intensively

-       Investigator(s) should have receptive attitude towards exploration

-       No experimental controls or manipulation are involved

-       Investigator(s) may or may not specify the set of dependent and independent variables in advance

-       Changes in site selection and data collection methods could take place as investigator(s) develops and/or updates hypotheses

-       Useful in the investigation of “how” or “why” questions

-       Focus on contemporary events

action research

interactive inquiry process that balanced collaborative problem solving with data-driven analysis to understand underlying causes and facilitate future predictions

5 phases of action research

1. diagnosing

2. action planning

3. action taking

4. evaluating

5. specifying learning

plan of research

1. research goals, objective, research question

2. research settings

3. data collection

4. sampling

5. data analysis methods

6. conceptual model

3 types of research goals

1. descriptions - describe what is happening but no explanation yet

2. theory development (exploration)- tries to understand meanings, identify factors and build theory

3. hypothesis testing (explanantion)- tests specific cause-and-effect relationshipd

research settings

1. comparative studies - compare multiple cases based on certain dimensions

2. snapshot studies - study a phenomenon at one specific moment in time

3. retrospective version - look back in time using data

4. case studies - deep detailed analysis of one case

5. longitudinal studies - study the same subject over time

6. panel longitudinal studies - explore micro-level change at individual level

7. cross sectional studies - explore macro-level change at context level (compare groups, not time)

sampling methods

1. purposeful (deliberate) sampling - choose participants based on criteria

2. convenience and opportunistic sampling

3. theoretical sampling

issues determining sample size

• heterogeneity of population

• number of selection criteria

• multiple samples within one study

• type of data collection methods

• budget and resources available

scientific applied research (design focused)

looks at problem solving questions, specific for one situation, interventions. Goal is to improve a specific situation. Had a regulative cycle, where diagonsis consists of PA, CA, EA.

• Regulative structure:

  • problem analysis

  • conceptual analysis - develop theory

  • empirical analysis - test conceptual model (theory)

  • design solution

  • implementation

  • evaluation

scientific theoretical research (theory based)

general explanatory questions, applicable to more than one situation. Aims to develop general theories.

• Structure: empirical cycle

  • observation

  • induction (build theory)

  • deduction (form hypotheses)

  • testing

  • evaluation

content validity problem

how do we know what the concept really is, are we measuring it right, did we choose the right indicators

threats to reliability

• subjectivity (unclear definitions, unstructured research process)

• inadequate documentation (others cant replicate it)

reliability criteria used in qualitative research

1. procedural reliability

• use low inference descriptors

• make the research as transparent as possible

• standardize procedures

2. inter-rate reliability

• different researchers get the same results

3 errors regarding validity

type 1: to see relations/concepts when they do not exist

type 2: to neglect relations/concepts when they exist

type 3: to ask the wrong questions

threats to internal validity

• observer bias - researcher attention distracted

• interviewer bias - tone/phrasing influences respondent

• Hawthorne effect - people act different because they know they are in a study

• social desirability - respondents give answers they think are expected

• group pressure - people conform to social norms in group interviews

generalization

transferring findings from one particular case study to more general relations

QUANT: straightforward because you work with large samples and statistics

QUALI: trickier because data is often context-specifc (e.g. based on interviews, observations, small case studies)

3 types of generalization

1. inferential - do findings apply to similar situations or other contexts?

2. theoretical - do findings support or challenge existing theories? Can explanations be generalized?

3. representative - are diverse perspectives captured? Does the study reflect symbolic or typical cases?

stages of a focus group

1. scene setting & ground rules

2. individual introductions

3. opening topic

4. discussion

5. ending

process of observational research

1. site selection

2. gaining entree

3. orientation

4. observation and note taking

5. identifying patterns (“funnel”)

6. theoretical saturation

approaches to validity for observational research

1. multiple observers

2. analytic induction

3. verisimilitude - rich descriptive writing that draws the reader into the world studied

5 quality criteria for observation

1. confirmability

2. dependability

3. credibility

4. transferability

5. utilization/action orientation

2 core principles of ethics of observations

1. unethical to deliberately misrepresent your identity to gain access to enter somewhere you otherwise would not have access to

2. unethical to deliberately misrepresent the nature of your research to those you are studying

5 categories of CAQDAS

1. text retrievers

2. textbase managers

3. code and retrieve programs

4. code-based theory builders

5. conceptual network builders

steps of grounded theory research

1. initiating research

2. data selection

3. initiation and ongoing data collection

4. data analysis

5. concluding the research

components of research design for case studies

1. study questions

2. study propositions

3. units of analysis

4. logic linking data to propositions

5. criteria for interpreting findings