Chapter 3: Planning and Conducting Empirical Studies

Flashcards to review key concepts from Chapter 3: Planning and Conducting Empirical Studies, covering empirical study design, PPDAC steps, and potential errors.

What is the aim of empirical studies?

To answer questions about populations or processes through data.

Why is it important to methodically design the collection and analysis of data in empirical studies?

To ensure we are addressing the question(s) of interest and maximize the accuracy and certainty of the answer(s) given the resource constraints.

What are some consequences of poorly planned or executed empirical studies?

Worthless results, misleading decision makers, unethical practices, and waste of money.

What does PPDAC stand for?

Problem, Plan, Data, Analysis, Conclusion.

What is the 'Problem' step in PPDAC?

• A clear statement of the study's objectives

• define the units; target population/process

• define the variate

• define the attributes

• 3 problem types

What is a target population or process?

The collection of units to which the experimenters conducting the empirical study wish the conclusions to apply.

Define 'variate'.

A characteristic of a unit.

Define 'attribute'.

A function of the variates over the population or process.

What is a descriptive problem type?

To determine a particular attribute of a population or process. (eg. what proportion of…)

What is a causative problem type?

To determine the existence (or non-existence) of a causal relationship between two variates.

What is a predictive problem type?

To predict a future value for a variate of a unit to be selected from the population or process.

What aspects are specified in the 'Plan' step of PPDAC?

• The units available for the study

• How the units will be selected

• What variates will be collected for each unit

• How the variates will be measured.

What is a study population or process?

The collection of units available to be included in the study.

Define study error.

The difference between the attributes of the study population/process and the target population/process.

What is a sampling protocol?

• The procedure used to select a sample of units from the study population/process.

• 2 ways to collect: random or convenience

Define 'sample size'.

• The number of units sampled.

• should have collected from enough to draw conclusion with a desired level of certainty

• can be limited by cost and availability

Define 'sample error'.

The difference between the attributes in the sample and the attributes in the study population/process.

What is measurement error?

• The difference between the measured value and the true value of a variate.

• From variate measurement, where me measure any variate corresponding to any attribution of interets defined in the problem step

Define 'response bias'.

Occurs when participants change some of their responses or do not respond at all to avoid potential consequences of revealing the truthful information.

What should the 'Analysis' step include?

• Numerical and graphical summaries of the data

• Selection of an appropriate model that describes the data

• Checking the fit of the model.

What should be addressed in the 'Conclusion' step?

Potential study, sample or measurement errors, departures from the Plan that affect the Analysis, and the limitations of the study.

What are the 3 problem types?

• Descriptive

• Causative

• Predictive

What to do if study error is suspected?

• error cannot be quantified

• literature and context experts should be consulted

What are some potential reasons for sample error?

• surveys in which participants are volunteers

• convenience sampling

What to do if sample error is suspected?

• sample error is unknown

• models can be used to estimate the error

What to do if measurement error is anticipated or suspected?

• use external organizations to promote confidentiality

• two people input the same data

• suspected → report