HOD 2500: Midterm

4 Ways of Knowing

1. Personal Experience
2. Intuition and Reason
3. Authority
4. Systematic Inquiry

What are personal experiences?

relying on your senses

5 senses

sense of smell, sound, sight, taste, and touch

The Linda Problem

conjunction fallacy: judging that two events will happen together rather than alone

If a woman's mammogram is positive, what is the likelihood she has cancer?

Surveys show that most people would guess 90% but it is actually less than 10%

What is this called in terms of the test?
Mammograms are 90% accurate in spotting those who have cancer.

sensitivity of the test

What this called in terms of the test?
Mammograms are 90% accurate in spotting those who don't have cancer.

specificity of the test

unequal probabilities

P(A|B) is not equal to P(B|A)

Examples of tests that could be used to detect the presence of something that could produce false positive or false negative results

pregnancy test, mammogram, metal detector, etc.

Examples of Authority

1. The same wine is two bottles, one fancy, one cheap
2. White wine and white wine with red food coloring

Seven 21st Century Survival Skills

1. Critical thinking and problem solving
2. Collaboration across networks
3. Agility and Adaptability
4. Initiative and Entrepreneurialism
5. Effective oral and written communication
6. Accessing and analyzing information
7. Curiosity and imagination

Skills developed in HOD 2500

1. Critical thinking and problem solving
5. Effective oral and written communication
6. Accessing and analyzing information

Systematic Inquiry

- The systematic process of collecting and logically analyzing information to provide explanations.
- The systematic, testable, and objective nature of research permits the careful examination of the process itself, as well as the results

Characteristics of Scientific Inquiry

objectivity, control for bias, verifiable, inductive, precise, truthful, willingness to change

HOD Systematic Inquiry

- Systematic Inquiry id the search for knowledge by using recognized methods in data collection, analysis, and interpretation
- The purpose of systematic inquiry is to explain natural phenomena, to understand the underlying relationships among phenomena, and to use this information to predict and influence behavior.

knowledge representation

knowledge is typically presented in the form of theories

theory

a means for simplifying and understanding complex realities - it is a model we use to explain what we observe

Hypothesis

clear, concise, testable statement - educated guess

Quantitative Research Designs

non-experimental (descriptive, comparative, correlational, ex post facto) and experimental (quasi experimental, true experimental, single subject)

Qualitative Research Designs

case study, ethnography, phenomenology, grounded theory

Non experimental

an investigation describing the current status of a variable or the relationship, other than causal, among two or more variables

experimental

an investigation characterized by the direct manipulation of one variable (the cause) so its effect can be seen on another variable (the effect) while controlling for other extraneous variables.

Descriptive

information about the frequency or amount of something

Comparative

descriptions of the differences between groups

correlational

description of the relationship between or among variables

ex post facto

description about the relationship between something that has occurred and correlated variables controlled through matching to show the cause and effect

true experimental

random assignment of subjects to groups

quasi experimental

non-random assignment of subjects to groups

single subject

one (or just a few) subjects

randomized pretest-posttest control group design

experimental design in which participants are randomly assigned to two groups, and each participant is tested on the dependent variable both before and after the manipulation

Qualitative Research Definition

"...a systematic approach to understanding the qualities, or the essential nature, of a phenomenon within a particular context"

What types of questions does qualitative research answer?

What is happening?
Why or how is it happening?

Case studies

exploration of a bounded system (a group, individual, setting, or event)

Ethnography

the study of a culture (people who share behaviors and beliefs)

Phenomenology

the study of a phenomenon (that transcends time, location, and culture)

grounded theory

using inductive logic to generate a model or theory to explain what is observed

Primary means of data collection

1. Interviews
2. Observations
3. Document analysis

similar characteristics between qualitative and quantitative research methods

- systematic ways to investigate the world
- use of subjects/participants
- formal and agreed upon research methods
- formal and agreed upon reporting formats
- attempt to reduce bias and error
- subject to IRB (protection of subjects)
- results are tentative

Quantitative assumptions

1. there exists an external universe that human beings can know
2. events are determined by a finite set of causes
3. the essential elements of events will recur
4. variables can be identified; relationships can be measured
5. etic perspective (outsider's point of view)

Qualitative Assumptions

1. reality is socially constructed
2. Human beings can examine their own experiences of reality and accurately describe them
3. variables are complex, interwoven, and difficult to measure
4. emic perspective (insider's point of view)

Goal of Quantitative

test theory, establish facts, show relationships, predict, and statistically describe

Goal of Qualitative

Establish grounded theory, develop understanding, describe multiple realities, and capture naturally occurring behavior

Key Concepts in Quantitative Research

variables, operationalization, control, reliability, validity, hypothesis testing, statistical significance, replication

Key Concepts of Qualitative Research

meaning, understanding, social construction, and context

Quantitative Research Design

uses designs that are highly structured, formal and specific (ex: pretest-posttest control group experimental design)

Qualitative Research Design

uses flexible, unstructured designs that evolve over the course of the research

Quantitative Subjects and Participants

involves MANY subjects representative of the population of interest. The ideal approach is random sampling and assignment

Qualitative Subjects and Participants

Involves a few participants who are particularly knowledgeable about the subject of interest. The ideal approach is purposeful sampling.

Quantitative - data, data collection, data analysis

uses numerical data collected at specific times from tests, surveys, observations, or interviews. Data is analyzed statistically

Qualitative - data, data collection, data analysis

uses narrative data that is collected over a long period of time from observations, interviews, or reviews of documents or artifacts

Quantitative Researcher's Roles

researchers are characterized as detached, objective observers of events; they remain distant from the subjects

Qualitative Researcher's Roles

researchers are characterized as participant observers; they must develop long-term, close, trusting relationships with participants

Quantitative Setting

researchers manipulate and control the setting to assure the isolation of the variables of interest and the control of extraneous variables

Qualitative Setting

researchers observe individuals in the natural setting in which events occur

Quantitative Approach

1. Begins with hypotheses and theories
2. Manipulation and control of variables
3. Use of formal instruments
4. Deductive
5. Seek consensus
6. Reduce data to numerical indices
7. Abstract language in write up

Qualitative Approach

1. Ends with hypotheses and grounded theory
2. Researcher as instrument
3. Inductive
4. Searches for patterns
5. Seeks pluralism, complexity
6. Makes minor use of numerical indices
7. Descriptive write up

Sources of Knowledge

personal experience, reason and intuition, authority, tradition, systematic inquiry

personal experience

- first hand knowledge is very powerful
- people can have very different perceptions of the same situation
- we often need to know things that cannot (or should not) be learned through experience
- seeing in believing; believing is seeing

reason and intuition

- what seems to be true as determined by informal (collective) or formal reasoning
- just because many (or most) people believe something, doesn't make it true
conclusions are only as sound as the underlying premises

people are subject to biases

confirmation bias, bandwagon effect, wishful thinking, repetition bias, framing effect, anchoring, dunning-kruger effect, bias blind spot

Unscramble this word: OCHSA

hindsight bias: CHAOS

Authority

- seeking answers from someone who has had experience with the problem or has some claim to expertise
- authorities can be wrong; they have no claim to infallibility
- authorities may be in disagreement among themselves

Tradition

- many people find answers to their questions through long established customs
- traditions are often based on an idealized past
- traditions can be distant from current realities and the complexities associated with them
ex: the guru's cat

The purpose of research questions

To communicate what will be studied in clear, concise, and unambiguous terms

Researchable Question

imply the possibility of empirical investigation
(ex: How are recent graduates with double majors perceived by employers in the United States?)

Non-researchable questions

include explanations of how to do something, vague propositions, and value-based concerns
(ex: is democracy the best form of government?)

Sources of Research Questions

1. causal observations
2. deductions from theory
3. literature related to current investigation
4. current social and political issues
5. practical situations
6. personal interests and experiences
7. replication of previous studies
8. clarification of contradictory research results

Three categories of quantitative research questions

1. descriptive
2. correlational/ comparative
3. experimental (causal)

Quantitative research Q formula

The question identifies specifically:
1. the variables,
2. the relationship between them, and
3. the subjects

Characteristics of Qualitative Research Questions

focus on a single, central phenomenon, open-ended, evolving, neutral as to what will be learned, no expected outcomes, written in "how" or "why" format

Qualitative Research Q Formula

The question identifies specifically:
1. the central phenomenon
2. the participants, and
3. the site

Weasel words/phrases to avoid

might, could, possibly, maybe, a chance, may, suggests, perhaps, its inconceivable

Words of Estimative Probability

- used by intelligence analysts in the production of analytic reports to convey the likelihood of a future event occurring
- A well-chosen WEP gives a decision maker a clear and unambiguous estimate upon which to base a decision. Ineffective WEPs are vague or misleading about the likelihood of an event. an ineffective WEP places the decision maker in the role of the analyst - that's your job.

A single question to evaluate quality

On a scale of 0 to 10, how likely is it that you would recommend us to a friend or colleague?

Net Promoter Score

Scoring
- promoters: 9 or 10
- passives: 7 or 8
- detractors: 0 to 6
The net promoter score is calculated by subtracting the % of detractors from promoters.

Measurement

assigning a value to a variable

Reliability

consistency of measurement

Validity

How meaningful, useful, and appropriate our conclusions are

Factors that influence Reliability

1. What you measure (physical, cognitive, and affective traits)(most reliable to least reliable)
2. the instrument you use
3. your technique

Scales of Measurement

1. Nominal: named categories (race, gender, religion)
2. Ordinal: ordered categories (race finish, grade level, SES, Likert Scale)
3. Interval: equal intervals (temperature in F and C)
4. Ratio: equal intervals with absolute zero (height, weight, time, scores, income)

Statistics

procedures used to summarize and analyze quantitative data

descriptive statistics

procedures used to summarize a set of numbers in terms of central tendency, variation, or relationships

inferential statistics

- procedures used to determine the error when estimating a value for a population based upon the measurement of the same value for a sample of that population
- the purpose of inferential statistics is to make conclusions about some value of a population on the basis of that same value measured for a sample
- allow us to estimate the magnitude of our error - the difference between the sample value and the population value - even though we dont know what the population value actually is
- one estimate of error is the "confidence interval" - a range within which the true value is likely (%) to be. the wider the range, the higher the confidence level.

Types of descriptive statistics

1. Central tendency: the typical score (best bet)
2. Variability: how different the scores are
3. Correlation Coefficient: a measure of the relationship between two variables
4. Z-Score: the relationship of one score to the norm group in terms of standard units
5. Effect Size: a measure of the magnitude and difference of the means of two groups

measures of central tendency

mean, median, mode

measures of variability

range, standard deviation

correlation coefficient

how related two variables are, predictability. sensitive to outliers (moving Pearson's r closer to zero)

Z score

The quantity z represents the distance between the raw score (of an individual's score, for instance) and the group mean in unites of the standard deviation. Z is negative when the raw score is below the mean and positive when above

Effect Size

the quantity ES represents the difference between the mean of the experimental group and the mean of the control group in units of the standard deviation

sampling error

error due to sampling, the larger the population that is samples, the smaller the error

Sample Fluctuation

occurs when we measure a value for samples repeatedly drawn from the same population. The value for each sample is different from the others (and different from the true value of the population)

statistical significance

mathematical test that gives a yes/no answer to the question: "Are the differences we see larger than we would expect than from sampling fluctuation alone?"
- based on the size of the sample

practical significance

- answers the question "So what?"
- tells us how important the differences are in terms of what people value

true/false
correlation implies causation

false

correlation

A measure of the strength of the relationship between two variables ("predictive power")
- strength from 0 to
- direction is positive or negative

What does an outlier do to a correlation coefficient?

moves it closer to zero

subject or participant

a person from whom data are collected (subject is quantitative, participant is qualitative)

sample

the collective group of subjects or participants from whom data are collected

population

a large group of individuals to whom the results of a study are to be generalized

two types of sampling procedures

probability and non probability