PSYC 321 - Research Methods

3 goals of science

1. description of reality

2. prediction of reality

3. explanation of reality

inductive vs deductive research

• Inductive research

  • Starts with data and ends with theory

• Deductive research

  • Form a theory and ends with data ( due to testing the theory )

common sense findings aren’t always that obvious

true

the emprical research cycle / the scientific method

Notice that sometimes drawing a conclusion means restarting and modifying the problem statement

why do we only have meso-level theories?

we dont have a law that can be applied to every single person in the world

theory

A statement that proposes to explain relationships among phenomena of interest.

what is parsimony ????

• ???

properties of a good theory

• Parsimony

• Precision

• Testability/ falsifiability

• Usefulness ( nothing as practical as a good theory )

• Generalizability

basic research vs applied research

• basic research might look at biology, chemistry, anxiety chemicals it is more theoretical and to derive principles

• applied research might look at conversations, actions it is more practical and solves a practical problem

• they should inform each other

is socio-psychology is core to psychology?

true

Levels of analysis in IO

1. organizaiton level (organisation climate, like uni policies and infrastructure)

2. group level ( team cohesion, deparment culture in a toxic manager)

3. individual level ( how they behave towards family, employees, supervisors )

the consequence of “the whole is greater than the sum of its parts”

though we study the individual level because most problems are top-down… we need to look at the other levels

when to use qualitative vs quantitiatve research?

qualititative research:

• dont know a lot about topic

quantitiative research:

• you have a theory you want to test and how to operationalize it

what are primary research methods?

A class of research methods that generates new information on a particular research question.

3 PRIMARY RESEARHCH DESIGNS used in IO Psych

True experiment, quasi-experiment, non-experiment/survey

true experiment

• what?

  • randomized assignment to groups…

  • highly controlled variables

  • Can exist outside the lab in natural settings

• generalizability ?

  • VERY questionablke gneralizability

• importance?

  • THE BEST FOR ESTABLISHING CAUSALITY

if you have true randomization in a true experiment you have all the control you need

TRUE

a survey can be used in a true experiment

TRUE

quasi-experiments

• what?

  • lack of randomization ( ex: gender-based assignment)

  • limited control ( ex: can’t control the independent variable )

• generalizability ?

  • higher generalizaibility

causal inferences

Aka A causes B

non-experiment/ survey design

• what ?

  • A type of research method for conducting studies that does not involve the manipulation of variables or assignment of participants.

  • instead data is just collected ( ex: randomly assigned surveys, observations )

• generalizability ?

  • higher generalizability to real-life

  • The lack of randomization or control can be accounted for statistically

csecondary research design

what?

• A class of research methods that examines existing information from research studies that used primary methods

types?

• archival research

• meta-anlysis

• data minig

con?

• less control

• garbage in and garvae out

data mining

• defined by volume, velocity, and variety

• CON: generalizability - how much is noise

data mining / big data is defined by 3 things

• 1. Volume — there are a lot of cases and many variables per case.

• 2. Velocity — data collection occurs at a rapid and continuously increasing pace.

• 3. Variety — data come in many forms, both inside and outside of an organiza-tion by both active and passive means.

how does big data restrain the types of data you can use ?

usually only cintains binary data like purchases/not-purchased

affinity index relation to big data

statistic that is based on the probability of two (or more) items being paired together.

helps to establish relationship between severa variables

meta analysis

• when 20 studies say one thing and 10 say another, how are we to determine what is the truth?

• M-A is a statistical procedure designed to combine the results of many indi-vidual, independently-conducted empirical studies into a single result or outcome.

• Need to adjust for variations within set up and samel sizes and level of analysis

how does the fie file drawer effect, effect meta-analysis ?

• Research studies that yield negative or non-supportive results are not published as often as studies that have sup-portive findings, and therefore are not made widely available to other researchers.

• this means that M-A might not have access to relevant studies

Is IO a quantitative heavy field ?

YES

what is qualitative research and the 2 types of .qualitative research in IO

• A class of research methods that involves collecting and analyzing data that are non-numerical in nature.

• 2 types:

  • thematic analysis

  • ethnogrpahy

thematic analysis

• idenitfying patterns within the data (ex: themes in their written responses)

• gather examples that fit within that patterns

ethnography

• Scientific study of a group of culture

• NOT particularly identifying set patterns but rather just describe

emic perspective vs etic perspective in ethnography

• they take thehe insider’s view is called the emic perspective ehen researching , whereas they take the  external view which is the etic perspective when generating conclusions  .

steps in thematic analyysis

• Familiarization with data

• Systematic encoding data

• Generate themes

• Check if any themese are missing

• Polish theme definitions

• Produce a report

what are constructs ?

Psychological entities or concepts of interests

ex: happiness

what is conceptual criteria and actual criteria ?

conceptual criteria - is the construct in question

actual criteria - is the operational definition

what is an operational definition?

Defining constructs in a way that is observable

ex: frequency of smiles within 2 hours of the movie

criterion deficiency and how do we account for this ?

• Degree to which actual criteria fails to overlap conceptual criteria

• We can reduce but not eliminate the criterion deficiency

• this appears because we’re not capturing an aspect of the actual concept

• we can control this using statistical analysis

criterion relevance

Degree to which actual criteria and conceptual criteria coincide

criterion contamination and how do we handle it ?

things that we’re capturing that are not a part of the concept is called criterion contamination

we can account for this using statistical analysis

criterian contaminations can be caused by 2 things distorting the conceptual criteria

Bias – extent actual criteria consistently measures something

Error – extent to which actual criteria is related to nothing at all

reliability in measurements

• is the test producing stable and consistent results ( ex: a clock that is always 3 minutes off )

validity measurments

• how well a judgement accurately measures a concept ( ex: a clock that is correct down to the miliseconds )

can something be reliable and not valid

TRUE

can something be valid but unreliable

FALSE

4 threats to internal validity

1. History effect

2. Maturation effect

3. Selection effect

4. Attrition effect

history effect

• historical event that has nothing to do what you’re trying ot measure but is eeffecting things ( ex: how much you hate math effecting your math scre )

maturation effect

• ???

selection effect

• selection bias… bias in people opting into something ( ex: motivated people will sign up to trial your new study method trial )

• there are probably some factors that differenitate uninterested people from the people who opt in, are they relevant factors idk

attrition effect

• bias in people opting out of something ( ex: 10y study… people wll drop out )

• there are probably some factors that differenitate those people from the people who stayare they relevant factors idk

sources of data

• Organizational record
• Questionnaire ( self-reports.. risk people lying and a small incentive is all you need )
• Observations ( to prevent people acting unnatural you need them to trust you )
• Interview/ Focus group (more group discussion-oriented)

correlation coefficent

refers to the mangitude of relationship between 2 variables

ranges from -1 to 1

the higher the absolute value the stronger the relaitonship

what does a negative correlation mean ?

as one variable goes up, the other goes down

what does a positive correlation mean ?

as one variables goes up, the other vairable goes up

3 criteria for causality

• covariance

  • when x is moving then y is also moving

• time order of events

  • x needs to move first and then y moves

• lack of alternative

  • no other secondary variable z, can be behind this

correlation is not equal to causation

Difficult to infer from non-experimental method

does one study prove anything in science?

FALSE. It takes several years of study. Research is an iterative process.

2 most important factors in the design of a research study

• Naturalness of the research setting

  • We want a "high fidelity" research setting

  • the greater the realism of the research setting, the more generalizable the results.

  • Fidelity: how close a setting is to reality

• Degree of control

  • Having control over how the study goes… but researchers are not God

  • An example of a study with a low degree of control: asking for participants that speak 2 rare languages…

internal validity

The degree to which the relationships evidenced among variables in a particular research study are accurate or true.

this type of validity depends largely on the study's procedures and how rigorously it is performed.

external validity

 The degree to which the relationships evidenced among variables in a particular research study are generalizable to real-life application.

another word for generalizability

external validity

predictor vs criterion variable s

predictor variables predict criterion variables

they don’t have a causal relationship! Just a predicitve one.

ex: , gpa predicting academic performance,

Correlation coefficient vs spread of data points on a scattter plot

• The tighter the spread the stronger the relationship

how often do we establish causality

Rarely. We focus on correlation. We cannot give proven to be effective strategies.