EVALUATING OPERATIONAL DEFINITIONS

RELIABILITIY, CONSISTENCY, DEPENDABILITY

_____ MEANS ____ AND _____

Good experiments are reliable, if we apply variables in more than one experiment, they ought to work in similar ways each time

• Measured operational definition should also be reliable. If we use the operational definition of the dependent variable in several sets of measurements, we should get the same results each time

• If possible, we select standardized test in measuring dependent variable because it shown to be reliable. The more accurate the variables, the more they are likely to be reliable.

1. Inter- rater reliability

2. Test- retest reliability

3. inter- item reliability

Types of Reliability

INTER-RATER RELIABILITY

• THE DEGREE OF AGREEMENT OR CONSISTENCY BETWEEN TWO OR MORE SCORERS (JUDGES OR RATERS) REGARDING A PARTICULAR MEASURE

  • It is also known as inter observer, inter coder (qualitative), inter scorers

  • When 2 or more raters score each response, scores given by different raters can be statistically compared

TEST-RETEST RELIABILITY, Time Sampling Reliability

• RELIABILITY OF MEASURES CAN ALSO BE CHECKED BY COMPARING SCORES OF PEOPLE WHO HAVE BEEN MEASURED TWICE WITH THE SAME INSTRUMENT.

  • It is also known as ________________

INTER-ITEM RELIABILITY

• DIFFERENT PARTS OF A QUESTIONAIRRE, TEST OR OTHER INSTRUMENTS DESIGNED TO ASSESS THE SAME VARIABLE ATTAIN CONSISTENT RESULTS.

  • can be assessed by split half method or Cronbach’s alpha (internal consistency)

VALIDITY

THE PRINCIPLE OF ACTUALLY STUDYING THE VARIABLES INTENDED TO BE MANIPULATED OR MEASURED.

FACE VALIDITY

THE DEGREE TO WHICH A MANIPULATION OR MEASUREMENT TECHNIQUE IS SELF EVIDENT

CONTENT VALIDITY

DESCRIBES A JUDGMENT OF HOW ADEQUATELY A TEST SAMPLES REPRESENTATIVES OF THE UNIVERSE OF BEHAVIOR THAT THE TEST WAS DESIGNED TO SAMPLE

PREDICTIVE VALIDITY

AN INDEX OF THE DEGREE TO WHICH A TEST SCORES PREDICTS SOME CRITERION MEASURES

CONCURRENT VALIDITY

AN INDEX OF THE DEGREE TO WHICH A TEST SCORES IS RELATED TO SOME CRITERION MEASURE OBTAINED AT THE SAME TIME

CONSTRUCT VALIDITY

IT IS ESTABLISHED THROUGH A SERIES IN WHICH A RESEARCHER SIMULTANEOUSLY DEFINE SOME CONSTRUCT AND DEVELOPS THE INSTRUMENTATION TO MEASURE IT

CONVERGENT VALIDITY

CONSTRUCT VALIDITY

___________ NEWLY DEVELOP TEST WILL BE CORRELATED WITH A RELATED CONSTRUCT

DIVERGENT/DISCRIMINANT VALIDITY

CONSTRUCT VALIDITY

____________ NEWLY DEVELOP TEST WILL BE CORRELATED WITH UNRELATED CONSTRUCT

INTERNAL VALIDITY

THE DEGREE TO WHICH WE CAN CONFIDENTLY INFER A CAUSAL RELATIONSHIP BETWEEN THE VARIABLES

EXTRANEOUS VARIABLE

VARIABLE OTHER THAN THE IV AND DV; A VARIABLE THAT IS NOT THE FOCUS OF AN EXPERIMENT BUT CAN PRODUCE EFFECTS ON THE DV IF NOT CONTROLLED

QUASI

SEEMINGLY, LESSER DEGREE

CANNOT PRACTICE RANDOM ASSIGNMENT

WHY QUASI IS LESS POWERFUL?

QUASI EXPERIMENTS = __________________

CONFOUNDING VARIABLES

NO RANDOM ASSIGNMENT IS IT BAD? WHY?

YES!

PRINCIPLE: AN EXPERIMENT WITHOUT RANDOM ASSIGNMENT IS PRONE TO ___________

LOWERS INTERNAL VALIDITY

INTERNAL VALIDITY, CERTAINTY

DEGREE OF ____ THAT CONCLUSION WILL BE _____

INCONCLUSIVE (LOW INTERNAL VALIDITY)

Unequal group characteristics

Cause of CV?

_____________ before the experiment

equal characteristics

HOW TO AVOID CONFOUNDING VARIABLES?

Groups have __________ before the experiment

CONTROL VARIABLES

• Characteristics, traits, conditions, situations, of the subjects that are held equal across all IV levels

  • PURPOSE? the more alike the participants are within each exp. group, the lower the chance to have confounding variable

FIND PARTCIOANTS, DIVIDE INTO 2 GROUPS, GIVE THE TEST, COMPARE

GENERAL STEPS IN EXPERIMENTAL RESEARCH

STEP 1:

STEP 2:

STEP 3:

STEP 4:

DIRECT CONTROL OF IV LEVELS

WHAT MADE THIS EXPERIMENT A TRUE EXPERIMENT?

TRADEMARK OF TRUE EXPERIMENTS ………………………

INDUCED BY THE EXPERIMENTER

MAIN INDICATOR OF DIRECT CONTROL

MANIPULATION CAN BE ………………….

NO DIRECT CONTROL OF IV LEVELS, PRE-EXISTING

WHAT MADE THIS EXPERIMENT A QAUSI EXPERIMENT?

TRADEMARK OF QUASI EXPERIMENTS ……………..

• IV LEVELS CANNOT BE INDUCED BY EXPERIMENTER.

IV LEVELS ARE ______

RANDOM ASSIGNMENT, CHANCE

_____________ EACH SUBJECT HAD AN EQUAL _________

POSSIBLE, NOT POSSIBLE

TRUE EXPERIMENT: RANDOM ASSIGNMENTS IS _____

QUASI EXPERIMENT: RANDOM ASSIGNMENTS IS _____

EATING, EATING FRUITS, EATING FRUITS, EATING VEGETABLES, HEALTHIER GUT, CLINICAL TEST, TRUE EXPERIMENT

1. Does eating fruits will lead to a healthier gut than eating vegetables?

a. IV -

b. IV LEVELS -

DV-

OP. DEF-

TRUE EXPERIMENT OR QUASI EXPERIMENT?

_________

SLEEPING TIME, 8PM, 11PM, POSITIVE MOOD, PSYCHOLOGICAL QUESTIONAIRE, TRUE EXPERIMENTS

2. Does sleeping at 8PM helps to develop positive mood than sleeping 11PM?

a. IV -

b. IV LEVELS - i. ii.

C. DV -

OP. DEF. -

TRUE EXPERIMENT OR QUASI EXPERIMENT?

_______

NATIONALITY OF SPOUSE, ASIAN SPOUSE, AMERICAN SPOUSE, HAPPINESS, PSYCHOLOGICAL QUESTIONAIRE, TRUE EXPERIMENT

3. Having an Asian spouse is happier than having American spouse?

IV -

b. IV LEVELS -

c. DV

OP. DEF. -

TRUE EXPERIMENT OR QUASI EXPERIMENT?

_________

COLOR, DARK, LIGHT, PSYCHOLOGICAL REACTIONS, PHYSIOLOGICAL REACTIONS, HORMONAL TESTING

4. Can color cause physiological reactions

a. IV -

b. IV LEVELS-

DV -

OP. DEF.-

TRUE EXPERIMENT OR QUASI EXPERIMENT?

_________

TYPES OF MUSIC, ROCK, POP, PHYSIOLOGICAL RESPONSE, HORMONAL TESTING, TRUE EXPERIMENT

5. Can different types of music lead to different physiological responses?

IV-

IV LEVELS-

d. OP. DEF. -

TRUE EXPERIMENT OR QUASI EXPERIMENT?

_________

AMOUN OF TIME SPENT IN SOCIAL MEDIA, LOW AMOUNT OF SPENT, MODERATE AMOUNT OF SPENT, SIGNS OF ADDICTION, SELF REPORT SCALE, TRUE EXPERIMENT

6. Does using social media exhibit signs of addiction?

a. IV-

b. IV LEVELS -


C. DV -

d. OP. DEF.

TRUE EXPERIMENT OR QUASI EXPERIMENT?

_________

EATING BREAKFAST, NOT EATING BREAKFAST, EATING BREAKFAST, PERFOMANCE OF STUDENTS, PERFORMANCE TEST, TRUE EXPERIMENT

7. Does eating breakfast help students do better in school?

a. IV-

b. IV LEVELS -


C. DV -

d. OP. DEF.

TRUE EXPERIMENT OR QUASI EXPERIMENT?

_________

TYPE OF PERSONALITY, TYPE A, TYPE B, STRESS LEVELS, PERSONALITY TEST, QUASI EXPERIMENT

8. Are people with a Type A personality more stressed than those with a Type B personativy?

a. IV-

b. IV LEVELS -


C. DV -

d. OP. DEF.

TRUE EXPERIMENT OR QUASI EXPERIMENT?

_________

EMOTIONAL INTELLIGENCE SCORE, HIGH, LOW, OVERALL WELLBEING, PSYCHOLOGICAL QUESTIONAIRE, TRUE EXPERIMENT

9. Do people who score higher on measures of emotional intelligence also score higher on measures of overall well-being?

a. IV-

b. IV LEVELS -


C. DV -

d. OP. DEF.

TRUE EXPERIMENT OR QUASI EXPERIMENT?

_________

PROCRASTINATION, PROCRASTINATING, NOT PROCRASTINATING, STRESS LEVELS, SELF REPORT SCALE, QUASI EXPERIMENT

10. How does procrastination impact student stress levels?

a. IV-

b. IV LEVELS -


C. DV -

d. OP. DEF.

TRUE EXPERIMENT OR QUASI EXPERIMENT?

_________