psy 235

Experimental control: (pg 228) - Experimental control includes ○ manipulate one or more IVs § At least one cant be none § Good idea to have more than one § Has to vary between 2 levels § Ideal one level control and other experimental but you want 2 levels § Strong operational definition ○ measure the types of DV(S) that will be measured and how and when they will be measured so that the effects of the independent variables can be assessed § Need to have good measures to make sure you know what is going on § Control situation § We want everyone to experience the experiment the exact same however only the manipulation will be different ○ regulate other aspects of the research context/environment including the manner in which participants are exposed to the various conditions in the experiment. ○ The goal of having such control is to enable researchers to conclude that the variable they manipulate rather than other controlled factors is the cause of any obtained effects on behaviour ○ Example: § Manipulated the colour of the ID code that was placed on the test § Chose to measure cognitive performance via an anagram task and controlled when the task was given and § Regulated other aspects of the research setting including how participants were exposed to the various colours (i.e. were they randomly assigned) ○ For anyone to make a causal statement everything needs to vary True experiment bare minimum for in-between - 1 IV - independent variable - 2 level of independent variable. ○ If you can have more than 2 then you should - Control and experimental - the levels you want and must have - If there are no confounds you can make c Causality and confounds: (pg 228-231) - What are the three criteria that must be met in order to make a causal inference? ○ Covariation of X and Y. as X varies, Y varies ○ Temporal order. The variation in X occurs before the variation in Y ○ Absence of plausible alternative explanations - In principle if X is the only factor in a situation that varies prior to a change in Y then the logical conclusion is that the variation in X must have caused the change in Y - What is a confounding variable? ○ A factor that covaries with the IV in such a way that we can no longer determine which one has caused the changes in the dependent variable. ○ Cannot tell whether the IV or the confound changes the DV ○ Example of potential confounding of an independent variable § Condition one Condition two Condition three IV colour of ID code Red Green Black Confounding variable proximity to midterm exams Midterm week 1 week after midterms 2 weeks after midterms - Research design Description Confounds minimized through Between -subjects Different participants in each condition The potential confounding effects of subject characteristics by using random assignment: procedure in which each participant has an equal probability of being assigned to any one of the conditions in the experiment Within-subjects Participants encounter all levels of experiment Counterbalancing a procedure in which the order of conditions is varied so that no condition has an overall advantage relative to the other conditions. --> ensures that participants experience the IV levels in different orders as the participants serve as their own control. --> makes a lasting impact. --> Between-Subjects design/randomly assigned - In theory randomly assigning participants to levels of the IV ( 1,2, or 3) will distribute individual differences throughout the conditions so experimental groups are equivalent in every way except the IV. ○ Everyone who can into the study are randomly assigned to a group ○ This is why we want more levels ○ Every participant has an equal chance in being in one of the levels. Between-subjects design terminology (in class) - Between-subject designs ○ General class of designs in which different research participants are used in each group ○ Involve comparisons between different groups of participants ○ Not repeated they only get that ○ Everyone got one score for their group ○ Exposed to only one and only one variable and only gets one score ○ Why we don’t do pretesting it can alert the participant about the study this is why we use random assignment. - Characteristics ○ Any given participant receives only one level of the independent variable ○ Only one score for each participant is used in the analysis of the results Control Achieved Through Experimental Design (in class) - Experimental group ○ The group(s) that receives the independent variable in an experiment - Control group ○ The groups that receives a zero level of the independent variable (or standard treatment) and that is used to assess the effects of the independent variable Between-subject Designs: Advantages pg: 236-238 - Single-factor experiments each participant either engages in only one condition or engages in all the conditions. They are called between-subjects and within subjects - Effects do not carry over to other conditions ○ This is b/c it only engages in one condition effects caused by exposure to one condition meaning it cant be carried over to other conditions - Less likely that participants will catch on to the hypothesis being tested - Exposure to multiple levels of the IV may be impossible or ethically and practically difficult - Good decision b/c participants will not have repeated exposure to the memory task Between-subject Designs: Disadvantages pg: 238-239 - Different people in each condition generate more "noise" (i.e., variability), making it more difficult to establish an effect of the IV on the DV ○ b/c you need to use different participants at different levels to make it different so you have multiple different people where as within subject they use the same participants over and over again at each level so it is less noise. Within same people participate at every level, between different people at different levels - Many participants required ○ So need to find more people. - The more people you have the nosier it will be, ○ If you are going to use an between subject we see that everyone is different Between-subjects Designs: Independent-Groups pg: 239-240 - Independent-groups (random-groups) design: (understand this is really strong) ○ Strongest type of between subject design you use random assignment . ○ Participants randomly assigned to conditions. ○ If those differences equated amongst our sample so a participant has an equal chance of being on either side of the experiment. ○ Large sample size just be chance ○ As soon as you cant use randomization you cannot make a causation ○ Example: in the colour experiment each participant was exposed to only one colour red or green or black and that determination was made randomly. Random assignment means that each participant has an equal probability of being assigned to any particular condition ○ Greater the amount of participants the better - block randomization helps overcome issues of random assignment (not on exam) ○ Run through random order of blocks (round of conditions) until desired sample size reached ○ Example: we have three conditions red, green, and black and we want 30 participants per condition. The experiment will begin next Monday and three morning sessions and three afternoon sessions. Each session involves one participants for Monday morning we randomly order the three condition say black-red-green. This constitutes the first block of conditions. Our 9 am participants is assigned to receive the black ink code the 10 am participant the red ink code and the 11 am participant the green ink code. Between-Subjects Designs: Natural-Groups pg: 241-243 - Natural-group design: (understand) weaker ○ Cannot say this caused this as it is no longer random assigned ○ Rather than manipulate IV, create different groups of participants based on naturally occurring attributes (subject variables) ○ Example ones self esteem. ○ To assign to groups it will create a confound due to it not being randomly assigned ○ This is weaker than random assignment ○ There is no way you can randomly assign ○ This is my variable but I'm using your natural characteristics ○ Low and high intelligence --> can't randomly assign you to low and high. ○ Still good just have to say more in your limitations - Subject variables often referred to as quasi -IVs Random Sampling vs. Random Assignment pg: 243-244 - You should be able to explain the differences between random sampling and random assignment in terms of defining and purpose ○ Random sampling § Description: each member of a population has an equal probability of being selected into a sample chosen to participate in a study § Example: from a population of 240 million adults in a nation a random sample of 1,000 people is selected and asked to participate in a survey § Goal: to select a sample of people whose characteristics (e.g. age, ethnicity, sex, annual income) are representative of the broader population from which those people have been drawn. § Don’t generally get to do § Cannot force someone into a study ○ Random assignment § Description: people who have agreed to participate in a study are assigned to the various conditions of the study on a random basis. Each participant has an equal probability of being assigned to any particular condition § Example: after a college students signs up for an experiment (e.g. to receive extra course credit or meet a course requirement) random assignment is used to determine whether that student will participate in an experimental or control condition § Goal: to take the sample of people you happen to get and place them into the conditions of the experiment in an unbiased way. This prior to exposure to the independent variable we assume that the groups of participants in the various conditions are equivalent to one another overall § Use it if you can use it ○ The difference between random sample and random assignment § Random sample: refers to the process of selecting a representative subset of individuals from a larger population to participate in a study (how you select individuals from the population to participate in your study) § Random assignment: is the method of placing those selected participants into different groups within an experiment. (how you place those participants into a group) Control Achieved Through Experimental Design (in class) (weakest experimental to the strongest) - one- group posttest only design (one-shot case study design) ○ Design diagrammed as: ○ Sample T. M ○ weak experimental design ○ T = treatment ○ M= measurement ○ Example: adding selfcare into class you use it then measure it however this will not tell you anything. Does not meet any requirement of a true experiment - Pretest: ○ A measurement occasion that occurs prior to the introduction of the independent variable ○ Example: test the stress before the test. - Posttest: ○ A measurement occasion that occurs at the conclusion of the presentation of the independent Variable ○ Then test it after treatment - one - group pretest-posttest design (better not as weak) ○ Design diagrammed as: § Sample M. T. M ○ weak experimental design ○ Example: do the test but they can not contribute to the self care they offered, need a control group, unsure what caused u to be stressed. Introduced a confound by testing before hand. ○ Pretesting extraneous variable . ○ Do not do this one just like the one before. - posttest- only designs with non-equivalent groups (more common) ○ Design diagrammed as: § Group A T. M § Group B T (zero level) M ○ Weak- okay experimental design ○ Example: one will get the treatment while the other will not get the treatment. However people did not get into the group through randomization. We are using groups that are already made ○ Example: selection issue, we cannot be sure if the participant are different. ○ If you use this don’t talk about causation ○ Better than the other two but still not strong - Posttest- only control group design (aim to do this ) this is the strongest true experiments ○ Design diagrammed as § R Group A. T. M § R Group B T(zero level) M ○ R= random assignment ○ Strong experimental design ○ The only way people get into the groups is through randomization ○ We have an independent treatment ○ We have two levels ○ This is a between subject design .using a T test (comparing two groups) how did this affect information of our dependent variable ○ Only problem is if you have small numbers - pretest-posttest control group design true experiments ○ Diagrammed as: § R Group A. M. T. M § R Group B M. T (zero level) M ○ Strong experimental design ○ What if the randomization did not work ○ Run a t test and compare if they are different then you have a confound ○ If you have a large group you can be sure that your randomization will work. § Might have more noise making a possible type 2 error Multilevel Completely Randomized Designs (in class) - Completely randomized design that contains more than two levels of the independent variable - Diagrammed as ○ R. Group A level 1 T. M ○ R. Group B level 2 T. M ○ R. Group C level 3 T M 3 questions we apply to every test: (has to be in this order - Are the results due to chance ? - important ○ Do a t test in a position to reject null hypothesis means that control is unlikely due to chance ○ Want to eliminate that this is done by chance ○ H0: μ1 = μ2 = μ3 ○ Use one-way ANOVA to compare the different samples ○ T test, ANOVA and statistical test can be the only way to determine if we reject null hypothesis § See if there are differences ○ If the F-value is significant , then reject null hypothesis - Use a post hoc test to determine whether there is a statistically significant difference between any combinations of groups ○ Use after if you have checked that you can reject the null hypothesis - If you perform a large number of post hoc tests, then expect more of them to be significant by chance than if you performed only a few tests ○ Familywise error rates: possibility of a type 1 error ○ Many post hoc tests control for both type 1 and type 2 error ○ Cannot keep running t tests that increases your type 1 error ○ Post hoc does the comparison that you cannot see. ○ Its good to do more variation ○ Good idea if you can test as much variation as possible to know if it is a linear or non linear effect - Are the results due to a confound? ○ Random assignment of participants to each study group control for known and unknown confounds § If we use random assignment - Are the results due to the independent variable? ○ If we reject the null hypothesis and rule out confounds, then we conclude that the independent variable influenced our results