Experimental concepts

• There are 3 key concepts for psychological experiments:

  • Constructs - Used to represent factors that are not directly observable, in a process known as operationalisation.

    • For example, intelligence (non-observable factor) can be operationalised as IQ (construct).

  • Reliability - This measures how well studies can be replicated. There are 3 types:

    • Internal - This is the extent that all the measures within a single test assess the same concept.

      • For example, in a personality test, do all the questions directly relate to personality?

    • Inter-rater - This measures whether multiple observers who witness the same thing reach the same conclusion.

      • For example, in a anger observation do both observers see traits of anger.

    • Test-retest - This measures whether the results of a measure are consistent for individuals the same individual when used multiple times or in similar conditions.

      • For example, does an individual score lower on an IQ test in the morning.

  • Validity: This checks whether measures are measuring the right thing.

    • Content - This checks whether the measure covers all aspects of the construct it intends to measure.

      • For example, whether different aspects of personality are properly addressed.

    • Face - This checks whether participants see the test as valid, which may influence their level of engagement.

      • For example, if a memory test asks questions about exercise, a participant may become confused and participate poorly.

    • Criterion - This checks how well results correlate with another related measure.

      • For example, whether another personality test had a similar number of introverts.

    • Construct - This measures how well a test measures the concept it was designed for.

      • For example, does a personality test accurately measure personality.

    • External - Measures how well the measure can be generalised across times, populations and environments. Two key elements are:

      • Experimental realism - Whether participants feel manipulation as real.

      • Mundane realism - Whether the experiment is similar to everyday life.

Approaches to research

• A variety of studies are necessary to fully understand a situation - like art.

  • Convergence is when multiple types of studies are used, and then conclusions are compared.

• There are three main types of studies:

  • Correlational studies - Investigate the relationship between 2 or more variables, and usually receive a linear result - meaning it was either positive or negative correlation (a straight line).

    • However, relationships between variables are often more complicated than this.

    • Can test hypotheses, predictions and reliability (inter rater reliability).

    • Cannot make causal statements, due to:

    • Directionality problem; cannot prove whether A = B, or B = A.

    • Third variable; a third unknown variable that = both A and B. Can often be more than 3.

    • Coincidence; if you look for patterns, you will find them even if there is no link.

  • Experimental studies - IV is manipulated, and all other variables are controlled for to see how IV affects DV.

    • They aim to minimise noise, which is random variation that cannot be removed and may obscure true effects.

      • Can be minimised by re-measuring over time to reduce variation, for example response time at different times throughout the day, therefore counterbalancing variables that may be present during different times (tiredness in the morning, hunger midday, etc.).

    • Participants are either randomly allocated or participant in both conditions of an experiment.

  • Quasi experiment - Participants cannot be randomly assigned, for ethical or plausibility reasons.

    • For example, depressed people vs neurotypical people.

• Two types of participant allocation can be used:

  • Between - where a participant only appears in one condition. Random assignment is used in experimental studies, where the experimenter has no say in which group a participant is assigned.

    • However in quasi experiments, random assignment cannot occur but between allocation must.

  • Within - Each participant is used in every condition, and a comparison between each individual’s results are used. Random assignment is not used, and requires less participants.

• There are another 2 classifications of studies:

  • Cross-sectional - Only looks at a single moment in time, for example hours of sleep at age 2.

  • Longitudinal - Studies an individual over a long period of time, for example hours of sleep at ages 2-4.

• Field experiments happen outside of a lab.

  • They are less controlled and have ethical issues.

  • Have higher external and ecological validity.

• Meta-analyses examine effect size (the size of an effect) by looking at multiple studies which examine the same thing.

  • They are systematic, and have very specific criteria.

• Factorial designs have multiple IVs.

  • This can be used to factor in other variables, for example a study which tests the effectiveness of a vaccine that also looks at the effect of gender.

• Interaction is when the effect of one IV depends on the level of the other IV.

Case studies

• Descriptive analysis of a person, a group or an event. It is an idiographic method.

• Many different techniques can be used, such as interviews and questions.

• Examples include:

  • HM.

  • Phineas Gage.

• They have limited generalisability and cannot establish cause and effect.

• However, they provide data into events they may not be able to be researched otherwise, such as brain injuries or traumatic events.

Observations

• Individuals are observed in their natural environments.

• Allows things to be observed that cannot be studied in a lab, for example children’s play.

• Can be structured, where the environment is structured by the experimenters and then watched, or can be completely random.

  • For example, perhaps certain toys are provided to children, or a unmodified home or school environment is used.

• Causality cannot be proved, there is limited control and susceptible to observer bias.

• However, high ecological and external validity due to realism.

Archival research and data mining

• Answers questions with data already collected (secondary data), normally for independent reasons.

  • This can avoid bias in data collection, however the researcher may still interpret the data with bias.

• Data scraping, a piece of code that will look across all databases for data that fits certain criteria.

• This is prone to data bias and has some ethical issues.

Survey

• Most widely used method for collecting data on people.

• Typically descriptive.

• Can be used as a part of many types of research, such as one shot, longitudinal, cross-sectional, etc.

• First, the population must be defined.

  • Census research means to ask everyone, but this is very rarely possible, so a subset/sample is used.

  • Then a researcher must decide what parameters it will be representative of.

    • For example, if studying university students the sample may be representative of age, subject, socioeconomic status, but not eye colour and favourite drink.

• There are four types of non-probability sampling:

  • Convenience, which samples available participants at a specific time and place.

    • This type of sampling mainly takes advantage of university students.

  • Purposive, which samples based on predetermined criteria or characteristics. This can be:

    • Quota, where the sample will have quotas for the percentage of the sample which accounts for certain characteristics.

      • For example, a third should be low anxiety, another be medium and another be high

    • Snowball, where members of the group to advertise to other members within the group, as these would be otherwise inaccessible to a researcher.

      • For example, illegal drug users.

• There are three types of probability sampling:

  • Random, people are selected randomly from a group.

    • Rarely used in practice as can be quite difficult.

  • Stratified, similar to quota where the sample must have a certain percentage of subgroups/stratas to make it similar to the real population.

    • For example, a certain amount of males and females.

  • Cluster, used when the population of interest naturally clumps into groups, such as hospitals.

    • The whole cluster is then sampled.

• This sampling can be done using:

  • Written questionnaires.

  • Interviews, structured or semi-structured.

    • Face-to-face interviews must have a strict protocol due to prevent biases and unconscious behaviour should be monitored.

  • Focus group interviews.

    • This will lead to different data compared to 1-to-1 interview, due to factors such as confidence.

• Bias:

  • Non-response - Those who respond are systematically different from those who don’t.

    • For example, if the survey is advertised on Instagram, only Instagram users will fill it in.

  • Self-selection - Those who chose to be in the study are different from those who don’t.

    • For example, a study on political biases may only have people already interested in politics willing to participate.

  • Social desirability - People give responses based on what they think is desirable to others.

    • For example, lying about how often they shower.

  • Researcher - Can be explicit or implicit bias.

    • For example, a researcher’s facial expressions could even influence participants.

• Many types of questions/items can be used

  • Closed ended:

    • List of options.

    • Likert - A type of scale that allows for a degree of agreement, for example asking people how much they agree with a certain question.

      • Often an even number to prevent a fully neutral response.

    • Checklists.

    • Rankings.

  • Partially open:

    • Option for other.

  • Open.

    • No options.

• Common issues:

  • Double-barrelled question, each question should only have one question.

    • For example, how often do you cook and wash your dishes?

  • No emotive language.

    • For example, is Coke the most delicious soft drink?

  • Shouldn’t ask thing which will likely not be remembered.

    • For example, how stairs did you walk up Monday?

  • Shouldn’t use absolutes, as they can be interpreted differently.

    • Do you always cook?

• Structure:

  • Order should be logical, for example anxiety questions should be grouped together and not intermixed with depression questions

    • Boring or sensitive items towards the end is a common suggestion, but this depends on the study

  • It shouldn’t be too short or long.

    • Asking too few questions does not provide enough data, while too many will overwhelm a participant and cause them to rush through.

  • Pilot study to ask participants how they understood the survey, and point out any potential issues.

• Internet based studies are becoming more utilised:

  • This allows for studies to become more personalised, such as asking participants smoking questions only if they admit they smoke.

  • Allows for videos and audio, either to show participants or ask them to record themselves.

  • Touchscreen can be used.

  • Many online websites can be used to advertise studies.

  • Over 60% of the human population has a smart phone, giving a large sample.

    • However, this does mean that online studies will often exclude those who do not have access to smartphones or have limited access, such as those living in poor and undemocratic regions.

  • Right now smartphones can measure audio, video, motion, location, social proximity, temperature and heart rate.

    • With additions, they can measure external sensors and peripheral, ECG, EEG, and bio samples.

  • This often results in big data, which is data that is too large and complex to be measured by normal data processing software.

  • Online information can understand human behaviour, for example studies have found that the length of phone calls can predict depression relapse.

    • Also analysed is phone calls, email, movements, financial transactions, etc.

    • Many big companies involved in this analysis, Google, YouTube, etc. They are often involved in publishing many studies using this data.

      • This could have many privacy implications.

  • Social media can also be manipulated, as done by a study into the emotion contagion effect.

    • This study manipulated the order in which social media posts were presented, which could be considered unethical as these individuals did not consent.

  • Interventions can even use social media to target physical and mental health.

    • Anti-smoking apps could identify when a person enters a location they normally smoke in and send them a warning notification.

    • Anxiety apps that track heart rates and notify with coping strategies.

    • These interventions are successful as they can be adapted to the individual, using their personal smoking locations and baseline heart rates.

    • AI could also be used for this.

  • Opportunities:

    • Cost effective and more efficient as advertising is easier.

    • No physical constraints, so easier to access deprived areas, disabled individuals, etc.

    • Automation makes processes it more transparent via code, but also less susceptible to experimenter bias.

    • New types of questions and areas of research.

      • This could be around the effects of social media, internet access and long distance communication.

    • Skills needed for research continue to change.

      • Coding, Excel, etc.

    • Democracy in action - Anyone can do surveys, and anyone can make them.

      • This can be a double edged sword, as it allows for access to stigmatised populations (LGBTQ+ populations) but also allows for people to lie without consequences.

        • However, why would they be motivated to do this?

    • Paradigm shifts, such as DSM alterations, epigenetics, brain organisation.

    • Multidisciplinary work.

      • Many scientists from different fields, or different areas within one field, can work together on research.

      • This can allow for trends to be noticed that wouldn’t be otherwise.

      • Participants are becoming more involved, such as patients, and even co-producing.

        • This is useful because the best person to understand the effects of a condition is a person with said condition.

  • Challenges:

    • Data access is a big issue.

      • A lot of universities do not have systems to collate, protect, store and analyse data.

      • Data is larger and faster, even tracking by seconds, making it harder to store and analyse.

      • Data can be biased as even AIs have bias due to their training being biased.

        • It can be inaccurate, such as poor location tracking.

        • Can be analysed incorrectly, such as using a poor application.

      • Ethics is a large issue.

      • Rules, procedures and technologies are ever changing.

Types of research

• Lab vs field research.

  • Controlled with low external validity, vs low control with high external validity.

• Basic vs applied research.

  • Basic research is mechanistic and has little obvious applications (for example, how do algae respire) vs applied research which aims to answer specific questions (for example, how to best treat depression).

• Quantitative vs qualitative.

  • Numerical vs word-based research, can often be combined.

Noise

• Noise is extraneous variables that influence the DV measurement and are evenly distributed across experimental conditions.

  • Statistics aim to detect effect through this noise.

  • Research designs aim to minimise noise.

  • Noise is also research specific, for example gender differences may be noise in a memory study focusing on age, but be the IV in a memory study focusing on gender.

• Confounds are ‘nuisance’ variables that:

  • Vary systematically with IV.

  • Consistently influence DV.

  • They threaten internal validity, as confounds will prevent accurate conclusions.

• Three types of confounds:

  • Person - Individual differences that vary and effect IV.

    • For example, if looking at age with gender biased groups, gender may effect results.

    • Avoided by random assignment or matching gender ratios.

  • Operational - Measure also measures another factor on accident, threatening construct validity.

    • For example, a questionnaire that measures OCD but also asks about depressive symptoms.

    • Can only be avoided by a refined operational definition.

  • Procedural - When a researcher accidently manipulates another variable alongside the IV.

    • Threatens internal validity.

    • For example, studying race but also using confederates of different genders.

    • One option is to repeat the study while controlling for this additional variable.

• Carryover effect:

  • Occurs when a previous experimental condition alters an individual’s behaviour and changes their results in the second condition. Examples include:

    • Practice - A person who in condition A does a spelling test may have had their spelling improved by condition B, causing their performance to increase.

    • Fatigue - Continuous experimentation will bore/tire the participant, causing their performance to worsen.

    • Order:

      • Framing - ??? - check lecture recording later

      • Priming - When the first question prepares a person for the rest of the measure.

        • For example, the first question asking about memory will prepare the participant for a memory test.

      • Interference - Being exposed to something can alter your ability to do something else, for example a study measuring chess ability will be affected if the person plays draughts.

• Longitudinal study issues:

  • Other major events have happened, such as a pandemic

  • Maturation - Over time a person will change in many ways, including age, symptom fluctuation (if measuring an illness such as Tourette's).

  • Instrumentation - The measuring instrument has changed or is not identical.

    • An example is it referring to out of date things, for example a payphone will have no relevance to a modern audience.

  • Attrition - People drop out of a study overtime for various reasons

    • Maybe these people are different to people who stay; more extreme symptoms, lazier, etc.

• Regression to the mean - If someone has an extreme score, this is likely to become less extreme when retested.

  • This can occur as extreme scores are often unlikely, people may have simply had a good or bad day.

Biases

• Experimenter - Issue where the experimenter does something to lead participants to act in a certain way, perhaps treating them differently or even looking at data in a particular way.

  • Dealt with double blind or strict protocol adherence; clothes, speech, drink breaks.

• Participant - Issue where participants behave systematically in a way the study does not expect.

  • Hawthorne effect - People change their behaviour when they know they are being observed or studied.

  • Evaluation apprehension - Testing can make people nervous and perform worse.

  • Good subject effects - People can attempt to meet the researchers aims and perform better

  • Dealt with by:

    • Double blind technique.

    • Manipulation checks - Checking IV manipulation actually works.

    • Reduction of demand characteristics, can be done by hiding the true aims or study hypothesis from the participants.

    • Debrief to understand what participants thought was going on.

    • A cover story.

    • Guaranteed anonymity.

    • Use of indirect measurement so the participant does not interact with experimenters.

      • WATCH LECTURE FOR AN EXAMPLE.

• Selection - Bias where the sample is not representative due to interactions between individuals with biological, behavioural and psychological conditions and their environment.

  • For example, if comparing in patient treatment to outpatient, it must be considered that those in in patient wards often already have worse psychological issues, and are therefore not representative of the general psychiatric population.

• Sampling - When sample does not correctly represent research population.

• Ceiling and floor effects - Not enough variation in the signal (tool or measurement).

  • Could be too easy or difficult.

  • For example, if it is too easy every participant will do well and no variation will be detected.

• Outliers - Unusual cases of extreme variation, which are not representative of the population.

• College sophomore problem - Major problem until 10 years ago.

  • Most psychology research is on college sophomores due to a convenience sample, which could affect external validity.

  • College students are often smarter, more self centred, more susceptible to social influences, less set in their ways, more informed and more experienced with research.

• WEIRD - Western, Educated, Industrialised, Rich and Democratic.

  • These populations can be outliers compared to others.

  • North America most often used.

  • Shouldn’t dichotomise by seeing populations as WEIRD and non-WEIRD.

  • Still variation within WEIRD populations.

Techniques to minimise

• Random assignment.

• Usage of a control/comparison groups.

  • These must be identical in every way except key anticipated variables, including IV.

  • Control groups can be decided by random assignment or matched by key variables such as age, gender, etc.

    • These variables must be decided upon based on relevance to the study.

  • Positive control groups have other manipulations, such as a placebo, sham operation, etc.

  • Double blind placebo controlled studies (RCT), seen as gold standard.

    • Both participants and experimenter are unaware of the groups, preventing demand characteristics.

      • Open label is when a participant knows their group.

• Counterbalancing is when conditions are not performed in the same order for all participants or conditions.

  • For example, some people do a cognition test first, some people do a memory test first.

  • Drowns out carryover effects.

• Pre-test measurements can be used to measure change after a study, but these can change overall results.

  • For example, this could prime the participant for a memory test if it asks about memory

• Pilot studies are rehearsals including data analysis to check for any surprises.