research methods- year 13

Reliability

Measure of consistency.

If a particular measurement is made twice and produces the same result, that measurement is described as being reliable.

Test- retest reliability

A method of measuring the reliability of a questionnaire or psychological test by assessing the same person on two seperate occasions.

This shows to what extent the test produces the same answers on two seperate occasions.

If a psychological test is reliable, then the results should be the same each time it’s administered.

Can be applied to questionnaires and also interviews.

There must be a sufficient time between test and retest to ensure participants don’t remember their answers, or that they aren't taken so far apart that they have changed their opinion.

Two sets of scores would be correlated to make sure they are similar.

Inter observer reliability

The extent to which there is an agreement between two or more observers involved in the observation of a behaviour.

If the total number of agreements divided by the total number of observations is greater than 80%, the data has high inter-observer reliability.

Observational research is based on one observer’s interpretation of events, which introduces subjectivity, bias and unreliability into the data collection process.

Validity

Whether a psychological test, observation, or experiment produces a result that is legitimate/ genuine.

Whether it measures what its supposed to measure and if it be generalised beyond the research setting where it was found

Internal validity

Whether the effects observed in the experiment are due to the manipulation of the independent variable or some other factor.

Demand characteristics impact internal validity.

external validity and ecological validity

Relates to factors outside the investigation such as generalising it to other settings, other populations of people and other eras.

Ecological validity is a type of external validity.

Ecological validity: the extent to which findings from the research study can be generalised to other settings and situations.

every day life: mundane realism.

Temporal validity

The extent to which findings from a research study can be generalised to other historical times and eras, a form of external validity.

Face validity

A basic form of validity in which a measure is scrutinised to determine whether it appears to measure what it’s supposed to measure.

This can be determined by simply eyeballing the measuring instrument or by passing it through an expert check.

For example, does a test of anxiety look like it measures anxiety?

Concurrent validity

The extent to which the psychological measure relates to an existing similar measure.

A particular test or scale in which the results obtained are very close to or match those obtained on another recognised and well-established test.

Close agreement between two sets of data would indicate that the new test has high concurrent validity.

Improving validity- experiments

Using a control group, the researcher can better assess whether changes in the dependent variable were due to the independent variable.

Experimenters may also standardise procedures to minimise participant reactivity and investigator effects that could affect the validity of the outcome.

Improving validity- questionnaires

Lie scale within questionnaires in order to assess the consistency of a respondent’s answers to control for effects of social desirability bias.

Validity may further be ensured by assuring respondents that all data submitted would remain anonymous.

Improving validity- observations

Minimal intervention by the researcher in the observation.

In covert observations, behaviours observed are natural and authentic.

If behavioural categories are too broad, overlapping, or ambiguous, this can have a negative impact on the validity of the data collected.

Improving validity- qualitative research

Has higher ecological validity than quantitative data.

Triangulation used- number of different sources as evidence; for example, data compiled through interviews with friends and family, personal diaries and observations.

The null hypothesis

The null hypothesis states there is no difference between the conditions.

The opposite of the alternative hypothesis (directional or non-directional) depends on how confident the researcher is in the outcome of the investigation.

Probability

A measure of the likelihood that a particular event will occur, where 0 indicates statistical impossibility and 1 is statistical certainty.

Levels of significance and probability

All statistical tests employ a significance level- the point at which the researcher can claim to have discovered a large enough difference in the data to claim an effect has been found.

The point at which the researchers can reject the null hypothesis and accept the alternative hypothesis.

The usual level of significance in psychology is 0.05 (5%)

p is smaller than or equal to 0.05 (p stands for probability).

This means the probability that the observed effect occurred when there is no effect is equl to or less than 5%.

Calculated and critcal values

To check for statistical significance, the calculated value must be compared with the critical value.

Critical value: when testing a hypothesis, the numerical boundary or cutoff point between acceptance and rejection of the null hypothesis.

Using tables of critical values

A one-tailed test hypothesis was directional

A two-tailed hypothesis was non-directional.

The number of participants in the study usually appears as the N value in the table of critical values

For some, degrees of freedom are calculated instead.

Level of significance (p value)

Type 1 and type 2 errors

Sometimes a wrong hypothesis may be accepted.

Type 1 error: the null hypothesis is rejected, and the alternative hypothesis is accepted when it should have been the other way around because the null hypothesis was true. Known as a false positive.

Type 2 error: the null hypothesis is accepted, but it should have been the alternative hypothesis because the alternative hypothesis is true. Known as a false negative.

We are more likely to make a Type 1 error if the significance level is too lenient (e.g. 0.1)

We are more likely to make a Type 2 error if significance level is too stringent (e.g. 0.01).

Case studies

An in-depth investigation, description and analysis of a single individual, organisation or event.

Involves the analysis of unusual individuals or events, such as a person with a rare condition.

Produces qualitative data- interviews, observations, questionnaires, or a combination of all of these.

Produces quantitative data- if they are subject to experimental or psychological testing to assess what they are and aren’t capable of.

Case studies tend to take place over a long period of time- longitudinal.

May involve gathering additional data about the individual from family and friends as well as from the person/ group themselves.

Case studies- strengths

Offers rich, detailed insights that may shed light on very unusual and atypical forms of behaviour.

Contribute to our understanding of typical functioning by looking at unsual behaviour.

E.g. the study of HM demonstrated existence of seperate stores in LTM and STM.

Case studies limitations

Prone to research bias because research is closely involved with the case being studied and may be subjective in its interpretations of the data.

This may reduce the validity of the study.

Another issue is that case studies often have small sample sizes, perhaps just one person or event. This makes it difficult to make generalisations from the data- low external validity.

Content analysis

A research method that enables the indirect study of behaviour by examining communications that people produce.

Analyses sets of qualitative data and turns it into quantitative data.

For example: tests, emails, films and other media.

The researcher goes through all the data collected and identifies a number of categories that can be used to classify the data.

Each category is then given a code (number).

Content analysis is used to study forms of communication such as spoken interaction, visual presentations or written forms.

The process of conducting a content analysis

Step 1: develop a coding system. The researcher goes through all of the data collected and identifies categories (called coding units because each category is given a code).

Step 2: code the data. The researcher now goes through the entire data set and counts instances that belong in each coding unit/ category.

Step 3: draw conclusions.

Content analysis- strengths.

Content analysis can get around many ethical issues associated with psychological research.

Many material analysts want to study material that already exists in the public domain; therefore, there are no issues in obtaining consent.

Content analysis- limitation

Content analysis may lack objectivity because the researcher can impose their own interpretations and biases onto the communication being studied, particularly when the original context is not considered.

Sections of a scientific report- abstract

Abstract:

The first section in a journal is a summary/ abstract (around 150-200 words in length).

It includes all the major elements:

aim and hypothesis, method/ procedure, results and conclusions.

When researching a particular topic, psychologists will often read lots of abstracts in order to identify studies that are worthy of further examination.

Sections of a scientific report- introduction

The introduction is a literature review of the general area of research, detailing relevant theories, concepts, and studies that are related to the current study.

The review should follow a logical progression- beginning broadly and gradually becoming more specific until the aim and hypotheses are presented.

Sections of a scientific report- Method

The method is split into several subsections and should include sufficient detail so other researchers can precisely replicate the study if they wish.

Design- The design is clearly stated.

Sample- information related to the people involved in the study: sample size, sampling method and target population.

Apparatus- details of any assessment instruments used and other relevant materials.

Procedure- a recipe-style list of everything that happened in the investigation. This includes a verbatim record of everything said to the participants- briefing, standardised instructions, and debriefing. This means another researcher can replicate the study.

Ethics- an explanation of how ethical issues were addressed within the study.

Sections of a scientific report- results

The results section should summarise key findings from the investigation

Likely to feature descriptive statistics such as tables, graphs, and charts.

Inferential statistics should refer to the choice of statistical test, critical and calculated values, and the level of significance.

Qualitative methods: results are likely to involve analysis of themes and categories and coding data.

Sections of a scientific report: discussion

The researcher will summarise the results in verbal rather than statistical form.

These results should be discussed in the context of the evidence presented in the introduction and other information that may be considered relevant.

The resarcher should discuss limitations of the present investigation, and this may include some suggestions of how these limitations might be addressed in a future study.

Wider implications of the research are considered.

This may include real-world applications of what was discovered and what contributions the investigation has made to the existing knowledge base within the field.

Sections of a scientific report- referencing

The references section includes full details of any source material cited in the report.

Journal references follow the format:

Author(s), date, article title, journal name (in italics), volume (issue), page numbers.

Book references:

Author(s), date, title of book (in italics), place of publication, publisher.

Paradigm and Paradigm shift defenition

Paradigm: a set of shared assumptions and agreed methods within a scientific discipline.

Paradigm shift: the result of a scientific revolution when there is a significant change in the dominant unifying theory within a scientific discipline.

Paradigm and paradigm shifts explanation

Thomas Kuhn (1962) suggested that what distinguishes scientific discipline from non-scientific discipline is a shared set of assumptions and methods- a paradigm.

Progress within an established science occurs when there is a scientific revolution: a handful of scientists begin to question the accepted paradigm; this critique gains popularity and pace, and eventually a paradigm shift occurs.

theory construction

The process of developing an explanation for the causes of behaviour by systematically gathering evidence and then organising this into a coherent account (theory).

Proposing a simple economic principle because it reflects reality.

An essential component of theory is that it can be scientifically tested.

A hypothesis is made, and then it can be tested using systematic and objective methods to determine whether it will be supported or refuted.

Supported- the theory is strengthened.

Rejected- the theory is abandoned.

Deduction: deriving a new hypothesis from an existing theory.

Falsifiability

The principle that a theory cannot be considered scientific unless it admits the possibility of being proved untrue.

Karl Popper (1934) suggested that genuine theories should hold themselves up for hypothesis testing and the possibility of being proven false.

Those theories that survive most attempts to falsify them become the strongest- not because they are necessarily true but because they haven’t been proven false.

Replicability

The extent to which scientific procedures and findings can be repeated by other researchers.

If a scientific theory is to be trusted, the findings from it must be shown to be repeatable across a number of different contexts and circumstances.

Used to assess the validity of a finding by seeing to what extent the findings can be generalised.

Objectivity and the empirical method

All sources of personal bias are minimised so as not to distort or influence the research progress.

Scientists must not allow their personal bias to discolour the data they collect or influence the behaviour of participants they are studying.

Lab experiments tend to be most objective; they have the most control.

Objectivity is the basis of the empirical method.

Theory cannot claim to be scientific unless it has been empirically tested and verified.

improving reliability- questionaires

Test- retest method.

Comparing two sets of data and finding out the correlation.

Questions may need to be removed if test-retest reliability is low.

E.g., the question is too complex or ambiguous.

Improving reliability- interviews

Using the same interviewer each time.

If that is not possible, then interviewers must be properly trained.

Interviewers mustn’t ask leading or ambiguous questions.

More avoided in structured interviews.

Improving reliability- observations

Making sure behavioural categories have been properly operationalised. so they are measurable and self-evident.

Categories should not overlap.

Observers may need further training.

Improving reliability- experiments

Procedures must be the same every time- standardised procedures.