Research methods #2 (complete)

What makes something a science ?

if it follows the scientific process

explain the scientific process diagram… (6 step)

1. observations

2. develop hypothesis

3. test hypothesis

4. analyse results

5. develop theory

6. repeat/modify

what are the 4 features of science?

1. theory construction and hypothesis testing

2. objectivity and the empirical method

3. replicability and falsifiability

4. paradigms and paradigm shifts

explain theory construction and hypothesis testing as a feature of science…

theory is an explanation for a phenomenon based on observations or empirical data (developed and modified through hypothesis testing)

hypothesis = testable statement making a specific prediction

What is objectivity and empirical method as a feature of science?

empirical evidence = gained through objective, real world experiments, observation and measurements.

objectivity = events and findings are not influenced by the individual experiencing them.

• high levels of objectivity increase confidence in the results regardless of who carried out the investigation - can be used to construct theories

what is replicability and falsifiability as a feature of science?

replicability = ability to check and verify the results of a study by repeating the method of the study to assess if similar findings are achieved, increased confidence in theory building if the results occur once or more

falsifiability = a proposition or theory can only be considered scientific if it is possible to show it as false.

What did Karl Popper (1902-1994) argue about falsifiability?

argued it was not possible to prove a theory, only to disprove it.

swans —>

if a scientist only ever saw white swans, develop a theory that all swans are white, but would only take one black swan to disprove that theory.

concluded that no amount of evidence can prove a theory to be right as one piece of evidence can prove a theory wrong. Therefore the null hypothesis of an experiment is tested to show if theory is incorrect - studies can only ever support a theory, not prove it

what are paradigms and paradigm shifts as a feature of science?

paradigms are a theory that is accepted by the majority of scientists in a field change over time as more evidence accumulated paradigm shifts

kuhn 1962 believed that science progresses in revolutions and called these revolutions paradigm shifts.

what are the three phases in science?

1. pre-science = variety of theories but not one generally accepted theory or paradigm

2. normal science = one theory (paradigm) remains dominant, despite occasional challenges

3. revolutionary science = disconfirming evidence accumulated until a theory/paradigm is overthrown, no longer maintained (paradigm shifts)

what are the two methods of analysis?

content analysis = statistical process that involves categorising and quantifying events and behaviour

• frequency and numerical coding of themes - finding categories or codes

thematic analysis = method to identify patterns of meanings and themes in data

• focus on finding recurrent themes in qualitative data.

What is coding and it’s importance in methods of analysis ?

• the process of placing data into categories

• potentially putting qualitative data into quantitative

• breaks down data into manageable categories for analysis

What are the 5 steps of content analysis?

1. get to know the data (repeated reading and viewing)

2. identify important categories

3. work carefully through the data (eg re-reading transcripts to make sure details are not overlooked)

4. count/tally the number of occurrences of each of the identified categories

5. check reliability by

   a) repeating content analysis (test-retest reliability)

   b) use second researcher to carry out content analysis on the same data independently (inter-rater reliability)

what are the two ways of checking reliability of findings?

1. test retest reliability - repeat content analysis on 1 or more occasions using same data set under same conditions, at different times

2. inter-rater reliability - use more than one researcher to carry out content analysis independently on same data.

in both cases compare results of the two analyses, calculate the correlation coefficient (+0.8 is generally accepted as indicating high levels of reliability)

What is the procedure for thematic analysis? (4 steps)

1. use coding initially to analyse data

2. review the data/codes looking for emergent themes

3. support for themes comes from direct quotes from the data

4. validity of themes is checked by collecting a new set of data

what is an emergent theme?

a theme a recurrent idea which is descriptive or qualitative

what are the strengths of content/thematic analysis?

• allow patterns in data to be analysed, and conclusions to be drawn

• can be replicated to improve validity

what are 2 weaknesses of content/thematic analysis?

• reducing data via coding can remove detail and reduce the richness of the data set

• subjective judgment is needed to define categories and coding units

  • opens up the possibility of researcher bias = looking for patterns to support a belief

what is reliability?

• consistency of a study or a measuring device within a study

• refers to when a study produces similar results if replicated

*different from replicability as there is a focus on the ability of the research method to provide consistent results, as a pose to the ability for another researcher to obtain the same results*

What 3 ways can reliability in experiments be improved?

• control investigation as much as possible keeping a constant environment (lab experiments with standardised procedures, standardised instruction)

• operationalise variables

• pilot test - to identify any problems

What 2 ways can reliability be improved in questionnaires?

• questions may need to be re-written or removed if they are too complex or ambiguous - leads to confusion and incorrect results

• Use more closed questions

What 3 ways can reliability be improved in Interviews?

• use the same interviewer each time - constant environment

• train interviewers to ensure questions are asked in the same way

• Use structured interviews/scripts

What 3 ways can reliability be improved in observations/content analysis?

• behaviour categories carefully operationalised and discussed to be clear on how to interpret different behavioural categories

• record the data so that it can be reviewed

• train observers/raters

what is validity and the two types?

concerns whether the test/measure/findings actually measures what it was intended to.

• internal validity = ability of the study to test hypothesis it was meant to, what is going on in the study.

• external validity = whether the findings of the study can be generalised beyond the study.

what are the 3 types of validity?

• ecological

• temporal

• population

Define ecological validity…

the degree to which the findings can be generalised to other situations, places or conditions

usually whether the behaviour of a study accurately reflects the way behaviour would occur in normal circumstances

define temporal validity…

where findings from research at the time it was taken still accurately reflect what would occur at the current time

define population validity…

where the findings from research can be generalised from the sample to other populations

What are the two ways of assessing validity?

• face validity - test appears to measure what it is supposed to

• concurrent validity - whether one test of a variable correlates highly with the results of another test of the same variable

what 7 factors can threaten validity?

• extraneous variables (variables other than the IV that impacts the DV)

• investigator effects (effects of researcher on research outcome)

• demand characteristics

• social desirability bias

• poor operationalisation of the variables

• order effects

• ppt differences

two ways to reduce

a. extraneous variables

b. Investigator effects

extraneous variables

• all variables except IV should be constant

• blind technique (ppts do not know what condition they are in)

investigator effects

• standardise procedures - eg use the same investigator

• double-blind technique (ppts/investigator now what each condition/group represents)

2 ways you can reduce…

a. demand characteristics

b. social desirability bias

demand characteristics

• conduct field experiments

• covert observations

social desirability bias

• include lie scale

• anonymity

how can you reduce…

a. ppt variables

b. order effects

c. poor opperalisation

ppt variables

• RM design, each acts as own control

• stratified sampling

order effects

• counter balancing (reverse order of tests)

poor operalisation

• clearly define variables and behavioural categories

What are descriptive/inferential statistics?

• descriptive = describes the findings making it easier to interpret (mean, median, mode, range)

• inferential = allow researchers to make inference (draw conclusions) about whole populations based on smaller samples

what is a statistical test?

statistical (inferential) tests are used to identify is results of a study were a genuine scientific outcome or due to chance factors

determines the likelihood that the difference or correlation a researcher has found had occurred by chance.

ie. Which hypothesis is most likely to be true

what do researchers do to start a statistical test?

write a experimental hypothesis of what they expect to find

write a null hypothesis stating there is no relationship between the variables/conditions

(then use statistical test to determine which hypothesis is more likely - accept or reject the null hypothesis)

what is the 4 step procedure of statistical tests?

1. collect data

2. identify appropriate statistical test to use

3. work out calculated value

4. compare to critical value to determine if result is significant

What are the three aspects research must be identified to determine which type of statistical test should be used.

• is it a test of difference or correlation

• if test of difference is it related = MP or RM, or unrelated = IG experimental design

• What level of measurement is used for DV/co-variables (nominal, ordinal, interval)

What are the three levels of measurement?

what measure of central tendency is most appropriate for each?

1. nominal

   data in categories and can be tallies, discrete form of data so is considered weak (mode)

2. ordinal

   data is ordered in some way/numerical,does not have equal intervals in between each unit. involves subjective human judgement or is in rank order (mode+median)

3. interval

   based on numerical scales (standardised), units of precise measurement like time or distances, objective form of data , most precise (mode+median+mean)

what mnemonic can you use to remember the order of statistical tests?

Carrots Should Come

Mashed With Swede

Under Roast Potatoes

what are the 8 statistical tests in order?

1. chi-squared

2. sign test

3. chi-squared (again)

4. Mann Whitney

5. Wilcoxon

6. spearman’s rho

7. unrelated t test

8. related t test

9. Pearson’s R

What statistical test may you have to find the calculated value in?

How is it found?

sign test

• data is converted to + or -, any ppts that stay the same are ignored

• count up +, - and same results (used to take away from the value of N)

• the least frequent result (the one there are least of) is the calculated value (s)

What is the critical value and how can it be found?

to see if calculated value is significant must compare it to the critical value found in a table:

to find appropriate critical value you need to know :

1. significance level

2. direction of the hypothesis (one or two tailed)

3. value of N or df (N = number of ppts, Df = N-1)

what is a significance level?

what level of significance is generally used?

the point at which a researcher can claim to have discovered a significant difference between conditions in an experiment or a significant correlation

otherwise known as a confidence level

generally 0.05 or 5% is used probability of result occuring by chance is less than or equal to 5% (95% confident that real result has been found)

What are one and two tailed hypothesis?

one-tailed = directional

two-tailed = non-directional

What must be included in a statistical conclusion?

• result is/isn’t significant

• because calculated value of… is less/more than the critical value of …

• for … df (degrees of freedom)

• in one/two-tailed test

• at… significant level

• means the null hypothesis is accepted/rejected

How do you calculate N in the sign test?

any ppts who stayed the same are disregarded

eg N = number of ppts - those who had no change

how is the df calculated in the chi-squared test?

create a contingency table

df = (number of columns -1) x (number of rows-1)

What does the calculated value in chi-squared represent?

The calculation compares the expected value with the observed value.

what are type 1 errors?

FALSE POSITIVE

• concludes results are significant when they are due to chance factors (not significant)

• rejected null hypothesis when it should have been accepted

• chances of making type 1 error are the same as the significance level

what are type 2 errors?

FALSE NEGATIVE

• concludes results are not significant when they are, real difference/association/correlation has been overlooked

• accept null hypothesis when should have been rejected

• Very difficult to predict the likelihood of type 2 errors

How can type 1 and 2 errors be minimised?

type 1

• replication - for a false positive to occur twice in unlikely

• increase significance level eg. from 0.05 to 0.01

  • however this may increase likelihood of type 2 errors so 0.05 is used as strikes a balance between the risk of making either errors

type 2

• increase sample size - larger sample = lesser likelihood that it will differ substantially from the population

(neither can ever be fully prevented)

What is peer review and its purpose?

When psychological research is subject to independent scrutiny by other psychologists that are experts in the field.

• important in scientific process = ensures research papers published in a scientific journal have integrity/can be taken seriously by other researchers.

• ensures quality and relevance of research

• ensures accuracy of findings

• used to evaluate proposed designs for research funding.

What is the 4 step process of peer review?

1. work is considered - in terms of validity, ethics, errors, significance, originality and improvements

2. reviewer makes suggestions for revisions - or rejects manuscript

3. author makes revisions - and the reviewer accepts the new revisions or rejects the manuscript

4. editor makes final decision - whether to accept/reject research report based on the reviewers comments/recommendations, decides if it should be published.

What are the 3 problems with peer review?

1. conservative - research that contradicts other research less likely to get published

2. lack of objectivity/anonymity - difficulty for peers to be objective as often know writer in small research fields

3. publication bias - journal editors have tendency to publish significant headline grabbing finding to increases credibility/popularity on their journals

What 3 aspects should be included on a consent form?

• outline of what the investigation entails: the general purpose and aims, what the study sill involve and how long it will take. anything that affects willingness to take part should be disclosed

• ppts informed of their ethical rights eg right to withdraw

• evidence the ppt has understood what is required - signature and date

What is the purpose of debriefing?

study is discussed with ppts returning them to the state they were in when they entered the study. especially important if deception has been used

What should be included in a debrief?

• written to ppts “You have taken part…”

• aims of the study

• other conditions that they were not involved in

• consideration of ethical issues

• opportunity to ask questions