Reliability and validity in schizophrenia

Reliability and validity

What is reliability in schizophrenia?

Means the consistency of diagnosis

Reliability and validity

What is diagnostic reliability?

• Test-retest: same clinician, same conclusion, two different points in time (eg: original diagnosis, and when watching a recording)

• Inter-rater reliability: different clinicians, same conclusions

Reliability and validity

What is a kappa score?

• measures inter-rater reliability

• 1 perfect agreement, 0 zero agreement

• 0.7 or above is considered good

• Regier et a (2013): DSM-V’s 0.46 kappa score for schizophrenia (moderate)

Reliability and validity

Reliability: what is cultural difference in diagnosis, and what does this indicate?

Cultural differences:

• Significant variation between countries in diagnosing schizophrenia,

Indicates that:

• Culture influences the diagnostic process

Reliability and validity

(Can be Evaluation): Reliability: examples of cultural differences in diagnosis → inter-rater reliability

Copeland et al (1971):

• 69% of the US psychiatrists diagnosed with schizophrenia vs 2% of British based on same description + same classification system (DSM-V) (low cross cultural inter-rater reliability)

• Suggests reliability can be consistent within, but inconsistent between cultures (so cultural variations impact reliability

Reliability and validity

(Can be Evaluation): Reliability: examples of cultural differences in diagnosis → symptom variance

Luhrmann et al (2015):

• Interviewed adults (20 Ghanian, 20 Indian, 20 American)

• Many African + Indian Ps reported positive experiences w/ voices BUT US patients violent/distressing voices

• This suggests manifestation of symptoms vary culturally, + using one universal classification tool (imposed etic) would result in inconsistent, and so less reliable, diagnoses due to the variance in symptoms.

• Can be overcome by using an emic approach (eg: indigenous classification systems)

Reliability and validity

Rosenhan (1973) (part 1):

• 8 pseudo patients, w/ fake name/job/one symptom (unfamiliar voice that said ‘hollow’, ‘empty’, ‘thud’) everything else said was real)

• Recorded treatment + operation of ward

• After admission - told staff they felt ‘fine’ and symptom was gone

• NO Ps detected as not having schizophrenia by staff

• Normal behaviour seen as symptoms → similar to real patients (eg: waiting outside of cafeteria = 'oral-accusative syndrome', NOT boredom)

• Were in hospital b/ 1-7 weeks - discharged w/ 'schizophrenia in remission), BUT some other patients suspected they were sane (35/118 patients voiced this)

Reliability and validity

Rosenhan (1973) (part 2):

• Told institutions pseudo-patients were being admitted (none were) and people were falsely accused of being one

• suggests → Diagnosis label = schema

• INSTEAD should focus on individual's specific behaviours + problems

Reliability and validity

What does Rosenhan (1973) tell us about the problems of reliability of diagnosis?

LACKS reliability:

• Definitions need to be more operationalised, rigid, and structures

• Demonstrates psychiatrists cannot reliably tell the difference b/w sane and insane people

COUNTER-POINT:

• Invalid - admitted when they didn't have schizophrenia but pretend they did - hospitals are more likely to say 'better safe than sorry' (not discharge someone who isn't ready)

Reliability and validity

What is validity in schizophrenia?

The extent to which a diagnosis represents the disorder it is measuring, that is distinct from other disorders

Reliability and validity

Validity: what is gender bias in diagnosis?

• Clinicians can base judgements of diagnosis on stereotypical beliefs about gender

• Eg: Boverman et al (1970) → women less likely to be diagnosed than men

• In USA, definitions of a mentally healthy adult = a mentally healthy male (androcentric) → led people to believe women = less healthy bcs deviate (from 'male' behaviour) BUT may be bcs differences in help-seeking behaviour → measuring gender behaviours NOT disorder

Reliability and validity

Validity: what is symptom overlap? What is the issue with it?

• ICD and DSM have been known to be unsuccessful when it comes to differentiating between symptoms of S and other disorders

• This lack of distinction calls into question validity of classification/diagnosis of S, + if S = UNIQUE disorder, or if S/DID/Bipolar = part of the same branch of disorder

Reliability and validity

Validity: examples of symptom overlap

Ellson and Ross (1995):

• People w/ DID have more S symptoms than people diagnosed w/ S

• both have positive and negative symptoms

Read (2004):

• most people with schizophrenia have sufficient symptoms to receive at least one other diagnosis (Eg: bipolar disorder shares positive symptoms like delusions, + negative symptoms like avolition.

Reliability and validity

Validity: what is co-morbidity?

• when two conditions co-occur - common in schizophrenic patients

• 50% have depression

• 12% have OCD

Reliability and validity

What does Rosenhan (1973) tell us about the problems of validity of diagnosis?

• They weren’t able to recognise what disorder it was

• Behaviour was often misinterpreted in terms of diagnosis (eg: patients waiting for lunch)

• SUGGESTING definition of schizophrenia is not clear enough

Reliability and validity

Evaluation: diagnosis has low reliability

diagnosis has low reliability, (consistency of a study) If a study = inconsistent → lack reliability. Reiger et al (2013): DSM-V's S kappa score = 0.46, suggesting low reliability → inconsistency b/w mental health professionals = limitation of the diagnosis → suggests DSM-V = invalid

Reliability and validity

Evaluation: diagnosis has low validity

Limitation of diagnosis = low validity →  standard assessment = is criterion validity (do different assessments systems give same diagnosis for same patient?) → Ellason + Ross (1995) found people w/ DID have more S symptoms than people diagnosed as S → questions if schizophrenia is a UNIQUE disorder, or if schizophrenia/DID/Bipolar are part of the same branch of disorder

Reliability and validity

Evaluation: symptom variations = issue w/ validity

two people w/ S diagnosis can share NO SYMPTOMS w/ each other → questions validity of classification tools in accurately representing/describing the disorders it attempts to

Reliability and validity

Evaluation: Gender bias

Longnecker et al (2010)'s meta-analysis of schizophrenia prevalence found men are diagnosed more often than women since the 1980s, so gender bias means that people with similar symptoms are receiving different diagnosis's

Reliability and validity

Evaluation: Culture bias

People of African origin are more likely to be diagnosed w/ schizophrenia in the UK, and rates are lower in Africa/West Indies - so not bcs of genetics, but rather bcs of culture bias meaning symptoms in the UK, but considered 'normal' in African cultures (eg: AVH) - issue w/ validity