Reliability and validity in diagnosis and classification

reliability and validity in diagnosis and classification of schizophrenia

Classification systems such as DSM-V are worthless unless they’re reliable. Reliability refers to the consistency of a classification system such as DSM e.g. to assess particular symptoms of schizophrenia. Reliability alone counts for nothing unless these systems and scales are also valid. Validity refers to the extent that a diagnosis represents something that is real and distinct from other disorders and the extent that a classification system such as DSM accurately diagnoses schizophrenia. Reliability and validity are inextricably linked because a diagnosis cannot be valid if it’s not reliable.

validity

Validity is the extent to which we are measuring what we are intending to measure; in the case of schizophrenia it concerns how accurate the diagnosis is.

One standard way to assess validity of diagnosis is concurrent validity (amount of agreement between two different assessments)

evidence investigating validity - cheniaux et al (2009)

Looking at the results from the Cheniaux et al. study above we can see that schizophrenia is much more likely to be diagnosed using ICD than DSM. This suggests that schizophrenia is either over-diagnosed in ICD or under diagnosed in DSM. Either way, this highlights an issue with concurrent validity. Different assessment systems do not arrive at the same diagnosis.

Reliability

Reliability means consistency of symptom measurement - an important measure being inter-rater reliability; this is the extent to which different clinicians agree on their assessments.

In the case of diagnosis inter-rater reliability means the extent to which two or more mental health professionals arrive at the same diagnosis for the same patients

Evidence investigating reliability - cheniaux et al (2009)

had two psychiatrists independently diagnose 100 patients using both DSM and ICD criteria.

Inter-rater reliability was poor, with one psychiatrist diagnosing 26 with schizophrenia according to DSM and 44 according to ICD, and the other diagnosing 13 according to DSM and 24 according to ICD. This evidence highlights weaknesses in the use of classification systems to diagnose schizophrenia. This is an issue as the external reliability is low as the psychiatrists failed to diagnose the patients consistently. This poor reliability is an issue for the diagnosis of schizophrenia.

HOWEVER, It is important to note that Cheniaux research was carried out using the DSM-IV and not DSM-5 and evidence generally suggests that the reliability and validity of diagnoses has improved as classification systems have been updated.

Jakobson et al (2005) tested the reliability of the ICD-10 classification system during the diagnosis of schizophrenia. 100 Danish patients with a history of psychosis were assessed using operational criteria, finding a concordance between clinicians of 98 per cent, demonstrating the high inter-rater reliability of clinical diagnosis of schizophrenia using up-to-date classifications.

Even if reliability and validity of diagnosis based on classification systems is not perfect, they do provide clinicians with a common language, permitting communication of research ideas and findings, which may ultimately lead to a better understanding of the disorder. They can then predict the outcome of the disorder and aid in the development of effective treatments.

symptom overlap

Symptom overlap is the perception that symptoms of schizophrenia are also symptoms of other mental disorders.

Despite the claim that the classification of positive and negative symptoms would make for more valid diagnosis of schizophrenia, many of the symptoms of the disorder are often found with other disorders, which makes it difficult for clinicians to decide which particular disorder someone is suffering from.

Read (2004) argued people diagnosed with schizophrenia have sufficient symptoms of other disorders that they could also receive at least one other diagnosis.

For example, Symptom overlap especially occurs with bipolar disorder, where negative symptoms e.g. depression and avolition are common symptoms, as well as positive symptoms e.g. hallucinations.

This highlights issues with the validity of trying to classify schizophrenia because a patient might be diagnosed as schizophrenic with the ICD, however, many of the same patients would receive a diagnosis of bipolar disorder according to DSM criteria.

A consequence of this issue could mean that individuals are misdiagnosed which can lead to years of delay in receiving relevant treatment, during which time suffering and further degeneration of symptoms can occur for the individual.

Co-morbidity

Co-morbidity is the phenomenon that 2 or more conditions occur together.

One issue which impacts the reliability and validity of a diagnosis is comorbidity. This is when two or more conditions occur together (e.g. Depression and Bipolar).

For example, Schizophrenia is commonly diagnosed with other conditions. Buckley et al. (2009) concluded that around half of patients with a diagnosis of schizophrenia

also have a diagnosis of depression (50%) or substance abuse (47%). Post- traumatic stress disorder also occurred in 29% of cases and OCD in 23%.

Therefore, comorbidity is an issue for the diagnosis and classification of schizophrenia. Different diagnoses could be given for the same person; in one instance they could be diagnosed with schizophrenia, diagnosed with bipolar in another instance or they could be diagnosed with both conditions. This issue could lead to inconsistencies in diagnoses between clinicians in relation to which disorder is diagnosed e.g. Schizophrenia or Depression, creating problems for the reliability of diagnosis.

Furthermore, comorbidity is also an issue for the classification of schizophrenia. Having simultaneous disorders suggests that schizophrenia may not actually be a separate disorder. A consequence is that it lowers the (descriptive) validity of schizophrenia, which can make effective treatment for schizophrenia difficult to achieve.

Cultural bias in diagnosis

Culture bias concerns the tendency to over-diagnose members of other cultures as suffering from schizophrenia.

Culture bias is another problem which affects the validity of diagnosis. Although cross-cultural research of schizophrenia suggest a similar prevalence across races, research has shown that Schizophrenia, despite culturally formulated updates to diagnostic manuals, is repeatedly diagnosed at a higher rate in the African American population.

Research by Cochrane (1977) reported the incidence of schizophrenia in the West Indies and Britain to be similar, at around 1%, but that people of Afro-Caribbean origin are 7 times more likely to be diagnosed with schizophrenia when living in Britain. Considering the incidence in both cultures is very similar this suggests that higher diagnosis rates are not due to a genetic vulnerability, but instead may be due to a cultural bias

Although there is not one explanation determining why African Americans are overrepresented. Two possible speculations are:

• Clinician bias - unconscious process stemming from stereotypes and biases which results in misdiagnosis (Schwartz, 2014).

• Under diagnosis of other disorders (Depression/Bipolar) in African Americans could contribute to the over-diagnosis of Schizophrenia.

Gara et al (2019) found that African American men with severe depression tend to be misdiagnosed with schizophrenia in comparison to other racial groups. The findings suggest that clinicians put more emphasis on psychotic than depressive symptoms in African-Americans, which skews diagnoses toward schizophrenia even when these patients show similar depressive and manic symptoms as white patients.

This is an issue as it suggests a lack of validity in diagnosing schizophrenia in people of African-American origin as differences in symptom expression are overlooked or misinterpreted by clinicians. The consequence of the misdiagnosis is that it prevents them receiving the optimal treatment for their disorder and puts them at risk of the side effects of medication taken for schizophrenia, such as diabetes and weight gain

gender bias in diagnosis

The tendency for diagnostic criteria to be applied differently to male and females and for there to be differences in the classification of the disorder.

Some critics of the DSM diagnostic criteria argue that some diagnostic categories are biased towards pathologising one gender rather than the other. For example, Broverman et al. (1970) found that clinicians in the US equated mentally healthy ‘adult’ behaviour with mentally healthy ‘male’ behaviour, illustrating a form of androcentrism. As a result there was a tendency for women to be perceived as less mentally healthy when they do not show ‘male’ behaviour. Also interestingly, some research has indicated that a psychiatrist’s gender might affect their ability to diagnose.

Loring and Powell (1988) - gender bias investigation

Loring and Powell (1988) randomly selected 290 male and female psychiatrists to read cases studies of patients’ behaviour and make a judgement on these people using standardized diagnostic criteria (e.g. DSM). When the patients were described as ‘male’ or no info about gender was given, 56% of psychiatrists have a schizophrenia diagnosis. When patients were described as ‘female’, only 20% were given a diagnosis of schizophrenia. Interestingly, the gender bias was not as evident among the female psychiatrists, suggesting that diagnosis is influenced not only by gender of the patient but also the gender of the clinician.

There is also gender bias in the fact that when making diagnoses, clinicians often fail to consider that males tend to suffer more negative symptoms than women (Galderisi et al., 2012) and women typically function better than men, being more likely to go to work and have good family relationships (Cotton et al. 2009).

This high functioning may explain why some women have not been diagnosed with schizophrenia when men with similar symptoms might have been; their better interpersonal functioning may bias clinicians to under-diagnose the disorder, either because symptoms are masked altogether by good interpersonal functioning, or because the quality of interpersonal functioning makes the case seem too mild to warrant a diagnosis.

These misconceptions could be affecting the validity of a diagnosis as clinicians are not considering all symptoms. This can be an issue and can lead to men and women who experience similar symptoms being diagnosed differently

Clinicians also have tended to ignore the fact that there are different predisposing factors between males and females, which give them different vulnerability levels at different points of life, which may impact the validity of diagnosis. The first onset occurs in males between 18-25 years whereas, females between 25-35 years. This difference may be related to differences in the types of stressors both sexes experience at different ages and to age-related variations in female menstrual cycle, which tends to be overlooked during diagnosis.