Research Methods

Week 1

Ethics

==Importance of good research==: Research is important in speech pathology to determine if treatment & assessment is safe, effective, better than the previous one, cost effective & working in clinical practice.

==Ethics in speech pathology==: Ethics is essential for the protection of human subjects.

==Examples of unethical research==:

==Principles of Ethics==:

• @@Beneficence@@: do good
• @@Non-maleficence@@: do no harm
• @@Autonomy@@: participants have the power to make moral choices & rational decisions
• @@Justice@@: fair, equitable and appropriate treatment of participants

==Declaration of Helsinki==:

• The World Medical Association (WMA) has developed the Declaration of Helsinki as a statement of ethical principles for medical research involving human subjects, including research on identifiable human material and data.
• “The wellbeing of human subject should take precedence over the interests of science and society.”
• Consent should be in writing
• use caution if participant is in dependent relationship with researcher
• Limit use of placebo
• Participants benefit from research

==Features of ethical research==:

• @@Valid research design@@: only valid research yields correct results; takes account of relevant theory, methods, and prior findings ~ B, R
• @@Competence of researchers@@: investigates capable carrying out procedures validity ~ B, R
• @@Identification of consequences@@: assessment of risks and benefits; respected privacy, ensure confidentiality, maximise benefit, minimise risk (B, R, J)
• @@Selection of subjects@@: subjects must be appropriate to the purpose of study, representative of the population to benefit from research, and appropriate number (B, R, J)
• @@Voluntary informed consent@@: obtained in advance
• @@Compensations for injury@@: researcher is responsible for what happened to subjects. Federal law requires subjects to be informed if harm is compensated but does not require compensation (B, R, J)

==Informed consent==: Is a process by which a subject voluntarily confirms his or her willingness to participate in a particular clinical trial, after having been informed of all aspects of the trial that are relevant to the subject’s decision to participate

Epidemiology

==What is epidemiology==: The science is concerned with the study of the factors determining and influencing the frequency and distribution of disease, injury and other health-related events and their cause in a defined human population

Epidemiology uses statistics to provide a snapshot of a populations diseases, illnesses and disability. This snapshot is then used by various people and groups to inform health promotion and guide research.

==Epidemiology describes==: a disease and all relevant factors, and the relationship between disease and these factors

Epidemiological observations relate to primarily large groups

==The purpose of epidemiology==:

• Identifies cause and risk factors for disease
• Determines the extent of disease in the community
• Studies natural history of disease and prognosis of disease
• Evaluates preventative and therapeutic measures
• Provides foundation for public policy

==Measurements of disease frequency==:

@@Incidence@@

• The estimated rate of new cases during a given time period. Usually measured in terms of 100,000 people
• e.g. the incidence of laryngeal cancer in Australia in 2002 was 8/100,000. the incidence in 2007 was 5/100,000 in men and 5/100,000 in women

@@Prevalence@@

• The estimate proportion of existing cases in a population during a given time period
• e.g. the prevalence of dysphagia in nursing home residents is 25%

==Benefits of epidemiological studies==

• Risk factors, allows us to determine big-picture scenario
• Identifying areas of strength and weakness for a population’s health. For example, Australia has a high life expectancy for both males and females is a strength, but we have an increasing number of people getting diagnosed with type 2 diabetes.
• Epidemiology helps us to identify the major causes of mortality and morbidity across Australia and smaller population groups, such as people of low socioeconomic status or Aboriginal and Torres Strait Islanders peoples.
• It gives us information about how people use our health care system and helps identify outbreaks in certain diseases or illnesses.

==Limitations of epidemiological studies==

• Bias, or representative sample
• Although it provides us with a snapshot of disability, disease and illness across our country, it does not measure the positive aspects of health, such as how socially connected people are.
• It may tell us how much physical activity people do but does not inform us of the quality of that physical activity.
• Nor does epidemiology tell us about the quality of life people are living.
• It can report on self-reported happiness, but beyond this, it is limited in how to determine what is making people happy.

==Clinical research==

• @@Sample size@@: the people within the studies. Represented by “n=”

  

==Variables==

• @@Independent variable@@: is the cause or predictor. Its value is independent on other variables in your study
• @@Dependent variable@@: is the effect or outcome. Its value depends on the independent variable. Cue: it is dependent on something (ie independent variable) to change

@@Examples of dependent and independent variables@@

==Confounding factors==

Common confounders are attributes of the participants; for example, body mass index, smoking status, age at onset of illness, socioeconomic status, educational status, and extent of support network. Life events are also potential confounders.

\n

Week 2

Research Design

==Primary research==: individual studies, attempting to answer a specific research question using data collected by researchers. They can be experimental or non-experimental.

==Secondary research==: “synthesized research” combined findings from primary research studies to provide strong conclusions about that body of evidence

~ systematic reviews, meta-analyses, clinical practice guidelines

==Experimental research==: The researcher manipulates one or more variables. The researcher compares those that received the manipulated condition versus those that did not.

@@Example of experimental research@@: Elderly nursing home residents are assigned to a control group (sham training) and an experimental group (respiratory muscle training). The speech pathology researchers compare cough peak flow measurements in each group after 4 weeks.

==Non-experimental / observational research==: The researcher does not control or manipulate any variables. The researcher may be made at different time points. Measurements may be made at different time points. The researcher may observe outcomes in response to different exposures/settings/situations.

@@Example 1 of non-experimental/observational research@@: A speech pathology researcher follows 100 patients with Parkinson’s Disease for 5 years to determine the factors that lead to the development of aspiration pneumonia. Every 6 months, participants’ cough strength and tongue strength are tested.

%%In this example, there is no independent variable. The researcher does not manipulate anything. We are simply observing outcomes.%%

@@Example 2 of non-experimental/observational research@@: A speech pathology researcher is interested in the relationship between physical activity and the development of aspiration pneumonia in patients with dysphagia. Data is collected on physical activity (hours of exercise per week) and development of aspiration pneumonia (over 6 months) from all outpatients with dysphagia.

==Retrospective==: Data is collected from studies of participants’ past. The data is already there.

@@Example of retrospective study@@: A speech pathologist conducts a chart review of language outcomes from 1000 with cleft palate.

==Prospective==: The researcher follows participants over time and collects data.

@@Example of prospective study@@: A researcher invites 50 patients with Parkinson’s disease to participate in a respiratory muscle strength training study. Participants are assigned to an intervention vs control group. Data is collected over 4 weeks.

==Levels of evidence==:

@@Systematic review & meta-analysis@@: The highest level of evidence. These are secondary research - which synthesises the findings of other studies.

@@Randomised control trial@@: High level of evidence & primary research (involves conducting a study to collect new data)

@@Control trial (non-randomised)@@: A study involving non-randomised groups to determine the effects of an intervention. Pilot study design & if participants cannot be randomised for ethical reasons (e.g. children - testing the effects of early intervention on children with severely delayed speech and language)

@@Randomisation@@: Helps to cancel out confounding factors, because there is equal chance they will occur in both groups.

@@Observational studies@@: include cohort studies, case controlled studies, cross-sectional studies & case studies

Cohort study: Follows participants over a period of time. Outcomes are often taken at multiple timepoints.

Cross sectional study: Evaluates participants at 1 point in time. They are not followed overtime. Provides a snapshot.

Research Questions

==The type of research design to use:== your clinical question determines the study design (e.g. randomised controlled trials, single-subject design)

==How to design your research question with PICO==:

@@P@@ = Population

@@I@@ = Intervention

@@C@@ = Comparison

@@O@@ = Outcome

@@Example of clinical research questions 1:@@

For children with severe to profound hearing loss, what is the effect of cochlear implants compared with hearing aids on speech and language development.

P = Children with severe to profound hearing loss

I = Cochlear implants

C = Hearing aids

O = Speech and language development

%%Best type of study to answer question%%: RCT

@@Example of clinical research questions 2:@@

What is the effect of cognitive rehabilitation on vocational outcomes in young adults who experience a stroke?

P = Young adults who experience a stroke

I = cognitive rehabilitation

C =

O = vocational outcomes

%%Best type of study to answer question%%: Case series / case study / cohort

Bias

==Cognitive bias==:

@@Confirmation bias@@: look for evidence to confirm what we know

The search for and use of information to support an individual’s ideas, beliefs or hypotheses. Example: A speech pathologist recommended an intervention for an adult with dysphagia. She cited one randomised control trial (RCT) which reported positive results. However she ignored other publisehd RCTs which reported no signification improvement.

@@“Hot stuff” bias@@: favouring fashionable topics

==Study design bias==:

@@Attrition bias@@: participants who stay/leave a trial

Randomised controlled trial involved 27 patients included, 6 weeks of exercising using shaker exercise. Final analysis: 11 patients in treatment group, 7 patients in control group.

@@Hawthorne effect@@: changing our behaviour to match those around us

@@Selection bias@@: choosing a certain group of people

@@Re-call bias@@: the ability to recall past events or experiences

@@Gender bias@@: excluding a particular gender

Healthcare data is bias towards men. The Lancet → Cardiology;’’s problem women - many women are missed for heart attacks, the onset of heart attack is different in women, risk factors may also be different

==Reporting bias==:

@@English language@@: publication of research in English

@@Positive results@@: publication of positive results only

==Publication bias==: Only publishing the research with statistically significant results.

==Blinding==: Is one method researchers can use to minimise bias.

@@Double blinded study@@: both the researcher and the participant are blinded to who is receiving the treatment. E.g. drug versus placebo.

@@Single blinded@@: Only the researcher doing the study knows which treatment or intervention the participant received.

@@Outcome assessor blinding@@: The individual taking the outcome measures (and/or analysing the data) is blinded to which participants received the intervention.

Power analysis

==Statistical significance:== Statistical significance quantifies whether a result is likely due to chance or a factor of interest. Is there a true effect, or is the result job just random? It also helps us to quantify whether the results from our study sample are representative of the population.

@@Examples of statistical samples in research questions@@:

• There was a statistically significant difference in intelligibility following the intervention.
• There was a statistically significant difference in measures of tongue strength between the two groups.
• Respiratory training had a statistically significant effect on improving cough peak flow.

P value > 0.05 = statistically significant result. There is a <5% probability that the null hypothesis is true (and the results are random)

Week 3

Hypothesis testing

==Null hypothesis==: The null hypothesis tells us that there is no relationship between two variables being studied.

@@Examples of null hypothesis@@:

1 RQ: What is the effect of oral motor exercises (independent variable) on tongue strength (dependent variable)?

Null: There is no effect of oral motor exercises on tongue strength.

If p < 0.05, we reject the null hypothesis (Oral motor exercises increased tongue strength). This means that there is a <5% probability that the null hypothesis is true i.e. a very small change the null hypothesis is true.

2 RQ: What is the effect of a stuttering intervention on levels of social anxiety?

Null: There is no effect of stuttering intervention on levels of social anxiety.

3 RQ: What is the effect of thickened fluids on rates of apsiration pneumonia in patients following stroke.

Null: There is no effect of thickened fluids on rates of aspiration pneumonia.

@@Why do we write null hypothesis@@: In order to show a result is statistically significant, a researcher must reject the null hypothesis. To reject the null hypothesis, the p-value must be <0.05. This means that there is a <5% probability that the null hypothesis is true.

==Errors in hypothesis testing==: Hypothesis testing is not fool proof and can lead to error. Specifically a type I error and a type II error.

==Type I error==: A false positive. This observes a difference when in truth there is none. We reject the null hypothesis in error. (e.g. “you’re pregnant” to a man). We try to avoid these errors as much as possible in research. !!! avoid type I errors at all costs!! but we may prefer false-positive results for screening

Example of Type I error:

RQ- What is the effect of the covid-19 vaccine on disease severity.

Null: There is no effect of the covid-19 vaccine on disease severity.

Type I error: Falsely reject the null hypothesis (p<0.05)

Conclusion: There is an effect of covid-19 vaccine on reducing disease severity. -→ Vaccine is approved for use.

==Type II error==: A false negative. Failing to observe a difference when there is one. Often due a small sample size or an “underpowered” study. e.g. “You’re not pregnant” to a pregnant woman.

This is riskier in events, and is worse & more dangerous. FALSELY FAILS TO REJECT THE NULL

Example of Type II error:

Q: what is the effect of the covid-19 vaccine on disease severity?

Null: There is no effect of the covid-19 vaccine on disease severity.

Type II error: Cannot reject the null hypothesis (p>0.05)

Conclusion: There is no effect of covid-19 vaccine on (reducing) disease severity. Vaccineis not approved for use.

Applying false-positives and false-negatives in screening tests.

@@What is the effect of a high false-positive?@@ The test identifies someone as positive when they are in fact negative.

@@What is the effect of a high false-negative? @@ The test identifies someone as negative when they are in fact positive. This is riskier in events, and is worse & more dangerous.

Power analysis

==Power==: “The probability to detect a statistically significant difference if it exists.” → higher power minimises Type II errors.

==Power analysis==: Is used prior to conducting the experiment to determine the size of the sample needed. The number of participants can determine whether type II errors exist. May also be used after conducting the experiment to determine if non statistically significant results were due to having too small a sample size.

==Post-hoc power analysis==: Will help identify how many people are needed for the study.

==Bad terminology in research==: “Borderline significance”, “a trend”, “the end towards significance”

==Large sample sizes==: Are highly powered. Results will likely be statically significant, but are they clinically significant or clinically meaningful. Statistically significant does not mean it is clinical significant.

! As clinicians, you need to evaluate between clinically and statistically significant findings.

==Cherry picking:== Cherry picking has a few different definitions, but it is most often thought of as the process of selecting a small amount of information or data to attempt to prove a point, while ignoring contradicting information.

@@How to identify cherry-picking@@: read the methodology and identify

• How many patients?
• What was their initial research question?
• What kind of materials were used?
• What questionnaire was used? Was it validated?
• Was all data included in the analysis and results? Did they leave some out?

Commonly used tricks occur may happen during a sub-group analysis.. For example, identifying a statistical difference in a certain group. If you plan to do sub-group analysis.. you must specify this in your research plan “a priori”. Examples include:

Week 4

Journal Club