Research Methods

Chapter 1 - Introduction to Research

Learning Outcomes:

• Discuss how the research process is important to understanding the current knowledge base in kinesiology

• Describe a variety of research methods used in kinesiology

• Identify implications that different philosophical worldviews have on the research process

Research In the Real World:

• When was the last time you did some research?

• How many times a day do you wonder about something?

• Has someone else wondered about that same thing and found an answer to it?

Current Knowledge in Kinesiology:

• Understanding how knowledge within a field is generated is important because it allows us the skills to critique an existing body of literature

• Understanding the research process provides the background needed to answer questions that no one knows the answer to yet

• How do facts become known as truths?

  • Ex. weight training -> strength gains

• Sometimes what we know is based on our own experiences

  • Personal experience is limiting and can vary substantially

• If we haven’t experienced something directly, we tend to rely on information gathered from outside sources

  • Ex. expert, professor, certified coach

• New discoveries are being made all the time

  • What was previously accepted as fact might actually be fallacy

• An understanding of science and research are essential

• There are a range of professionals with kinesiology-related backgrounds

  • Ex. fitness trainers, exercise therapists

• Many who graduate with kinesiology degrees also go on to other professional programs such as physical therapy, dentistry or medicine

• Other professions include the police force, border service, business, clinical psychology

• Digby Elliott (2007) provided a historical review of kinesiology programs developed in Canada

• Emergence of kinesiology programs in Canadian universities began in the 1960s at Simon Fraser University and the University of Waterloo

• Kinesiology: the study of movement

  • Hence, research is easily recognized as core to the discipline

Introduction to Research Methods:

• The terms science and research are commonly used by researchers in kinesiology

• Science: the discovery of knowledge

• Research: specific method used to discover knowledge

  • These terms are thus intimately tied together

• Cresswell (2014) identified three approaches to research:

  • Quantitative research designs

  • Qualitative research designs

  • Mixed methods research designs

• Each approach differs in underlying assumptions, the types of questions asked, the specific methods used, the type of data that results and the analysis of data

Chapter 2 - Research Methods & Worldviews

Learning Outcomes:

• Describe a variety of research methods used in kinesiology (quantitative, qualitative, mixed methods)

• Identify implications that different philosophical worldviews have on the research process

Research Methods:

• Cresswell (2014) identified three approaches to research:

  • Quantitative research designs

  • Qualitative research designs

  • Mixed methods research designs

• Each approach differs in underlying assumptions, the types of questions asked, the specific methods used, the type of data that results, and the analysis of data

Quantitative Research:

• The generation of numerical data to answer research questions

• Suited to questions related to the testing of theory

  • Ex. social cognitive theory: what dictates a person’s behaviors, theories and interactions? 

  • Can then use theory to design interventions to help people

• Differences among groups and relationships among variables

• Validity is fundamental so that any conclusions that are made are accurate and can be applied to populations beyond the study sample

  • Works for people outside of study

• Researchers attempt to be as objective as possible

• Typically use large sample sizes

• Focus heavily on the measurement of variables

  • The idea that there is an objective truth, focusing on this truth gains us knowledge

• Use statistics for data analysis

Qualitative Research:

• Einstein: not everything that can be counted counts. Not everything that counts can be counted

• Generation and interpretation of non-numerical data

• Sources of data include open-ended interviews, direct observations, written documents, art-based methods

• Well suited to understanding people’s meanings of experience

• Design is emergent and flexible

• Data is typical collected in the participant’s natural settings

• Themes are generated from the data

  • Trying to gather themes that summarize data

• The researcher is an integral part of the research process

  • Should acknowledge worldview in research

• Sample sizes tend to be much smaller than in quantitative research

  • Logistically speaking

  • Many interviews vs an impossible amount

  • Saturation point: enough data that it starts to get repetitive

Mixed Methods Research:

• Combines quantitative and qualitative research methods

• This can take many forms

  • Designs that prioritize either one or both quantitative and qualitative methods

  • Designs in which quantitative and qualitative methods are conducted simultaneously or one following the other

  • Larger programs of research combining quantitative and qualitative research studies

Components of a Research Design:

• The research approach and methods employed in a study are inherently linked

  • Ex. the quantitative approach to research requires the application of appropriate quantitative methods

• There are several decisions to make and steps to take when planning and designing a research study

• A researcher’s philosophical worldview represents a set of beliefs related to her or his general orientation of the world and the nature of research

  • Dictates what a researcher believes counts as knowledge

  • Permeates the entire research process from beginning to end

• Ex. Eastern medicine vs Western medicine

  • Discredit and do not accept things as knowledge

• Ontology and epistemology are two concepts aligned closely with the philosophical worldview

  • Ontology: general orientation to the world, refers to someone’s belief in the nature of truth and reality

    • What you believe is true and real

    • The study of reality

  • Epistemology: nature refers to someone’s belief about how we acquire knowledge about truth and reality

    • How do you believe we acquire knowledge

    • The study of knowledge

• Philosophical worldview > research approach > research methods

• Cresswell (2014) identified four worldviews that are common in the literature

  • Postpositivism: premise on the notion that there is a single reality or objective truth

    • Includes assumptions of determinism (causes determine effects) and reductionism (ideas can be reduced to testable research questions - deductive reasoning)

    • It relies on theory to uncover objective reality

    • It is guided by the scientific method (very structured)

    • Characteristics of the quantitative approach align with postpositivism

  • Constructivism: based on the notion that multiple realities exist and that meaning is varied and complex

    • The meaning of experience is subjective and socially constructed (inductive reasoning)

      • Reality is different for each of us

      • We create our reality based on our experiences

    • Researchers seek complexity of views

      • Things are complicated and can’t be simplified into a single number

    • Researchers recognize that their own realities and views shape the research process as there is no way that they are not influencing it

    • Characteristics of the qualitative approach align with constructivism

  • Pragmatism: premised on the idea that researchers need to be concerned with solutions to problems

    • No commitment to any single notion of reality

    • Researchers are concerned about application, or doing what works

    • Researchers incorporate all research approaches that are appropriate and necessary to understand their research problem

    • Characteristics of mixed methods approaches align with pragmatism

    • Want research to make something work, serve a purpose

      • Addressed at solving a specific problem

  • Transformative: based on the notion that research needs to be closely connected with politics and have an action agenda

    • Researchers focus on reform and change to better the lives of participants, make positive change in world

      • Ex. social movement that is happening

    • Challenging existing structures in society

    • Research is inherently collaborative

    • Characteristics of the qualitative approach align with a transformative philosophical worldview

      • Though the quantitative approach can add important elements

  • Two-eyed seeing (emerging and not as common): rooted in the belief that there are many ways of understanding the world, some represented by indigenous knowledge systems and others by European-derived sciences

    • Reflects the bringing together of knowledge from the strengths of indigenous ways and the strength of western ways of knowing

    • All knowledge systems are equitable

      • Many ways of understanding the world

      • Gather what is important from community and mix with what is important for them

    • Particularly valuable in allowing for diversity of perspectives

    • Two different backgrounds and ways of seeing things are combined to form new perspective

• Each worldview has inherent assumptions about knowledge that drive the research approach and distinct methods or procedures used in research

  • Different perspectives on what is true and what is knowledge

Philosophical Worldviews as Guiding Frameworks of Research:

• What counts as knowledge to researchers will influence their entire program of research including their:

  • Research approach

  • Strategy of inquiry

  • Methods used

Review Questions:

• Why is the research process important to understanding the current knowledge base in kinesiology?

• What are three characteristics of:

  • The three research approaches

  • The five philosophical worldviews

Chapter 3 - Identifying a Research Question and Study Purpose

Learning Outcomes:

• Describe the process of identifying your research topic 

• Identify strategies to narrow your interests to a specific research question and purpose for your study

• Discuss why literature is used and how it is used in research

The Research Topic:

• Narrowly focused and represents a clearly defined focal area related to an important complex problem

• Identification of a research topic usually comes from a researcher’s 

  • Interests

  • Experiences

  • Coursework

  • Academic background

• Once the research topic is identified, it is important to come up with a problem and related research questions

• The research problem

  • Represents the foundational need for the study

  • Describes the context for the study

  • Describes issues that exist in the literature, theory, and/or practice

• A well-developed research problem should be challenging, worthwhile, important and feasible

• Research problems can have descriptive, predictive, and explanatory bases

  • Descriptive: describe an event or condition without explanation

    • Not getting at the cause, just describing

    • This can lead to other research

  • Predictive: look at relationships between variables

  • Explanatory: claim about the relationship between variables and why events or behaviors happen

    • Cause and effect

    • Hardest type

Use of Theory:

• Theory: an explanation of observed patterns or a supposition about a relationship among phenomena

• Compromised of verifiable and testable statements or propositions and is generally derived from

  • Observation

  • Experimentation

  • Reflective thinking

• Theory can be used as a 

  • Framework to guide research problems and questions

  • Foundation for connecting other frameworks

• Within quantitative research, theory is often used to guide the entire research process

• Within qualitative research, theory may be used 

  • To inform the research problem and purpose

  • As an outcome of data gathering (ex. Grounded theory)

• Ex. the theory of planned behavior

  • Attitudes + subjective norms + perceived behavioral control -> intention -> behavior

  • What drives human behavior?

Types of Reasoning:

• Within the scientific approach, there are two types of reasoning

  • Inductive: observations of specific events and circumstances are used to make predictions about general principles that are tied together and united into theory

    • Using a bunch of information gathered from interviews, data and coming up with a theory based on what you know

  • Deductive: researchers start with concrete generalized information often contained within a theory, and use this information to explain specific events or circumstances

    • Using a theory that is already made to explain an observed phenomenon

The Literature Search and Review:

• Literature review: a synopsis of what researchers know based on studies that have already been done on similar, relevant topics

• Important for identifying

  • Research topic

  • Research problem

  • Purpose statement

  • Variables and concepts of interest

  • Hypothesis and research questions 

• Identify keywords and search library catalogs, search engines, databases, theses and dissertations, journals, etc.

• Main sources include research articles, book chapters, meta-analyses, conceptual articles, and published reports

• Many ways to organize the literature

  • Literature map: visual representation that draws together the existing studies and identifies how your topic is situated within the broader body of research

  • Annotated bibliography: summary and critique of articles, book chapters and documents you have identified

• Writing the literature review

  • Read articles

  • Identify 20-40 relevant articles

  • Summarize articles

  • Include introduction, body, and conclusion

• Purpose of the literature review

  • The literature review concludes with a statement of the study's purpose

  • The purpose statement is the most important statement in the entire research study

    • Clear and concise

    • State the intent of the study

    • Identify variables or phenomena/concepts in the study

Variables:

• Within a quantitative study, several types of variables should be considered

  • Independent variables: variables that are manipulated

    • Changed by researcher

  • Dependent variable: variable that is being affected

    • Thing we are trying to change

  • Moderator variable: variable of interest that cannot be manipulated

    • Can’t change, but need to keep track of 

    • May affect the dependent variable, findings of the study

  • Mediator variable: variable that is proposed to at least partially explain the relationship between the independent and dependent variable

  • Control variable: variable that could influence the outcome or results of the study

    • Need to be aware of and control for 

  • Extraneous variable: unmeasured variable that is not controlled for in the study

• Within a qualitative study, the focus is on a central phenomenon

Quantitative Research Purpose Statement:

• Begin with words that identify the intent of the study being described

• Identify the important study variables

• Identify the theoretical framework or model that is guiding the study

• Identify the participants targeted for the study

• Mention the strategy of inquiry

Qualitative Research Purpose Statement:

• Begin with words that identify the intent of the study being described

• Identify and describe the central phenomena or idea

• Recognize the philosophical worldview and research approach

• Identify the participants targeted for the study

• Mention the strategy of inquiry

• Identify the research site where the study is being conducted

Mixed Methods Research Purpose Statement:

• Begin with words that identify the intent of the study being described

• Identify the important study variables for the quantitative aspect of the study

• Identify and describe the central phenomena or idea for the qualitative aspect of the study

• Identify the philosophical worldview

• Identify the theoretical framework or model that is guiding the study

• Identify the participants targeted for the study

• Identify the strategy of inquiry for both the quantitative and qualitative studies

• Identify the research site where the study is being conducted

The Research Question and Hypothesis:

• Following the purpose statement, it is important to present research hypotheses or question

• Hypothesis: a prediction derived from theory, literature or speculation about the outcome of a study

  • Research hypothesis

  • Null hypothesis

• Research question: broad inquiry statement about the central phenomenon

Review Questions:

• How would you describe some uses for theory in a research study?

• How and why is the literature search used in a research study?

Chapter 4 - Ethics

Learning Outcomes:

• Discuss reasons why ethical standards are important to the research process in kinesiology

• Describe the core ethical principles and ethical guidelines and how they shape research planning, data generation, data analysis and knowledge translation

• Identify unique ethical considerations for engaging in research with indigenous peoples

• Describe a process of ethical decision making as a professional in kinesiology-related disciplines

• Apply the ethical decision making process to an ethical dilemma in kinesiology

Ethical Standards in Research:

• Ethical research is perhaps the most important responsibility of researchers

• The terms ethics and respect are often used interchangeably

• Ethics should be viewed as an ongoing process and not a single event

  • It should include considerations for research planning, data generation, data analysis, and knowledge translation

• Ethical policies in canada have been developed, in part, in response to historical practices of unethical treatment of peple in research

  • The nuremberg code (1947) -> medical experiment by nazi researchers

  • Belmont report (1979) -> tuskegee syphilis study

Nuremberg Code:

• Voluntary consent: the voluntaty consent of the human subject is absoloutley essential. Participants should be fully informed about the nature, duration and purpose of the study as well as potential risks and benefits

• Fruitful results: the experiment should aim to produce results that are valuable to society and cannot be obtained though other means. Research should have a clear scientific purpose

• Avoiding unnecessary suffering: the experiement should be designed to avoid unnecessary physical and mental suffering and injury. Risks should be minimized to protect particpants

• Qualified personnel: the experiment should be conducted by qualified and scientifically competent individuals who are capable of safeguarding the health and rights of participants

• Informed risk assessment: participants should be informed of any potential risks involved in the research, and the anticipated benefits must outweigh these risks

• Right to withdraw: participants should have the right to withdraw from the study at any time without facing any negative consequences

• Preliminary animal studies: prior to conducting experiments on human subjects, adequate knowledge should be obtained from previous studies, including experiments on animals

• Safety measures: the experiment should be conducted in a way that protects participants form harm, safeguards should be in place to ensure their safety

• Emergency plans: if unexpected adverse events occur, there should be plans in place to address them, including the ability to terminate the experiment if necessary

• Ethical review: the experiment should be subject to thorough review to ensure that it adheres to ethical standards and respects the rights and welfare of participants

Ethical Policies in Canada:

• Various policies have been developed to guide the ethical practices of canadian researchers

• Tri-Council Policy Statement: ethical conduct for research involving humans (TCPS2)

  • Informed by three core principles that convey the value of human dignity

    • Respect for persons  = autonomy, voluntary consent, self-determination, right to refuse consent

    • Concern for welfare = avoid unnecessary risks, maximize benefits

    • Justice = fairness and equity, no group should be burdened by research harms

  • Three core principles serve as a foundation to guide all research processes

• Canadian universities (ex. Alberta Research Information Services System - ARISE)

• Research Ethics Boards (human/animal)

Conflicts of Interest:

• Conflicts of interest may occur when the study places the researcher or the university they are associated with in a conflict with their duties related to research, personal or institutional interests

• They could arise from

  • Researchers economic investments

  • Interpersonal relationships

Process of Consent:

• Researchers must ensure participants free, informed and ongoing consent throughout the research process

• Various influences could compromise the extent to which participation is free or voluntary

  • Ex. incentives

• Additional ethical considerations related to consent is necessary for vulnerable populations

  • Children

  • Those with cognitive impariments

Privacy and Confidentiality:

• Two terms that are particularly meaningful within the context of research ethics

  • Privacy: his or her right to be free from intrusion by others

  • Confidentiality: researchers' obligation to safeguard entrusted information

Research With Indigenous peoples:

• Many Indigenous peoples in Canada and globally, are distrustful or research

  • Nutrition research by canadian researchers

  • Unethical research with people from the Nuu-chah-nulth first nations in BC

• No single best ethics policy to guide researchers

  • Chapter nine in the TCPS2 is focused on engaging in research with indigenous people

• OCAP principles

  • Ownership

  • Control

  • Access

  • Possession of knowledge

  • The info was obtained from Indigenous people, so they want OCAP of it

Animal Research Ethics:

• Animal models are used extensively in kinesiology research

  • Offer insight into biological structure and function

• Canadian council on animal care (CCAC)

  • Developed policy statements for the experimental care and use of animals in research

• Three Rs tenet

  • Replace = try not to use animals, use something other than a living thing

  • Reduce = least amount as possible

  • Refine = least amount of harm done, treat humanely

Ethical Principles:

• Autonomy: the right to self-determination

  • Respect the individuals right to make informed decisions

• Non-maleficence: duty to not cause unnecessary pain/suffering

• Beneficence: obligation to act for the benefit of the patient/research participants

• Justice: the obligation to treat people fairly and equitably

• Fidelity: the obligation to be trustworthy/loyal

• Veracity: the duty to be honest with research participants

Ethical Decision Making:

• Kinesiology professionals need to have an effective ethical decision-making process

• Ethical dilemma: situation that requires ethical action, but the professional is required to perform two or more mutually exclusive actions

• Many different ethical decision-making models

1. Develop ethical sensitivty

   1. Critical to be sensitive to ethical issues

   2. Recognize that the chocies we make affect other people

      1. Recognizing this early in situations is one of the most important facets to being an ethical professional

   3. Being proactive can be useful in helping to avoid more challenging ethical situations in the future

2. Consult code of ethics

   1. Professional code of ethics: designed specifically to offer professionals a guide in their ethical-decision making

   2. Helps ensure the credibility of a profession 

   3. Offers protection for the clients they serve

   4. Does not (and cannot) cover all situations

3. Search ethics literature

   1. Ethics texts and journal articles are also useful sources to support ethical decision-making

      1. Particularly useful when a profession’s code of ethics does not address the specifics of a situation that requires ethical decision-making

   2. Truscott and Crook (2013) - Ethics for the Practice of Psychology in Canada

      1. Much applications for a wide range of professions in kinesiology

4. Develop self-awareness

   1. Critical to acknowledge that the bias you bring to the situation could impact your decision making process

   2. Are there any self-interests you should recognize

   3. Being aware of self-interests needs to be considered when trying to make the most defensible ethical decision possible

5. Apply ethical principles

   1. Ethical principles: help to shape most codes of ethics in kinesiology-related professions

   2. Important to apply fundamental ethical principles to a situation requiring ethical decision-making

      1. Autonomy

      2. Non-maleficence

      3. Beneficence

      4. Justice

      5. Fidelity

      6. Veracity

6. Develop alternative courses of action

   1. When deicsding a course of action professionals should clearly detail the probable costs and benefits of each possible course

      1. All options should be brainstormed

   2. Provides a chance to reflect on who might be affected in the situation and how each action could influence them

7. Consult with colleagues

   1. Can offer perspectives and options that a professional has not already considered

      1. May provide a more objective lens

   2. Need to consider how to consult with a colleague about an ethical dilemma in a way that does not disclose the identity of a client

8. Take action

   1. After the first 7 steps have been completed, it is time to implement the chosen action plan

   2. This should include:

      1. Informing the people who are going to be affected by your actions

      2. A commitment to assume responsibility for actions

   3. Take actions that are most defensible to other professionals

      1. Ask yourself: “would i act this way in a well-lit room?”

9. Reflect upon the results of your actions

   1. The final step is recommended in many ethical decision-making models

   2. Actions should be constient with what someone demonstrating outstanding ethical virtues would characteristically do

   3. Ethical virtues include qualities such as

      1. Compassion

      2. Discernment

      3. Trustworthiness

      4. Integrity

Review Questions:

• Why are ethics standards important to the research process?

• What are some examples of common ethical dilemmas kinesiology-related professionals face?

Chapter 5 - Quantitative Study Design

Learning Outcomes:

• Describe types of research designs used in quantitative studies

• Identify various methods of assessment and instruments used within different study designs

• Describe internal validity and external validity and associated threats

Quantitative study design:

• The purpose statement is the foundation for the design of your study

• The design informs the methods of measurement and assessment, as well as the statistical analyses

• Quantitative designs can be used to address descriptive research, predictive and explanatory research problems

• Important terminology within a quantitative study design includes:

  • Validity: degree to which researchers can have confidence in conclusions based on the research they conduct

  • Logical validity: quality of the researchers’ arguments, their application of theory to support the needs of the study, and the appropriate interpretation of results based on the data

  • Construct validity: refers to whether the measures used by researchers asses/test what they intended to measure

  • Reliability: an important part of construct validity and indicates that the measure is consistent or repeatable

• Five main research elements that researchers consider to distinguish their study design:

  • Are the people in the study assigned to groups?

  • How many measurements are being used?

  • What types of measures or observations are being used?

  • Is there an interest in generalizing the findings to other populations or settings

  • Can you conclude that the findings are based on the manipulation of the independent variable?

Experimental Research Design:

• True experimental design may be the most powerful means of generating new knowledge

• The only quantitative design that can be confidently used to identify a cause-and-effect relationship

• Typically a true experiment is conducted in a laboratory within a controlled environment

  • This level of control helps to ensure the study’s internal validity

  • External validity is not as easy to claim

• A true experimental design includes

  • An experimental group

  • A control group

• Group membership is defined using random assignment

• A randomized control trial is a type of experimental research design

  • Pre and post-test design

  • Post-test only

Quasi-Experimental Research:

• Studies that are sort of experimental in design

• No randomization of groups

• Low on internal validity, but often higher in external validity

• The basic formula for a quasi-experimental study:

  • People are studied in real-world settings

  • An independent variable is introduced or manipulated

  • There is a dependent variable (the effect) that is measured

• Pre and post-test design methods are often used within a quasi-experimental research framework

• Repeated measures design is also common

  • Includes only an experimental group with multiple levels of the independent variable and usually only one dependent variable

  • Participants are essentially their own control group

• Single subjects design

  • Single person or small number of people studied over a long period of time

• Lower internal validity

• Less confident that the change in the dependent variable is an outcome of the manipulation of the independent variable

• Threats to internal validity

  • Variable not always controlled for

Pre-Experimental Research Design:

• A study in which researchers examine one group of idnviduals and provide an intervention during the study

• There is either the absence of a control group for comparison or a non-equivalent comparison group is identified during the study or after the intervention

• This type of design poses challenges to internal and external validity

• 
  

Non-Experimental Research Design:

• Characterized by

  • No randomization to groups

  • No groups to randomize

  • No manipulation of independent variable

• Type of study used to report correlations among variables on interest

• Often times these designs are descriptive in nature

  • Correlational

  • Longitudinal

Threats to Experimental Validity:

• Important study design questions that should be considered

  • Question related to internal validity: can you conclude that the findings are based on the manipulation of the independent variable?

  • Question related to external validity: is there an interest in generalizing the findings to other populations, settings or time

Threats to Internal Validity:

• Factors that may alter the dependent variable separate from the effects of the independent variable

• Threats related to the experimental procedure

  • Testing

  • Instrument accuracy

• Threats related to the manipulation

  • Diffusion of treatments

  • Halo effect

• Threats related to the participants

  • Maturation

  • History

  • Regression

  • Selection bias

  • Experimental drop-out

  • Placebo and Hawthorne effects

Threats to External Validity:

• External validity can be threatened by interactions arising from timing, sample and settings characteristics, and treatment manipulation

  • Selection and treatment interaction

  • Setting and treatment interaction

  • History and treatment interaction

Chapter 6 - Data Analysis in Quantitative Studies

Learning Outcomes:

• Identify and apply different data types and sampling methods

• Discuss the importance of statistics

• Apply appropriate statistical analysis techniques to various types of research questions

• Identify appropriate measures of central tendency and variability

• Describe the process of hypothesis testing

Measurement in Quantitative Research:

• Data collected in quantitative studies can be described as

  • Discrete

    • Nominal (categorical data - education level, smoking stat)

      • Assignment of numbers for classification purposes

    • Ordinal (ranked categories - injury severity, flexibility from low to high)

      • Quantitative values provide a classification according to order or magnitude

  • Continuous

    • Interval (ranked equal distance between categories - body temperature in degrees)

      • Classification using an ordered continuum with equal intervals and increments is known, consistent and measurable

    • Ratio (ranked, intervals, true zero - body weight, height)

      • Classification using an ordered continuum with equal intervals and a true zero

Sampling:

• Probability sampling

  • Any method that assures that the different units in the population have equal probabilities of being chosen

  • Includes random selection, stratified random sampling, and systematic sampling

• Non-probability sampling

  • Ny method that does not use random sampling

  • Includes convenience or purposeful sampling methods

• Random selection

  • The sample can be randomly selected from the larger sampling frame or populationusing procedures such as

    • Random numbers table

    • Computer program

    • Pulling names out of a hat

  • The key to random selection is that the sample is representative of the population to which it is being generalized

• Stratified random sampling

  • If the population or sampling frame is divided and goruped on a characteristic before random sampling takes place

  • This approach can be particularly important if there is a certain characteristic that needs to be represented in the sample to enhance generalizability or external validity

  • Ex. skittle sampling

  • Steps

    • Identify population

    • Define strata

    • Count how many of each

    • Determine sample size

    • Calculate the sample from each stratum

    • Random selection from each stratum

    • Record and analyze

• Systematic sampling

  • A process whereby researchers use lists or inventories of units in a population to select every Nth entry for the sample

  • There are some concerns with the use of lists since they have a characteristic in common and might have inherent biases

• Purposive sampling

  • Involves identifying units that represent a characteristic of interest

  • Several methods of purposive sampling

• Convenience sampling

  • A process of drawing a sample from groups of people that are familiar or convenient

  • Clinicians asking patients, professors asking students, school teams etc.

Data Analysis:

• The design of the study offers a foundation for the type of statistics that are needed to analyze the data

• Statistics: objective ways of interpreting observations used to help researchers address questions related to description, prediction and association, and explanation of differences

• Researchers hypotheses are also integral to determining the type of statistical analysis

Descriptive statistics:

• Researchers need to have a way of summarizing data to describe and identify patterns, direct further analyses, and explain their results

• Frequency distribution or histogram is the simplest graph

• Frequency polygon is another type of graph whereby the frequency of scores is represented by dots that are connected by a line

Types of Distributions:

• Normal distribution: symmetrical data distribution around a center of scores

• Skewed distribution: deviate from the symmetry seen in normal distribution

  • Positively skewed distribution

  • Negatively skewed distribution

• Kurtosis: when the tails of the distribution have a clustering of scores, the peak of the distribution can be effected

  • Playkurtic

  • Leptokurtic

  • Mesokurtic

• Modality: the number of peaks in the distribution, with each peak referred to as a mode

  • Unimodal

  • Bimodal

  • Multimodal

Central Tendency:

• Represents how scores in a distribution most commonly cluster together

• Three main measures of central tendency

  • Mean (interval/ratio): sum of all the scores divided by the total number of scores

  • Medial (ordinal): middle score of a distribution when the scores are ordered in magnitude

  • Mode (nominal): most frequently occurring score

Measures of Variability:

• An index of how the scores vary or disperse

• Common measures of variability include:

  • Range: distance between the most extreme scores in a distribution

  • Interquartile range: scores that are the boundaries of the lowest and highest of the distribution, or the middle 50% of the scores

  • Standard deviation: interpreted as being the spread of the scores above and below the mean

  • Variance: the square of the standard deviation

Research Questions Focused on Group Differences:

• T-test

  • Used if the comparisons are focused on two groups

  • Tests the difference between group means

• Independent t-test

  • Used when the study has two groups of different participants (each participant contributes one score to the data)

• Dependent t-test

  • When two data points are collected over time (e.g. a pre and post test design) or when participants are exposed to two experimental conditions

• Analysis of variance

  • Used when the research question involves two or more groups

    • Factorial anova

      • Used when more than one independent variable is manipulated

    • repeated measures anova

      • Used when there are two ore more time points

• Chi-square test

  • Used to examine the discrepancy in frequencies between, or a comparison between the frequency data outcome and what would be expected by chance

  • Common practice is to report data in contingency table

Research Questions focused on Relationships Between Variables

• If the research questions pertain to associations between variables or prediction, the type of analysis is focused on correlation and regression

• Correlation: statistical technique that allows researchers to determine the relationship between two or more variables

  • Correlation coefficient ranges from 0-1 and coefficient can be positive or negative

• If researchers are interested in prediction ( a concept related to correlation) a regression analysis is used

Hypothesis testing:

• Probability of error in statistics is usually depicted as a “p-value”

  • P-value = 0.05 (5% chance that findings were by chance OR 95% confident that there is a real difference)

• Type 1 error (false positive)

  • When researchers make the decision that a manipulation or treatment has been succesfful when in fact it has not been

• Type 2 error (false negative)

  • When researchers make the decision that the manipulation has failed, when in reality it did work

Statistical vs Practical Significance:

• Although scores may look practically different, they might not be statistically different and vice versa

• Practical significance: the amount of change needed in a variable to be considered meaningful in the real world

• Decisions around practical significance are more subjective compared to testing for statistical significance

Chapter 7 - Merits of Quantitative Studies

Learning Outcomes:

• Apply the standard of validity to evaluate the merits of quantitative research studies 

• Discuss why researchers are left to make judgements on the merits of a study based on logical validity and construct validity in non-experimental designs

• Discuss the strengths of experimental designs

• Provide examples of logical validity, construct validity, internal validity and external validity in kinesiology

Validity:

• High-quality research is very hard to conduct and often hard to identify

• Knowing how to critique research is a necessary skill

  • Enables you to understand the merits of researcher’s research questions, justifications provided for the research, choice in methods, conclusions made, and potential implications and applications of findings

• Understanding of the concept of validity is essential

• Logical validity: quality of researchers arguments, their application of theory to support the needs for the study, and the appropriate interpretation of results based on the data

  • Have the researchers provided a convincing justification for all aspects of the research project

• Construct validity: refers to whether the measures used by researchers do indeed measure what they intended to measure

  • Measurement is at the heart of quantitative research

  • Without strong measures of variables, the research will be of little use

  • Researchers need measures that are both consistent/reliable and accurate/valid

• Internal validity: refers to researchers abilities to claim that any change in an outcome is the result of a treatment or intervention and not a result of other factors

  • One of the types of valiidity that is relevant to experimental designs

• External validity: refers to researchers abilities to see similar successes of the treatment or interventin with other populations, in other contexts, and across time

  • One of the types of valiidity that is relevant to experimental designs

• In order to evaluate the internal and external validity of a study, a manipulation of an independent variable (as a result of a treatment or intervention) is required

• Four criteria are used to evaluate the merits of experimental studies in quantitative research

  • Logical validity

  • Construct validity

  • Internal validity

  • External validity

• Only logical and construct validity can be used to evaluate the merits of non-experimental quantitative studies

• Generlaizibility is still important to non-experimental research

  • Generlizing study findings to other people, places, and times

• All studies have limitations

• It is best to consider the merits of any research study on a continuum ranging from high quality to low quality

Application of Logical Validity:

• To asses logical validity, we must examine the logical flow of researchers arguments and decision-making from start to finish

• We need to read and critically evaluate the entire research report

• Another way to support logical validity is to ensure previous research is being cited appropriately

Application of Construct Validity:

• Goal for researchers is to develop a nomological network

  • A web of evidence descirbing how researchers measures should be related to other measures of different constructs and behaviors

  • Evidence is used to justify researchers choices to use particular measures, and support that the measures are indeed measuring what they intent to measure

• The evidence for and against construct validity will range on a continuum

• The task for researchers is to carefully examine construct validity evidence and make a decision as to whether or not a measure is sufficient

Application of Internal Validity:

• Researchers want to be confident that any chang ein an outcome is a result of their intervention and not the result of other factors

• Potential ways to increase internal validity

  • Include a control condition and an experimental condition

  • Randomly assign participants to conditions

• These approaches help researchers rule out potential threats to internal validity

• Including a control condition and random assignment do not guarantee internal validity

• Researchers need to consider many details of a study to provide additional protection against threats to internal validity

• There is no perfect study

  • It is impossible to control for all threats

Application of External validity:

• The key to generalizing study findings to other people, context or time is ensuring that the sample chosen for the study is representative of the population that rresearchers wish to generalize their results to

• Another factor is the extent to which the conditions of the study reflect the real world

• One way to enhance external validity is to clearly identify inclusion criteria specifciyng who can and cannot be part of the sample studied

• Limits to the external validity of most studies include

  • Reliance on non-randmom selection of pariticapnts - volunteer

  • Reliance on data from weird sample

    • WEIRD = western, educated, industrialized, rich and democratic

Chapter 8 - Qualitative Design

Learning Outcomes:

• Describe strategies of inquiry commonly used in qualitative research

• Dicuss the various applications of theory in qualitative research

• Identify sampling methods for qualitative research and demonstrate application

• Discuss processes of data generation in qualitative research

Qualitative Strategies of Inquiry:

• Qualitative study design is an overarching term that includes

  • Strategy of inquiry

  • Philosophical worldview

  • Methods - data generation and data analysis

• Near impossible to identify all qualitative strategies of inquiry

  • Described and classified differently depending on research disciplines and philosophical assumptions

Defining Features of Qualitative Research:

• Researchers are key instruments and play an integral role in generating data

• Use of the term data generation rather than data collection

  • Emphasizes the manner in which researchers and participants work together to generate data

• Research occurs in the natural setting

• Qualitative research is emergent

Narrative:

• Stories that are used to bring understanding or meaning to the lived experiences of individuals

• Stories can be analyzed in great depth or stand alone as a description of experiences

• Various specific forms of narrative enquiry

  • Life history

  • Oral history

• Stories typically generated via indepth and structured interview

  • Observations and participants journaling may also be used

Ethnography:

• Driven by questions that seek to understand cultures or a cultural group

  • Describe and interepret shared features of a cultural group, such as behaviors, values and beliefs

• Various specific forms of ethnography

  • Critical ethnography

  • Autoethnography

• Participant observation is typically the primary process used for data generation

  • Interviews and documents like journals, may also be used

Phenomenology:

• The study of a phenomenon or a concept through the exploration of lived experiences

• Phenomenology can be understood as a philosophy and a qualitative strategy of inquiry

• Various specific forms of phenomenology

  • Interpetatve phenomenological analysis

  • Empirical phenomenology

• Data is typically generated via multiple in-depth interviews with participants

Case Study:

• The study of the complexity and distinctiveness of a case within important circumstances

• The case or bounded system is bound by time and place

  • Cases of interest: people, team, event, organization or community

• Often fit into one of three categories

  • Intrinsic, instrumental or collective case study

• Data generation is typical extensive

  • Interviews, observations, visual methods

Qualitative Description:

• Used by researchers who want to develop a comprehensive description and summary of a phenomenon or event

  • Results in a description of the event or phenomenon in everyday language

• May take on the tones of other strategies of inquiry

• Individual or group interviews are typically used to generate data

Grounded Theory:

• Focused on the generation and analysis of data to construct a theory

• Iterative strategies are often employed

  • Researchers use comparative methods to examine the data against each other in an effort to identity similarities and differences

• End product is a theory

• Data typically generated via one-on-one interviews with pa

Role of Theory in Qualitative Research:

• Theory could be used in a number of ways

  • Theoretical lens: theory could shape the research question, participant selection, data generation, and data analysis

  • Interpretive framework: theory is drawn upon to interpret the research findings

Sampling For Qualitative Studies:

• Purposeful sampling

  • To recruit a sampe of information-rich participants who will purposefully inform an understanding of the topic being studied

• Researchers ma chose to identify a specific form of purposeful sampling

  • Extreme case: qualitative research method that involves selecting unusual or exceptional cases to study

  • Maximum variation: qualitative research technique that involves selecting a sample that includes a wide range of perspectives on a topic

    • Look at topic from wide range of angles

    • Help identify common themes

    • It's also known as heterogeneous sampling

  • Snowball sampling: a non-probability sampling technique that involves asking current study participants to recommend other people who might be interested in participating in research

Sample Size:

• Sample sizes in qualitative studies are relatively small compared to most quantitative studies

• Morse (2000) identified a number of considerations for determining sample size

  • Scope of the study

  • Nature of the study

  • Quality of the data

  • Number of interviews per participant

  • Study designs

• General guidelines about sample sizes for each strategy of inquiry

  • Narrative: 1-2

  • Phenomenology: 5-8

  • Grounded theory: 30-50

  • Ethnography: single group of people that share culture

  • Case study: single case or 4-5 cases

  • Qualitative description: often takes on tone of another and uses those sample sizes

Data Generation in Qualitative Research:

• Interviews are the most common method for generating data in qualitative research 

• Qualitative studies often use more than one method of data generation

• New or refiend methods of qualitative data generation are continuously being published in the research literature

Interviews:

• Often thought to be synonymous with qualitative research

• Relational nature of interviews

  • Important to build and maintain rapport

• Can take the form of

  • One on one interviews

  • Group interviews

• Typically comprised of three main phases

  • Introduction, questioning, conclusion

• Interviews can take place

  • In person

  • Over the phone

  • Through computer software

• Interview guide can be

  • Strucutred

  • Semi-structured 

  • Unstructed or conversational

• Often audio recorded

Observation:

• Researches go into the natural setting to try and better understand the topic of study

  • Use various senses including sight, tough and hearing to study the phenomenon

• Observation can take various forms

  • Complete participant

  • Participant as observer

  • Observer as participant

  • Compete observer

• Field notes are often used

• Features that should be included

  • Header, descriptive notes, reflective notes, sketch

Written Docuemnts;

• Typically used in addition to other forms of data generation, such as itnerviews

• Various types of written documents can be used to generate data in qualitative research

  • Public documents: policies, newspaper articles, histroical archives

  • Written documents from participants: logbooks, personal diaries or letters, performance reports

Visual Data:

• Photographs, drawings, mappings, diagrams or videos can provide valuable insight into a phenomenon or event

• Presumptuous of researchers to think that all participants express themselves comfortably through words

Chapter 9 - Qualitative Data Analysis

Learning Outcomes:

• Discuss various methods of qualitative data analysis

• Apply the principles of qualitative data analysis to qualitative data

• Identify strengths and challenges researchers often face in the qualitative data analysis process

Introduction to Qualitative Data Analysis:

• Qualitative data analysis is used when researchers generate non-numerical data

• There are a wide range of qualitative data analysis methods to choose from

• The specific goal of qualitative analysis is important

  • Create a taxonomy: a formal system for classifying multi-faceted and complex phenomenon

  • Develop themes: a way to characterize the responses of participants and provide insight into the essential components of their experience

  • Develop theory: a set of interlocking casual variables to explain some aspect of personal, social or phyiscial realities

• It is important for researchers to think about the types of results they want from their analysis

• The types of results should be informed by the goal or purpose of the research

Inductvie and Deductive Data Analysis:

• Researchers need to decide if they will utilize inductive or deductive processes in qualitative data analysis

  • Inductive data analysis: researchers identify taxonomies, themes or theory from the data generated in the research

  • Deductive data analysis: researchers use an existing framework or list of categories to code the data

• Researchers conducting qualitative data analysis often use a combination of both

Qualitative Data Analysis as Immediate, Ongoing or Spiral:

• Immediate: data analysis begins immediately as part of the development of the research

  • Since the investigator is the primary data collection instrument, analysis begins the moment the investigator begins thinking about research

  • Researchers need to reflect on the role they play as a part of the data analysis process

• Ongoing: data analysis doesn’t take place at one moment, it is something researchers engage in throughout the entire research process

  • Embracing analysis as an ongoing process can add a level of depth and richness to the final interpretations

• Sprial: data analysis is not a fixed linear approach

  • Researchers flow through the data analysis process in analytic circles, often returning to earlier steps in the analysis as new insights and reflections emerge

  • Without embracing the spiral nature of data analysis, much insight can be missed

Steps in Qualitative Data Analysis:

• There is no one way to best approach conducting a qualitative analysis

• The ideal approach is one in which the steps in data analysis match the goals of the research and the strategy of inquiry

• It is always best practice to consult key texts to identify specific data analysis strategies that might be particularly important within a particular strategy of inquiry

• Creswell (2014) presents a useful and practical data analysis process

  • Organize and prepare the data

    • Transcribing is a key part of organization and preparation

    • Taking oral data and reproducing it as faithfully as possible

    • Not all data requries transcription, but non-written data needs to be organized and prepared

    • Ex. photographic data printed

  • Read or look at all the data

    • Go through the data multiple times

    • Spend sufficient time reviewing the data as it is less likely something will be missed in the formal coding process

    • This results in a more rich and insightful final interpretation

    • It allows researchers to focus on the larger picture that can get missed in the minute details of a data set

  • Start coding all the data

    • Coding: represents a systematic organizing of the data into meaningful chunks that become the significant themes of the research

    • Segmenting data into categories and labelling them

    • Many strategies exist (highlighting, spreadsheet, sticky notes)

    • Different approaches to code for categories of interests

      • Ex. expected, surprising and unusual

      • Ex. conceptual, relationship, setting, perspective

  • Generate descriptions or themes

    • Look for organizing frameworks that tell the story of their data in a way that best informs their research question

    • Common categories might be grouped, isolated data points discarded, and unique elements highlighted

    • Types and numbers of themes can vary widely

    • Descriptions might be particularly important to allow the reader direct entry into the research 

  • Decide how the findings will be represented

    • Many researchers chose to publish their findings as a journal article

    • Benefits: journal articles have a long history within academia, experts in the field often review research before publication, many researchers are trained in writing final documents

    • Challenges: page limitations, may not reach intended audience, written work may not be the desired outcome

    • Many formats for representation exist 

  • Interpret the findings

    • Interpretation of the findings can differ substantially depending on researcher theoretical inclinations, knowledge of literature, and level of analysis

    • Multiple interpretations are okay, expected and valued

    • Imagination and creativity are embraced

Chapter 10 - Merits of Qualitative Research

Learning Outcomes:

• Identify strategies that can be used to contribute to the trustworthiness, rigour, and validation of a qualitative study

• Explain the principle of methodological coherence

• Apply criteria from a comprehensive checklist to judge qualitative research

• Adopt a relativistic approach when considering the characterizing traits for evaluating qualitative research

• Ethically assess qualitative research

Evaluating Qualitative Research:

• How to determine if a qualitative study is good?

• qualitative research inherently different from quantitative research

• Requires different approaches than criteria used to evaluate quantitative research

  • No measures or instruments to assess

  • No intervention or treatment to examine for threats to internal validiity

• Evaluation of qualitative research can include a number of considerations

Qualitative Evaluation Approaches:

• Trustworthiness: convincing an audience that study findings are worth paying attention to

  • Used synonymously with rigour and validation

  • One place to start when evaluating qualitative research

  • Four aspects

    • Truth value: credibility of truth

      • Confidence in the truth of study findings for participants

    • Applicability: transferability of a study

      • Forming understandings that may be relevant to other contexts or participants

    • Consistency: consistency of a study

      • Seek to understand variability of study findings

    • Neutrality: findings are based on participants meanings and experiences

      • Findings are not a mere function of researchers biases, interests and perspectives

  • Some similarities between aspects of trustworthiness and quantitative terminology

    • Ex. truth value and controlling for threats to internal validity

  • Trustworthiness is a starting place to evaluate qualitative research

• Methodological coherence: having all components of a research design align with one another

  • Imperative to have a well planned research study

  • Armchair walkthrough: reflect on all aspects of qualitative study, consider alternative approaches, provide a road map

  • Indicator of quality research

  • Requires alignment within a research design, important to consider during design

  • Coherence among philosophical assumptions, research questions, study design, data generation, data analysis and interpretation

• Consolidated criteria for reporting qualitative research: using a checklist to consider what is good research (COREQ)

  • Include criteria about research team, study design, analysis and findings

  • Comprehensive protocol to assess qualitative research

    • Three domains/categories with a total of 32 items

  • Some items are particularly relevant to certain forms of qualitative research

    • Interviews and focus groups

  • Multiple purposes: a guide to inform researchers of important aspects to include in their research

  • Identifies a variety of details that qualitative research can be evaluated on

• Relatvisitc approach to characterizing traits: identifying study characteristics that may suggest high quality research, depending on the context

  • Study quality is context dependent

  • Up to resrarchers and audience to evaluate and determine merits of a study

  • Characterizing traits: criteria that may allude to quality of research

    • What is deemed high quality in one study may not be on high quality in another

    • Fluid and dynamic

    • Criteria to deem qualitative research as good is relative

• Ethics: being an ethically minded researcher

  • All research designs should have ethics as a foundation 

    • Engaging in ethical research is of utmost importance

    • Ethics is an ongoing process

    • Researchers need to ensure respect for peoples, concern for welfare and equity

  • Need to consider merits of research in an ethical way

  • Ethically minded reserchers set foundation for a strong study

    • Utilize ethical-decision making model and basic ethical principle

  • Ethics are integral to the way researchers think about, approach and do research

Strategies to Enhace Trustworthiness, Rigour and Validation

• Strategies can be used to support trustworthiness, rigour or validity of a qualitative study

  • Audit trail: researchers maintain a detailed record of entire research process

    • Someone external to study examines various components of study

• Member check: study participants review data or study interpretations

  • Opportunity to add, alter, delete

• Peer debrief: researchers pushed by professional peer to critically reflect on study

• Present negative or discrepepant information: presenting information that counters main study findings

  • Highlights opposing views and unique experiences

• Prolonged engagement: sustained time spent with participants in the field

• Purposeful sampling: recruiting information-rich participants who can best inform research question

• Researcher reflexivity: researchers position themselves

  • Reflect on biases, experiences and backgrount to consider how these shape research

• Rich, thick descriptions: collecting through descriptive data

  • Presenting findings in a rich manner

• Triangulation: crosscheck study findings and interpretations

  • Use a variety of data sources, perspectives and methods

• Certain number of strategies does not guarantee strong study

  • There is no magic formula

• Just as validity in quantitative studies is a matter of degree, so to are merits of qualitative research

• There is more to consider when evaluating qualitative research 

Additional Considerations:

• Alignment among various components of research study

• Crosschecking study with comprehensive checklist

• Sometimes using a checklist is critiqued

A Good Article:

• See if there is a section on quality

  • Many words can be used: trustworthiness, validation, rigour

• Were methods used to ensure trust in this process

• Checklist

  • Title

  • Quality discussed

  • Main word of quality

  • Specific methods to ensure quality

Chapter 11 - Qualitative Interviews and Analysis

Interviews

• The way we interview depends on what we want to know

• Process of finding out what other think and feel about their worlds

• Result is to understand major points of their message and how it compares to your own situation

• Need to be a good conversationalist, and a good listener

• Purpose of interviews

  • Generate insights and concepts

  • Expand our understanding

  • Search for exceptions to the rule

  • Document historical idiosyncratic cases

  • Results validated elsewhere

• Types of interviews

  • Survey - Structured

    • Strict set of questions

    • Aims to measure facts, knowledge, behavior

    • Minimal interaction with interviewer

    • No flexibility

    • Useful for people with limited interviewing experience

    • May miss what interviewee believes to be important

  • Semi-structured interviews, Open ended, Focus group or narrative 

    • Conversational style

    • Allows for free responses or elaboration

    • Potential for rich data

    • Time consuming

    • Little control

    • Guide? Reliability?

  • Interview guides

    • List of questions to ask participants

    • Ensures interviewer covers every topic

    • Identify appropriate topics and questions

    • Ecide on the level of detail

    • Draft and order the questions

    • List and probes or prompts

    • Pilot the questions

  • Interview suggestions

    • Questions should be clar and easy to understand

    • Ask only one question at a time

    • Avoid leading questions

    • Start with easier or more simple questions before moving on to challenging or nuanced questions

  • Types of questions

    • Introducing 

    • Probing

    • Specifying

    • Direct

    • Indirect 

    • Structuring

    • Silence

    • Interpreting

  • Ethical considerations

    • Confidentiality

    • Informed consent

    • Risk of harm

    • Reciprocity

  • Additional considerations

    • Sensitive topics

    • Disclosures

    • Grief

  • Quality criteria for Interviewer

    • Knowledgeable - familiar with topic

    • Structuring - outline the procedure

    • Clear - simple, easy, short questions

    • Gentle - let people finish, give time to think

    • Steering - avoid digressions

    • Critical - be prepared to challenge what is said

    • Remembering - retain info from interviews

    • Interpreting - clarify and extend answers

Thematic analysis:

• Offers a method for identifying patterns in a data set

• Allows for the description and interpretation of meaning

• One of the most common processes

  • Acts as an umbrella term to capture many ways to work with qualitative data

• Types of analysis

  • Rich description of entire data set: the identified themes are an accurate reflection of the content contained in the entire set

    • Useful for under-researched areas

  • Detail account of one theme or group: relates to a specific question within the data or to a particular theme

• Semantic themes: identified within the surface meanings of the data

  • Engagement occurs with the obvious meaning

• Latent themes: you begin to examine underlying ideas, assumptions, and concepts that are shaping the semantic content

• Phases of analysis

  • Familirization

    • Transcribing

    • Reading

    • Re-reading

    • Taking notes

    • Marking ideas

    • Re-reading 

  • Coding

    • Identifying features of the data that are interesting, unique, common or meaningful

  • Theme development

    • Sorting the codes into higher level patterns

    • Use tables, mind maps etc.

  • Reviewing themes

    • Review each theme and consider whether they can be presented in a coherent pattern

    • Do the themes allow you to tell the story in a compelling and coherent way?

    • Look through data again to ensure you haven’t missed codes

  • Naming and defining themes

    • Identify what each theme is all about

    • Define the themes in a detailed analysis

    • How does the theme fit with the overall story?

  • Writing the report

    • Tell the story in a way that is convincing to the reader

    • Choose example from the data that captures the points you are trying to convey

Chapter 12 - Levels of Evidence

Learning Outcomes:

• Discuss what we mean by the best evidence

• Framing research questions into search strategies

• Database examples in applying limits/filters and identifying resources that match the research question

Levels of Evidence:

• In general, you should seek evidence in the following order

  • Filtered resources - unbiased and comprehensive synthesis or summaries of the research (systematic reviews, meta-analysis)

  • Unfiltered - individual research study reports

  • Expert opinion and background - commentaries, opinions

• The best evidence will depend on the following

  • The topic of interest (effectiveness of intervention vs experience)

  • How common or rare the condition is

Filtered Summaries and Guidelines:

• Meta-analysis systematic reviews

  • Meta-analysis: common and emerging technique for research synthesis

  • Preferred reporting items for systematic reviews and meta-analyses (PRISMA)

  • A well documented reporting system for research synthesis and meta analysis

  • Research question involves an evaluation of published literature (what is known on topic)

    • Identifies a problem

    • Systematic literature review

    • Integrates findings from multiple studies

    • Determines an effect size

• Critically appraised article summaries, guidelines

  • Not all practice guidelines are evidence based

Primary Research Studies:

• Double blind randomized controlled trials

  • Best research method for intervention/treatment questions

• Cohort studies

  • Good for prognosis research questions

• Case control studies (quasi-experimental design)

• Case studies (rare cases - all that is available)

• Qualitative research (outside of hierarchy)

  • Good for questions related to experience

• Expert opinion

  • Can be helpful - we learn a lot from others

  • Experts disagree

  • Often correct, but just because something has always been done a certain way does not mean it is the best way

Chapter 13 - Mixed Methods Research

Learning Outcomes:

• Describe mixed methods research

• Explain when researchers should use mixed methods research

• Identify various ways to combine quantitative and qualitative research into a mixed methods design

• Consider how to integrate quantitative and qualitative data analysis approaches for analyzing data in mixed methods research

• List the strengths and challenges of mixed methods research

• Appreciate that mixed methods research is more than simply adding quantitative and qualitative research together

Mixed methods:

• Quantitative research designs are often used to

  • Test theory, examine status of variables, investigate differences among groups, explore relationships among variables

• Qualitative research designs are often used to

  • Gain in-depth understanding of complex phenomena, meanings of experiences

• Mixed methods research combines qualitative and quantitative forms of research

• More than simply collecting and analyzing quantitative and qualitative data

• Systematic and purposeful mixing of quantitative and qualitative research

• The overall strength of the mixed methods study is greater than either quantitative or qualitative on its own

When to Use:

• Becoming increasingly common within kinesiology

• Can be used to answer a range of questions

• Example areas

  • Evaluation of exercise programs

  • Measurement of physical activity

  • Development of sport programs

  • Informing physical education strategies

• Research question drives appropriate direction of research

• Researchers characteristics also play a role in determining research design

  • Philosophical worldview

  • Education

  • Training

  • Skills

  • Past research experience

• Researchers can come together to form research teams for diverse research designs

• Requires a lot of planning

• Many decisions to make including

  • Implementation sequence: timing of the quantitative and qualitative components within mixed methods research

    • Concurrently - quantitative and qualitative data collected at essentially the same time

    • Sequentially

  • Priority: refers to weight or emphasis assigned to quantitative and qualitative components

    • Whether one will have priority over the other

  • Integration: mixing the quantitative and qualitative components

    • Can occur at various stages including data collection, data analysis, data interpretation, knowledge translation

    • Deciding when and how to integrate

    • Often includes merging or converging data during analysis 

    • Options:

      • Side-by-side comparison: analyzed separately, one set of findings presented followed by the other

      • Independent analysis: results from first phase inform development of next research phase

      • Data transformation: converting one form of data to another

        • Qualitize numerical data

        • Quantitize qualitative data (count codes or themes)

    • Researchers may need findings from one phase to plan the next phase

Mixed Methods Research Designs:

• Combination of quantitative and qualitative research approaches can take many forms

• Concurrent research designs: include quantitative and qualitative data collected at essentially the same time

  • Concurrent nested designs: one dominated while other is nested or embedded within it

  • Data given unequal priority

• Sequential research designs: involves two distinct phases

  • Data collected in first phase is analyzed and used to inform second phase

  • Explanatory: quantitative data collected in first phase, qualitative data collected in second

  • Exploratory: qualitative data collected in first phase, quantitative data collected in second phase

Benefits of Mixed Methods Research:

• Neutralizing weaknesses and maximizing strengths of quantitative and qualitative designs

• Triangulation 

• Comprehensiveness

• Instrument development and testing

• Assisting sampling

• Enhancing generalization

Challenges of Mixed Methods Research:

• Blending philosophical worldviews

• Bringing together diverse researchers

• Timelines and resources

• Sampling and analysis decisions

• Publication and evaluation

Chapter 14 - Participatory Action Research

Learning Outcomes:

• Summarize the defining features of participatory action research

• Apply a participatory action research design

• Justify participatory action research as a respectful approach for engaging in research with underrepresented research populations

• Asses the inherent challenges of engaging in participatory action research

Participatory Action Research:

• Participatory action research emerged in part, in response to a distrust of traiditional postpositivst research

  • Typically approached by researchers with political and social agendas

• Unique and emerging research process

  • Co-generates knowledge with participants

  • Results in practical outcomes for participants

PhotoVoice:

• Using photography

• Ex. using photo elicitation to explore youth experiences and positive youth development in sport

Defining Features:

• PAR has been described as the sum of its individual terms

  • Participatory: researchers and participants are involved in all phases of the research process

  • Action: primary goal is to create positive change to address injustices

  • Researchers: social process of gathering and asserting knowledge

• Seven key defining features

  • Social process - examining relationships and how they are formed

  • Participatory - collaboration throughout research process

  • Practical and collaborative - shared action strategies, real life issues

  • Emancipatory - liberating or freeing from social structures

  • Critical - addressing irrational or unjust constraints

  • Reflexive - action and reflection

  • Process to transform both theory and practice

• PAR studies do not need all seven

• Characterizing traits that support researchers in making judgments about the quality of research 

  • Community driven

  • Prolonged engagement and consultation

  • Project deliverables

Research Process:

• No one right way to do, no distinct set of guidelines to follow

• Not a linear process

  • Typically conceptualized as a cyclical and iterative process

Action Research Spiral:

• Specifically, researchers will:

  • Plan for change

  • Act and observe with respect to the change process and anticipated consequences

  • Reflect on such processes and consequences

• Then re-plan, act and observe again, and reflect again

Five-Phase PAR Approach:

• Compared to the action research spiral, this approach is more detailed

• Frisby and colleagues (2005) outlined a five-phase PAR approach that includes

  • Setting the research question

  • Building trust

  • Data collection

  • Data analysis

  • Communicating

• The five phases should be constantly negotiated among researchers and participants

Data Generation and Anaylsis:

• There are no specific measures or processes that must be used in PAR

• Interviews are the most common process for generating data within PAR

  • Observations, personal diaries, photographs, and visual artwork can also be used

• Data analysis should align with research questions and processes of data generation

  • Remember, methodological coherence

Role of Theory in PAR:

• Similar to the use of theory in qualitative research, theory can be used in PAR in a variety of ways

  • Theoretical lens: theory is used to guide or inform PAR

  • Interpretive framework: theory helps to make sense of findings and to identify action strategies

• Theory can play a central role in informing PAR practices and, at the same time, theory can be generated through practice 

Process for Engaging Underrepresented Populations in Research:

• PAR provides an opportunity to respectfully engage underrepresented populations in mutually beneficial, relevant and respectful research

• Those who may be weary of traditional research approaches may be drawn to PAR because of its defining features

  • Participatory

  • Collaborative

  • Transformative

Inherent Challenges of PAR:

• PAR is accompanied by a number of inherent challenges

• PAR is flexible and emergent, and therefore it is difficult to provide specifics as to how researches should work to address such challenges

  • Should draw on the shared experiences of other researchers who have navigated such challenges

Establishing and Nurturing Relationships:

• Establishment and maintenance of trust and relationships are central to the success of PAR

  • Relatively few researchers have written about the actual process by which relationships are developed and nurtured

• Establishment of relationship can take a significant amount of time

  • Particularly challenging for researchers who have numerous other university-related commitments

• Researchers who engage in PAR try to foster a sense of shared ownership and control of research processes and outcomes

  • Ensuring that participants are actively engaged throughout the research process can be challenging

• Contributions of researchers and participants may vary depending on the specific phase of the research

  • Critical to acknowledge that the researchers and participants have differing expertise and interests

• Researchers do not always clearly articulate the various levels of action that have occurred in their PAR study

• Researchers should be aware that action can occur on various levels, from individual-level action to broader societal or policy-level action

Chapter 15 - Knowledge Translation

Learning Outcomes:

• Define knowledge translation

• Illustrate the anatomy of a journal article

• Identify a variety of traditional and innovative knowledge translation strategies

• Discuss how researchers choose knowledge translation strategies for their research

What is Knowledge Translation:

• Generally speaking: sharing research findings, insights, experience

• A process that includes the synthesis, dissemination, exchange, and application of knowledge

  • Often abbreviated as KT

  • Also referred to as knowledge mobilization and knowledge sharing

• Takes place within a complex system of interactions between researchers and knowledge users

• Knowledge user: any individual who is able to use the knowledge gained through research

  • Ex. practitioners, educators, healthcare administrators, patient groups

Core Elements of Knowledge Translation:

• Canadian Institutes of Health Research

  • Four fundamental elements

  • Some overlap between the four elements, but they all are integral to the knowledge translation process

• Synthesis

  • Contextualizing and integrating study findings within larger body of knowledge

  • Ex. systematic review, narrative review, meta-analyses, practical guidelines

• Dissemination

  • Tailoring knowledge exchange to a particular audience

  • Tailoring both the message and the medium

  • Ex. providing summaries for knowledge users, delivering educational sessions with patients

• Exchange

  • Engagement between researchers and knowledge users

  • Results in mutual learning through planning, producing, disseminating, and applying existing or new research

• Ethically sound application of knowledge

  • Putting knowledge into practice

  • Should be consistent with ethical principles, social values and legal regulatory frameworks

Knowledge Translation Process:

• Two ways to plan timing of knowledge translation activities:

  • Activities occur at the end of research study

  • Activities integrated throughout duration of research

• Integrated knowledge translation

  • Active engagement between researchers and knowledge users throughout entire research process

  • Researchers and knowledge users work together and collaborate

  • Sometimes referred to as collaborative research, action-oriented research, participatory action research

• Tangible strategies used by researchers to synthetize, disseminate, exchange, and apply knowledge

• Two broad approaches:

  • Traditional approaches

    • Publication

      • Published journal articles are probably the most familiar form

      • Researchers submit manuscript to a journal in hopes of having it accepted for publication

      • Manuscripts often undergo peer-review process

        • Experts in respective fields critically review manuscripts

      • Many journal articles contain similar elements

      • Open Access Online Journals

    • Anatomy of a journal article

      • Follow fairly standard format: title, abstract, key terms, introduction, methods, results, discussion

      • Sections and subsections labeled with headings and subheadings

      • Not all journal articles have this organization

      • Peer review process

    • Conference proceedings

      • Researchers present and discuss their research in person to an audience

      • Provides interaction between researcher and audience

      • Usually one of thee formats

        • Verbal presentation

        • Poster presentation

        • Symposium

    • Art-based

      • Using art and performance

      • Requires researchers to extend beyond traditional training and experiences

      • Ex. short film, interpretive dance

      • Sometimes the line between data generation and data presentation is blurred

  • Innovative approaches

    • Text-based

      • Stories are a medium through which meanings can be communicated to make sense of experiences and convey knowledge

      • Narratives allow researchers to share research through storytelling

      • Fictional narratives may reach general public to exchange information

      • Poetic representation takes place when researchers transform data into poem-like presentation

    • Media-based

      • Social media permits researchers to immediately disseminate findings to general public

      • Websites developed about research labs, include updated information about previous and current research

      • Development of online tools to serve as resources

      • Ex. TEDTalks that spread ideas in short online talks

    • Relationship-oriented

      • Gatherings of people involved and interested in research

      • Premised on relationship-building and interactions between researchers and knowledge users

      • Provide forum to share information and engage with knowledge users

• Not necessarily mutually exclusive

  • There is some crossover between the approaches

Selecting Knowledge Translation Strategies:

• The choice of which knowledge translation strategy to use should be driven primarily by research question

• Other important factors

  • Intended audience

  • Ethically sound strategies

• Opportunities for knowledge translation are seemingly endless