HSCI 207 Midterm 3

Main goals of qualitative research

• Seeing through the eyes of the studied

• Emphasis on process

• Flexibility and limited structure

• Ultimate goals such as rich understanding? Social change? Social justice?

Theoretical saturation

The point in qualitative research, particularly in grounded theory, where collecting more data yields no new insights into the emerging theory

Seeing through the eyes of the people being studied

• Empathy

  • Probing “beneath the surface” of social behaviour

  • Seeing through the eyes of the people studied

• In-depth description and emphasis on context

  • Behaviour that may seem odd or irrational may become more understandable if the context is described

  • Naturalism is an approach if the context is described

  • Observing people in their own environment

Emphasis on process

• Showing how events and patterns unfold over time

• A long time spend in the field allows the researcher to understand individual and social change its context

• This can also be done with semi-structured interviewing, unstructured interviewing, and life history approach

Flexibility and limited structure

• Questions should be quite general

• There is usually little or no theory driving the research

• The topics explored in the research may change as the study processes

• Allows the researcher to find new directions of study

Main steps in qualitative research

• Establish a general research question

  • What interesting health issue are you studying

• Select a relevant site and subjects

  • Where is the research being conducted and who are the research subjects

• Collect the data

  • Determine which methods to use

  • As suggested earlier, it may be more appropriate to use more than one method

• Interpret the data

  • Determine the meanings that the research subject put to activities that occur in the social environment

• Conceptual and theoretical work

  • Evaluate the data related to your research question

  • How does available data answer this question

• Writing up and findings/conclusions

  • The researcher must demonstrate the credibility of the research and why the research matters

Motivation for dumpster diving

• Using whatever data, literature, or evidence you happen to find without a systematic, rigorous or purposeful method

• Why its a problem

  • Leads to bias

  • Produces inconsistent and unreliable findings

  • Weakens the study’s credibility

Theory and Concepts in qualitative research

• Qualitative research often involves the “grounded theory” approach : the use of data to develop theories

  • This may involve an iterative process: going back and forth from data to theory, revising the theory in the process

• Qualitative research may involve testing theories

  • This can be done through an iterative process, or occasionally through theory testing in the conventional sense

Criteria for evaluating qualitative research

• Trustworthiness: made up of four criteria

  • Credibility

  • Transferability

  • Dependability

  • Confirmability

• Authenticity: the degree to which the research is transformative and emancipatory for the people studied and society at large

Credibility

• Do the people studied agree with the interpretation of their thoughts and actions offered by the researcher

• Conducted through respondent validation

  • People studied may become defensive and try to censor research

  • People studied may not give genuine feedback on what the researcher produced, but may instead try to please the researcher

  • People studied may not have the expertise to provide meaningful comments

Transferability

• “Thick” description helps to determine whether transferability is possible

  • Provides enough information to conduct later comparison to findings from other studies

Dependability

• Were proper procedure followed

• Can the study theoretical inferences can be justified

Confirmability

• Did the researcher sway the results dramatically

• “Auditing” can be used to examine this

When should qualitative research be used

• Nature of the question

• Topic needs to be explored

• A detailed view of the topic is needed

• Have sufficient time and resources

• Audience is receptive to qualitative research 

Sampling

• Purposive sampling

  • Selection of participant, sites that will best help the researcher to understand the problem

  • Frequently used in exploratory research

  • Appropriate to select unique cases that are especially

• Snowball sampling

  • Seeking referrals from other participants

  • Each person is connected

Critiques of qualitative research

• Too subjective 

• Difficult to replicate 

• Generalization

• Lack of transparency

Ethnography

• Study of people and their culture in naturally occurring settings

  • Behaviour is observed in an unstructured way by carrying out in depth discussions and interviews with the people studied

  • Goal is to describe the life of the community from the point of view of participants and with as little impact from outside as possible

  • The researcher is immersed in a particular social setting for a long period of time, sometimes even years

Ethnography versus Participation observation

• Participant observationis an observational part of ethnography

  • The terms “ethnography” and “participant observation” are essentially synonymous, although we consider “ethnography” to be the more inclusive term

  • Ethnography includes participant observtions, but also individual interviews, studying documents, and artefacts from the community

  • Ethnography also refers to a written account a particular qualitative research

  • Ethnography research has its origins in anthropology but is adopted in sociology

Overt ethnography

People being studied know they are being observed by a researcher

Covert ethnography

People being studied do not know they are being observed by researcher

Access to closed settings

• Provide a clear explanation of your aims and methods

• Be willing to negotiate the terms of access

• Be open about how much time your research would take

• In covert research: adopt and support a suitable social role

Access to open settings

• Similar process to access closed 

  • May need to get sponsors and gatekeepers on side

Ongoing access

• People get suspicious of the researchers motives

• Group members fear that what they say or do will get back to bosses or colleagues 

• People being studied may decide to sabotage the research

• To maintain access:

  • play up your credentials 

  • do not give people a reason to dislike you

  • play a role and construct a “front”

Key informants

• Participants who are particularly knowledge and cooperative

• Drawbacks to using them:

  • Researcher may ignore other group members

  • Key informants view may not be representative of the group as a whole and gets unduly represented in the research

Roles for ethnographers

• Complete participations

• Participant as observer

• Observer as participant

• Complete observer

Complete participation

• Covert operations

• The researcher adopts a secret role in the group

• This method gets the closest to participants and their activities both there is a risk of over identification or developing a strong dislike of the participants

Participant as observer

• Researcher adopts a role in the group

• Participants are aware who the researcher really is

• Risk of reactivity

Observer as participant

• Researcher observes and interviews from the edge

• Risk of reactivity

• Risks incorrect interpretation of activity

Complete observer

• Researcher does not engage the participants at all

• No risk of reactivity

Field notes

• Detailed notes of events, conversations, and behaviour, and the researcher’s initial reflection on them

• Types of field notes

  • Mental notes

    • remember and write later

  • Jotted notes

    • Brief notes made at the time to refresh the memory when writing detailed notes later

  • Full field notes

    • Detailed notes made at the time to refresh the memory when writing detailed notes later

Analytic memos

• Link observations to concepts

• Notes on data, but not the data notes

  • Must be kept separate to avoid influencing the data notes

Visual ethnography

Uses visual materials as sources of data, documents, or illustrations or participants stories

Diificulties using visual ethnography

• Different context of when, where, how, and by whom the visual material was taken

• Different meanings may be ascribed to the visual material by the researcher and by different participants 

• Potential for researcher to influence the perceptions of the visual by the subject

Institutional ethnography

• Study the daily practices in institutions and how those reveal power inequalities or ruling relations in organizations

  • Looks at how institutional discourses relate to peoples everyday experiences with these institutions

  • Studying daily activities is important because institutional representations, and in particular written texts

• Institutional ethnography explores how institutional discourses relate to peoples everyday experiences with institutions 

Theoretical sampling

• Types of purposive sampling

• Meant to be an alternative strategy, used “order to discover categories and their properties and to suggest the interrelationship into a theory”

• The researcher:

  • simultaneously collects and analyzes the data

  • decides what data to collect next and where to find them

  • develops a theory in the process

• Data collection continues until the point of theoretical saturation is reached

Analytic induction

• Qualitative analysis is an iterative process

  • Analysis starts after some data have been collected

  • Further data are gathered on the basis of that analysis

• Difficulties with analytical induction

  • All cases must be explained, the hypotheses generated may be too broad to be useful

  • There are usually no guidelines on how many cases must be reviewed before the validity of the hypothesis is accepted

Grounded theory

• Theory that was derived from data, systematically gathered and analyzed through the research process”

• Also an inductive, iterative process

  • Systematically gather data and develop theory out of the data

  • Analysis throughout the research process

• Features

  • Constant comparison; documented through the creation of analytic memos

Types of coding in Grounded Theory

• Open

  • Identifies initial concepts that will be categorized together later (emotions)

• Axial

  • Data are reviewed for linkages and are reorganized according to those connections

• Selective

  • Selecting the core categories

    • Validating the relationships

    • Identifying gaps that need to be filled in

Outcomes of grounded theory

• Concepts (building blocks of theory)

• Categories (encompass two or more concepts)

• Properties (attributes of a category)

• Hypothesis (initial hunches)

• Theory (substantive or formal)

Substantive theory

Observed patterns are related to each other and a theory is developed to explain the connections in that setting

Formal theory

Theory formulated at a higher level; requires data collection in different settings; applicable to a variety of settings

Steps in analysis

• Step 1: Researcher begins with a general research question

• Step 2: Relevant people and or incidents are theoretically sampled

• Step 3: Relevant data are collected

• Step 4: Data are coded, which may, at the level of open coding, generate concepts

• Step 5: Through constant comparison of indicators and concepts categories are generated

• Step 6: Categories become saturated in the course of the coding process

• Step 7:Relationships between categories are explored in such a way that hypotheses about connections between categories emerge

• Step 8 and 9 : Further data are collected via theoretical sampling

• Steps 10 and 11 : The collection of data is likely to be governed by the theoretical saturation principle and the testing of the emerging hypothesis which leads to specification of substantive theory

• Steps 12 : The substantive theory may eventually be explored using grounded theory processes in a different setting from the one in which it was generated

Criticisms of grounded theory

• Differences between concepts and categories may be vague

• Observation and data gathering may not be as “theory neutral” as claimed

• Practical difficulties

• It may not result in theory; that is, what which explains something

• Coding may result in fragmentation, loss of narrative flow

Steps and consideration in coding

• Code and transcribe as soon as possible

• Read through the data before considering any interpretation

• Read through the data again

• Do not be concerned with producing too many codes, this is normal at the beginning

• Review the codes to consider association, redundancy, relationships to existing concepts

• Consider general theoretical ideas regarding codes and data

• Keep coding in perspectives

Problems with coding

• Risk of losing the context

  • By selecting parts of the text, qualitative coding weakens the connection between description and social setting in which the events occur, resulting in a loss of context

• Fragmentation of data

  • Breaking the data into codes and small chunks of text results in a loss of narrative, creates partial interpreations’s 

Turning data into fragments

• Basic coding

  • Getting the simplest labels for the material, such as negative and positive consequences of the analysed activity

  • Produces a superficial analysis, and needs to be followed by other steps

• Deeper awareness of the content in the text

  • rework original codes to more fully reflect the context of the text, relate codes to the focus of the research, to what is included and what is missing

• Exploring broader analytic themes

  • Researcher moves away from the context of specific interviews and looks for broader analytical themes

Advantages of using software

• Proposes new visual ways of looking at data which stimulate its holistic perception and point to connection between ideas and concepts (excel graphs yk)

• Allows to estimate how representatives different quotes from qualitative interviews are

• Improves the transparency of qualitative analysis, as researchers are more explicit and how they conduct the study

Criticisms of using software

• It primarily works to quantify coded text and negates the qualitative, thematic interpretation of meaning important qualitative analtysis

• It fragments textual material into very small pieces of data where the natural flow of the story is broken

• It is too closely built around the grounded theory and this diminishes another key strength of qualitative research is it flexibility 

Narrative analysis

• Researching the stories people tell to understand their life and world

• Four models

  • Thematic : examines what is said rather than how it is said

  • Structural: examines the way a story is told and what is emphasized to increase persuasiveness of the story

  • Interactional: examines the dialogue between the teller and the listener

  • Performance: examines narrative as a performance - explore the us of words and gestures to get the story across

Criticisms of narrative analysis

• Over relies on the story and gives preference to the accounts of participants as the only explanations of social phenomena

• Stories may be accepted at face value by the researcher

• Taking a critical stance may help the researcher to understand the motives, the social situation of the teller, and broader social conditions in which the story is told

Focus groups

• 6 - 10 people

• A moderator or facilitator 

  • Makes sure the discussion stays on the issue without directing too much

  • Makes sure all the people participants

• Naturalistic 

  • Brings out how individuals collectively make decisions and interpretations

• 10-15 groups

Selecting participants

• Natural groups: people who already know each other or already have had some interaction

  • May be appropriate, depending on the goals of the research

  • Useful if the research is actually about how social interaction occurs

• Disadvantages

  • Pre-existing styles of interaction or status hierarchies may affect the discussion

  • The group may have taken for granted assumptions that are not challeneged

Limitations of focus groups

• Less control over discussion than interview

• An unwieldy amount of data may be produced

• The data may be difficult to analyze

• Personalities traits in room

• Difficult when sensitive issues, social hierarchy, strongly opposed positions

Qualitative interviewing

• Is less structured and more likely to evolve as a natural conversation 

• is often conducted in the form of respondents narrating their personal experiences or life histories

• Can be a part of ethnography or stand alone

• Is unstructured or semi structured

• Aims to get a story

• More open ended

• Greater interest in the interviewees perspective and concerns

• Tangents are encouraged

Structured interviews

• Pre-established questions with limited set of response options

• Coding scheme is pre established

• Same questions and same order

• Each interviewee treated in same manner 

Unstructured interviewing

• Researcher uses only a brief set of points to introduce topics

• Only thing defined is the broad topic of interest

• Conversational 

• No more than a short interview guide

• Starts with a single broader question

• Respondents answers are in a free form

Semi structured

• Researcher has a lit of questions or topics to be covered

• offers a somewhat longer interview guide

• Interviewees are still free to reply in any way they choose

• Questions may be asked out of order

• New questions may be devised and asked on the spot

• There is a clear focus on the topic of interest

• Useful when more than one interviewer

• Useful when there are several interviewees

Benefits of interviews for interviewees

• Catharsis

• Self acknowledgment/validation

• Sense of purpose

• Self awareness

• Empowerment

• Promotion of healing 

• Voice

Preparing an interview guide

• Unstructured : short list of issues

• Semi-structured : longer list of actual questions 

• Always be open to new issues that may arise

• Establish some degree of loose order to questioning process

• Language that is understood by the participants

• No leading questions

• Prompts to ensure sufficient personal information about the individual participant is collected to contextualize the data

Kinds of question

• Introduction questions

• Follow up questions

  • Rephrase the interviewee answer and ask them to elaborate on the answer

• Direct questions : interviewee perceptions

  • “Do you find it easy to keep smiling when serving customers”

• Probing questions

  • Short general questions inviting the respondent to reflect more deeply on what they already said without particular reference to details

• Indirect questions : perceptions of others

  • “How do you feel about..?”

• Structuring questions 

  • “Now I would like to move on to a different topic”

• Interpreting questions

  • “Do you think that your leadership role had to change from one of encouraging others to a more directive one?”

• Silence

  • A pause will give the interviewee an opportunity to reflect and amplify an answer

  • Dont pause for so long that the interviewee feels embarrassed

Interview preparation

• Familiarize yourself with the setting to contextualize the data

• Have a good tape recorder and know how to use it

• If possible, use a quiet and private setting for interviews

• Know your interview guide

• Prepare to be an active listener

The interview

• Primary intent to listen to your interviewee

• Avoid bringing anxiety to the interviewee

• Vignette questions can be used to ground interviewees ideas and accounts of behaviour in particular situations

During the interview

• Be knowledgeable about the topic of the study; clear in asking simple and understandable questions

• Be gentle, sensitive, and open, so that the interviewee can freely express their opinion and not be interrupted

• Be flexible and steering: flexible enough to respond to the new themes raised by the participant, but steering to redirect discussion 

Flexibility in the interview

• Be able to vary the order of questions and clear up inconsistencies in answers

• Flexibility is also important when audio recording equipment breaks down, or when an interviewee refuses permission for recording to take place

• Knowing when to switch off their recording equipment

Online interviews and focus groups (advantages)

• Usually smaller

• Several responses at once

• Overcomes geographical issues and sensitive topics

• Visual biases reduced

• Less reactive response to moderator 

• Often a safe and friendly environment

Online interviews and focus groups (disadvantages)

• Takes longer

• More difficult to establish rapport

• More difficult to probe

• Higher non response

• Moderator cannot read body language

• Online connection may be loss

Advantages of ethnography over qualitative interviewing

• Better for “seeing through the eyes of others”

• Better for learning the “native language”

• Can discover “things taken for granted”

• Better for uncovering deviant

• Can establish context of peoples behaviour

• More naturalistic

Disadvantages of ethnography over qualitative interviewing

• Some issues are not observable

• Can’t reconstruct past events and future plans

• More intrusive

• Longitudinal research harder

• Less breadth of topics can be covered

• Doesn’t Addresses specific issues

Knowledge translation

A dynamic and iterative process that includes the synthesis, dissemination, exchange and ethnically sound application of knowledge to improve the health of Canadians, provide more effective health services and products and strengthen the healthcare system

What is knowledge translation

• Making users aware of knowledge and facilitating their use of it

• Ensuring that research is informed by current available evidence and the experiences and information needs of users

• Closing the gap between what we know and what we do

Who is the audience of knowledge translation

• Researchers within and across disciplines

• Policymakers, planners, and managers in health care and public health

• Health care providers 

• General public, patient groups 

• Private sector

Why is knowledge translation important

• The production of new knowledge often does not, by itself, lead to it widespread adoption or impact health

• Moving knowledge into action

Factors influencing research use

• Availability of research results 

• Ability of practitioners to critically review research evidence

• Resources to implement research findings

• Relevance of the research to the decision being made

• Timeliness

• Quality and credibility of the evidence

The health information gap

• Consistent evidence of failure to translate research findings into clinical practice

  • 30-40% patients do not get treatments of proven effectiveness

  • 20-25% patients get care that is not needed or potentially

  • Cancer outcomes could be improved by 30% with optimum application of what is currently known

  • 10% reduction in cancer mortality with widespread use of available therapies

The health information gap

• Patients and health practitioners

  • Do not always receive research results needed to make more informed decisions about patient care

• Health researchers

  • Need feedback from users

• Policy makers, health administrators

  • Need information to make decisions around future research funding
    

Reasons for the “Gap” Policy makers Vs. Researchers 

• Complex policy problems

• Speed

• Feasible and pragmatic solutions

• Reducing uncertainties

• Difficulty in framing researchable questions

• Simplification of problems

• Finding the truth

• Time to think

• Publish or perish

• Thoughtful deliberations

• Focus is not on policy process and how to influence

Knowledge translation process

• Research

  • Knowledge generation

• Publish

  • Peer review as quality check

• Develop

  • End user material

• Disseminate

  • Teaching, conferences, workshops

• Evaluate

  • Adoption, impact

Knowledge translation models/frameworks

• Push or knowledge

  • Researcher are responsible

• Pull model

  • Problem solving model where decision makers are responsible

• Interactive model

  • Emphasizes the importance of reciprocal relationships between knowledge producers and knowledge users

• Five question framework

Five question framework

1. What should be transferred (clear, compelling idea, related to audiences decision making)

2. To whom should the research knowledge be transferred

3. By whom should the research knowledge be transferred

4. How should research knowledge be transferred

5. With what effect should research knowledge be transferred

What is the peer review process

• Getting the funding

• Before the article is published

  • Journal organized the review of manuscript

  • Typically 2 reviews

  • Usually a round or two of revision

• After the article is published

  • Letters to the editor, challenges in future studies

  • Academic discussions

Imperfections of peer review system

• Editorial boards and readership prone to their own biases

• Potential sources of publication bias

  • Reader want certain kinds of news

  • Positive or significant findings

• Reviewers are human (busy)

• Delays in publication

• Overwork reviewers

• New ideas

Selecting a journal

• Who do you want to read your paper

• Timeliness

• Articles should not be submitted to more than one paper at a time

• Follow the guidelines in the “instructions for authors”

Online submission

• Many publishers now offer a completely electronic submission process

• Article is submitted online and the entire review procedure also happens online

• Speeds up the editorial process

• Is invaluable for authors in low income countries

Authorship criteria

1. Conception or design of work, data collection, and/or analysis and interpretation

2. Drafting the manuscript or reviewing and revising sections

3. Assuming responsibility for the final version of the manuscrip

Reflexivity and role of researcher

• Can influence data in multiple ways

  • Personal aspects

  • Qualififcations

  • Experiences 

  • Relationships with participants

  • Assumptions

  • Epistemological orientations

Reflexivity

• Asking how the researcher has had an influence at all stages

• Account for how the researcher may have had an influence

• Explore how interpersonal dynamics between researcher and participants influences what is know

• Share the researcher motives, background, preconceptions, preliminary hypothesis and thoughts in the data analysis process

Triangulation

Strategy where you use multiple sources

1. Methods = goal is not necessary to get to same answer; task is to compare

2. Data sources = interviews vs. observations; teachers vs. students

3. Analyst = multiple interviewers/researchers; member checks

4. Theories = what is gained from applying different theoretical approaches

Enhancing reliability and validity in qualitative studies

• Audiotaping

• Detailed transcription

• Multiple coders

• Member checks

• How have contradictions been handled

• Rich data - use of quotes

Strategies to deal with the threats to validity

• Prolonged involvement (reduces reactivity, increases researcher bias, reduces respondent bias)

• Triangulation (reduces reactivity, reduces researcher bias, reduces respondent bias)

• Peer debriefing (no effect for reactivity, reduces research bias, no effect to responder bias)

• Member checking (reduces reactivity, reduces researcher bias, reduces respondent bias)

• Negative case analysis (no effect for reactivity, reduces researcher bias, no effect for respondent bias

• Audit trail (no effect for reactivity, reduces researcher bias, no effect for respondent bias

What things need to be analyzed

• Words, including the pairing of certain words

  • Frequency of words and pairing

• Subjects and theme

  • Includes both manifest and latent context

• Value positions

  • Positive or negative view

    • for example multiculturism 

Potential pitfalls in devising coding schemes

• Categories must be mutually exclusive

• Categories must be exhaustive

• Instructions must be clear

• Unit of analysis must be clear

• Pilot test will reduce risks of error

• Reliability of coding is important 

  • Inter coder reliability (consistency between coders)

  • Intra coder reliability (consistency within coders)

Content analysis without a pre existing coding scheme

• Qualitative content analysis looks for underlying themes present in a unit of analysis

  • Researcher decides herself what themes are to be extracted and how the presence of a theme is established

  • Themes are recurrent topics mentioned in a text or in a recording

  • Themes could be anticipated or emergent in the text

Watch lecture for content analysis (active or passive)

Conversation analysis

• Detailed analysis of the structure of talk

  • Indexicality = Things like pauses and sounds have meaning that depend on the context in which they exist

  • Reflexivity = examines how social order is created through communication

• Fits into qualitative research

• Focuses on reflexivity

• Data from naturally occurring situations

• Link to quantitative research

  • Fine grained detailed analysis of talk

  • Structure of talk analyzed

  • Positivist orientation on replicability 

  • Context means just the words said prior to a response

• Assumptions

  • Talk is structured 

  • Talk is forged contextually

  • Analysis should be grounded in data

• Characteristics of how talk is organized 

  • Turn talking 

  • Adjacency pairs

    • How the talk moves between parties

  • Preference organization

    • Preferred responses and non preferred responses

Discourse analysis

• Studies how a view of the world or understanding of an object is produced through 

  • Includes elements of conversation analysis

  • Broader and more flexible: covers communication other than talk

  • How linguistic categories shape people’s understanding of the world

  • Also how the relationships of power are reproduced in a discourse

• Anti-realist

  • No objective reality waiting to be found, no objective account of social world is possible

• Tends toward a constructionist orientation

  • Priority to accounts by actual participants

  • Recognizes that many different accounts are possible

  • Oriented action

• Strategies: producing facts

  • Quantification rhetoric: statement involving numbers or quantities made to support/refute arguments

  • Using variation in numbers to highlight contrast

  • Attention to specific details: emphasizes supportive evidence to the argument

  • Attention to rhetorical detail: sensitivity to the way argument is constructed: construct convincing arguments, discount a possible counter argument

• Critical discourse analysis

  • Exposes the political nature of the examined texts, considers the issue of power hierarchies, structural inequalities, and historical political struggles

Advantages of content analysis

• Very transparent in its quantitative form, therefore easily replicated

• Allows for longitudinal analysis

• Unobtrusive method

• Flexible: it can be used with several kinds of unstructured information

• Overcomes social barriers to researcher access

Disadvantages of content analysis

• Limitations due to the texts analyzed

• Usually some intercoder and intra coder unreliability 

• Potential for invalid conjecture, especially in discussion of latent meanings 

• Difficult to answer “why” questions using this method

• Emphasis on measurement way make it atheoretical in nature