Mending Fences: Defining the Domains and Approaches of Quantitative and Qualitative Research
Foundational Perspectives and the "Good Fences" Metaphor
Rationale for the Article: Given the increasing ubiquity of qualitative research and mixed-method designs, the authors investigate whether and how qualitative and quantitative research models can be integrated. The paper is prompted by the National Institutes of Health (NIH) adoption of best practices for mixed methods research (Creswell, Klassen, Plano Clark, & Smith, 2011).
The Metaphor: The title and opening draw on Robert Frost’s (1919) poem "Mending Wall," specifically the line: "Good fences make good neighbors." The authors argue that a productive relationship between research models depends on understanding the "fence" or the boundary between them, predicated on an insight into their fundamentally different guiding questions and domains.
Keywords: Best practices, methodology, mixed methods research, qualitative research, quantitative research.
Knowledge Claims and Philosophical Suppositions: Qualitative and quantitative research are not merely different data types (numeric vs. non-numeric) but stem from different philosophical suppositions (Garza, 2004, 2007, 2011; Giorgi, 2009; von Eckartsberg, 1998). Misunderstanding these foundations can "create a terrible mess" in research (Greener, 2011, p. 3).
Defining Quantitative and Qualitative Domains and Approaches
The Concept of "Approach": Following von Eckartsberg (1998) and Giorgi (1970/2009), the term "approach" refers to the implicit interpretative frames of reference and philosophical foundations brought to bear on subject matter and methods.
Qualitative vs. Quantitative Definitions: - Quantitative: Positioned at one end of a continuum, it involves the numeric analysis of data to test and verify relations of magnitude between variables measuring quantities (e.g., height, weight, hippocampal volume, number of behaviors). - Qualitative: Positioned at the other end, it makes descriptive knowledge claims about meaning using "descriptive" data, typically expressed in linguistic narratives.
The Research Continuum: The authors represent the interface of knowledge and frame of reference as a continuum (see Figure 1 in transcript). While "pure" cases exist at the poles, many studies occupy a "middle ground" involving the mixing of data and approaches.
Middle Ground Definitions: - Quantitizing: Occurs when research claims knowledge of an order of magnitude but uses a qualitative interpretive framework (e.g., performing numerical analyses based on the frequency of themes or ratings of intensity; Teddlie & Tashakkori, 2009). - Qualitizing: Occurs when research claims qualitative knowledge but uses a quantitative interpretive framework (e.g., defining categories based on range in magnitude or treating frequency counts as dimensions of importance; Hesse-Biber, 2010).
Epistemological Challenges in the Middle Ground
The Rubric of Measurement: Special care is needed when knowledge claims do not match the interpretive frame. Johnson and den Heyer (1980) emphasize the distinction between a statistical question and a psychometric question.
Example of Ratio Data: A regression coefficient of between Facebook friends and profile photos is purely quantitative. Each unit (1 friend, 1 photo) represents a consistent quantity on a ratio scale.
The Likert Scale Critique: Likert-type data often represent "quantitizing." - Concerns exist regarding whether Likert data is interval or ordinal (Carifio & Perla, 2008). - Knapp (1990) posed the question " what?" in response to a value circled on a scale. It is unclear if steps are equidistant or if the "degree" of agreement is a quantity that remains identical across different participants. - These values cannot escape the subjective understanding of the participant.
Dummy Coding and Regression: Merenda (n.d.) highlights a troubling use of quantitizing where dichotomous categories (e.g., male and female) are treated as continuous variables through dummy coding. This violates the assumption of continuity in regression, as there are no values between the two categories.
Case Studies in Quantitizing and Qualitizing: - Cialdini et al. (1976): Qualitizing example where frequency of "we" statements was rendered as dimensions of subjective ownership of victory or defeat. - Pollard, Nievar, Nathans, and Riggs (2014): Quantitizing example where chi-squared analyses of theme counts were used to conclude that the experiences of Hispanic and Caucasian mothers did not differ.
The Hegemony of Approach and Methodological Pitfalls
Hegemony of Perspective: Quantitative frameworks often dominate qualitative practice. For instance, applying quantitative "reliability" (correlation and sameness) to qualitative analysis (which focuses on relatedness) can hamper the discernment of complex meanings.
Fluid Dynamics Analogy: Fredrickson and Losada (2005) used formulas from fluid dynamics in physics to explain attitude changes, resting on the presumption that attitudes follow the same laws as fluids. Critics (Brown, Sokal, & Friedman, 2013) raise serious conceptual concerns about such mathematical applications to human phenomena.
Participant Validation: Giorgi (2008) critiqued the practice of asking participants to "verify" qualitative analyses. Since participants are not trained in the specific phenomenological approach or procedures, they cannot assess the validity of the technical analysis.
Incursions: The authors note that quantitative incursions into qualitative research are more common because everyone speaks in narratives, but few speak in the specialized language of statistics.
Concerns Regarding Counting in Qualitative Research
Acontextual Counting: Sandelowski (2001) warns against equating frequency with importance. What a participant does not say can be as revealing as what they do say.
The Virginity Narrative Example: In a workshop described by Garza (2004, Spring), a participant described losing her virginity but never mentioned her partner. This absence was more meaningful than any count of words present.
Analytic Overcounting: This refers to the tendency to count everything (e.g., precise number of themes or participants exhibiting a theme) to the detriment of describing the actual patterns of meaning.
Domain Respect: The authors argue that once one begins to count themes as the primary mode of analysis, they are strictly speaking no longer conducting qualitative research, nor are they conducting traditional quantitative research.
Confirmation versus Augmentation
The Problem with Confirmation: Researchers often use one method to "confirm" or "verify" the findings of another. The authors argue this is inappropriate because it implicitly holds one data type as more valid.
Biological vs. Descriptive Examples: Hippocampal volume differences (Hampton et al., 1995) between hoarding and non-hoarding species are complementary to qualitative data on memory, not confirmatory. They are "two different languages."
Concordance Rates: Riegel et al. (2010) measured the percentage of agreement between self-care themes in narratives and survey cutoff scores. - If self-care maintenance had agreement and self-care confidence reached , the difference simply means 20 more people circled a high number and spoke of the theme. - This does not make the data more valid; it merely renders qualitative data into a dimension of magnitude.
The Mountain Task Analogy: Based on Piagetian developmental psychology, a child (egocentric) cannot imagine a different perspective of a mountain from the other side of a table. An "approach-centric" researcher similarly seeks only confirmation of their own view. A methodologically pluralistic researcher seeks "augmentation"—using multiple perspectives to provide a more complex and full description.
Methodological Pluralism and Integration Types
The Case for Methodological Pluralism: The authors advocate for an approach akin to "methodological multiculturalism" involving respect for the boundaries and currencies of both domains.
Integration Types (Creswell et al., 2011): 1. Connecting Data: One analysis informs subsequent data collection (e.g., qualitative analysis reveals a new variable for quantitative study). This preserves boundaries. 2. Embedding Data: One method is primary, the other secondary/supplemental. This preserves boundaries. 3. Merging Data: Comparing or confirming findings by transforming qualitative themes into counts. The authors warn this often violates technical boundaries.
Case Study: Program Implementation (Trend, 1979): - Quantitative data showed a program was successful. - Qualitative data showed it was not. - Reconciliation revealed a contextual variable: Urban versus Rural site location. Nuances in family income, ethnicity, and recruitment ease were only visible when both data types were used to augment each other.
Case Study: Facebook and College Satisfaction (Landrum & Garza, 2011): - Quantitative Step: Used Structural Equation Modeling (SEM). Heavy FB users connecting with high school friends reported less satisfaction with college than those connecting with college friends. - Qualitative Step: Focus groups revealed that for some, the "meaning of home" had transformed from their parent's house to their college residence. - Conclusion: Satisfaction was not about the amount of usage (magnitude) but the meaning of the connection (qualitative). This led to new research avenues.
Final Conclusions and Recommendations
No Privileged Position: Neither method holds a privileged perspective on the world. Even natural science is not value-free.
Validity: In qualitative research, validity is defined as coherence between the researcher’s frame of reference, the question, the data, and the findings.
Admonition for "Methodological Adventurers": Researchers visiting other domains must learn the local languages and customs. Only through understanding and respecting the "fences" can the uniqueness of both approaches be appreciated to make "great strides" in psychology.