Research Methodology in Management: Current Practices, Trends, and Implications for Future Research

RESEARCH METHODOLOGY IN MANAGEMENT: CURRENT PRACTICES, TRENDS, AND IMPLICATIONS FOR FUTURE RESEARCH

Authors

  • Terri A. Scandura, University of Miami

  • Ethlyn A. Williams, University of Colorado at Colorado Springs

Introduction

  • This study compares research strategies in management between two periods: 1985-87 and 1995-97.

  • Conducted a content analysis of articles from key journals:

    • Academy of Management Journal

    • Administrative Science Quarterly

    • Journal of Management

  • Investigated the use of research methods addressing issues of:

    • Triangulation

    • Validity (internal, external, construct, statistical conclusion validity)

  • Results indicate:

    • Notable trends in research strategies

    • Decreases in validity measures across studies

  • Emphasizes the importance of research method assessment for management studies to advance.

Theoretical Overview

  • Triangulation:

    • Defined as utilizing multiple methods to corroborate research findings.

    • Relates to the idea originated by McGrath (1982) regarding inherent flaws in all research methods, necessitating corroborative evidence.

    • Can apply to:

    • Strategies (e.g., qualitative vs. quantitative methods)

    • Data collection settings

    • Data sources (single vs. multiple)

    • Example: Jick (1979) details a study assessing anxiety and job insecurity during a merger using multiple data sources (self-reports, interviews, observations).

    • Advocates (Webb et al., 1966; Smith, 1975) suggest various methods to study phenomena to achieve a well-rounded understanding.

Research Strategies

  • Increased triangulation may enhance conclusion validity.

  • McGrath’s (1982) framework categorizes research strategies into eight types:

    1. Formal theory

    2. Sample surveys

    3. Laboratory experiments

    4. Judgment tasks

    5. Computer simulations

    6. Experimental simulations

    7. Field studies

    8. Field experiments

  • Adjustments for this analysis expanded to nine categories by combining formal theory/literature reviews and differentiating field studies based on data collection type.

  • Discuss how each method correlates with internal and external validity.

Validation Types

Internal Validity

  • Concerned with causal relationships. Requires:

    • Covariation between variables

    • Evidence that the cause precedes the effect

    • Alternative explanations dismissed

External Validity

  • The ability to generalize findings across different populations, settings, and contexts.

Construct Validity

  • Validity of measures relative to the theoretical constructs they are intended to reflect.

  • Required assessments for consistency and reliability of measures (Sackett & Larson, 1990).

Statistical Conclusion Validity

  • Ability to draw reliable conclusions based on statistical evidence of relationships among variables.

  • Involves consideration of sample size and the appropriateness of statistical tests used for data analysis.

Methodology

  • Content analyses conducted on research methods sections of articles across both time periods:

    • Articles selected if they utilized empirical methods or formal theory/literature reviews.

  • Total articles analyzed:

    • 1985-87: n = 280 empirical studies coded.

    • 1995-97: n = 334 empirical studies coded.

  • Results analyzed by 13 coding dimensions across various aspects of each study, including:

    • Research strategy employed

    • Substantive content area

    • Level of dependent variable analysis (individual, group, organizational)

Results Overview

Triangulation Trends

  • Comparison of the two decades showed:

    • Decline in sample surveys and laboratory experiments

    • Increased use of field studies, particularly those using secondary data

  • Significant shifts in substantive content, with a decrease in studies focusing on strategic planning, careers, and turnover, yet rising studies on groups/teams and international management.

Internal and External Validity

  • Increase of cross-sectional studies at the expense of longitudinal ones

  • Rise in usage of private sector samples while public sector studies declined.

Construct Validity

  • Less emphasis was placed on construct validation measures between the periods; reports on discriminant, convergent, and predictive validity decreased while reliability reports were low but stable.

Statistical Conclusion Validity

  • Sample size medians increased, which may suggest improvements in statistical power.

  • A notable trend includes a decline in usage of analytical methods such as ANOVA while increasing tendencies in regression techniques and time-series analysis.

Discussion

  • General trends indicate a shift toward methods compromising validity and triangulation.

  • Findings stress an existing gap between rigor and relevance in management research, highlighting issues of validity.

  • Recommendations from prior literature (e.g., Mitchell, 1985) on practices to enhance validity appear neglected over the periods studied.

Future Directions

  • The need for improved triangulation in management studies is emphasized to build robust research findings.

  • Discussion of innovative research methodologies, such as online samples and simulation tools to enhance methodological rigor.