Chapter 18 - Developing a Research Report
the research report is the permanent record of counseling research
if a research project is poorly designed and has serious methodological flaws, the quality of the write-up is irrelevant
a well-designed and implemented study that is reported ineffectually loses credibility
Types of Research Reports
research reports can take different forms, depending on what point a counseling researcher is at in her or his career, and the audience for which the topic is best suited
three common types are journal articles, dissertations or theses, and dissertation or thesis proposals
Journal Article
the most common and widely-accessed avenue for publishing research reports is through refereed journals
each journal has an editorial staff and a large number of professionals in the field who serve as peer reviewers of submitted manuscripts
Dissertation/Thesis
some master’s students and all doctoral students are required to complete a research project culminating in a thesis of dissertation as a final requirement before graduating
most theses and dissertations consist of four or five chapters, depending on whether the intro and review of literature are included in different chapters
Dissertation/Thesis Proposal
before proceeding with the study, graduate students typically write a research proposal that includes the first three chapters of the dissertation or thesis (introduction, literature review, and methodology)
Research Presentation
many researchers present their work at professional conferences
formats include educational sessions, roundtable discussions, and poster presentations
Format of the Research Report
the format and organization of a research report can vary, however, you should expect to find the following basic components in a research report: title page, abstract, introduction, review of literature, methods, results, discussion or conclusions, and references
title page
research is only useful to future researchers and practitioners if it can be accessed
the best titles are informative, catchy, and should provide details such as the independent and dependent variables, the population of the study, the design, and clues to the outcome of the study
abstract
intended to give readers a brief, but thorough, summary of the article
readers often use the abstract to determine if they will read the rest of the article
the abstract should be succinct, well-organized, and should accurately represent the content of the article
most sections of the report should be represented in the abstract: intro, methods, results, conclusion
introduction
a clear and well-organized introduction provides the framework for the rest of a report
the intro usually consists of a statement of the problem, a brief overview of related research and theoretical underpinnings, and a list of the research questions and hypotheses
statement of the problem
the statement of the problem is a description of the current state of knowledge related to the research topic
the problem statement should provide answers to the following questions:
why is this issue important?
who does this issue impact?
what is the purpose of this study?
what practical outcomes does the researcher hope will emerge from the study?
overview of relevant research
provides a framework for how this particular study will address the problem
the overview of the literature is a brief, but thorough description of previous research findings, relevant methodological issues in previous research and important conclusions that inform the researcher’s decisions about how to study the issues at hand
research questions and hypotheses
after developing a rationale for the study, the next step is to state the research questions and/or hypotheses
the research questions focus the report, by centering on the purpose of the study
the questions should be specific, clearly worded, and an extension of the introduction and rationale
clear, operationalized definitions for each variable should be provided
methods
the methods section should be clear and detailed enough to provide info to critique the validity of the study and to replicate it
the methods section walks through each step of the research study
the four main areas in the methods section are participants, instruments, research design, and procedures
participants section answers the following questions:
who are the participants of this study?
how were they selected?
how many participants make up the sample?
are there different groups of participants (an experimental and control group, or other types of groups being compared)?
how many participants are in each group?
what are the demographics of the sample, such as age, gender, ethnicity, region of residence, or other relevant characteristics?
instruments or measures
address each construct or variable and describe the instrument used to measure it
the instrument description should include, the full name of the instrument, authors, appropriate citations, previous evidence of reliability and validity, info about the sample on which it was normed, number and type of items (e.g., Likert scale)
research design
identify the research design that is being used
indicate how the variables are organized in the study in order to answer the research questions? is the design correlational, ex-post facto, causal-comparative, experimental, or quasi-experimental in nature?
procedures
provides the reader with a detailed, step-by-step guide to how the study was conducted
describes precisely what was done, and when, where, and how each step was completed
includes how the participants were contacted, hwo informed consent and parental permission was addressed, how the instruments were administered and in what order, instructions that were given to the participants, how experimental conditions were manipulated
results
this is the section in which the actual findings of the study are presented
what statistical tests were run and the results of these analyses are presented
a summary of the stat results, along with the statistical significance, effect size, and power should be provided
connect the findings with the stated research questions and illuminate if findings support or refute your hypotheses
discussion
provides an opportunity to draw conclusions about the results of the study and connect the findings to the body of existing research described in the intro
address the following questions
what limitations of the study impact the conclusions that can be drawn?
what ethical concerns impact the interpretation of results?
what are some implications of these results for practitioners?
what now, that is, what are some directions for future research?
Qualitative Research Reports
the purpose of qualitative research is to understand personal experiences
although an attempt to discover patterns and themes exists, excerpts of participant statements are often reported verbatim
if you conduct a qualitative research project, you will not attempt to condense your data into numbers or statistics, but allow access to the emotion and personal quality of the participants’ voices
qualitative research
interactive, subjective, and participatory and this is evident in the tone and style of writing in the report
often reports are in first person and disclose personal investments in the research topic
the report addresses philosophy, purpose, methods, and presentation and reflects the values of constructivism and critical theory
Building Skills for Effective Research Report Writing
in addition the typical sections of the research report, there are a number of general guidelines to support research report development
these include 3 major sections entitled Developing as a Writer, Preparing in Advance, and Writing Guidelines
developing as a writer
skill of developing research reports is developed over time and through predictable processes
to hone report writing skills over time, important to practice with opportunities for feedback, anticipate opportunitites to re-write, and avoid writer’s block
practice with feedback
develop collaborative relationships that will provide candid report feedback
reviewers without content expertise can be useful
reviewers with expertise in research report writing may be particularly helpful
reviewers with content expertise may add useful insights about targeting the report to a particular audience, and whether a report might be appropriate for the selected outlet
anticipate opportunities to rewrite
journals rarely accept research reports without requiring revisions
even if a manuscript receives a favorable review, opportunities for revision and improvement often are available
most editors do not assume that all suggested revisions will necessarily be implemented
it is useful to go through the reviews and carefully consider which suggestions will be implemented and which will not
avoiding writers block
the process of developing a research report can be difficult to start and ubiquitous writer’s block may occur
a general guideline is, just write
is it much easier to edit and revise a first draft than it is to develop the first draft
by getting the ideas onto paper, you can begin to solicit the support of others who can provide an informal review to help refine your work
prepare in advance
know audience in advance
starting well before the research report is begun, it is important to consider a target audience and a potential outlet for the research report
in this manner, is it possible to tailor language specifically to the audience of interest
understand guidelines for authors for your chosen outlet
often specific journal Guidelines for Authors for particular publications
failure to follow these guidelines could preclude publication or make for extensive revisions
particular guidelines, such as maximum page length and other formatting requirements should be closely adhered to in report preparation
be familiar with publication guidelines for your profession
most journals adhere to specific publication styles
in counseling, most often based on the publication manual of the American Psychological Association (APA, 2001)
prior to developing a research report, it is important to be familiar with these publication guidelines
writing guidelines to strengthen actual writing of the report
build a bridge to your study
interpret and recommend within existing data
be transparent in your writing
Build a Bridge
the methodology of your study should leave no questions as to why this study was conducted and why it is imporant
this bridge occurs through an overview of the literature in which you build a clear argument for the study
the key question that has to be answered is, Why this study now? How does this study provide a logical extension of the current knowledge base?
Interpret and Recommend with the Current Data
in the discussion section, the author(s) commonly interpret the results to make meaning of the results and provide recommendations for counseling practice and future research
these interpretations and recommendations are important aspects of the research report and should be consistent with the findings of this study and previous research on the topic
making interpretations and recommendations that extend beyond the available data is a common problem in report writing
Be Transparent
reports often omit limitations of their study, reframe limitations to suggest they are not really limitations, or minimize these limitations
all research projects have methodological limitations that exist a priori or evolve over the course of the study
limitations should be addressed explicitly, as well as how interpretations of findings might be influenced by these limitations
Common Mistakes in Research Reports
there are a number of common mistakes that routinely plague research reports
can be categorized as general writing problems, problems in the review of existing literature, problems in reporting methodology, and problems in writing discussion sections
general writing problems
many common mistakes made in overall writing style and tone of the research report
failure to follow author guidelines
virtually all journals publish author guidelines and are typically available in back issues and via the Internet
common guidelines include maximum page length and restricting the use of Tables and Figures, both related to publication costs
failure to adhere to guidelines may result in failure of acceptance for publication
failure to follow publication style guidelines
one detail that typically is chronicled in author guidelines is what publication style should be followe
APA style is most common for counselors
minor style problems typically do not substantially hinder the review process but major style problems in the writing may negatively impact the review process
use of discriminatory language
when research report authors include language, often inadvertently, that is discriminatory in nature it may be problematic if a reviewer has a strong reaction that may create a negative bias in the review process
transition and heading problems
a common problem in reports is inattention to the overall flow of the report
if sections are written without sufficient attention to transitions individual sentences and paragraphs may be well written, but the report as a whole does not flow well
this can be corrected by attending to introductory sentences and phrases, and in closing sentences and phrases in a section to segue into the next section
the appropriate application of headings also helps the reader transition to new sections
a weak abstract
the abstract should follow the author guidelines, particularly for length, and briefly describe the study
the abstract will often be the only text available to make a decision about whether to download the full text of the article
both hard copy and on-line full text articles, should find that the abstract accurately and fully describes the study
common problems in the literature review
problems may also emerge in the review of the literature
common problems in this section of the report
failing to build a logical argument for the study
entering the methodology section of the report, it should be clearly understood “Why this study now?”
the argument should be grounded in existing empirical and theoretical literature as to why this study was conducted
the argument should be clearly and explicitly delineated and is important as a context for the discussion section of the report, providing a foundation on which to discuss the results of the study and make recommendations for practitioners and future research
bias in reviewing existing literature
in some instances, report writers will only review studies that support the argument that they hope to make
bias may be introduced by exaggerating limitations of studies with findings contradictory to the bias of the author(s), and minimizing or ignoring limitations of studies with findings consistent with this bias
it is the responsibility of the report writer to provide a balanced and fair review of existing literature, including an objective account of methodological limitations in previous research
common problems in methodology sections
the methods section of the report provides the blueprint of how the study was carried out
common problems include
insufficient detail
the goal of the methods section of the report is to provide sufficient detail to allow replication
sufficient detail should be provided about participants, instrumentation, and procedures to be easily replicated by others
if page limitations preclude inclusion of all info necessary to allow replication, this should be noted explicitly in the report, along with contact info that allows a reader to obtain additional info
failure to provide evidence of validity and reliability
a common mistake made in research reports is to provide inadequate evidence of instrumentation validity and reliability
writers may fail to provide any evidence of validity and reliability
writers may provide evidence of instrument validity and reliability only from previous research
research instruments are valid and reliable only for a specific sample this it is necessary to report psychometric info for the sample of each study
common problems in results section
issues related to statistical versus clinical significance
statistical significance is not the same as clinical significance
significance tests are influenced by sample size
researchers have turned to reporting effect size as an alternative to significance testing
providing discussion within the result section
a common problem occurs when the report writer offers discussion content, including interpretations and implications of findings, within the results section
unless guidelines request this, it is necessary to include all interpretations and implications of findings in the discussion section rather than the results section
common problems in discussion sections
failing to delineate limitations to the study
discussion of the methodological limitations will inform future research and provide a context for the results of the study
failing to discuss limitations of the study leaves the reviewer to determine
whether the study is flawless
if the writer intentionally chooses not to discuss limitations
if the report writer is unsophisticated in the importance of discussing limitations
going beyond the data
it is important that the findings be consistent with the findings of the current study and previous research findings
it is a mistake to make interpretations or recommendations that go beyond the data or are inconsistent with existing data
it is indicative of limited rigor in the report and is a common “fatal flaw” in the report that precludes publication
failing to note implications for practitioners
the opposite of going beyond the data also may occur when the report writer fails to discuss implications for counseling practitioners and recommendations for future research
the implications for practitioners might be considered the “so what” of the study, and recommendations for future research the “now what” of the study
recommendations for future research
manuscript review process
submit to journal editor
determined whether the manuscript will be distributed for peer review
reviewers read and study the report, and provide a detailed review to the editor, including feedback on the merits of the study and the quality of the written report
the editor generates an action letter, detailing the adjudication of the manuscript and recommended changes
the most common decisions are Accept As Is, Accept pending Minor Revisions, Revise and Resubmit, and Reject
if revisions are requested, a time frame in which these should be received is provided
if a manuscript is rejected, it may be resubmitted to another journal, although ethically it may be under review by only one journal at a time