Lesson 8: Authority Is Constructed and Contextual 2 - Examining Source Characteristics and Bias
Fundamentals of Source Appropriateness and Bias
Examining source characteristics is the primary method for determining if a source is suitable for academic research. Appropriateness is defined by three core pillars:
Credibility
Reliability
Validity
Evaluation of bias is a mandatory component of source analysis. Bias involves examining how the opinions, perspectives, or prejudices of those involved in publication influence content choices.
The analysis of bias focuses on two distinct components:
The "Who": The individuals and organizations involved in the publication.
The "Why": The intended audience and purpose of the work.
Context matters in evaluation; conventions for what is considered appropriate vary significantly according to the specific academic field.
Framework for Evaluating Sources: The "Wh-" Questions
A deeper evaluation of sources can be performed using a hierarchical structure of "wh-" questions. The order of these questions is intentional, moving from foundational intent to specific content details.
The Hierarchy of Evaluation
Why (Purpose): This establishes the foundation for all other characteristics.
Where (Context) and When (Currency): These are the next priority. If a source was not published in a reliable place or the date is not acceptable for the topic, it does not require further consideration.
Who (Authority): Evaluation of the creators, publishers, and sponsors.
What (Content): Evaluation of the specific data, arguments, and comprehensiveness.
Categorization of Appropriateness (Table 8.1)
Most Likely Appropriate:
Why: To inform.
Where: Scholarly/peer-reviewed journals, or / websites.
When: Up-to-date information.
Who: Expert in the field.
What: Comprehensive coverage, facts, statistics, charts, graphs, photographs, and citations.
Possibly Appropriate:
Why: To persuade.
Where: Scholarly/not peer-reviewed, trade publications, newspapers, or reputable / sites.
When: Not current but contains valuable historical or background information.
Who: Journalist or professional writer.
What: Commentary on an issue or personal narrative.
Most Likely Not Appropriate:
Why: To sell something, to present personal opinions, or to promote a particular political agenda.
Where: Social media.
When: Obsolete. When new discoveries make the source information irrelevant, it should be discarded.
Who: Non-professional writer.
Deep Dive: Why – Audience, Purpose, and Bias
The "Why" underlies every choice made by those involved in publishing and can lead to bias in various ways.
Assessing Intended Audience: One must determine if the target audience is scholars in the field, college-educated readers, or any adult reader.
Assessing Intended Purpose: One must ask if the author intends to inform, persuade, sell, or present a specific viewpoint.
The author's intentions influence where they choose to publish and what specific information they include or omit.
Deep Dive: Where – Context and Authoritative Domains
Determining the appropriateness of context involves assessing the authority of the publication (journal, news source, or website) and identifying potential biases.
Domain Analysis:
or sites: Often considered more authoritative as the authors typically have strong qualifications and the publishers are less likely to promote an agenda.
sites: May carry more bias depending on the commercial interests of the host.
Caveat: Bias is never entirely absent. Even and sites may promote specific agendas based on political ideologies or their stated organizational mission.
Deep Dive: When – Currency and the Information Cycle
Currency influences source appropriateness based on how information evolves over time. This is linked to the Information Cycle, where data becomes more comprehensive as an event evolves.
Contextual Currency:
Historical/Background: Credentials and reliability are maintained regardless of recentness.
Statistical Data: Must be strictly up-to-date.
The Three-Year Rule: In many professional fields, to achieve credibility, sources should not be more than approximately years old. A specific cut-off year should be determined based on the individual research topic.
Deep Dive: Who – Assessing Author, Publisher, and Sponsor Authority
Identifying bias requires looking at affiliations and professional contexts (Table 8.2).
Criteria for the Author
Education level.
Work context and professional experience in the field.
The number of previous publications on the specific topic.
Active memberships in professional organizations.
General reputation and status as an expert.
Criteria for the Publisher
Age of the publishing company.
Reputation for quality.
Impact Factor: For journals, this is measured by how often the work is cited by others.
Criteria for the Sponsor/Promoter
Reputation and mission statement.
Formal connections to other organizations.
Case Study: Evaluation of Plastic Sustainability
Source Profile
Title: Plastics of the Future? The Impact of Biodegradable Polymers on the Environment and on Society
Journal: Angewandte Chemie International Edition, ,
DOI:
Author Analysis
Tobias P. Haider: Works at Max Planck Institute for Polymer Research (world-leading research center); holds a Master of Science in Chemistry; widely cited.
Carolin Völker & Johanna Kramm: Work at Institute for Social-Ecological Research with the junior research group PlastX. They focus on transdisciplinary sustainability and the societal role of plastic.
Potential Bias: Their support for biodegradable plastics may be influenced by personal interests, financial needs for research funding, or institutional mission.
Publisher and Sponsor Analysis
Publisher: Angewandte Chemie is a journal of the German Chemical Society. It has a high impact factor and covers a broad range of professional topics, suggesting a lack of narrow bias.
Sponsor: The German Chemical Society is a professional, credentialed organization that must meet field standards, which supports its credibility.
Deep Dive: What – Relevance, Accuracy, and Comprehensiveness
Relevance Questions
Does it fit the academic context?
How does it enter the existing conversation?
Which specific part of the research discussion does it support?
Does it share the same audience as the researcher?
Accuracy Strategies (Table 8.3)
Source Verification: Look up references cited in the source; check they are valid.
Timeline Verification: Check publication dates of the source and its references to ensure they align with the research timeline.
Process Verification: Examine the primary research; look for evidence of peer-review or a "refereed" status. Consult with professors or experts to verify research validity.
Comprehensiveness and Bias
A source lacks comprehensiveness if it is limited in scope or fails to provide a thorough look at all sides of an issue.
Factors causing a lack of comprehensiveness include:
Strategic omission of facts.
Biased language used to promote a single perspective.
Limitations in information accessibility.
Statistical analyses that fail to account for all variables, leading to skewed results.
Case Study: The Science of Learning (Brookings Institution)
Evaluation Components
Publication Date: Tuesday, .
Publisher: Brookings (nonprofit public policy organization).
Target Audience: Educators/teachers interested in how learning science informs teaching. Evidence includes simple, explanatory language and a simple essay structure.
Authority (Authors):
Helyn Kim: PhD in Education (University of Virginia), Postdoctoral Fellow.
Eileen McGivney: Former Research Associate with extensive research history.
Esther Care: Senior Fellow who directs research centers.
Accuracy/Comprehensiveness Detail:
Uses links instead of a formal Bibliography section.
Cites the authors' own previous works (potential for point-of-view bias).
Includes a broad range of questions rather than promoting a single course of action, which suggests comprehensiveness.
References include TED talks and social science organizations.
Language Maturity: Uses objective phrasing such as “we need to explore” and “does not consistently inform.”
Synthesizing a Complete Source Evaluation
A strong evaluation summary should be at least sentences long.
It must clarify both the strengths and weaknesses of the source.
It should explicitly define in what situation the source is appropriate and when it should be avoided.
Comparison across multiple sources (at least ) is recommended to ensure a complete evaluation and recognize systemic bias.