Information Organization, Database Searching, and Algorithmic Bias in Historical Research

Information Organization and Source Typology in History

  • The Challenge of Knowledge Organization:

    • Information must be organized systematically for humans to process, understand, and utilize it effectively.
    • In 20022002, Dr. Vartan Gregorian (former president of the New York Public Library and the Carnegie Corporation) stated that the greatest challenge facing society is organizing information into structured knowledge, emphasizing that society must rise above an obsession with quantity and transmission speed to focus on organizing, assimilating, finding meaning in, and ensuring the survival of amassed information.
    • Information systems function as ecosystems analogous to the taxonomy of species or the periodic table of elements, which attempt to categorize animals and elements into logical groupings.

  • Categorization of Historical Sources:

    • Primary Sources: Original evidence utilized by historians to analyze and interpret the past, including:
      • Artifacts and physical objects.
      • Documents and manuscripts.
      • Architectural building plans and city plans.
      • Statistics and quantitative data.
      • Images, videos, maps, and unforeseen material evidence.
    • Secondary Sources: Works that discuss and interpret historical meaning, such as monographs and peer-reviewed journal articles.
    • Tertiary Sources: Reference materials that aggregate definitions and summaries, such as encyclopedias and multi-volume historical reference sets.

  • Complexities of Information Retrieval Systems:

    • Knowledge organization is non-hierarchical, unlike biological taxonomy, making categorization complex because human cognitive processes and experiences vary widely.
    • There is no single uniform type of library user or information seeker.
    • Database: An organized collection of structured information or data stored electronically.
    • Database Management System (DBMS): Software that interfaces between users and raw data, applying structural organization to render data accessible.
    • Search Engine: Hardware or software program designed to search for and identify items in a database corresponding to user-specified keywords (as defined by the Oxford English Dictionary).
    • Bibliographic Databases: Databases containing organized references to research works, journal articles, monographs, or primary source collections.
    • The Internet: A global system of computer networks that serves as a delivery pipeline for diverse sources, exhibiting mixed searching performance relative to curated bibliographic databases despite perceived search engine efficiencies like Google.

Search Engine Mechanics and Algorithmic Operations

  • Data Shedding and Granular Tracking:

    • Modern society relies on algorithms and big data, with data shed continuously from smartphones, web browsing, database searching, and health tracking systems used to triage medical treatment and outcomes.
    • Artificial Intelligence (AI) uses granular user data to model and predict user search behavior.
    • Technological advances accelerate data availability and automated processing with minimal human intervention.

  • Algorithmic Definition and Result Ranking:

    • An algorithm is a protocol determining how a search engine queries a database and ranks results matching its interpretation of search terms.
    • Algorithms are unique to each database, causing search results and rankings to fluctuate from day to day, hour to hour, and across geographical locations.
    • Ranking factors include:
      • Frequency and location of query words within the record.
      • Geographic location of the user.
      • Previous search history of the user.
      • Commonly searched terms and phrasing patterns learned by AI.
    • Google's Ranking System: Combines the frequency and positioning of query words on a page with the total number of external web pages linking to that page.

Strategic Searching: Keyword versus Subject Headings

  • Keyword Searching:

    • Functions across nearly all databases and serves as the standard initial search technique.
    • Recommended when exact subject headings are unknown or when searching for novel concepts lacking established subject terms (e.g., "sportswashing" or "history of borderlands").
    • Executes exact character matching against fields such as titles, abstracts, notes, tables of contents, and subject fields.
    • Limitations:
      • A keyword query for children and history will fail to retrieve a work titled A History of Childhood in the Middle Ages unless the exact string children appears in the bibliographic metadata.
      • Does not automatically search for linguistic variations or synonyms (e.g., adolescents, teens, or teenager).
    • Some systems offer keyword auto-suggestions based on query patterns from previous researchers.

  • Subject Searching and Controlled Vocabulary:

    • Provides higher relevance in academic research by querying structured index fields within library catalogs, periodical databases, and primary source collections.
    • Employs a controlled vocabulary (a dictionary of standardized, approved terms applied uniformly across indexed sources).
    • Specialized subject periodical indexes utilize discipline-specific jargon reflecting specialized fields of study.
    • Most academic library catalogs rely on the Library of Congress Subject Headings (LCSH).
    • Recommended Workflow: Begin with a broad keyword search, inspect the hyperlinked subject headings in relevant bibliographic records, and click those subject headings to execute precise subject searches.
    • Example: Searching the keyword child excludes books containing only teenagers; selecting official LCSH terms like Adolescents--United States--History, Teenagers--United States--History, or Children--United States--History narrows the field precisely to relevant monographs.

  • Natural Language Searching:

    • Utilizes AI models to interpret queries entered in everyday conversational speech syntax, altering traditional distinctions between keyword and subject searching.

Algorithmic Bias and Systemic Inequities

  • Bias in Algorithmic Systems:

    • Algorithms make assumptions using historical user patterns; query suggestions can reproduce societal prejudices, including racist and sexist phrasing.
    • Demographics of Software Engineering: Mathematical search formulas are predominantly designed by white, male programmers from wealthy Western nations whose priorities center on system performance and speed rather than social impact.
    • Biased retrieval outputs are frequently mischaracterized by developers as accidental glitches rather than structural flaws.
    • Certain mathematical search formulas are deliberately manipulated by external actors to promote racism, violence, white supremacy, and anarchy.
    • Examples of Algorithmic Bias:
      • Search queries such as "black girls" on web search engines have historically generated hyper-sexualized, racist, and sexist results.
      • Facial recognition algorithms consistently demonstrate higher error rates and mislabeling when processing images of Black men.
      • In 20202020, Apple removed legacy technical terminology including master, slave, whitelist, and blacklist from its software code and applications.

  • Database Bias in Academic Literature Coverage:

    • A comparative evaluation study by Rafols, Ciarli, and Chavarro analyzed research on Indian rice production across three primary scientific databases: CAB Abstracts, Web of Science, and Scopus.
    • Findings demonstrated that CAB Abstracts provided superior journal coverage and retrieved significantly more literature originating from low-income countries in the Global South.
    • Takeaway: Researchers must search across multiple databases and employ varied search strategies rather than relying on a single commercial interface.

Structural Bias in Library Subject Headings (LCSH)

  • Origins and Institutional Bureaucracy:

    • LCSH cataloging frameworks originated within a cultural framework dominated by Western, white, male worldviews.
    • Updating LCSH terminology is a slow, highly bureaucratic process that attempts to balance neutral, accurate description without directly dictating public policy.

  • Examples of Cultural, Gender, and Political Bias in LCSH:

    • Gender Asymmetry: Historical presence of subject headings for Women farmers without a corresponding heading for Male farmers (subsumed under Farmers).
    • Racial Terminology: Progression from legacy terms such as Afro-Americans to the current authorized heading African Americans.
    • Immigration Terminology (Illegal aliens vs. Noncitizens):
      • In 20142014, the Library of Congress initiated steps to replace Illegal aliens with Undocumented immigrants.
      • In 20162016, the U.S. Congress intervened by introducing the Stopping Partisan Policy at the Library of Congress Act to block the change, arguing illegal alien was established statutory legal language.
      • While the Congressional bill failed to pass, the Library of Congress officially replaced Illegal aliens in 20212021 with Noncitizens and Illegal immigration.
      • In April 2022\text{April } 2022, the Biden administration mandated that U.S. federal immigration agencies discontinue using the term alien.
    • Armenian Genocide Terminology:
      • In 20202020, the Library of Congress initially declined a proposal to update Armenian massacres, 1915-1923 to Armenian genocide, 1915-1923, asserting that using genocide constituted a sensitive diplomatic and foreign policy determination reserved for the U.S. President and Department of State.
      • Following bipartisan Congressional support, the official heading was successfully updated to Armenian genocide, 1915-1923.

  • Historical Comparison of Subject Heading Revisions:

    • Afro-Americans →\rightarrow African Americans
    • Jewish question →\rightarrow Jews
    • Armenian massacres, 1915-1923 →\rightarrow Armenian genocide, 1915-1923
    • Vietnamese Conflict, 1961-1975 →\rightarrow Vietnam War, 1961-1975
    • Japanese Americans--Evacuation and relocation →\rightarrow Japanese Americans--Forced removal and internment
    • Tulsa Race Riots, Tulsa, Okla., 1921 →\rightarrow Tulsa Race Massacre, Tulsa, Okla., 1921
    • Cripples / Handicapped →\rightarrow People with disabilities
    • Illegal aliens →\rightarrow Noncitizens / Illegal immigration
    • Race question →\rightarrow Race relations
    • Slaves / Slave →\rightarrow Enslaved persons
    • Unresolved Antiquated Headings: Headings such as Sexual minorities and Indians of North America (rather than Indigenous peoples) remain in official usage.

  • Remapping and Corrective Initiatives:

    • African American Subject Funnel Project: A joint effort by the African American Studies Librarians' Interest Group of the Association of College and Research Libraries (ACRL) and the Library of Congress to update and create non-biased headings.
    • TRLN Discovery Subject Remapping: Initiated by the Triangle Research Libraries Network to map outdated official LCSH terms to more accurate descriptive headings at the local catalog level.
    • Individual academic libraries frequently insert local subject metadata to correct systemic bias and improve source discovery.

Constructing Search Statements and Boolean Logic

  • Search Planning Protocol:

    • Deconstruct a complex research topic into core component concepts.
    • Brainstorm lists of exact synonyms, variations, and related historical terminology for each concept set.

  • Sample Concept Breakdown Matrix:

    • Research Topic: Participation of women in the scientific revolution in early modern Europe.
    • Concept Set 11 (Gender): women, gender, female
    • Concept Set 22 (Domain): science, maps, mathematics, chemistry, physicians, surgeons, medicines, physics, physicists, botany
    • Concept Set 33 (Chronology): early modern, 16\text{th century}, 17\text{th century}, 18\text{th century}, enlightenment
    • Concept Set 44 (Geography): Europe, France, England, Great Britain, Germany, Italy

  • Syntax Rules for Database Querying:

    • Parentheses (): Enclose synonyms and alternate terms within parenthetical sets to establish logical grouping.
    • Truncation Root Symbols: Truncation operators expand a root word to capture lexical variations (e.g., botan* retrieves botany, botanist, botanical, and botanically).
    • Boolean AND Operator: Joins discrete parenthetical concept groups, requiring retrieved records to contain at least one term from each defined set across metadata fields (title, abstract, subject terms).

  • Discovery Layers:

    • A discovery layer is a single-search-box interface that aggregates disparate library databases, online catalogs, periodical indexes, digital reference sources, and government documents into a single index.
    • System Limitations:
      • Does not index every specialized database subscribed to by a library.
      • Subject headings are not standardized across aggregated sources (e.g., local catalogs use LCSH, whereas Historical Abstracts uses its own proprietary controlled vocabulary).
      • Filtering by subject in a discovery layer may inadvertently drop relevant items indexed under differing controlled vocabularies.

Advanced Search Strategies, Facets, and Form Subdivisions

  • Faceting Search Results:

    • Facets appear along the sidebar of a search results page, displaying common metadata categories attached to retrieved records.
    • Clicking facets applies Boolean-style filters to progressively narrow broad result sets into manageable, highly targeted research corpora.
    • Common Facet Categories: Subject term, language, publication date range, specific database source, geographical region, physical item location, format/source type.
    • Example Application: A broad keyword query for civil war and women filtered by subject facets Social conditions and Sex role isolates targeted monographs such as Daughters of the Union: Northern Women Fight the Civil War and Occupied Women: Gender, Military Occupation, and the American Civil War.

  • Advanced Searching with Person, Place, and Form Headings:

    • Biographies: Execute subject searches on proper names (e.g., Subject = Gandhi), which can be narrowed via standard subdivisions like Bibliography, Correspondence, Philosophy, and Views on life.
    • Geographical Regions: Search specific political entities (e.g., Subject = Berlin (Germany)). To find historical studies, run an advanced search combining Subject = Berlin with Subject = history.
    • Form Subdivisions: Standardized subject qualifiers that refine regional/historical focus:
      • Social life and customs
      • Politics and government
      • Civilization
      • Description and travel
      • Antiquities

  • Nomenclature Shifts and Historical Search Term Selection:

    • Language in historical cataloging is inherently political, requiring researchers to test multiple historical terms:
      • French and Indian War versus Seven Years' War
      • American Revolution versus American War of Independence
    • Vietnam War Case Study:
      • Because the U.S. never formally declared war, the original LCSH heading was designated as Vietnamese Conflict, 1961-1975.
      • Researchers initially had to execute keyword searches on "Vietnam War" to uncover the official heading Vietnamese Conflict.
      • LCSH has since updated its vocabulary to Vietnam War, 1961-1975 and added related entries such as Cambodian-Vietnamese Conflict, 1977-1991 and Sino-Vietnamese Conflict, 1979.
    • Yugoslav Wars Case Study: Literature on the Bosnian War of the early 1990s1990\text{s} is cataloged under Yugoslav War, 1991-1995--Bosnia and Herzegovina.

  • Key Conclusions and Recommended Literature:

    • Database retrieval models are non-standardized; no single universal tool searches all academic information simultaneously.
    • Mastery of search engine algorithms, controlled vocabularies, and systemic biases is essential for exhaustive historical inquiry.
    • Recommended Professional Reading:
      • Center for Humane Technology
      • Library Subject Headings
      • Perspectives on History
      • Weapons of Math Destruction
      • Masked by Trust (analysis of library discovery layers)