Data and Databases

Software Systems and Environments

Web-Based Software Systems:

• Definition: Applications accessible directly through a web browser without requiring local software installation or specialised computational hardware.

• Key Platforms & Tools:

  • GenBank: Web portal for querying publicly available DNA sequence records.

  • EMBL: European molecular database Web interface for data access.

  • BLAST (Basic Local Alignment Search Tool): Web-based sequence alignment tool for finding similarity between biological sequences.

  • WebLogo: Graphical Web application for generating sequence logos from biological alignments.

Stand-Alone Software:

• Definition: Programs installed directly on local machines that leverage local processing resources to perform biological analysis.

• Key Programs:

  • MEGA (Molecular Evolutionary Genetics Analysis): Software suite for conducting statistical analysis of molecular evolution and constructing phylogenetic trees.

  • PyMOL: Molecular visualization system used to render $3D$ protein and nucleic acid structures.

  • Bioconductor: An open-source software project based on the R programming language providing tools for the analysis of genomic data.

Command-Line Tools and Operating Systems:

• Definition: High-performance environments where commands are executed via shell scripts or terminal interfaces without graphical user interface overhead.

• Key Components:

  • Linux & Unix: Primary operating system architectures for running scalable bioinformatics pipelines and server utilities.

  • Perl & Python: Scripting languages used to parse, manipulate, and process genomic file formats.

  • Command-Line BLAST: Local executable version of BLAST used to query massive databases efficiently.

Integrated Software Environments:

• Definition: Comprehensive platforms combining multiple sequence analysis tools, graphics, and database connections into a single framework.

• Key Platforms:

  • RStudio: Integrated development environment (IDE) for the R programming language used for statistical modeling and computational biology.

  • Geneious: Commercial integrated software suite for sequence alignment, assembly, and molecular cloning analysis.

  • BioJava: An open-source Java framework for processing biological data, alignments, and $3D$ structures.

Biological Data Formats

Overview and Rationale for Data Formats:

• Determinants of Format: The specific storage format depends on the biological data type (e.g., nucleotide sequences, amino acids, quality scores, structural coordinates), downstream experimental goals, and software compatibility.

• Format Proliferation: The existence of numerous biological formats is driven by software compatibility requirements, processing and reading speed, and storage efficiency.

• File Interconversion: Some formats can be freely converted between each other via software utilities, whereas others cannot due to non-overlapping metadata models.

• Accessibility: Plain-text formats (such as FASTA, FASTQ, and GFF) can be opened and edited using standard text editors like Notepad, whereas specialized formats require domain-specific software (such as PyMOL for PDB files).

FASTA Format:

• File Extensions: .fa, .fasta, .fa

• Data Type: Stores raw nucleotide (DNA/RNA) or amino acid (protein) sequences.

• Structure: Composed of two distinct line types per entry:

  • Header Line: > gives basic information about the sequence

  • Sequence Line: Placed on the line directly following the header, containing the sequence as a continuous, uninterrupted string of single-letter standard codes.

• Nucleotide Sequence Example:

  • >fasta sequence A

  • AACTAGATCGATAGCTAGAGCGCGAGATCGCGAGATCTA

  • >fasta sequence B

  • GTGACTGATCGACTATGGGGCATATATCTCTTTAGCGCG

  • >fasta sequence C

  • CCCTATAGCGTAGCATGCTAGTGACTATATACTATATATC

• Protein Sequence Example:

  • >fasta protein sequence a

  • LGKCFEPSGSFFYWDAIVDEWTVPPALVMCTFVRMGWFNCWDEAMKSVWNFLAWDETERY

  • >fasta protein sequence 2

  • NLWYNSVSLKKFQYNEYAMKGMQDDCKQTTMWIGQHHNNQPTCTCTMSIMQTAYSADYSV

  • >fasta protein sequence 3

  • DLVNMGHYSPDCHPGELPSQMHEKHEKRTAHSMQYPYYVFRRSPTYCKAKMFMMMFATYP

FASTQ Format:

• File Extensions: .fastq, .sanfasta, .fq

• Origin: Developed by the Sanger Institute to combine raw biological sequence strings with their corresponding per-base quality scores.

• Structure: Exactly four lines per sequencing read:

  • Line 1: Begins with an @ character, followed by a unique sequence identifier and optional descriptor metadata (e.g., @K00188:208:HFLNGBBXX:3:1101:1428:1508 2:N:0:CTTGTA).

  • Line 2: The raw nucleotide sequence read (e.g., ATAATAGGATCCCTTTTCCTGGAGCTGCCTTTAGGTAATGTAGTA).

  • Line 3: Begins with a + character, optionally followed by a repeat of the sequence identifier.

  • Line 4: Quality score encoded as ASCII characters, corresponding base-for-base to the nucleotides in Line 2 (e.g., AAAFFJJJJJJJJJJJJJJJJJFJJFJJJJJFJJJJJJJJJJJJJ).

• Phred Quality Score Scale ($Q$):

  • The Phred quality score ($Q$) represents the probability ($P$) that a corresponding base call is incorrect, calculated via the formula:     

  •        $Q = -10 \times \log_{10}(P)$

• Higher the score = higher chance if the base cell being correct

• FASTQ Encoding Variations:

  • Sanger (S): Uses Phred+33 offset; raw quality scores typically range from $0$ to $40$ (ASCII characters $33$ to $73$).

  • Solexa (X): Uses Solexa+64 offset; raw quality scores typically range from $-5$ to $40$ (ASCII characters $59$ to $104$).

  • Illumina 1.3+ (I): Uses Phred+64 offset; raw quality scores typically range from $0$ to $40$ (ASCII characters $64$ to $104$).

  • Illumina 1.5+ (J): Uses Phred+64 offset; raw quality scores typically range from $3$ to $41$ (ASCII characters $67$ to $105$), where score $0 = \text{unused}$, score $1 = \text{unused}$, and score $2 = \text{Read Segment Quality Control Indicator}$.

  • Illumina 1.8+ (L): Uses Phred+33 offset; raw quality scores typically range from $0$ to $41$ (ASCII characters $33$ to $74$).

General Feature Format (GFF):

• File Extensions: .gff2, .gff3, .gff

• Purpose: Tab-delimited plain-text format used to annotate specific structural and functional elements associated with genomic sequence coordinates. The current standard version is GFF3.

• Example

• Nine Tab-Separated Columns:

  

Protein Data Bank (PDB) Format:

• File Extensions: .pdb

• Purpose: Standard text format for encoding primary sequence data and three-dimensional ($3D$) atomic spatial coordinates of biomacromolecules.

• Used to store three-dimensional structural data of biological macromolecules such as:
      • Proteins
      • DNA
      • RNA
      • Protein-ligand complexes
      • Macromolecular assemblies

• PDB files are obtained from structural biology techniques including:
      • X-ray crystallography
      • NMR spectroscopy
      • Cryo-electron microscopy (Cryo-EM)

• Example

• Column Key

• Atom columns

Biological Databases: Foundations and Concepts

• Definition: A biological database is a organized digital archive designed to store, manage, and curate biological datasets so that information can be easily retrieved by researchers and automated systems.

• Core Architectural Terminology:

  • Entry: Each individual record or file stored within a database.

  • Value: An indexed parameter or attribute that a user submits to locate a specified set of data.

  • Query: The process of searching a database to extract targeted records.

Primary Sequence Databases and Infrastructure

GenBank:

• URL: https://www.ncbi.nlm.nih.gov/genbank/

• Organization: Maintained by the National Center for Biotechnology Information (NCBI) at the National Institutes of Health (NIH).

• Scope: Open-access, annotated collection of all publicly available DNA and RNA sequence records.

• International Collaboration: Part of the International Nucleotide Sequence Database Collaboration (INSDC) alongside the DNA DataBank of Japan (DDBJ) and the European Nucleotide Archive (ENA). These three entities exchange data daily.

• Release Schedule: A major release occurs every two months and is hosted on NCBI's FTP server.

• Access Mechanisms:

  • Entrez Nucleotide: System for sequence identifiers and annotation queries.

  • BLAST: Sequence alignment search tool against GenBank databases.

    • 
      

      

    • 
      

      

  • NCBI E-utilities: Programmatic API for automated sequence searching, linking, and downloading.

  • FTP Distribution: Downloadable in ASN.1 and flatfile formats via ftp://ftp.ncbi.nlm.nih.gov/ncbi-asn1 and ftp://ftp.ncbi.nlm.nih.gov/genbank.

• Data Usage Policy: NCBI places no restrictions on use or distribution of GenBank data, though individual submitters may claim intellectual property rights.

EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute):

• URL: https://www.ebi.ac.uk/services/data-resources-and-tools

• Scope: Maintains a comprehensive range of freely available, updated molecular data resources and computational biology tools.

Protein Knowledgebases and Structural Archives

UniProt (Universal Protein Resource):

• URL: https://www.uniprot.org/

• Scope: Premier resource for protein sequence and functional annotation.

• Core Components:

RCSB Protein Data Bank (RCSB PDB):

• URL: https://www.rcsb.org/

• Scope: Primary archive for three-dimensional structural data of biological macromolecules.

• Content:

  • Experimentally Determined Structures: $209,611$ structures determined by X-ray, NMR, or cryo-EM.

  • Computed Structure Models (CSMs): $1,068,577$ predicted models integrated from AlphaFold DB and ModelArchive.

• Educational Resources: Includes PDB-101 training resources and educational features such as the September Molecule of the Month (e.g., Histone Deacetylases).

Protein Family and Domain Databases

Structural Databases

Additional specialised bioinformatic Databases

Databases at a glance

Pathway and Functional Annotation Systems

KEGG (Kyoto Encyclopedia of Genes and Genomes):

• URL: https://www.genome.jp/kegg/

• Organization: Developed by Kanehisa Laboratories.

• Scope: Integrated database for understanding biological system functions from large-scale molecular datasets generated by high-throughput technologies.

• Primary Sub-Databases:

  • KEGG PATHWAY: Graphical wiring diagrams of cellular metabolic and signaling pathways.

  • KEGG BRITE: Functional hierarchies and classification tables for biological entities.

  • KEGG MODULE: Defined functional collections of genes or metabolic reaction steps.

  • KEGG ORTHOLOGY (KO): Functional ortholog groups linked to genes and proteins.

  • KEGG GENES / GENOME: Organismal and viral genomic sequence data (KEGG Virus).

  • KEGG COMPOUND / GLYCAN / REACTION / ENZYME: Small molecules, glycans, biochemical reactions, and enzyme nomenclature.

  • KEGG NETWORK / DISEASE / DRUG / MEDICUS: Human disease networks, approved drugs, drug target interactions, and clinical health information.

• Analysis Tools: Includes KEGG Mapper, KEGG Taxonomy, KEGG Synteny, BlastKOALA, GhostKOALA, KofamKOALA, BLAST/FASTA similarity searches, and SIMCOMP structure comparison.

Highly specifc database

Silva

• Comparative online resources fir quality check and alignment ribosomal RNA sequence data

Model Organism Databases

• TAIR (The Arabidopsis Information Resource):

  • URL: https://www.arabidopsis.org/

  • Scope: Dedicated genetic and molecular biology database for the model plant Arabidopsis thaliana.

  • Content: Complete genome sequence, gene structure, gene product function, gene expression data, DNA/seed stocks, genome maps, genetic markers, and weekly literature updates.

  • Technical Infrastructure: Phasing out legacy tools (SeqViewer, GBrowse) to focus on JBrowse/JBrowse2, with integrated support for PhyloGenes.

• FlyBase:

  • URL: https://flybase.org/

  • Scope: Master database for Drosophila species genes and genomes (Drosophila melanogaster, Drosophila virilis, and comparison species like Apis mellifera).

  • Features: Includes BLAST search, JBrowse genome viewer, Gene Ontology (GO) vocabularies, ID Validator, Batch Download, and GAL4 driver expression data.

  • Funding Model: Implements a user website access fee of $150.00\,\text{USD}$ per person per year alongside tax-deductible community contributions.

Literature and Bibliographic Databases

• PubMed:

  • URL: https://pubmed.ncbi.nlm.nih.gov/

  • Organization: Maintained by the NCBI / NLM.

  • Content: Contains over $36\,\text{million}$ citations for biomedical literature from MEDLINE, life science journals, and online books.

  • Features & Tools: Provides E-utilities API access, FTP downloads, Single/Batch Citation Matcher, Clinical Queries, and Medical Subject Headings (MeSH) indexing.

• Europe PMC:

  • URL: https://europepmc.org/

  • Scope: Partner repository of PubMed Central (PMC) and an ELIXIR core data resource.

  • Content: Indexes over $42,866,678$ life-sciences articles, preprints, and related bibliographic entries.