Comprehensive Overview of Software Testing Fundamentals

Definition and Components of Software

Conceptual Definition of Software (ISO Standard): According to ISO definitions, software is composed of four fundamental components that are essential for its operation:
- Computer Programs (Code): The actual executable instructions or source code.
- Procedures: The operational steps or methods required to use the software.
- Documentation: All written materials, manuals, or guides associated with the software.
- Necessary Data: The data required for the software to operate effectively.

Software Quality Concepts

General Definitions of Software Quality:
- According to IEEE: Software quality is defined as the degree to which a system, system component, or process meets specified requirements.
- Customer-Centric Definition (IEEE): Software quality is the degree to which a system, system component, or process meets the needs and expectations of the customer or user.
The ISO 9126 Software Quality Model: Quality is categorized into six main characteristics, each containing specific sub-attributes:
- Functionality:
  - Suitability
  - Accuracy
  - Interoperability
  - Security
  - Compliance
- Reliability:
  - Maturity
  - Fault tolerance
  - Recoverability
  - Compliance
- Usability:
  - Understandability
  - Learnability
  - Operability
  - Attractiveness
  - Compliance
- Efficiency:
  - Time behavior
  - Resource utilization
  - Compliance
- Maintainability:
  - Analyzability
  - Changeability
  - Stability
  - Testability
  - Compliance
- Portability:
  - Adaptability
  - Installability
  - Co-existence
  - Replaceability
  - Compliance

Software Errors: Causes, Impacts, and Costs

Defining Software Error: Simple definition suggests a software error is a mismatch between a program and its specification.
Common Causes of Software Errors:
- Incorrect definition of customer requirements.
- Lack of accurate information.
- Misunderstandings between stakeholders or team members.
- Human factors: Carelessness, fatigue, and time pressure.
- Lack of professional experience.
- Environmental conditions.
- Errors in how the software is used.
Harms Caused by Software Errors:
- Wasted time and financial resources.
- Negative impact on company reputation.
- Potential for physical injury or death.
The Cost of Fixing Software Defects:
- The cost of finding and fixing defects increases significantly over time through the development lifecycle stages:
  1. Requirements
  2. Design
  3. Build
  4. Test
  5. Live use
- Determining and correcting a bug during the "Live use" phase is far more expensive than correcting it during the "Requirements" phase.
Historical Cases of Software Failure:
- Disney’s Lion King ( $1994$ - $1995$ ): Known software compatibility issues affecting gaming experience.
- NASA - Mars Polar Lander Failure ( $1999$ ): Resulted in the loss of the spacecraft.
- The Y2K (Year 2000) Bug: Concerns regarding dates and century changes in legacy systems.
- The Patriot Missile Failure ( $1991$ ): Resulted in fatalities due to a software tracking error.

Defining Software Testing

General Definition: Software testing is the activity of evaluating software and its components to ensure the software meets expected requirements.
ISTQB Definition: According to the International Software Testing Qualifications Board, software testing is a process comprising all activities, including planning, preparation, system evaluation, and related tasks to determine if the system satisfies requirements.
Authoritative Historical Definitions:
- Glenford Myers ( $1979$ ): In "The Art of Software Testing," he defined testing as "the process of executing a program or system with the intent of finding errors."
- Bill Hetzel ( $1983$ ): In "Complete Guide to Software Testing," he defined testing as "any activity aimed at evaluating an attribute of a program or systems" and stated that "Testing is the measurement of software quality."
Standards-Based Definitions:
- IEEE 610 (Software Engineering Terminology): "The process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component."
- IEEE 829 (Test Documentation): "The process of analysing a software item to detect the difference between existing and required conditions (i.e. Bugs) and to evaluate the features of the software items."
- BS 7925-1 (Software Testing Vocabulary): "Process of exercising software to verify that it satisfies requirements and to detect errors."

Objectives and Benefits of Software Testing

Core Objectives:
- Find defects and failures in the product to reduce software quality risks.
- Verify that all requirements have been met.
- Check for completeness and validate if the product operates as the end-user expects.
- Build confidence in the quality level of the product.
- Provide stakeholders with sufficient information to make decisions, particularly regarding quality.
- Ensure compliance with contractual, legal, or regulatory requirements and standards.
Key Benefits:
- Reduction of product risk and increase in software quality.
- Increased customer satisfaction.
- Enhanced security.
- Savings in both time and money.

Testing in Software Development Life Cycles (SDLC)

SDLC Overview: A set of all activities and phases required to develop a system or software product. Various models are applied during the process.
Waterfall Model:
- Software is built through a clearly sequenced chain of activities.
- Suitable for projects with few requirement changes.
V-Model:
- Similar to Waterfall, but every development phase has a corresponding testing activity.
- Saves costs on fixing errors and has a higher success rate than Waterfall.
- Lacks flexibility; project risk remains relatively high.
Iterative - Incremental Development Model:
- Software development is divided into small cycles (iterations).
- Each cycle includes analysis, design, build, and test steps.
- Suitable for projects with complex requirements or systems that change/add requirements frequently.
- Typical example: Agile methods like Scrum.
- Workflow Stages: Initial Planning $\rightarrow$ Planning $\rightarrow$ Requirements Analysis & Design $\rightarrow$ Implementation $\rightarrow$ Testing $\rightarrow$ Evaluation/Deployment.

The Seven Principles of Software Testing

Testing shows the presence of defects: Testing can prove bugs exist but cannot prove that no bugs remain.
Exhaustive testing is not possible: Testing every single combination of inputs and preconditions is impossible except for trivial cases.
Early testing: Testing activities should start as early as possible in the SDLC to find defects early.
Defect clustering: A small number of modules usually contains most of the defects discovered during pre-release testing.
Pesticide paradox: If the same tests are repeated over and over, they will eventually no longer find new defects.
Testing is context-dependent: Testing is performed differently in different contexts (e.g., safety-critical software vs. an e-commerce site).
Absence of errors fallacy: Finding and fixing defects does not help if the system is unusable and does not fulfill the users' needs and expectations.

Quality Assurance (QA) vs. Quality Control (QC)

Quality Assurance (QA):
- Focused on the process to ensure the development process is appropriate and the system will meet objectives.
- Responsibilities:
  - Defining standards and testing procedures for quality.
  - Monitoring compliance with defined processes and standards.
  - Adjusting and improving processes for specific products.
  - Note: QA does not directly test the software functionality.
Quality Control (QC):
- Focused on the product to ensure it meets requirement specifications.
- Responsibilities:
  - Directly checking product quality before delivery to the customer.
  - Following the procedures defined by the QA department.

Verification and Validation

Verification:
- Evaluating work products of a specific development phase to see if they meet the requirements set at the start of that phase.
- Key Question: "Are we building the product right?"
Validation:
- Evaluating a product to see if it meets business requirements and customer expectations.
- Key Question: "Are we building the right product?"

Software Defect Terminology

Error / Mistake: A human action that produces an incorrect result.
- Defect / Bug: A flaw in a component or system that can cause the component or system to fail to perform its required function (found in code or documentation).
- Failure: An event in which a component or system does not perform a required function within specified limits (the difference between actual and expected results during execution).

Core Testing Concepts and Classifications

Static Testing: Testing software without actually executing the program or application code (e.g., reviews, inspections).
Dynamic Testing: Testing software by executing the application in a run-time environment.
Black Box Testing:
- Testing without knowledge of the internal source code or system architecture.
- Focuses solely on inputs and outputs against expected results to evaluate quality.
White Box Testing:
- Testing based on the internal logic and code of the system.
- Requires the tester to have programming knowledge to examine code paths and internal structures.
Gray Box Testing: A combination of both Black Box and White Box testing techniques.
Manual Testing: Testing performed by hand, from designing test cases to executing them manually.
Automation Testing:
- Using software tools to execute test cases.
- Requires the development of scripts and specific source code to facilitate the automated process.