1/34
Software Testing (Definition)
The process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation, and evaluation of software products to determine that they satisfy requirements, demonstrate fitness for purpose, and detect defects.
Core Purpose of Testing
To gain confidence that the software will work as intended in its environment, raise quality and reliability, and detect defects.
Limitation of Testing
Testing can only show that defects/failures are present; it cannot prove that a program is completely free from errors.
Development Cost of Testing
Testing activities often contribute to 30% to 50% of total software development costs.
Good Test
A test that has a high probability of finding errors.
Why Testing is Necessary
1) Gain confidence with acceptable risk, 2) Prevent and detect errors early, 3) Discover deficiencies and capabilities, 4) Assess software quality.
Principle 1: Testing shows presence of failures
Testing can demonstrate that failures/defects exist, but cannot prove that software is error-free.
Principle 2: Exhaustive testing is impossible
Testing all combinations of inputs and preconditions is not feasible, except for trivial (non-complex) programs.
Principle 3: Early testing
Testing activities should start as early as possible in the software development life cycle, rather than as a phase at the very end.
Principle 4: Defect clustering
A small number of modules usually contain most of the defects; testing effort should focus proportionally on expected defect density.
Principle 5: Repetitions have no effects
Repeating the exact same tests brings no new insights; test cases must be regularly reviewed, revised, and modified.
Principle 6: Testing is context-dependent
Testing is done differently in different contexts (e.g., safety-critical software vs. mobile entertainment apps).
Principle 7: Absence-of-errors fallacy
Fixing defects and finding no failures does not guarantee that the system will satisfy user expectations or business needs.
Fundamental Test Process Steps
1) Planning and control, 2) Analysis and design, 3) Implementation and execution, 4) Evaluating exit criteria and reporting, 5) Test closure activities.
Test Policy
A high-level document describing the rules, principles, approach, and major objectives of the organization regarding testing.
Test Strategy
The overall high-level approach describing the test levels to be performed and the testing within those levels for an organization or program.
Test Planning Tasks
Determine scope, risks, objectives; determine approach, test resources, and schedule; determine exit/coverage criteria.
Test Control
An ongoing management activity that involves developing and applying corrective actions to get a project back on track when monitoring detects deviation from plan.
Test Analysis and Design
The activity where general testing objectives are transformed into tangible test conditions and test designs/scripts.
Test Implementation Tasks
Transform test conditions into test cases and procedures/scripts; create test data; build test suites; set up and verify the test environment.
Test Execution Tasks
Execute test suites/procedures, log outcomes and versions of testware, compare actual vs. expected results, report discrepancies as incidents, and re-test fixes.
Exit Criteria
Defined objectives against which test execution is assessed to decide whether testing is complete (e.g., coverage percentage or defect thresholds).
Test Closure Activities
Collecting data from completed test activities to consolidate experience, archive/handover testware, resolve incident reports, and document lessons learned.
When are Test Closure Activities done?
When software is delivered, when a project is cancelled, when a specific milestone is reached, or when a maintenance release/update is completed.
The V-Model
A software development model that illustrates how testing activities correspond directly to each development activity.
V-Model: Key Advantage of Early Test Design
Designing test cases early detects defects in specifications and requirements before the software is actually built, when fixes are much cheaper.
Component (Unit) Testing
Checks whether individual software components (module, unit, class) satisfy the requirements of their component specification.
Integration Testing
Checks whether groups of interacting components operate together as intended in the technical system design.
System Testing
Checks whether the completely integrated system as a whole meets all specified functional and non-functional requirements.
Acceptance Testing
Checks whether the system satisfies customer/user requirements and shows contractually approved performance from a customer perspective.
Requirement Definition (V-Model)
Development stage where customer/user requirements are collected and specified; pairs with Acceptance Testing.
Functional System Design (V-Model)
Development stage representing requirements as functions and user dialog sequences; pairs with System Testing.
Technical System Design (V-Model)
Development stage designing technical interfaces, architecture, and subsystem decomposition; pairs with Integration Testing.
Component Specification (V-Model)
Development stage defining tasks, behavior, internal structure, and interfaces of each subsystem; pairs with Component Testing.