INF 115 Midterm Study Guide

What is the relationship between “testing” and “quality assurance”?

Testing is a subset of quality assurance, and quality assurance is the idea of reviewing code and choosing particular languages/libraries to ensure quality.

Waterfall Model

The foundation for many other software development process models. Incorporates testing at the very end of the development lifecycle.

V Model

Built upon Waterfall model, but instead incorporates testing into every stage of the development lifecycle.

Spiral Model

Development follows a series of waterfall models implementations, starts with the most risky thing to implement and ends with the least risky thing to implement. Helps save time by getting rid of the riskiest things first.

Verification

Are we building the thing correctly?

Can be thought of as a “checklist”.

Checking with users to make sure that the thing being built meets their needs.

Validation

Are we building the right thing?

Can be thought of affirmations, and affirming someone’s feelings.

Mistake

An action that a human does without intention that produces an incorrect result

I.e. Someone’s finger slips while typing, or someone makes a false assumption.

Fault

Incorrect code; anomaly in the source code that may or may not lead to an error.

Error

Internal manifestation of executing a fault, may cause deviation from the correct behavior and result in a failure. 

Failure

Manifestation of an error

Observable discrepancy between the expected functionality of the program and its actual functionality.

Test Case

Set of inputs that cause a program to take some defined action and have expected outputs

Test Suite

Software that facilitates faster and easier execution of test cases

A set of test cases

Test Oracle

Strategy to verify the correct answer

Something used to decide what outputs are expected from a particular test case

Test Plan

A document describing the scope, approach, resources, and schedule of intended testing activity

Black Box vs. White Box Testing

Black Box tests the specifications and functionality of the software to ensure that it meets the requirements; does not look at the code itself.

White Box testing involves looking at the code itself.

Why not exhaustively test, or fully prove for correctness?

Expensive and time consuming; there are theoretically an infinite number of inputs. We would need an oracle for every test case.  

What is the basic power of testing to find bugs? Given the answer to this question, why do we then test at all?

“Testing can only prove the presence of bugs, not the absence of them.” 

We catch bugs and test software because some detection is better than one; it’s the best that we have.

Partitioning

Divide and conquer

Turn a large problem into smaller problems that are easier to solve

Restriction

Making choices that make the problem easier

Programming in a way that make it impossible for certain bugs to occur

Redundancy

Making intentions explicit in multiple ways to allow for consistency checking

Sensitivity

Better to fail every time than only sometimes

Visibility

Making information accessible and observable

A person can see what is happening in the code at any given time

Feedback

Learning from past mistakes 

Tuning the development process

What is exploratory testing? What is its strengths? What is its limitations? Who would want to do it, and when, and how?

Unscripted, unautomated, unrepeatable testing; Guided by the tester’s curiosity and a test charter.

Allows for real-time testing that is guided by the tester; has a greater chance of discovering bugs due to the experimental and unexpected nature of testing. Allows the tester to observe and evaluate any unspecified behaviors and encourages note-taking for future testing.

Encourages note-taking for future test cases

Test Charter

The mission statement of testing, guidelines and “a plan” to follow when testing, where the focus should be placed.

Acceptance Testing

Assessment of whether agreed-upon requirements are met, Often employ checking of the business contract between client and contractor

Types: Smoke testing, User acceptance testing, End-user testing, Alpha and beta testing

Smoke Testing

Very minimal form of testing, usually only ran to ensure that the program can run.

Test Driven Development (TDD)

Write the test case, auto-fail it, make code to pass the case, and test again.

Behavior Driven Development (BDD)

Evolution of TDD, Provides guidance on what test cases to write, how many, how to name them, how to structure the test cases

Uses a format of “given, when, and then” → Given a scenario, when an event happens, then do a certain thing.

Random Testing vs. Symptomatic Testing

Random Testing is unlikely to produce/discover boundary errors. Symptomatic testing allows for explicitly testing edge cases.

Why Functional Testing?

We want to find bugs, and we want to derive test cases from program specifications (i.e. specification-based testing or black-box testing)

Equivalence Partitioning

Dividing inputs into partitions and/or categories

Choose a representative value for each partition

Combinatorial Testing

Combining partitions for multiple categories as an input set.

I.e. 2-way (pairwise), 3-way, 4-way, etc.

Purpose of Pairwise Testing & N-way Testing?

Ensures that each test input has been tried with every other possible input in some combination

Higher levels of testing produce stronger testing overall, but there is an exponential increase with the number of tests needed with a diminishing return of bugs found.

State Transition Table

Rows: States

Columns: Events/Transitions

Intersection of the row/columns: Resulting State

Purpose of having the a FSM?

Another way of capturing the specifications

Informs/guides your testing

Tells you when you can stop

JUnit Structure

@Test: Denotes a test method

@BeforeEach: Runs a method to run once before EACH test case

@BeforeAll: Runs a method to run once IN TOTAL before ALL test cases

assert(input): Checks to ensure the input resolves to be true

assertEquals(input1, input2): Checks to ensure that the input is equal to a test oracle.