GIT NOTES.txt

Testing

Process of executing a program with the intent of finding errors.

Software testing

A process where the software is evaluated to determine that it meets the user needs and that the process to build the software was followed correctly.

Software Testing Cycle

Validation - Software is evaluated to determine that it meets the user needs.

Verification - process to build the software was followed correctly.

Main purpose of a tester

Find bugs, replicate them, report them, verify that the bugs have been fixed and make sure the game is fun.

Why software testing is important?

Testing is critical to make sure that the software has appropriate quality and works correctly.

ISTQB

International Software Testing Qualifications Board.

Principle 1 - Testing shows presence of defects

Testing reduces the probability of undiscovered defects remaining in the software.

Principle 2 - Exhaustive Testing is impossible

Testing everything is not feasible except for trivial cases. risk analysis and priorities should be used to focus testing efforts.

Principle 3 - Early Testing

Testing activities should be started as early as possible in the software or system development life cycle (SDLC).

Principle 4 - Defect clustering

Testing effort shall be focused proportionally to the expected and later observed defect density of modules.

Principle 5 - Pesticide Paradox

test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to find potentially more defects.

Principle 6 - Testing is Context Dependent

Testing is done differently in different contexts.

Principle 7- Absence of errors fallacy

Finding and fixing defects does not help if the system built is unstable and does not fulfil the user’s needs and expectations.

Testing Rules

A programmer should avoid attempting to test his or her own program.

A programming organization should not test its own programs.

Test cases must be written for input conditions that are invalid and unexpected, as well as for those that are valid and expected.

Avoid throwaway test cases unless the program is truly a throwaway program.

Testing is an extremely creative and intellectually challenging task.

Unit Testing

It is the execution of a complete class, routine or small program that has been written by a single programmer or team of programmers, which is tested in isolation from the more complete system.

Tested individually and independent of each other.

To test the smallest units or software components in a program.

Advantages of Unit Testing

Helps you isolate bugs faster and fix them quicker.

Prevents future updates from adding new bugs to old working code.

Provides confidence that a method behaves as expected.

Disadvantages of Unit Testing

Bad or inaccurate tests create false confidence.

Requires more knowledge to implement correctly.

UI is hard to test.

Unit tests don’t catch integration errors.

Test Suite

A test suite houses all unit tests related to a logical grouping of functionality.

Integration Testing

Validates how modules of code work together.

It is the combined execution of two or more classes, packages, components, or subsystems.

Ways to do integration testing

Top-down Integration - Top level components created first.

Bottom-Up Integration - Low level components created first.

Ad-Hoc Integration - Integrated in the order in which they are finished.

Backbone Integration - Skeleton is built and components gradually integrate into it.

System Testing

Execution of the software in its final configuration.

Checks if integrated product meets specified requirements.

Quality of the data used should be consistent, complete and up to date.

Acceptance Testing

Created during design phase and actual testing is done after system is fully developed and tested and ensure the software meets the requirements.

Basically Beta Testing.

Non-Functional Categories of Testing

Stress Test

Load Test

Performance Test

Volume Test

Security Test

Stability Test

Robustness Test

Compatibility Test

Configuration

Usability Test

Documentation Checks

Maintainability Checks

Code Inspection(Static Testing) Benefits

4 Moderators.

Programmer gets feedback.

Other participants gain by being exposed to programmer’s errors and programming style.

Identifies most error-prone sections of program.

4 Types of Testing in order

Unit > Integration > System > Acceptance

Walkthrough(Static Testing)

Participants simulate how a computer would execute the code.

Done by 3-5 People(Moderator, Secretary, Tester, Programmer).

Desk Checking(Static Testing)

Reads programs, checks with an error list and walks test data through it.

Done by 1 Person.

Peer Ratings(Static Testing)

Evaluate anonymous programs in terms of overall quality, maintainability, extensibility, usability and clarity.

Provides programmer self-evaluation.

Black Box Testing

Test the system against external factors responsible for software failures.

Usability Issues, Performance, abrupt application failure and response time of the system.

White Box Testing

Exercise the source code directly.

Testing code modules that will become part of a reusable library.

Differences between Black Box Testing and White Box Testing

White box testing provides the tester with better prospects to exercise the source code directly and in ways that no player ever could but black box testing doesn’t.

Regression Testing

software which was previously developed and tested still performs the same way after it was changed or interfaced with hardware.

Load Testing

Test heavy activity and whether the application is able to function properly under load.

Game Debugging

Track reported software bugs in game development projects.

Bug tracking systems are integrated with other project management software.

Low Priority Bug

Hardly matter and makes no difference whether they are fixed or not.

Medium Priority Bug

Should be fixed as it annoys the player but not affect gameplay.

High Priority Bug

Must be fixed as they seriously affect gameplay.

A game that ships with high bugs is a bad game or has been rushed to the market.

Critical Bug

Very special bugs that demands immediate attention which can cause crashes, freezes and data corruption.

Must be fixed at all costs and game is sent back to developer if spotted by publisher.

Bug Categories

Visual

Audio

Level design - Badly constructed game level

Artificial intelligence - pathfinding and NPC behaviour

Physics

Stability - Predictability of the code

Performance - Speed with which the hardware processes the code

Networking - Server-client connectivity and bandwidth issues

Compatibility - Relate to how a game runs on certain hardware

Bug reporting system status

Open

Assigned

Resolved

Verified

Closed

Quality Assurance Team Roles

Producer - Oversees the game after months of development

QA Manager - Assesses the situation and considers how to plan

Lead Tester

Floor Lead - Unofficial leader tester

Version Control System

Records changes to a file or set of files over time.

Allows you to revert selected files or the entire project back to a previous state.

Benefits of Version Control System

Backup and Restore

Sync with multiple computers

Working in a team

Safely create and test new features

Ownership / Credits / Blame

VCS Terminologies

Repository - Database where files are stored

Server - The computer storing the repository

Client - The computer connecting to the repository

Working copy -Your local directory of files

Trunk/Main - Master location for code in the repository

Head - The latest revision in the repository

Add - Place a file under version control

Check in - Send local changes to repo

Check out - Download from a repo to your working copy

Ignore - Allows files to exist in your working copy but not in the repo

Revert - Throw away your working copy and restore last revision

Update/Sync - Update your working copy to the latest revision

Diff / Change - Specific modification to a document

Branch - Duplicate copy of code used for feature development

Merge - Integrate changes from two different branches

Conflict - Inability to reconcile changes to a document

Resolve - Manual fixing of conflicted document changes

Locking - Prevents other developers from making changes

Local Version Control System

A simple database that keeps all changes to files

Centralised Version Control System

Need to collaborate with developers.

A single server that contains all the versioned files.

Everyone can check what everyone else on the project is doing to a certain.

Administrators have fine-grained control.

Lose everything if the central database on the hard disk becomes corrupted.

Distributed Version Control System

Fully mirror the repository including its full history.

If any server dies, any of the client repositories can be copied back up to the server to restore it.

Every clone is a full backup.

Deal well with having several remote repositories.

Allows several types of workflow setups.

GITHUB

Modified - Changed the file but have not committed to database yet

Staged - Marked a modified file in its current version into your next commit snapshot

Committed - Data is safely stored in your database

Unity Version Control System Perforce VCS vs Plastic SCM

Perforce Supports remote activity but PlasticSCM does not

Test Plan

A document describing the software testing scope, approach, resources and schedule of intended test activities.

Software Requirement Specification (SRS)

Test Case

A set of conditions or variables under which a tester will determine whether a system under test satisfies requirements or works correctly.

Good Test Case

Detects an as yet undiscovered error

Shows that a program does not do what it was designed to do

Balance Testing

Ensures that gameplay is fair to both human and AI player alike.

Multiplayer games need to be balanced or all matches will be skewed.

Compatibility Testing

Game is tested with different hardware configurations

Game is full compatible with parts and peripherals found on the market

Need to be proficient in PC assembly and troubleshooting

Soaking

A sub-discipline of testing, leaving the console or computer running for extended periods of time

It is necessary when memory leaks or rounding errors might hurt the game stability in the long run

Compliance Testing

Game has to go through certification before being sold on the market and is conducted by hardware developers

PEGI (Pan European Game Information)

ESRB (Entertainment Software Rating Board)

Localisation Testing

Involves converting a game from one region to another

Translation mistakes are a bad sign for a foreign game

Play Testing

Involves going over all possible varieties of play and making sure all of them are fun

Watch for gameplay mechanics

Playability Test

To provide feedback to allow us to improve the product for the players who will play this game.

Includes the usability test.

Usability Test

Process of observing user's reactions to a product and adjusting the design to suit their needs

Gameplay mechanisms need to make sense and be easy to use

Purpose of Playability Testing

Identify issues with the layout and errors

Identify broken links

Evaluate the content

Check user reaction

Evaluators

Age

Experience

Gender

Education

Prior Training

Technical Expertise

Mock-up/Prototype

Checklist - Items to test based on requirements and standard ease of use criteria

Survey - Questionnaires with a rating for different aspects of interaction and usability

Scenario - Testing tasks to test the typical interaction of user with the game

Evaluating player enjoyment in games

Concentration

Challenge Player Skills

Control

Clear goals

Feedback

Immersion

Social Interaction

Heuristics to evaluate the playability of games

Gameplay

Game Story

Game Mechanics

Game Usability

Types of Playability Test

Functional Playability - Ability to explicitly attempt to do something in the game

Structural Playability - Measure of if the gameplay is sufficiently demanding to engage

Sensory Playability - Information presentation and availability

Social Playability - Measure of how well the gameplay facilitates a certain type of social interaction practice

Goal or Task(Levels of Complexity)

Give as much information as needed for the tester and provide task descriptions without steps

Detailed Scenario(Levels of Complexity)

Describes the user characteristics that the user interface should be designed for

Full Scale Scenario

Has exact steps to acommplish the task that needs to be followed and tested

Alpha testing

Find defects before the code is released to key customers during the beta test or general population

Ensure that majority of the most common tasks and scenarios work well

Beta Testing

Closed Beta - Selected group of customers, company works closely with them during the test.

Open Beta - Software is available on request.