Understanding Software Performance Testing Techniques

Key Performance Indicators (KPIs)

Metrics that help assess the performance of a system.

Availability

The amount of time an application is available to the end user.

Throughput

Number of application-oriented events the software can process (within a given time).

Capacity Utilization

The percentage of the capacity of a resource that is being used.

Baseline Testing

Establish a point of comparison for further tests by executing a single transaction as a single user.

Load Testing

Understand the behavior of the system under a normal/expected load.

Efficiency-oriented indicators

Metrics that focus on the effective use of resources in the application.

Service-oriented indicators

Metrics that focus on the service availability and performance from the user's perspective.

Undue perceived delay

A delay experienced by the user that is considered excessive or frustrating.

Seamless manipulation

User interaction with the application that feels instantaneous, typically <= 0.1s.

Free navigation

User experience where interactions feel fluid and responsive, typically <= 1s.

User switch/interrupt operation

The point at which users may abandon an operation due to delays, typically <= 10s.

Application-oriented events

Specific actions or requests that the software processes, such as user requests.

Network bandwidth consumption

The amount of data transmitted over a network by application traffic.

Memory usage

The amount of memory consumed when a specific number of visitors are active.

Normal conditions

The standard operating environment under which baseline tests are conducted.

Maximum load

The highest level of demand the application can handle before failing or behaving unexpectedly.

Database capacity

The amount of data the database can handle before experiencing delays or crashes.

Network-related issues

Problems that affect the performance of the application due to network conditions.

Soak Testing

Identify performance problems that appear over extended periods of time by supplying expected load to the application continuously.

Stress Testing

See how the application performs under unfavorable conditions and ensure it fails and recovers gracefully by overwhelming system resources.

Spike Testing

Make sure the system can handle bursts of higher user or system activity, focusing on high load for a short duration.

Performance Testing Process

1. Decide on the testing environment 2. Identify performance metrics 3. Plan and design performance test 4. Configure the test environment 5. Implement the tests 6. Execute tests 7. Analyze, report, retest.

Testing Environment

What kind of environment can you afford? Options include a subset of production system with lower-spec servers, replica of production, or actual production.

Latency

How long it takes to receive the first byte after a request is sent.

Error Rate

Percentage of requests resulting in errors.

Concurrent Users

How many active users at any point in time, which is the most common measure of load.

CPU Utilization

The percentage of CPU resources being used.

Memory Utilization

The percentage of memory resources being used.

Performance Test Checkpoints

Specific points of interest during a test, such as logging into a website, clicking checkout, processing payment information, and showing final confirmation.

Test Environment Configuration

Ideally, automate the creation and tear down of the testing environment to allow repeatability using tools like Chef, Ansible, Spinnaker, or Heroku.

Response Time

Measured in seconds, indicates how long it takes for a system to respond.

Elapsed Time

Measured in seconds, refers to the total time taken for a process to complete.

Dynamic Analysis

Involves instrumentation/profiling to analyze performance.

Profiling

Uses tools to instrument code and record performance information at runtime.

Continuous Integration

Profiling can be integrated into this process, though it may slow down test execution.

Performance Code Review

Involves looking for inefficient coding patterns and resource management.

Inefficient Coding Patterns

Examples include repeated calls to a database inside a loop.

Parallelism Potential vs. Overhead

Overhead of splitting execution may outweigh performance gains.

Method Concatenation

Inefficient pattern of concatenating strings using '+' in a loop.

Performance Testing Tools

Various tools are available to assist in performance testing.