JMH (Java Microbenchmark Harness)

What is JMH?

Java Microbenchmark Harness, a toolkit for building, running, and analyzing Java benchmarks.

Who developed JMH?

The same team at Oracle responsible for the Java Virtual Machine (JVM).

What is the main purpose of JMH?

To create accurate and reliable benchmarks for Java code.

Why not use System.nanoTime() directly for benchmarking?

It is prone to noise, JIT optimizations, and can yield misleading results without warmup and control.

What does JMH help avoid?

Common benchmarking pitfalls such as dead-code elimination, loop unrolling, and JVM warmup effects.

How do you include JMH in a Maven project?

Use the jmh-generator-annprocess and jmh-core dependencies.

What is the annotation used to mark a benchmark method?

@Benchmark

What does the @Benchmark annotation indicate?

That the annotated method should be measured by the JMH harness.

What is @State in JMH?

Marks a class that holds benchmark state; it determines the lifecycle of benchmark instances.

What are the common @State scopes in JMH?

Thread, Group, Benchmark.

What does @State(Scope.Thread) mean?

Each thread gets its own instance of the benchmark class.

What does @BenchmarkMode do?

Defines how the benchmark method’s performance is measured.

What are the common BenchmarkMode options?

Throughput, AverageTime, SampleTime, SingleShotTime, AllModes.

What is BenchmarkMode.Throughput?

Measures how many operations are performed per time unit.

What is BenchmarkMode.AverageTime?

Measures the average time taken per operation.

What is BenchmarkMode.SampleTime?

Measures time for individual samples and provides distribution.

What is BenchmarkMode.SingleShotTime?

Measures time for a single invocation, used for cold-start benchmarking.

What does @OutputTimeUnit do?

Specifies the time unit for the benchmark output (e.g., seconds, milliseconds).

What is the purpose of @Fork?

Specifies how many times the benchmark should be forked in a new JVM instance.

What is the default fork count in JMH?

1

Why does JMH fork JVMs?

To ensure a clean environment and isolate benchmarks from JVM optimizations.

What does @Warmup do in JMH?

Defines how many iterations should run before measurement begins to allow JVM warmup.

Why is warmup necessary in benchmarking?

To allow the JVM to optimize the code and reach steady-state performance.

What does @Measurement do in JMH?

Specifies the number of measurement iterations and duration of each.

What is @Param in JMH?

Used to define parameters that can be injected into benchmark methods to test with multiple values.

What is the purpose of blackhole in JMH?

To consume values in a way that prevents the compiler from optimizing them away.

What class is used for blackhole consumption?

org.openjdk.jmh.infra.Blackhole

How do you run a JMH benchmark from command line?

Use the generated JAR with java -jar benchmarks.jar

What does the JMH plugin do in Maven?

Generates the benchmark source files and executable JARs.

How do you benchmark multiple methods?

Create multiple methods annotated with @Benchmark in the same class.

Can JMH benchmark private methods?

No, benchmark methods must be public.

Can JMH benchmark static methods?

Yes, as long as they are annotated with @Benchmark and public.

What is a common pitfall in benchmarking Java code?

Letting the JIT compiler eliminate or optimize away the code under test.

What is the recommended iteration time for warmup and measurement in JMH?

At least 1 second per iteration is recommended.

How do you configure JMH benchmark parameters programmatically?

Use OptionsBuilder and Runner classes.

Can JMH benchmark multithreaded code?

Yes, using @Threads or @Group annotations.

What does @Threads do?

Defines the number of threads to run a benchmark method concurrently.

What is @Group used for in JMH?

To define group benchmarks for measuring interaction of multiple methods.

What is false sharing, and how can JMH detect it?

False sharing occurs when threads write to variables on the same cache line; JMH can help detect it via @Contended (Java 8+) or careful structuring.

What is the use of -prof perfasm option in JMH?

To use the perf profiler and disassemble JIT-compiled code for analysis.

What are common profilers available in JMH?

gc, stack, perfasm, async, jfr, dtrace.

Can JMH benchmarks be used in production?

No, they are intended for microbenchmarking in test environments.

How does JMH prevent dead-code elimination?

By consuming outputs using Blackhole or returning results from benchmark methods.

What does the @Setup annotation do?

Initializes state before each iteration, trial, or invocation.

What does @TearDown do in JMH?

Cleans up resources after each iteration, trial, or invocation.

What levels are available for @Setup and @TearDown?

Level.Invocation, Level.Iteration, Level.Trial

What does Level.Invocation mean?

The method runs before/after every single benchmark invocation.

What does Level.Iteration mean?

The method runs before/after each benchmark iteration.

What does Level.Trial mean?

The method runs once per forked JVM instance.

Can JMH benchmark allocations?

Yes, using the -prof gc or -prof perfasm profilers or analyzing GC metrics.

How to benchmark with different input sizes in JMH?

Use @Param to vary input values across iterations.

How to include multiple benchmark classes in one run?

Use includes in OptionsBuilder or wildcards in CLI.

Can JMH benchmark across different JVMs or vendors?

Yes, it's common to test performance across OpenJDK, Oracle JDK, GraalVM, etc.

Is JMH suitable for benchmarking large-scale applications?

No, it's best for fine-grained microbenchmarking.

What is the format of JMH output?

Typically includes score, error margin, unit, and optional throughput or time per op.

Can JMH benchmarks be used for regression tracking?

Yes, to detect performance regressions across code or JVM versions.