RELIABILITY, REPLICABILITY, and VALIDITY

Reliability

The extent to which an experiment, test, or measuring procedure yields the same results on repeated trials.

Inter-rater

Degree of agreement among observers who rate or assess the same phenomenon.

Inter-observer

The consistency between two researchers watching the same event; e.g., whether they will produce the same records.

Test-retest reliability

Measures the consistency of results when you repeat the same test on the sample at a different point in time.

Replicability

The ability of a scientific experiment or trial to be repeated to obtain a consistent result.

Validity

The quality of being logically or factually sound.

Ecological validity

The realism with which a design of evaluation setup matches the user's real work context.

Subjectivity/Objectivity

Objective means verifiable information based on facts and evidence; subjective means information or perspectives based on feelings, opinions, or emotions.

Demand characteristics

An experimental artifact where participants form an interpretation of the experiment's purpose and semi-consciously change their behavior to fit the interpretation.

Generalisability

The extent to which the findings of a study can be applicable to other settings.