RELIABILITY, REPLICABILITY, and VALIDITY

Reliability: The extent to which an experiment, test, or measuring procedure yields the same results on repeated results

• inter-rater: degree of agreement among observers who rate or assess the same phenomenon

• inter-observer: The consistency between two researchers watching the same event; e.g whether they will produce the same records

• test-retest reliability: measures the consistency of results when you repeat the same test on the sample at a different point in time

Replicability: The ability of a scientific experiment or trial to be repeated to obtain a consistent result.

Validity: the quality of being logically or factually sound

• ecological validity: The realism with which a design of evaluation setup matches the user's real work context

• subjectivity/objectivity: Objective means verifiable information based on facts and evidence. Subjective means information or perspectives based on feelings, opinions, or emotions 

• demand characteristics: refers to an experimental artifact where participants form an interpretation of the experiment;s purpose and semi-consciously change their behavior to fit the interpretation

• generalisability: extent to which the findings of a study can be applicable to other settings