Nested Level Model for Validation

Learn about using the nested level model for validation.

how do you know your visualisation works 1

does it amplify cognition?

how do you know your visualisation works 2

does it create insight?

how do you know your visualisation works 3

how can we measure amplified cognition or insight?

how can we argue that one design is better than another 1

what does better mean? faster? more effective? more engaging?

how can we argue that one design is better than another 2

what does effective mean? how do we measure engagement? what sort of task does one tool do better than the other?

domain situation threat to validity

you misunderstood their needs

data/task abstraction threat to validity

you’re showing them the wrong thing

visual encoding/ interaction idiom threat to validity

the way you show the data does not work

algorithm threat to validity

your algorithm is too slow

immediate validation

done while or just after work on a certain level has finished

downstream validation

can be done after work on subsequent levels is finished

immediate validation approach for domain situation 1

observe or interview the target audience and verify characterisation of practices

immediate validation approach for domain situation 2

field study in the real world setting, contextual inquiry

downstream validation approach for domain situation

measure adoption rates and observe the target audience

downstream validation approach for domain situation question

what do users do with the visualisation on their own account, without being prompted?

immediate validation approach for visual encoding/ interaction approach

make sure no known guidelines are being violated, justify your design with respect to known principles

downstream validation approach for visual encoding/ interaction approach

controlled experiment with qualitative discussion and comparison of resulting visualisations