CS3009 Human Computer Interaction: HCI Research
CS3009 Human Computer Interaction: HCI Research
Main Themes
Evaluating research quality
Finding relevant research
Literature survey methods
Originality
Contribution Types:
Empirical: New knowledge through observation/data gathering
Artefact: Creation/invention of new system/product/tool, etc.
Methodological: New methods
Theoretical: New concepts/principles
Dataset: New corpus of data
Survey: Literature review or meta-analysis
Opinion: Essay or argument
Significance
Significance Questions:
Why is the research important?
Who does it benefit?
How do they benefit?
Typical Beneficiaries:
Other researchers
End users
Organisations
Typical Benefits of HCI Research:
Increased efficiency
Increased effectiveness
Increased user satisfaction
Resulting benefits include:
Decreased cost of service
Increased uptake of service
Academic Significance:
Citation counts indicate academic significance (frequency with which other academics reference work)
Brunel Computer Science publications statistics:
9th in UK in NTU performance ranking of scientific papers for world universities in Computer Science
9th in UK in 2024 Shanghai Global Rankings of Academic Subjects in Computer Science
Home to nine of the world's top 2% cited scientists (Elsevier 2025)
Two staff among the top 30 cited computer scientists in the UK (#2 and #27) (Research.com)
Home to two highly cited researchers in Clarivate listing (top 1% of citations for the field)
Rigour
Approach Components:
How data is collected
How data is analysed and interpreted
Assumptions underlying the method
Methods Definition: Formalised procedures/tools guiding the process of gathering and analyzing information.
Key Attributes of Rigour
Validity:
Can results be generalized to other situations/people?
Types:
External validity
Ecological validity
Are effects due to the variable of interest or due to other factors?
Internal validity
To what extent are conclusions based on statistical tests correct/reasonable?
Statistical validity
Includes Construct, Content, and Criterion validity
Measurement validity
Reliability: Consistency/repeatability of measures used.
Types of Reliability
Type: Test-retest
Description: Consistency over time – same result when repeated.Type: Inter-rater
Description: Consistency between people – same result from different individuals measuring.Type: Internal consistency
Description: Consistency between different elements of a test designed to assess the same construct.
Reliability vs. Validity
Unreliable & Unvalid: Not consistently measuring what is intended.
Reliable, Not Valid: Consistently measuring but not accurately reflecting the intended construct.
Both Reliable & Valid: Consistently measuring and accurately reflecting the intended construct.
Visual Aid: Based on Image: © Nevit Dilmen, Wikimedia Commons.
Rigour in Qualitative Research
TABLE III. Criteria for Judging Quality of a Research Study:
Quantitative Terms:
Truth value: Internal validity
Applicability: External validity or generalizability
Consistency: Reliability
Neutrality: Objectivity
Qualitative Terms:
Credibility
Transferability
Dependability
Confirmability
Literature Survey Methods
Where to Find HCI Research:
Journals:
HCI
International Journal of Human-Computer Studies
Computers and Human Behavior
Conferences:
CHI
British HCI
Interact
Ranking Research Quality Exercise
Poll Location: Complete the poll at PollEv.com/katehone721
Quality Indicators
Key indicators include:
Peer review
Respected journal publishers or learned societies:
ACM
IEEE
Elsevier
Taylor and Francis
Springer
Google Scholar Rankings
Categories > Engineering & Computer Science
HCI Publication Rankings:
Computer Human Interaction (CHI)
Proceedings of the ACM on Human-Computer Interaction
International Journal of Human-Computer Interaction
Behaviour & Information Technology
IEEE Transactions on Affective Computing
International Journal of Human-Computer Studies
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Virtual Reality
International Journal of Interactive Mobile Technologies
ACM Transactions on Computer-Human Interaction
ACM Designing Interactive Systems Conference
ACM Symposium on User Interface Software and Technology
ACM/IEEE International Conference on Human Robot Interaction
Frontiers in Virtual Reality
IEEE Virtual Reality Conference
International Conference on Intelligent User Interfaces (IUI)
Universal Access in the Information Society
IEEE Transactions on Human-Machine Systems
H5-Index and H5-Median values provided for context.
Search Approaches
Typical Methods:
Typically keyword-driven searches.
Most facilitate citation searches.
Databases:
ACM Digital Library, IEEE, Academic Search Complete, Google Scholar
Search by author/research group
AI academic search tools
Academic social media sites (ResearchGate, Academia.edu)
Understanding References
Example Format: Hone, K. (2006) "Empathic agents to reduce user frustration: The effects of varying agent characteristics." Interacting with Computers, 18(2), 227-245.
Structure includes: Author’s name (surname, initial(s)), Date of publication, Title of paper, Title of journal, Volume number, Issue number, Page numbers.
Harvard Format for References
Reference list must be alphabetical by first author's surname.
In-text citations use only author surname(s) and publication date (no initials).
Formats include:
In-line: 'Brown and Perry (2024) found that…'
End of sentence: 'It has been found that….(Brown and Perry, 2024)'
'Et al' is used as an abbreviation for 'and others'.
Literature Surveys
Most academic publications include a background literature section to:
Set the context for the study
Demonstrate the 'gap' that the research addresses
Some academic publications are solely based on the review of past literature and provide a survey contribution type.
Types of Literature Survey
Narrative/Traditional Review: No formal methodology.
Systematic Review: Uses a standardized, structured methodology.
Meta-analysis: Statistical analysis applied to research data identified through systematic review.
Stages in a Systematic Review
Each stage must be documented ensuring:
Transparency
Repeatability
Stages include:
Scope and map
Plan and protocol
Inclusion and exclusion criteria
Search and screen
Quality appraisal
Data extraction & synthesis
Comparison of Review Types
Traditional Review:
May bias material selection
Informal approach lacks transparency & repeatability
Can be executed by a single researcher
Relatively quick to complete
Systematic Review:
Designed to minimize bias
Systematic approach that is transparent and repeatable
Ideally involves multiple researchers
Takes significant time and effort to complete
AI Tools for Literature Review Process
Rapidly evolving ecosystem providing:
Literature search (e.g. Consensus, Elicit, Scite)
Syntheses of answers to research questions based on identified papers
Visualisation of the research landscape (e.g. Connected Papers, Research Rabbit)
Pros and Cons of AI Tools
Pros:
Significantly speeds up the search process.
Bespoke tools link directly to publisher abstracts/citations.
May find papers outside traditional search approaches.
Cons:
Possible bias due to the databases used for underlying data.
Not all papers included will have undergone peer review.
Lack of transparency regarding approach.
Summary
Developed understanding of:
Research quality
Finding research
Literature review methods
Next week: Discuss the early days of HCI and experimental methods.