Experimental Research Methods for Human-Computer Interaction

Knowing how to design and conduct experiments (“user studies”) is a skill required of all researchers in human-computer interaction. This course will examine the scientific foundations of experimental research, the rudiments of designing and conducting an experiment, and the tools and procedures for hypothesis testing.

Registration has ended

Time and location

Monday 3rd and Tuesday 4th of December 2012
University of Tampere (Kanslerinrinne 1). In building Pinni B, hall B3111.


One feature of this course is participation in an experiment (lasting about 30 minutes).  Working in pairs and using a hand-out prepared by the presenter, attendees take turns acting both as participant (performing tasks while data are collected) and investigator (instructing the participant, administering the tasks, collecting and recording data).  Although only using a paper mockup, the experiment will have all the components of a real experiment.  There will be independent variables, dependent variables, counterbalancing, and so on.  Testing the research questions through an analysis of variance (ANOVA) is presented and explained.  Note that prior knowledge of statistical tests is not required.

Publication is the final step in research.  And so, this course also includes some pointers and tips on writing-up a research project – including the methodology and results of the experiment – in a paper that is formatted and ready for submission to an HCI conference.

About the Presenter

Professor Scott MacKenzie's research is in human-computer interaction with an emphasis on human performance measurement and modeling, experimental methods and evaluation, interaction devices and techniques, text entry, language modeling, gaming, eye tracking, touch input, and mobile computing. He has more than 130 publications in the field of Human-Computer Interaction (including more than 30 from the ACM's annual SIGCHI conference) and has given numerous invited talks over the past 20 years. Since 1999, he has been Associate Professor of Computer Science and Engineering at York University, Canada.  Home page: http://www.yorku.ca/mack/

Organizer: Professor Kari-Jouko Räihä

Schedule (tentative)

Day 1 – Session 1: What is research  experimental research  empirical research  publication  citations and impact  research methods (experimental, observational, correlational)  observation and measurement  in-class user study

Day 1 – Session 2: Formulating research questions  internal validity  external validity  comparative evaluation  circumstantial vs. causal relationships  ideas  finding a research topic

Day 1 – Session 3: Methodology  ethics approval  independent variables  dependent variables  control variables  random variables  confounding variables  experiment task  procedure  participants  questionnaire design

Day 2 – Session 1: Within-subjects vs. between subjects  order effects, counterbalancing, latin squares  group effects, asymmetric skill transfer  longitudinal studies  running the experiment

Day 2 – Session 2: Hypothesis testing  parametric vs. non-parametric procedures  analysis of variance  why analyse the variance  more than two test conditions  post hoc comparisons  two-way analysis of variance

Day 2 – Session 3: ANOVA tool, API, demo  testing for a group effect  non-parametric tests  writing and publishing a research experiment  parts of a research paper  preparing the manuscript  writing for clarity

Tentative timetable

Day 1: Monday, 3.12.2012 (hall B3111)

10.00-10:15    Morning coffee
10:15-11:45    Session 1
11:45-12:45    Lunch break *
12:45-14:15    Session 2
14:15-14:45    Coffee break
14:45-16:15    Session 3

18:30    Informal get-together

*) Lunch can be bought at own expense.

Day 2: Tuesday, 4.12.2012 (hall B3111)

  9:15-10:45    Session 1
10:45-11:15    Coffee break
11:15-12:45    Session 2
12:45-13:45    Lunch break *
13:45-15:15    Session 3   

*) Lunch can be bought at own expense.


This seminar is free-of-charge for INFORTE.fi member organization's staff and their PhD-students. For others the participation fee is 700 €. The participation fee includes access to the event and the event materials. Lunch and dinner are not included.


Doctoral students participating in the seminar can obtain 1-5 credit points. This requires participating on both days and completing the assignments given at the seminar.