You are here: Home » Events » Running Crossover Experiments in Software Engineering

Running Crossover Experiments in Software Engineering

 

Schedule and location

Tuesday March 13th – Wednesday March 14th


University of Oulu, Linnanmaa campus (visiting address: Pentti Kaiteran katu 1)  Please, find the map of the campus here. 
  • 13th March class HR144
  • 14th March Software Factory Lab, class TS136

Registration 

Registration is open February 7th - March 6th .

Speaker

Associate Professor Sira Vegas, Universidad Politécnica de Madrid, Spain.

Organizer

Professor Markku Oivo, University of Oulu, Finland.

 

Overview

In experiments with crossover design subjects apply more than one treatment. Crossover designs are widespread in software engineering experimentation: they require fewer subjects and control the variability among subjects. However, some researchers disapprove of crossover designs. The main criticisms are: the carryover threat and its troublesome analysis. Carryover is the persistence of the effect of one treatment when another treatment is applied later. It may invalidate the results of an experiment. Additionally, crossover designs are often not properly designed and/or analysed, limiting the validity of the results. In this seminar, we aim to make participants aware of the perils of crossover experiments and provide risk avoidance good practices. We study how another discipline (medicine) runs crossover experiments. We review the SE literature and discuss which good practices tend not to be adhered to, giving advice on how they should be applied in SE experiments. We illustrate the concepts discussed by requesting participants to design a crossover experiment and analyse a real crossover experiment. 

Detailed Program

Day 1: Designing a SE Crossover Experiment

9:00-12:00    How to design a SE crossover experiment:

  • Introduction (reminder of basic concepts in SE experimentation)
  • What is a crossover design?
  • Crossover experiments in SE: State of the practice
  • Design issues: define periods, define sequences, deal with carryover
  • Summary 

12:00-13:00   Lunch Break 

13:00-16:00    Laboratory. Students will be asked to design a SE crossover experiment. The proposed designs will be discussed.

 

Day 2: Analysing a SE Crossover Experiment (with SPSS)

9:00-12:00    How to analyse a crossover experiment:

  • Take into account subject variability
  • Deal with carryover
  • Match analysis with design
  • Beware of effect size
  • Summary

12:00-13:00   Lunch Break

13:00-16:00    Laboratory (with SPSS). Students will be asked to analyse a real SE crossover experiment. They will perform several analyses (more and less appropriate ones), and will compare the different outputs in order to discuss the best approach that should be used.

 

Credit points

Doctoral students participating in the seminar can obtain 2 credit points. This requires participating on all of the days and completing the lab assignments.

Registration fee

This seminar is free-of-charge for Inforte.fi member organization's staff and their PhD students. For others the participation fee is 400 €. The participation fee includes access to the event and the event materials. Lunch and dinner are not included.