sfu siat

IAT 432 Design Evaluation

Spring 2011 Course Schedule

week week week week week week week week week week week week week
Week 1
Topic
Required Reading and Materials

Monday, Jan 10

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

 

Lecture Slides

Course Outline

Lecture: Introduction to course

  • Review Academic honesty policy
  • Overview of course format
  • Overview of course work
  • Course textbooks
  • Overview of Evaluation
    • Evaluation Frameworks
    • Why evaluate?
    • Analytic vs. Participative evaluation methods
    • Objective vs. subjective evaluation methods
    • Formative vs. summative methods
    • Quantiative vs. qualitiative data

Studio Labs: Introduction to Studio Labs

  • Intro to Studio Labs
  • How in-class studio preps work
  • How in-class exercises work
  • How team assignments work
  • Teams for Assignment 1
  • Give out handout -- Dix et al. Chapter 9: Evaluation

 

 

Lecture Readings: Start on Rubin, J. Handbook of Usability Testing, Part I.

Handout: Dix, A. et al. Human Computer Interaction, Pearson, London, England, 2004, Chapter 9 (paper version given out in Studio Lab).

The two course textbooks are on hold in library (Burnaby and Surrey) and available at bookstore.

See TA if you miss getting the Dix et al. Chapter 9 reading handout in today's studio labs.

Week 2
Topic
Required Reading and Materials

Monday Jan 17

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

Lecture Method: Heuristic Evaluation

  • Analytical Methods
  • History
  • Types
  • Pros and Cons
  • Process of applying
Studio Labs Team Assignment 1: Heuristic Evaluation (5%)
  • Introduction to Assignment 1
  • Team formation
  • Exercise: Applying heuristics and comparing evaluators

Lecture Reading: Read all five bulleted entries for Useit's Heuristic Evaluations.

Skim through Dix, Chapter 9 (given out in the labs). See TA if you need a copy of this paper handout.

Studio: Review Team Assignment 1 & review the Project Briefing document.

TA will email team lists for d101, d102, d103 for Assignment 1.

Extra Reading for Assignment 1: Nielsen's Ten Usability Heuristics.

Week 3
Topic
Required Reading and Materials

Monday Jan 24

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

 

 

Lecture : Usability Testing I

  • What is Usability?
  • Usability testing
  • Usability test plans
  • Issues, goals and audience
  • Team and roles
  • User profiles & tasks
  • Usability metrics, data & measurement.

Studio Lab: Team Assignment 2: Usability Testing (15%)

  • Discussion based on individual studio preparation #1: Naive Usability
  • Introduction to Team Assignment 2
  • Exercise observational studies

Lecture Reading: Rubin, J. Handbook of Usability Testing, Part II.

Team Assignment 1 DUE Wed Jan 26 at 2:30 pm in IAT 432 drop box.

Studio: Prepare Studio prep exercise #1: Naive Usability.

Studio: Review Team Assignment 2, the Client Briefing and the NDA.

TA will email team lists for d101, d102, d103 for Assignment 2.

Extra reading: Hornbaek, Current practice in measuring usability:
Challenges to usability studies and research
, Int. J. Human-Computer Studies 64 (2006) 79–102.

Week 4
Topic
Required Reading and Materials

Monday Jan 31

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

 

 

Lecture: Usability Testing II

  • Usability Planning cont'
  • 3 Methods: silent, think aloud, constructive interaction
  • Ethics of working with human subjects
  • Test setup
  • Test materials

Studio Lab: Team Assignment 2 cont

  • Usability testing preparation
  • Preparing test tasks
  • Work on preparation tasks (e.g., timeline & scripts, note sheets)
  • Exercise on think aloud technique

 

Lecture Reading: Rubin, J. Handbook of Usability Testing, Part II -- continue.

Extra Lecture Reading: Jokela, When good things happen to bad products, Interactions December 2004.

Team Assignment 2 DUE Mon Feb 21 (extended due to reading week) by 2:30 pm in the IAT 432 drop box.

Studio: Extra Reading for Assignment 2: http://usability.gov/basics/

 

Week 5
Topic
Required Reading and Materials

Monday Feb 7

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

 

 

Lecture: Usability Testing III Query Methods

  • Query Methods: Asking Users through Interviews and Surveys
  • Data analysis
  • Report Writing

Studio Lab: Team Assignment 2 cont'

  • Exercise on questionnaire vs. interviews
  • Discussion based on individual studio preparation #2: Writing useful reports.
  • Review of report structure for Usability Testing Assignment 2.
  • Time for Q&A Assignment 2.

Lecture Reading: Rubin, J. Handbook of Usability Testing, Part II.

Lecture Reading: Surveys -- Martin, D. Doing Psychology Experiments, Wadsworth, UK, 2004, Chapter 10 (pp. 215-226)

Extra Reading: AskTog, 2001: How to Deliver a Report Without Getting Lynched.

Studio: Prepare Studio prep #2: Writing useful reports.

Week 6
Topic
Required Reading and Materials

Monday Feb 14

Reading Week

 

 

 

Week 7
Topic
Required Reading and Materials

Monday Feb 21

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

Lecture: Usability through Controlled Lab Experiments

  • Scientific Method
  • Hypothesis testing explained
  • Independent & dependent variables

Studio Lab: Team Assignment 3: Controlled Experiment (10%)

  • Team Dynamics Coaching
  • Introduction to Assignment 3
  • Introduction to tcl environment and tcl kb script.
  • Work through data collection with kb script.
  • New team meet & greet.

Lecture Reading: Martin, Chapters 1, 2, 7, 8

Lecture Reading: Dix et al. Chapter 9 (Handout from first week ) Read Section 9.4.2 pp 329 340 Experimental Evaluation

Extra Reading: Lawson, Chapter Measurement, criteria and judgement in design. (hand out available from Instructor).

Final exam questions given out.

Team Assignment 2 DUE Mon Feb 21 (extended due to reading week) by 2:30 pm in the IAT 432 drop box.

Studio: Review Team Assignment 3. Download and install tcl environment. Download tcl script.

TA will email team lists for d101, d102, d103 for Assignment 3.

Team Assignment 3 DUE Wed March 9 at 2:30 pm in IAT 432 drop box.

Week 8
Topic
Required Reading and Materials

Monday Feb 28

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

 

Lecture: Class cancelled.

Read Reliability & Validity in Martin.

Studio Lab: Cancelled. Studio Prep due next week in Studio.

Lecture Reading: Martin, Chapters 7 , 8, 12, 13.

Extra Lecture Reading: Statistics Tutorial
7 common errors in Statistics.

Studio: Prepare Studio Prep #3: Controlled Experiment Design.

Team DATA SHEETS DUE Friday Feb 25, 5:30 by email to TA.

Whole class data sheet for assignment 3 will be sent out by email Monday Feb 28.

Week 9
Topic
Required Reading and Materials

Monday March 7

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

 

Lecture: Controlled Expts & Statistics

  • Statistics for designers.
  • Descriptive statistics
  • Basic Inferential tests
  • Graphing basics
  • Lecture notes pdf.
  • Lecture stats samples excel file.

Studio: Go over studio prep #3 and and work through studio prep #4: statistics in-class exercise.

Complete Reading: Martin, Chapters 1,2 7 , 8, 12, 13.

Studio Studio Prep 3 from last week AND

Studio Prep #4: Statistics. We will do this in class and it is due at the end of your workshop.

You will need the data file.

Week 10
Topic
Required Reading

Monday March 14

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

 

Lecture: Field Studies, Emotion & Affect (Assignment 4)

Review Possible Exam Questions.

Studio Labs: Team Assignment 4 Field Evaluation: Nintendo Wii (15%)

  • IMI survey scoring exercise
  • Discussion of Studio prep exercise #5: Enjoyment/Engagement

Lecture Reading: Martin pp 14-19 on Qualitative Designs; Rubin Chapter 14; Dix et al. Chapter 9 section on field studies.

Extra Reading: Jarrrett, Caroline. Better Reports: How to Communicate the Results of Usability Testing.

Studio: Review Team Assignment 4

Studio: Prepare Studio Prep #5: Enjoyment/Engagement.

TA will send the teams list for d101, d102 and d103 for Assignment 4.

Inter-rater reliability example for Assignment 4.

Team Assignment 4 DUE Wed March 30 at 2:30 pm in IAT 432 drop box.

Week 11
Topic
Required Reading

Monday March 21

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

 

Lecture: Physiological Measures and Eye Tracking -- Veronica Zammitto

Studio Labs: Team Assignment 4 Field Evaluation: Nintendo Wii (15%)

  • Cued Recall Debrief introduction
  • Coding exercise

 

Lecture & Studio Reading: Read Bently et al. Evaluation using cued-recall debrief to elicit information about user's affective experiences, OzCHI, 2005. (Focus on Cued Recall Debrief Method).

Team Assignment 4 DUE Wed March 30 at 2:30 pm in IAT 432 drop box.

Week 12
Topic
Required Reading

Monday March 28

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

 

Lecture: Interface Criticism & Aesthetics

Course Evaluations.

Studio Labs

  • Introduction to Team Assignment 5: Interface Criticism (5%)
  • Interface Crti exercises.

Lecture Reading: Bertelsen & Pold, Criticism as an approach to interface aesthetics, NordiCHI '04, ACM press (2004), 23-27.

Studio: Review Team Assignment 5.

TA will email team lists for d101, d102, d103 for Assignment 5.

Team Assignment 5 DUE Wed April 13 by 2:30 pm in the IAT 432 drop box.

Week 13
Topic
Required Reading

Monday April 4

10:30 - 11:20 Lecture D100 SUR3310

11:30 - 1:20 Studio Lab D101 SUR3130

1:30 - 3:20 Studio Lab D102 SUR3130

4:30 - 6:20 Studio Lab D103 SUR3130

 

Lecture: Usability Testing in the Real World - Guest Lecture.

Studio: Review of all methods & exam questions. Bring questions to class.

 

Studio Reading: Dix Chap 9 Matrix of methods

 

Exam
   

Monday April 11

 

 

Exam D100 TUE 19 APR 12:00 - 15:00

 

Hour Offices by appointment this week.

 

 

FINAL EXAM: Closed Book

Room: SUR 2600.

 

Team Assignment 5 DUE Wed April 13 by 2:30 pm in the IAT 432 drop box.

button button button Teaching Assistant: Allen Bevans.

Teaching Courses

IAT 432
Design Evaluation


IAT 834
Mixed Methods in Design Research


IAT 884
Tangible Computing

Contact Info:

  • Email: aantle@sfu.ca
  • Phone: 778.782.7438
  • Fax: 778.782.9422
  • Office Location: Podium 2-2832