In addition, each poster presenter will participate in a poster “teaser” session. The teaser session gives you the opportunity to speak for 90 seconds about your poster to excite the other HRI attendees about your work.  In order to participate in your teaser session, you must email one slide in PDF format to the late-breaking abstracts co-chair, Kristen Stubbs (kristen.stubbs@gmail.com), on or before Monday, March 9. The name of the file should be the presenter’s last name, an underscore character, and the HRI submission number (i.e., Stubbs_205.pdf).

You can find out to which poster session and which teaser session you have been assigned by visiting:
https://spreadsheets.google.com/ccc?key=prMLRjo_iIQVQQ5XC8TAILQ

Late-Breaking Abstracts will be available after the conference in the ACM digital library. These library entries would be on-archival, thus enabling you to take this work and re-submit it to other venues, while still allowing this early version to remain visible to researchers worldwide.

Authors will be notified of the acceptance decision by January 8th, 2009.  IMPORTANT: If accepted, authors must be prepared to submit both the source file and a PDF version of the paper to Sheridan Printing (https://www.sheridanprinting.com/typedept/hri2.htm) by January 13th, 2009 or risk not having their paper included in the proceedings.

2. Background of participants

This course assumes a familiarity of evaluating HRI such as was provided in earlier HRI tutorials or experience with basic experimental evaluation techniques.  Participants should be familiar with human-robot interaction but we will not assume prior knowledge of specific HCI evaluation techniques.

3. Motivation

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

As HRI becomes a more mature discipline, expectations are rising for validating the work that we are doing to design robot interfaces.  HRI is its own sub-discipline, but there are similarities to HCI because robots are computerized applications — even though they are usually mobile, often remote from users, sometimes fragile, and potentially unpredictable in their behavior.  Rather than start from scratch, a number of researchers have taken HCI evaluation techniques and have adapted them to be more compatible with the practical realities of robots.  This tutorial describes several example techniques and their adaptations and provides guidance for using these techniques with HRI applications.

The instructors are active in the field of search and rescue robotics, assistive robotics, and unmanned aerial vehicles and will use examples from these domains in the presentations.  Having examples from multiple domains will enrich the discussion.

4. Conducting the tutorial

The tutorial will use lectures and exercises to give participants experience with using four example techniques.  The tutorial will begin with a lecture and discussion of how HRI differs from HCI and what roles robot users take on when working with robots (Scholtz 2003).  We will discuss situation awareness (SA) measurement techniques (e.g., Endsley’s (1998) Situation Awareness Global Assessment Technique) and which ones are appropriate for various types of robot evaluation tasks. The heart of the tutorial consists of modules on each of the three categories of HCI evaluation methods (inspection, empirical, and formal/ analytical) and exercises that illustrate them, as described below.

4.1 Inspection Method: GDTA

Goal Directed Task Analysis (GDTA) was used by Adams (2005) to better understand how users will want to employ a robot interface.  A GDTA analysis can also be used as a means of inspecting an interface to evaluate whether it allows users to meet their goals efficiently.  We will provide attendees with a fragment of a previously-developed GDTA that they will use to evaluate whether the interface is compatible with the identified goals.

4.2 Inspection Method: Heuristic Evaluation

Heuristic evaluation for HRI (Drury et al. 2003) is an inspection method based on Nielsen’s (1994) technique, but modified with heuristics that apply more directly to HRI.  Attendees will practice comparing the heuristics to the example interface, identifying the parts of the interface that violate the heuristics.

4.3 Empirical Method: LASSO

The LASSO method (Drury et al. 2007a) is based on usability testing and the Think Aloud method (Ericsson and Simon 1980).  Users perform typical tasks while thinking aloud, and evaluators analyze the users’ utterances to determine whether users had awareness of the robot’s Location, Activity, Surroundings, and Status as well as progress towards completing the Overall mission (LASSO).  We will show a video clip from a LASSO evaluation and attendees will practice coding a transcript fragment.

4.4 Formal/Analytical Method: GOMS

The Goals, Operators, Methods and Selection rules (GOMS) technique (John and Kieras 1996) is a formal/analytical method.  In conjunction with Kieras, we have extended GOMS for HRI (Drury et al. 2007b).  We will guide attendees in writing a fragment of a GOMS model using the simplest variant of GOMS, the Keystroke Level Model, for the example robot interface.

5. Tutorial schedule

If there is more time available than is scheduled below, we will allow more time for exercises and discussion.

6. Instructors’ backgrounds

Dr. Jill Drury is an Associate Department Head at The MITRE Corporation, Adjunct Assistant Professor of Computer Science at the University of Massachusetts Lowell, and Visiting Scientist at MIT.  Her research interests include human-robot interaction, evaluation methods for human-computer interaction, and awareness support for collaborative applications.  She regularly teaches Evaluation of HCI at the graduate level.  She is a member of ACM, SIGCHI, and SWE (Society of Women Engineers) and has had organizing committee and reviewer roles for many conferences, including HRI and Human Factors in Computing Systems (CHI).  She co-organized a tutorial with Dr. Yanco and Dr. Jean Scholtz on Introduction to HRI at the Intelligent User Interfaces (IUI) 2006 conference.  Her publications in the area can be found at jldrury.googlepages.com.

Dr. Holly Yanco is an Associate Professor of Computer Science at the University of Massachusetts Lowell.  Her research interests include human-robot interaction, artificial intelligence for robotics, assistive technology, and urban search and rescue.  She has doctorate and master’s degrees from the Massachusetts Institute of Technology and a bachelor’s degree from Wellesley College, all in Computer Science.  She received the Award for Teaching Excellence from the University of Massachusetts Lowell in 2002 and the Frederick C. Hennie III Teaching Award from the Electrical Engineering and Computer Science Department of the Massachusetts Institute of Technology in 1996.  She is an elected member of AAAI’s Executive Council and has served as the Exhibitions and Sponsorship Chair of HRI-07, HRI-08 and HRI-09.  With Dr. Jean Scholtz, she co-organized a tutorial on Introduction to HRI at the CHI 2004 and CHI 2005 conferences, and (also with Jill Drury) at the IUI 2006 conference.  She also presented a tutorial introducing HRI to the AAAI 2008 conference.  See a list of her publications at https://robotics.cs.uml.edu/publications/.

7. References

[1]    Adams, J. A. (2005). Human-Robot Interaction Design: Understanding User Needs and Requirements.  In Proceedings of the 2005 Human Factors and Ergonomics Society 49th Annual Meeting, 2005, Orlando, FL, USA.
[2]    Drury, J., Riek, L. D., Christiansen, A. D., Eyler-Walker, Z. T., Maggi, A. and Smith, D. B. (2003).  Command and Control of Robot Teams.  In Proceedings of the 2003 Association of Unmanned Vehicle Systems International, Baltimore, July 2003.
[3]    Drury, J. L., Keyes, B., and Yanco, H. A. (2007a).  LASSOing HRI: Analyzing Situation Awareness in Map-Centric and Video-Centric Interfaces.  In Proceedings of the Second Annual Conference on Human-Robot Interaction, Arlington, VA, March 2007.
[4]    Drury, J. L., Scholtz, J., and Kieras, D. E. (2007b).  Adapting GOMS to Model Human-Robot Interaction. In Proceedings of the Second Annual Conference on Human-Robot Interaction, Arlington, VA, March 2007.
[5]    Endsley, M. R., Selcon, S. J., Hardiman, T. D., and Croft, D. G. (1998).  A Comparative Analysis of SAGAT and SART for Evaluations of Situation Awareness.  In Proceedings of the 42nd annual meeting of the Human Factors and Ergonomics Society, Chicago, October 1998.
[6]    K. A. Ericsson and H. A. Simon (1980). Verbal Reports as Data. Psychological Review, Vol. 87, pp. 215 – 251.
[7]    John, B. E. and Kieras, D. E. (1996).  The GOMS Family of User Interface Analysis Techniques:  Comparison and Contrast.  ACM Transactions on Human-Computer Interaction, 3(4), December 1996.
[8]    Nielsen, J. (1994). Heuristic Evaluation.  In Nielsen, J., and Mack, R.L. (Eds.), Usability Inspection Methods.  John Wiley & Sons, New York, NY.
[9]    Scholtz, J. (2003).  Theory and Evaluation of Human Robot Interactions.  In Proceedings of the Hawaii International Conference on System Science 36, January 2003

• HRI Pioneers Workshop – Organizers: Alan Wagner, Rosemarijn Looije, Maxim Makatchev, Marek Michalowski, Emily Mower; March 10th, 2009 (full day)
• Social responsibility in HRI: Conducting our research, changing the world – Organizers: Nathan G. Freier, Vanessa Evers, Takayuki Kanda, Victoria Groom, Bigle Mutlu, Peter H. Kahn; March 10, 2009 (morning)
• Societal impact: How socially accepted robots can be integrated in our society – Organizers: Astrid Weiss, Manfred Tscheligi, Aude Billard; March 10, 2009 (afternoon)
• Full-day tutorial: Using and adapting Human-Computer Interaction methods for HRI – Organizers: Jill Drury, Holly Yanco; March 10, 2009 (full day)

Guidelines and Requirements

• Videos should be self explanatory. There will be no live narration or introduction.
• As this is peer-reviewed and archival, the chairs strongly discourage videos which contain advertisements or are heavily promotional in nature.
• No revisions will be accepted. Submit the final version.
• Videos should not exceed 3 minutes in length. We recommend titles and credits last less than 3 seconds each.
• The video cannot exceed 30MB (upload capped).
• The video should play on standard PC and Macintosh computers with no 3rd party codecs added. Use Windows .avi, QuickTime.mov, or .mp4.
• Do not use DRM.
• Start the filename with the lead author’s last name.
• The videos will be played on a projected screen. Therefore, please use a horizontal resolution of at least 640 pixels. If you can use a higher resolution and still stay under the MB cap, please do so.
• If accepted, ACM requires video authors to sign a release. Copyright remains with author but 3rd party material must have verified copyright. Therefore, copyrighted media (music, video, etc) should not be included unless permission has been obtained. Authors are responsible for obtaining such permission.

Submission instructions
Submitting a video to the HRI 2009 Video Session is a two part process.

1. Upload video: Use the upload method on https://www.hri-metrics.org/hri09videos/ to submit your movie. Please make sure the workspace below reports a successful upload.
2. Email details to the chairs: Send the following in a plain text email to Aaron Steinfeld, astein+videos@cs.cmu.edu. This content will be included in the ACM Digital Library with the video.
   • Video title
   • Author 1, Affliation and email address
   • Author n, Affliation and email address
   • 350 word abstract

   For example:

   • A Happy Robot
   • Aaron Steinfeld, Robotics Institute, Carnegie Mellon University, steinfeld@cmu.edu
   • This video shows a robot which is happy when surrounded by friendly people. It becomes sad when everyone leaves.

The submission deadline is December 1st, 2008!

a) Be relevant to the field of human-robot interaction.  So, for example, a paper that describes a new face tracking algorithm needs to demonstrate how it is of direct use to human-robot interaction.  A paper contributing a face recognition technology should use standard recognition metrics (e.g., ROC curve) as well as demonstrate or highlight a path to “feasibility” in human-robot domains with regard to interactive performance, sufficient accuracy, integration with closed loop control, etc.  Similarly, a study of the elderly must show how insights from the study directly inform the design of robots for this population and a wizard-of-oz experiment should show how findings contribute to our understanding of how people might interact with robotic capabilities that are plausible (if not currently available).

b) Clearly articulate:  1) the contribution to HRI, 2) how the contribution advances the state-of-the art or knowledge in HRI, and 3) how the contribution relates to other work in HRI as well as the fields of study on which the paper draws (e.g. psychology, cognitive science, anthropology, computer vision, artificial intelligence, speech recognition, etc.).

c) Be technologically and methodologically sound based on the criteria generally used for that technology/method within a given field.  For example, conventions used in psychology for conducting experiments with people and analyzing the data, and reporting the study (e.g. hypotheses, manipulation checks, the creation of scales, ANOVA analyses, correlation tables, etc.) should be applied.  Authors should take care to use correct terminology for their methods to avoid being evaluated against the incorrect set of criteria.  For example, a user study of 5 people should be referred to as a user study or evaluation and not an experiment.

d) For empirical papers, provide adequate detail for readers to understand what was done, how the data were collected, from how many people, what were the characteristics of these people, what questions were people asked, what type of robot was involved (if a robot was used), etc.

e) Be written to be accessible for a broad, interdisciplinary/multidisciplinary HRI audience.

We particularly encourage papers that bring together subfields and investigate problems that have not been explored and are novel to HRI.

The categories and keywords were revised this year based on the abstracts for previous HRI papers.  In particular, we have identified groups according to 1) topic (high level group), 2) method, and 3) domain.  (Note that authors will need to pick a high level group as well as a method and domain.)  We sent these materials to 6 diverse members of the community and iterated based on their comments to get to this final set.

They are meant to focus on the main contribution of the paper to HRI, not on the technology, the social scientific theories, or the methods being applied.

HRI Communication – refers to any studies primarily concerned with how people and robots communicate, including interfaces used to command robots.  This includes (but is not limited to):
•    Feedback modalities
•    Dialogue
•    Language processing
•    Conveying intentions
•    Building common ground
•    Response time
•    Gaze and gestures
•    Situation awareness
•    Interface design and usability

Robot Perception & Prediction – refers to any studies primarily concerned with how robots perceive and learn about their environments, including embodied agents (humans, robots, etc.) and dynamic phenomena.  This includes (but is not limited to):
•    Following people
•    Anticipating human behavior
•    Learning about the environment
•    Modeling social situations
•    Learning from interaction
•    Task/activity modeling and monitoring
•    Cognitive architecture

Human’s Responses to Robots – refers to any studies primarily concerned with how people perceive robots and their attitudes toward robots.  This includes (but is not limited to):
•    Beliefs about robots
•    Responses to autonomy
•    Cross cultural differences in responses to robots
•    Affective & emotional responses
•    Mental models of robot behaviors

Robots in Context – refers to any studies primarily concerned with how robots affect the broader context in which they are embedded and handle the challenges of real world environments.  This includes (but is not limited to):
•    Roles that robots can/should take in social and organizational settings
•    How robots affect work practices
•    How robots affect the domestic environment
•    Teamwork & group dynamics
•    How a robot can/cannot be used in a particular context

Conceptual/Foundational – refers to non-empirical studies that are primarily concerned with presenting a concept for consideration by the HRI community.  This includes (but is not limited to):
•    Ethics of HRI
•    Theoretical contributions the tradition of the social sciences, e.g. using existing theories and evidence to make predictions about particular aspects of HRI
•    Identifying unifying themes or useful paradigms for HRI
•    Philosophical foundations of HRI

Methods & Metrics – refers to contributions primarily focused on improving methods and metrics for HRI.
•    Architecture (software framework, control architecture, etc.)
•    Design methodology (heuristics, practices, etc.)
•    Assessment techniques
•    Task metrics (efficiency, fan-out, MTTI, etc.)
•    Common metrics (system, operator, robot performance)
•    Methods for empirical studies (development of measures, new date collection techniques, etc.)

Keyword Subgroups

In addition to selecting a primary “group,” participants will be asked to identify the research methods used and the domain/application of use.

Research Methods keywords include:
•    Experimental
•    Ethnography/Qualitative field study
•    Quantitative field study
•    User study/Evaluation
•    Simulation
•    Mathematical modeling
•    Enabling technologies
•    Not applicable (or other)

Domain/Application keywords include:
•    Assistive robots (elder care, therapy, etc.)
•    Domestic robots
•    Entertainment robots
•    Exploration robots
•    Robot intermediaries (e.g. telepresence, proxies, avatars)
•    Service robots
•    Robots for/with children
•    Education
•    Search and rescue or military
•    Not applicable (or other)

Tutorials

We invite experts in different areas to propose half-day or full-day tutorials relevant to the themes of HRI’2009. Example of topics are state of the art overviews of particular HRI areas or design/research methods. Interdisciplinary and user-centered approaches are particularly welcome. Please submit the following information (up to 2 pages) to the tutorial/workshop chair, Kerstin Severinson Eklundh (email: kse@csc.kth.se) no later than September 15, 2008:

• Title of tutorial
• Tutorial speaker(s), including short CV’s
• Motivation or background
• Target audience or prerequisites
• Overview of tutorial including topics covered
• Links and references relevant to the tutorial

If the tutorial is accepted, the submitted material will be used for advertising it as part of HRI’2009.

Workshops

Workshops are an opportunity for participants to meet other members of the HRI community, to discuss problems and to present their ideas around a common topic. The workshops can be half-day or full-day, and could cover any topic relevant to HRI research, development or education. Interdisciplinary and user-centered approaches are particularly welcome. The format of the workshop may vary depending on the topic and audience. The responsibilities of workshop organizers include: (1) setting up a website for the workshop that will be linked to the HRI’2009 website; (2) organizing a high-quality peer review process of workshop contributions as well as decision of acceptance/rejection, and (3) (when applicable) putting together proceedings of the workshop consisting of accepted papers and/or materials.

Please send your workshop proposal in the following form to Kerstin Severinson Eklundh (kse@csc.kth.se) no later than September 15, 2008:

• Title of workshop
• Organizer(s), including contact information and short CV’s
• Abstract of about 250 words with suggested length and format of the workshop
• Prerequisites for participation
• Required format of workshop contributions
• Plan for documentation of the workshop
• List of potential attendees