Speech and Gesture Multimodal Control of a Whole Earth 3D Visualization Environment

Krum, David M.; Omoteso, Olugbenga; Ribarsky, William; Starner, Thad; Hodges, Larry F.

Speech and Gesture Multimodal Control of a Whole Earth 3D Visualization Environment

Files

195-200.pdf (1.61 MB)

Date

2002

Authors

Krum, David M.
Omoteso, Olugbenga
Ribarsky, William
Starner, Thad
Hodges, Larry F.

Publisher

The Eurographics Association

Abstract

A growing body of research shows several advantages to multimodal interfaces including increased expressiveness, flexibility, and user freedom. This paper investigates the design of such an interface that integrates speech and hand gestures. The interface has the additional property of operating relative to the user and can be used while the user is in motion or standing at a distance from the computer display. The paper then describes an implementation of the multimodal interface for a whole Earth 3D visualization which presents navigation interface challenges due to the large magnitude of scale and extended spaces that are available. The characteristics of the multimodal interface are examined, such as speed, recognizability of gestures, ease and accuracy of use, and learnability under likely conditions of use. This implementation shows that such a multimodal interface can be effective in a real environment and sets some parameters for the design and use of such interfaces.

        @inproceedings{:10.2312/VisSym/VisSym02/195-200
,
booktitle = {Eurographics / IEEE VGTC Symposium on Visualization
},
editor = {D. Ebert and P. Brunet and I. Navazo
},
title = {{Speech and Gesture Multimodal Control of a Whole Earth 3D Visualization Environment
}},
author = {Krum, David M. and 
Omoteso, Olugbenga and 
Ribarsky, William and 
Starner, Thad and 
Hodges, Larry F.
},
year = {2002
},
publisher = {The Eurographics Association
},
ISSN = {1727-5296
},
ISBN = {1-58113-536-X
},
DOI = {/10.2312/VisSym/VisSym02/195-200
}
}