SpeechActs: A Spoken Language Framework

Paul Martin, Fredrick Crabbe, Stuart Adams, Eric Baatz, and Nicole Yankelovich, (IEEE Computer, Vol. 29, Number 7, July 1996)

Introduction by Paul Martin and Nicole Yankelovich

SpeechActs was a prototype system designed for traveling professionals who required access to online information while they were away from their computer. SpeechActs provided a natural, speech-only interface to a suite of integrated applications, including e-mail and calendar. Other applications provided speech access to dynamic data feeds for weather forecasts, stock quotations, currency exchange data, and international time. Without having to train the system, a user could telephone SpeechActs from an airport, a hotel room, or a colleague's office and speak requests naturally without having to memorize commands. For example, SpeechActs understood phrases such as "I'd like mail please," "How's the weather in Chicago?" or "What's on Bob's calendar the day after tomorrow?"

The SpeechActs paper selected for this collection introduces the SpeechActs Framework and focuses on the techniques used to create a conversational, consistent "sound and feel" across applications. The paper also touches on compiling speech-recognizer and natural language grammars from a single source, dynamically altering grammars for better recognition, and handling recognition errors with medium-grained semantic analysis.

While the SpeechActs project never had a direct impact on Sun's products, it was unique for its time in the speech field. In 1996, when this paper was published, almost all existing speech applications had been built by speech vendors and worked only with the vendor's specific technology. SpeechActs was one of the first large-scale, continuous speech projects that considered recognizers and synthesizers as pluggable components. In addition, the grammar work done as part of the SpeechActs project had a substantial influence on portions of the Java(TM) Speech API.

SpeechActs never had a large user population, but those that did use the prototype system were heavily studied through laboratory usability tests and longitudinal field trials. This data enabled the team to develop an in-depth understanding of conversational speech user interface design techniques. The lessons learned from this research are still being referenced in the current speech user interface design literature and have had a substantial impact in the field.

Link to Paper

REFERENCES:

Using Natural Dialogs as the Basis for Speech Interface Design, Nicole Yankelovich, submitted to MIT Press as a chapter for the upcoming book, Automated Spoken Dialog Systems, edited by Susann Luperfoy. http://www.sun.com/research/speech/publications/mit-1998/MITPressChapter.v3.html
How do Users Know What to Say? Nicole Yankelovich, ACM Interactions, Vol. III, Number 6, November/December 1996. http://www.sun.com/research/speech/publicationsacm-interactions-1996/Interactions.html
The "Casual Cashmere Diaper Bag": Constraining Speech Recognition Using Examples, Paul Martin, Proceedings of the Association of Computational Linguistics, Madrid, July 7, 1997. http://www.sun.com/research/speech/publications/acl97/cashmere.html
Office Monitor, Nicole Yankelovich, Cynthia D. McLain, CHI '96 Conference on Human Factors in Computing Systems, Vancouver, British Columbia, Canada, April 14-18, 1996. http://www.sun.com/research/speech/publications/chi96/CHI96.html
Designing SpeechActs: Issues in Speech User Interfaces, Nicole Yankelovich, Gina-Anne Levow, Matt Marx, CHI '95 Conference on Human Factors in Computing Systems, Denver, CO, May 7-11, 1995. SMLI 94-0394. http://www.sun.com/research/speech/publications/chi95/CHI95.html
Speechacts: A Testbed for Continuous Speech Applications, Paul Martin, Andrew Kehler, Proceedings of the AAAI Workshop on Integration of Natural Language and Speech Processing, pages 65-71, Cambridge, MA, August. 1994 MIT Press. http://www.sun.com/research/speech/publications/aaai-workshop94.ps