The role and use of speech gestures in discourse

Abstract

This paper describes how a secondary level of discourse information can be processed in a speech signal for the automatic annotation of discourse progress and for producing an estimate of a speaker's participation status. In a semi-formal round-table meeting situation there is typically only one main speaker at any given moment, but several participants may be speaking simultaneously, expressing agreement (or otherwise), chatting, translating, etc., in addition to the main speaker. We are currently performing research into technology to process this 'audio landscape' in order to detect the main speaker and to categorise the competing forms of speech in a given situation. Several speech gestures such as laughter, agreement, and feedback-responses can be recognised, isolated, and used to determine the progress of the meeting and the degrees and types of participation status among the members present. The technology exists to recognise these discourse events, but we still lack a model of their function in the mutual transfer of information through speech interaction.