Detecting the Noteworthiness of Utterances in Human Meetings
Satanjeev Banerjee and Alexander Rudnicky
SIGDIAL Workshop on Discourse and Dialogue (SIGDIAL 2009)
Queen Mary University of London, September 11-12, 2009
Summary
Our goal is to make note-taking easier in meetings by automatically detecting noteworthy utterances in verbal exchanges and suggesting these to meeting participants for inclusion in their notes. To show feasibility of such a process we conducted a Wizard of Oz study where the Wizard picked automatically transcribed utterances that he judged as noteworthy, and suggested their contents to the participants as notes. Over 9 meetings, participants accepted 35% of these suggestions. Further, 41.5% of their notes at the end of the meeting contained Wizard-suggested text. Next, in order to perform noteworthiness detection automatically, we annotated a set of 6 meetings with a 3-level noteworthiness annotation scheme, which is a break from the binary "in summary"/"not in summary" labeling typical used in speech summarization. We report Kappa of 0.44 for the 3-way classification, and 0.76 when two of the 3 labels are merged into one. Finally, we trained an SVM classifier on this annotated data. We show that this classifier’s performance lies between that of trivial baselines and inter-annotator agreement.
