Newest papers

Conducting Reliable Keyword Estimation with Unreliable ASR in Police Audio Data

Robert Cole Molloy, Rhiannon N. Miller, Stephen Morgan, Anjalie Field · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26614 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Mengekstrak wawasan interval dari kumpulan data audio besar menggunakan transkripsi manusia mahal dan memakan waktu. Off-the-shelf Automatic Speech Recognition (ASR) sistem menawarkan alternatif yang mudah untuk digunakan, tetapi kesalahan transkripsi sistematis, terutama dalam pengaturan khusus domain, dapat memprioritaskan analisis downstream dan mengarah ke kesimpulan yang tidak dapat diandalkan. Kami menyelidiki apakah transkripsi ASR yang tidak sempurna masih dapat digunakan untuk mendapatkan perkiraan frekuensi kata kunci yang valid secara statistik dengan menggabungkan output model dengan serangkaian kecil contoh yang ditandai manusia, mengikuti prediksi Inference Powered dan kerangka pembelajaran yang didasarkan pada desain. Kami mengevaluasi pendekatan ini pada komunikasi radio polisi dari Baltimore dan Chicago, sebuah domain di mana distribusi kata kunci yang tidak valid dapat memberikan wawasan berharga tentang aktivitas polisi, tetapi sistem

Machine translation (Meta M2M100); the English original follows.

Abstract

Extracting insights from large audio data sets using human transcription is expensive and time-consuming. Off-the-shelf Automatic Speech Recognition (ASR) systems offer an easy-to-use scalable alternative, but systematic transcription errors, especially in domain-specific settings, can bias downstream analyses and lead to unreliable conclusions. We investigate whether imperfect ASR transcripts can still be used to obtain statistically valid keyword frequency estimates by combining model outputs with a small set of human-labeled examples, following Prediction Powered Inference and Design-based Supervised Learning frameworks. We evaluate this approach on police radio communications from Baltimore and Chicago, a domain in which keyword distributions could provide valuable insights into police activity, but off-the-shelf ASR systems exhibit systematic errors on operationally important terminology. Using bootstrap-based confidence interval coverage as our evaluation metric, we find that our approach substantially increases the number of domain-specific keywords that can be analyzed with statistically valid confidence intervals, from 43.0% using ASR alone and 27.2% using only a small set of human-labeled transcriptions to 56.2% in the Baltimore dataset, with similar improvements observed in Chicago data. Our results demonstrate that post-hoc error correction can improve statistical validity of downstream analyses even for poor-quality transcriptions, offering an avenue for better leveraging off-the-shelf ASR models for large-scale speech analysis.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Inference, Speech recognition, Transcription (linguistics), Scalability, Set (abstract data type)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange