ویراساینس.
سند علمی

Automatic self-supervised learning of associations between speech and text

Juha Knuuttila, Okko Johannes Räsänen, Unto K. Laine

چکیده

Discovery of statistically significant patterns from data and learning of associative links between qualitatively different data streams is becoming increasingly important in dealing with the so-called Big Data problem of the modern society. In this work, a methodological framework for automatic discovery of statistical associations between a high bit-rate and noisy sensory signal (speech) and temporally discrete categorical data with different temporal granularity (text) is presented. The proposed approach does not utilize any phonetic or linguistic knowledge in the analysis, but simply learns the meaningful units of text and speech and their mutual mappings in an unsupervised manner. The first experiments with a limited vocabulary of childdirected speech show that, after a period of learning, the method is successful in the generation of a textual representation of continuous speech.

اسناد مشابه

  • Two Self-supervised Learning Techniques for Speech Recognition

    سند علمی۲۰۱۰Damianos Karakos, Haolang Zhou, Puyang Xu, Sanjeev Khudanpur

    for predicting the uttered word sequence W, given the acoustic information A. The acoustic model is represented by the conditional distribution P (A|W), while the language model is represented by the prior P (W), and the bulk of the research in speech recognition is on training p

  • Self-Supervised Learning for Automatic Text Summarization by Text-span Extraction

    سند علمی۲۰۰۱Massih-Reza Amini, Patrick Gallinari

    We describe a system for automatic text summarization that operates by extracting the most relevant sentences from documents with regard to a query. The lack of labeled corpora makes it difficult to develop automatic techniques for summarization. We propose to use a self-supervis

  • Efficient Personalized Speech Enhancement Through Self-Supervised Learning

    مقاله‌ی دسترسی آزاد۲۰۲۲IEEE Journal of Selected Topics in Signal Processing['Aswin Sivaraman', 'Minje Kim', '']

    This work presents self-supervised learning methods for monaural speaker-specific (i.e., personalized) speech enhancement models. While general-purpose models must broadly address many speakers, personalized can adapt to a particular speaker's voice, expecting solve narrower prob

  • The Un-normalized Graph p-Laplacian based Semi-supervised Learning Method and Speech Recognition Problem

    سند علمی۲۰۱۷Loc Tran, Linh Tran

    Speech recognition is the classical problem in pattern recognition research field. However, just a few graph based machine learning methods have been applied to this classical problem. In this paper, we propose the un-normalized graph p-Laplacian semi-supervised learning methods

  • TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech

    مقاله‌ی دسترسی آزاد۲۰۲۱IEEE/ACM transactions on audio, speech, and language processing['Andy T. Liu', 'Shang-Wen Li', 'Hung-yi Lee', '']

    We introduce a self-supervised speech pre-training method called TERA, which stands for Transformer Encoder Representations from Alteration. Recent approaches often learn by using single auxiliary task like contrastive prediction, autoregressive or masked reconstruction. Unlike p

  • A WAV2VEC2-Based Experimental Study on Self-Supervised Learning Methods to Improve Child Speech Recognition

    مقاله‌ی دسترسی آزاد۲۰۲۳IEEE Access['Rishabh Jain', 'Andrei Barcovschi', 'Mariam Yahayah Yiwere', 'Dan Bigioi', 'Peter Corcoran', 'Horia Cucu', '']

    Despite recent advancements in deep learning technologies, Child Speech Recognition remains a challenging task. Current Automatic (ASR) models require substantial amounts of annotated data for training, which is scarce. In this work, we explore using the ASR model, wav2vec2, with