Speech-to-Retrieval, Google unveils a new approach to voice search

Between artificial intelligence models and advanced voice assistants, search is also evolving with a new approach coined by Google as Speech-to-Retrieval: the announcement comes from a blog article published by Ehsan Variani and Michael Riley, scientific researchers at Google Research.

Available for many years now, voice search for information on the web is still used by many people today. However, while the voice search technology used by Google initially relied on automatic speech recognition (ASR) to transform an audio input into a textual query and then search for matching documents, it has been verified that even the smallest errors in voice recognition can alter the meaning of the query and return the wrong results to the user. The researchers explain in the article how a voice search can give incorrect results, giving a concrete example: a user pronounces the query 'the Scream painting' with the intention of obtaining information about Edvard Munch's famous work, but if the ASR system changes the 'm' to 'n', the query is transcribed to 'screen painting' and thus gives results related to painting techniques instead of the artist's masterpiece. A

The model announced is based on a dual-encoder architecture with two neural networks learning from huge amounts of data in order to understand the relationships between what the user says and the available information: one encoder processes the audio by converting it into a vector representation from which it captures the semantic meaning, on the other hand a second encoder processes a vector representation for the documents. In other words, when a user pronounces a voice search query, the audio is streamed to an encoder, i.e. a pre-trained model capable of transforming sounds into data: from this process a sort of fingerprint is created that captures the deep meaning of the query and is then used to identify a set of relevant results through a process of search classification. Obviously this is only the first step since the final result is always managed by the sys

In conclusion, voice search based on S2R is not just an academic experiment but an evolution that is now operational. In fact, thanks to the close collaboration between the Google Research and Search teams, these next-generation models are now implemented in multiple languages, offering a clear improvement in terms of accuracy and latency compared to traditional systems. Furthermore, in order to support the advancement of research in this field, Google has decided to open source the SVQ (Spoken Query Dataset) dataset as part of the Massive Sound Embedding Benchmark (MSEB): by sharing these resources, in fact, the Mountain View company intends to stimulate the global scientific community to experiment, compare models and contribute to the creation of the next generation of intelligent voice interfaces, capable of understanding and responding naturally to the nuances of human language.