What Is Lip Reading?

Lip reading (lip-reading / speech-reading) can "read" or "partially read" what it says by observing the speaker's mouth shape changes. The purpose of lip reading research is to use visual channel information to supplement auditory channel information and improve the comprehension of computer systems. British scientists have developed lip-reading computers that can distinguish between different languages, and can recognize English, French, German, Arabic, Mandarin Chinese and Cantonese, Italian, Polish, and Russian. This technology has practical implications for deaf patients, law enforcement agencies, and workers in noisy environments.

Lip reading

discuss

Lip reading (lip-reading / speech-reading) can "read" or "partially read" what it says by observing the speaker's mouth shape changes. The purpose of lip reading research is to use visual channel information to supplement auditory channel information and improve the comprehension of computer systems. British scientists have developed lip-reading computers that can distinguish between different languages, and can recognize English, French, German, Arabic, Mandarin Chinese and Cantonese, Italian, Polish, and Russian. This technology has practical implications for deaf patients, law enforcement agencies, and workers in noisy environments.

Chinese name

Lip reading

Foreign name

lipreading / speechreading

Overview

British scientists have developed lip-reading computers that can distinguish between different languages. In fact, researchers have been developing computers that can read lip language, but it is the first time that they have been "churched" to recognize various languages. Scientists point out that the invention has practical implications for deaf patients, law enforcement agencies and workers in noisy environments.

The study was directed by Stephen Cox and Jake Newman, School of Computer Science, University of East Anglia, UK. The technology was developed by statistically modeling the lip movements of 23 people who speak two or three languages. The system can recognize the language spoken by a single speaker with high accuracy. These languages include English, French, German, Arabic, Mandarin Chinese and Cantonese, Italian, Polish, and Russian and Japanese.

Cox said: "This is an exciting advance in automatic lip reading technology. It is the first scientific confirmation of what we have intuitively concluded that when people speak different languages, they appear differently in different orders. Cox said. "For example, we found that French speakers tend to have round lips, while Arabic speakers have obvious tongue movements." Cox added: "For years, scientists Basic research on speech and language processing principles, such as speech recognition in noise, formulaic language modeling, language processing by speech synthesis, and development of speech processing applications. "

According to researchers, there are many scientific challenges in automatic lip reading technology. Several major scientific issues have been solved, including what the relationship between facial movements and perceived speech is; how the speaker's language and the content of the conversation affect this relationship; the language, the posture of the speaker, and the accuracy of the content of the conversation in identifying What effect is there and so on.

This research is part of a broader automated lip reading project. The project will build on advanced computer video and speech recognition technologies to research and evaluate automatic lip reading in videos, thereby developing tools and technologies that can perform automatic, language independent lip reading on targets from video. The project also seeks to quantify human and automated machine capabilities. The next step for researchers will make this system more suitable for personal physiology and the way people speak.