Ehsan Hoque, assistant professor of computer science, is in rare company. The editors of MIT Technology Review have named Hoque one of this year’s “innovators under 35”—an honor that has previously gone to the likes of Larry Page and Sergey Brin, the cofounders of Google; Mark Zuckerberg, the cofounder of Facebook; and Jonathan Ive, the chief designer of Apple.
Hoque, who is also an assistant professor of electrical and computer engineering and an assistant director for research outreach at the University’s Goergen Institute for Data Science, is being recognized for his cutting-edge work in human-computer interaction.
Hoque is interested in understanding the secret code of human behavior that is, in the words of anthropologist Edward Sapir, “written nowhere, known by none, and understood by all.” He applies a computational lens to understand and model the ambiguity that language, facial expressions, gestures, and intonation introduce in human communication. In addition to expanding the understanding of human behavior, Hoque developed new systems that allow individuals to practice and receive feedback on social skills in a repeatable, objective, and respectful way; help musicians practice singing vowels; and provide live feedback public speakers, while they’re engaged with audiences.
“The area of human-computer interaction is exciting because it allows us to develop a bridge between artificial intelligence and humanity,” said Hoque. “The way we smile, pause, speak and look at someone can reveal a great deal about the point we’re trying to get across. Much of my work involves developing systems that allow computers to engage in communication in a very natural way.”
-
MIT Technology Review, “35 Innovators Under 35”
profile by Julia Sklar ’14
Toward that end, he developed MACH and then LISSA, two computerized conversation assistants. Both systems provide real-time feedback on a person’s social speaking skills. How often does the person say “um” or “ah?” Are the gestures natural and appropriate? Is the person’s voice properly modulated?
While MACH (My Automated Conversation coach) was initially designed to help MIT undergraduate students improve their interview skills, an improved version, LISSA (Live Interactive Scoial Skills Assistant) was developed with the intention of helping individuals with developmental disorders, such as autism. His motivation comes from personal experience, as he is the primary caregiver for a non-verbal brother, 16, with Down syndrome. “Taking care of my brother gave me an insight that individuals with special needs do not necessarily have to become nondisabled. Instead, effort and awareness should go in helping them to live effectively as people with needs.”
Being overwhelmed with public responses from his initial work, Hoque has taken the effort to develop a social training module that is widely accessible via a computer browser. Called ROCSpeak, the system analyzes word choice and body language, including the use of smiles, filler words, and intonation.
In addition, Hoque and his team have developed Rhema, an intelligent user interface for “smart glasses” that gives real-time feedback to the speaker. Rhema is able to record a speaker, transmit the audio to a server that automatically analyzes the volume and speaking rate, and then present the data to the speaker in real time. At that point, the speaker decides whether or not to make adjustments.
While the work of analyzing the full range of a person’s communications was motivated to help individuals with social difficulties, it has attracted a lot of interest from the U.S. Department of Defense and the U.S. Army. Hoque is leading a $2.5 million dollar project by the Defense Department to harness the many ways humans communicate with each other so that computers can become actual collaborators. A separate grant from the Army is allowing Hoque to research ways of using the technology to study deceptions. “When a person’s non-verbal communication is out-of-sync with the language being used, we sense something being off,” said Hoque. “It could be a matter of stress, nervousness, or deception, and automated techniques may be able to detect it more objectively.”
Hoque earned his PhD in 2013 from MIT. His PhD thesis, “Computers to Help with Conversations: Affective Framework to Enhance Human Nonverbal Skills,” was the first of its kind to show that people can improve their social skills by interacting with automated systems.
