Impact Journalism Day 2016

Look Who’s Talking: The App That Deciphers Babies’ Cries

Infant Crying Translator offers help for stressed parents who can’t interpret their infant’s wailing.

Nearly 300,000 sounds from 100 newborn babies were gathered at National Taiwan University Hospital’s branch in Yunlin County.
Chuan-Yu Chang

What’s baby saying? A team in Taiwan has made an app to tell parents if their baby is hungry, tired, in pain or needs a fresh diaper.

Over three years, researchers collected about 300,000 sounds from 100 newborn babies at the National Taiwan University Hospital.

They then made a smartphone app called Infant Crying Translator, a cloud-based program that can decipher the crying of babies from the moment they’re born until they are about 6 months old.

It takes a mere 15 seconds. Tap “Record” in the app and a clip of the infant’s cry is uploaded to a cloud database. The file is quickly compared to an audio library and a verdict pops up onscreen.

After winning an innovation award from Taiwan’s government in 2014, Infant Crying Translator for Android and iPhone went on sale in 2015 and now has some 10,000 users worldwide.

The app isn’t perfect, but results are getting better all the time. According to user feedback, accuracy reaches 92 percent for babies under 2 weeks old.

For babies under 2 months, the accuracy of the app was once low but now can be as high as 85 percent. And for a 4-month-old, accuracy is 77 percent and rising.

“When we first launched, the app wasn’t particularly accurate for babies older than 2 weeks. Back then, the database only had audio of very young babies that we gathered from a hospital,” says head researcher Chuan-Yu Chang.

“Now the library has many, many more sound files uploaded by users, and it can make increasingly better judgments for a wider range of ages,” he adds.

In addition, a machine-learning algorithm lets parents personalize the app for their baby.

“Based on the actual situation for each baby, the cloud database will make unique data models to improve the recognition rate,” says Chang.

High-tech, low birthrate

Chang is from Taiwan, a high-tech island responsible for assembling some of the world’s top information- and communication-technology products.

It’s also a place where the birthrate has slipped to one of the lowest in the world, as a high cost of living prompts more Taiwanese to put babies on hold in favor of careers.

The Population Reference Bureau reported that the 2015 fertility rate – the average number of babies that women have during their childbearing years – was 1.2 in Taiwan, behind the Asia average of 2.2 and world average of 2.5.

The Taiwanese government has enacted measures like childcare subsidies and free day care, but to little effect.

Chang’s team at Yunlin University of Science & Technology hopes that baby translation can play its part to reverse the decline, which Taiwan’s president has declared a matter of national security.

“There are many ways to apply the program so that it’s even more helpful for parents,” notes Chang.

Developers are working with the government to incorporate the baby translator in a mobile suite for expectant mothers, which is aimed at making parenthood less intimidating.

They are also working with a leading Taiwanese original equipment manufacturer to create a bedside appliance that lets parents monitor their baby from afar.

The device, slated for release this summer, has a built-in microphone that can detect crying and then automatically switch on the translation program.

The result is beamed to the parents, who can come running with exactly what the child needs. “Even if parents are busy and cannot be right beside the baby, they can know how their child is doing,” says Chang.

Babies without borders

Chang says that while developing the app, a surprise finding has been that babies born in different countries sound mostly the same – at least in the earliest stage of life.

“Newborn babies of different nationalities cry in mostly the same ways,” he says, adding that the same goes for male and female newborns.

Chang’s team is convinced that a reliable baby reader has an important place beyond Taiwan and in the global market.

Of course, the world is full of experienced parents who, even on a bad day, could outperform the app. But for first-time parents of any nationality, the app could offer some clarity and relief when faced with an inconsolable newborn.

“From my own experience as a father, I know that sometimes when the baby cries, the parents feel a bit like crying, too,” says Chang. “Humans have emotions and they make mistakes. The app doesn’t get flustered. It simply reads the data.”

This article first appeared in Taiwanese daily The China Post.