Logo of Phnom Penh Post newspaper Phnom Penh Post - Robotic ‘Assistant’ chats in Khmer



Robotic ‘Assistant’ chats in Khmer

Content image - Phnom Penh Post
Srun Channareth, an electronics engineering professor and NPIC faculty member, teaches his students. Heng Chivoan

Robotic ‘Assistant’ chats in Khmer

The man says “chhum reap sour” (hello) and the machine replies hello. Then he says “chhum reap lear” (goodbye) and the robot answers “lear heoy” (bye) . When the man says “Cambodia” the robot tells him “I love Cambodia”.

The conversation was between a small box-like robot with the Cambodian national flag printed on it and the words “National Polytechnic Institute of Cambodia” (NPIC) on its 3.2 inch screen and one of its student-creators from the institute.

“We want to create a Khmer automatic speech recognition system that can turns spoken Khmer spoken into a written Khmer transcript generated by itself without using the internet,” Ny Virbora or Bora, lead student on a team with three other members, told The Post.

Since they study software programming related to artificial intelligence (AI) their professor recommended that the four-member group – Cheat Chea, Sokheang Ching, Ny Piseth and Ny Virbora – work together to create a robot that recognises Khmer speech.

‘Assistant’ gets entry level job

The robot – named “Assistant” – is only 22cm in height and works automatically when the power is on. It can communicate in Khmer through the Khmer Automatic Speech Recognition System using Carnegie Mellon University’s (CMU) Sphinx speech recognition system with a limited dataset.

“The robot can transcribe speech to words and can reply to some specific questions from us. Therefore, we interrogate the robot in Khmer and it replies in Khmer with scripts on screen,” says Bora, who studies electronics engineering.

The team is considering marketing the robots to private companies or corporate clients for customer service purposes but they also believe it could be developed for the education sector and that has been the group’s initial focus.

“We can use automaton in kindergarten and primary school levels while it can handle some inquiries, display pictures and videos to students,” said Bora.

Assistant uses noise reduction technology to help it understand what is being said to it and its limited dataset includes 85 speakers and 157 words selected from the Khmer language that it uses to create sentences when replying to people.

To evaluate the speech recognition accuracy, 100 Khmer transcripts were randomly created from the training dictionary and used for calculating the word and sentence error rate, according to the research team.

Content image - Phnom Penh Post
Ny Virbora, lead student on the four-person team that has created a robot which can turn Khmer speech into Khmer writing posted to its screen with no internet use. Heng Chivoan

“The recognition accuracy for Khmer speech was up to 89.91% of word recognition accuracy and 90.02% of sentence recognition accuracy,” Bora said.

He said the ability to reply to inquiries depends on the database used to train Assistant. Train in this context means a set of software used for training scripts and acoustic models for creating a speech recognition model for any language by providing sufficient acoustic data that can run with the CMU Sphinx speech recogniser.

“This automaton is not that different from a toddler. To teach young kids to recognise words, firstly we need to show that word and we speak the word to him several times. But the machine is unlike a human in that if a man talks to it then it would recognise that man only. So we need to input more data to make it efficiently recognise a wide range of human voices,” Bora said.

“Voices from more than 80 people were used to train this robot to recognise various accents. If you train with only one person’s accent then the automaton of course recognises only that person,” said Srun Channareth, a professor at National Polytechnic Institute of Cambodia and the Department of Electronics Master of Engineering who led the “Assistant” project.

‘Assistant’ finds success at work

Though Assistant is only knows 157 words, Nareth said, the accuracy rate with them is higher than 90%.

“For this model, it is considered our first success because we can trust that it will get it right. The tested result for accuracy and the ability to transcribe speech to text is at 90% and the ability to use its set of words is at 89% because sentences made up from those words can be confusing sometimes,” said Bora. He cited the results of Microsoft’s research, which estimates that a robot with 90 per cent accuracy can be used in the field.

His team is now beginning to recruit volunteers to help integrate more data into the robot to enhance its use of words and phrases.

“As for the number of words, it depends on the location or sector where the robot will be used,” Bora said. “If we use it for the education sector, we’ll see what words need to be used by looking at textbook lessons. We only include Khmer words, as they will be transcribed into sentences.”

“What’s special about our robot is that it works offline without using the internet. Working offline means that it runs faster and it is local computing that processes the data in the robot without having to connect to a server,” said Bora, 22.

The next step is making a larger database because Assistant now stores a limited set of data and increasing the size of its database will take time.

“From the beginning of the project to the first phase robot, our team spent more than a year, because we needed time to do a lot of research on intelligent technology (AI), even if the robot design itself was effortless,” said Bora.

In terms of hardware materials, the team only had to spend about $200 since the robot’s body was 3D-printed by the university’s own 3D-printer.

‘Assistant’ gets a promotion

“Soon, we will create a new robot model and develop new software that acts as an assistant for use by companies and restaurants as well as customer service locations,” Bora said. “The new one will be smaller and more portable, but with an expanded screen size of up to 7 inches.”

The robot can act as an information source for customers without them having to rely on human services or contact. For instance, at the airport passengers could ask the robot to show them the scheduled flights from Phnom Penh to Japan. Once the recognition system for the Khmer language is fully developed its output can then be translated to other languages as well.

In order for the robot to function properly, Bora added, there are still two major stages left. First, updating the software to make the system run faster, and second, increasing its ability to recognise words.

“We will also equip it with a camera so that it has the ability to recognise people and be able to call them by their names directly,” said Nareth. “We have a lot of projects, but what we lack is support for further research.”

‘Assistant’ goes back to school

Bora told The Post that training robots to recognise words is very difficult as the first step is to create a database that includes both voice and text data.

“In the coding system, we cannot write in Khmer directly because the computer does not know Khmer script,” he said. “So we use the Latin alphabet represented in Khmer as a unicode format.”

Content image - Phnom Penh Post
The ‘Assistant’ robot can communicate in Khmer through the Khmer Automatic Speech Recognition System using Carnegie Mellon University’s Sphinx software and a limited dataset. SUPPLIED

It took the team almost half-a-year to create their initial database since they had to collect the data and create a list of words. But for the new words being added, Bora says they have written software that speeds up the data collection and makes it less time consuming.

“Before, there were 10 of us. We collected data and that took us up to two days, but then with the software I wrote, I spent just over an hour on it and we collected from 10 to 20 words,” he said.

The team struggles with both hardware and software issues and bridging the gap between theory and practice. They can design robot bodies but it can be a hassle to produce real-world parts from computer-aided designs.

“We need to study theories and lessons related to the development of AI and speech recognition, but when we put all those theories into practice, obviously, the practice is not the same as in the theory at all, so we need more time to experiment,” Bora said.

Khmer-supported software takes a long time to get started, although there are some theory books and software support available now. But they cannot use other countries’ software completely because they want what they are doing to be accessible to Cambodians who only know the Khmer language.

“Whether or not there is a sponsor,” he said, “it is important that we focus on promoting strategies for further research. Although we cannot afford to buy supplies and materials the school can help some. Not only that, teachers or professors can also help find additional funding.”

‘Assistant’ explores new careers

Prof. Channareth said that if a company develops an advanced enough robot that can assist in communicating with customers or answering questions, they could use them to replace human workers and thereby save money and increase efficiency.

Industries from telephone companies to banks to restaurants and cafes or customer service providers can benefit from this approach and in fact already are in some parts of the world.

“When we enter a bank now, they have a machine that requires customers to press the number and choose the type of service they need and it then puts them in line to speak with a human teller to receive that service. What if that first machine they interacted with could just provide the service?

“However, we are mainly focused on educational applications that will be used by donors. Through working with charities the students can strengthen their programming skills and write code and become skilled in electronics engineering while capably contributing to the donors’ companies,” he said.

MOST VIEWED

  • ‘Education’ a priority traffic-law penalty

    A top National Police official on June 21 neither rejected nor confirmed the authenticity of a leaked audio message, which has gone viral on social media, on a waiver of fines for a number of road traffic-related offences. General Him Yan, deputy National Police chief in

  • Pursat Ford assembly plant opens

    The Kingdom’s first Ford assembly plant was inaugurated on June 16 in Pursat province amid rising demand for brand-new vehicles among Cambodians. The facility is seen as a game changer for the domestic automobile industry, which could bring a wave of investors seeking to cash

  • Volunteer scheme to foster ‘virtuous’ humanitarian spirit

    A senior education official said volunteer work contributes to solidarity and promotes a virtuous humanitarian spirit among the youth and communities. Serei Chumneas, undersecretary of state at the Ministry of Education, Youth and Sport, made the comment during the opening of a training programme called “

  • Siem Reap’s $18M zoo said to educate public, help wildlife

    Angkor Wildlife and Aquarium Co Ltd has invested $18 million in a zoo in Siem Reap province, which will be opened in October to educate and promote animal conservation as well as attract national and international tourists. Currently, the Angkor Wildlife and Aquarium is building the

  • $50B infrastructure plan en route

    The government’s upcoming $50 billion,10-year infrastructure master plan will provide tremendous investment opportunities for domestic and foreign entities, transport experts and economists say. Minister of Public Works and Transport Sun Chanthol revealed the plan to Japanese ambassador to Cambodia Masahiro Mikami on June 15. At

  • Chinese firms unveil preliminary results on metro, monorail for capital

    Minister of Public Works and Transport Sun Chanthol and representatives from China Road and Bridge Corp (CRBC) and its parent company, the state-owned China Communications Construction Co Ltd (CCCC), met on June 24 for talks on results of the firms’ preliminary study on a potential metro