That interpretation app has been upgraded! We asked the developer about it!

Hello everyone who reads the Kodensha blog!

I'm not very good at science!
This is the blog of Kodensha's Development Office, where our sales assistant, Satake, who is not familiar with technical terms, interviews our technical developers and presents them in a conversational style!

This is a blog where I try to provide you with information about our service development, releases, and various other information in as easy-to-understand a manner as possible, while learning about technology myself! This blog is to provide you with information on our service development, releases, etc., in as easy-to-understand a manner as possible, while also learning about technology myself!

Thank you for your patience! The theme of the second issue is here☆
"That interpreter application has been upgraded!

That's right, our real-time voice interpretation application "My Interpreter Assist" has actually been upgraded to an even easier-to-use version!

And here's the app icon for "My Interpreter Assist" ↓.

The icon is quite cute.

Now, today, we have this person!
He is a super active person who planned an overseas trip the day before his departure and went to Taiwan on short notice!
We will interview Mr. Shibata of Development Section 3 about his development episodes.

Let's start with an interview with Mr. Shibata at -----!　But first, a brief explanation of what "My Interpreter Assist" is.

What is the real-time voice interpretation application "My Interpreter Assist"?

Under the Japanese government's new tourism vision of "increasing the number of foreign visitors to Japan to 40 million by 2020," local governments, lodging facilities, restaurants, retailers, and other businesses are vigorously implementing inbound tourism measures.
In Osaka and other parts of Kansai, we often see many foreigners enjoying sightseeing.
In such a situation, one of the most common problems that foreign tourists have during their trip is "difficulty in communicating with staff at facilities, etc." (Reference: Japan Tourism Agency)
(Reference) Japan Tourism Agency The issues of multilingual display and communication for each travel situation have been clarified.
-A questionnaire survey was conducted among foreign visitors to Japan regarding the acceptable environment for multilingual displays and communication.

We have prepared a multilingual website as well as free Wi-Fi environment in our facilities.
But we are having a hard time communicating with customers onsite..."

We are sure that you have many problems like this.
This communication problem is a problem for both foreign visitors to Japan and those who accept them.
My Interpreter Assist" solves such problems.

Simply place a tablet or other terminal with the real-time voice interpretation application "My Interpreter Assist" installed at the reception desk or in a storefront, and speak into the terminal!

When you speak in a foreign language, it is translated into Japanese and then spoken in Japanese.
When you reply in Japanese, it is translated into the foreign language and spoken in the foreign language.

This is a very convenient and easy-to-use application that recognizes speech and enables instant translation and voice output.

Improvement of 'that point' of the previous version!

How has this "My Interpreter Assist" been upgraded?
I'd like to get down to business.

Satake: Mr. Shibata, please give us a hand! （Translator.)

Shibata: Nice to meet you.

Satake: You seem nervous, but please relax and talk to me!
Now, I would like to start the interview! First of all, please tell us why you decided to upgrade the real-time voice interpretation application "My Interpreter Assist V2".

Shibata: Well, first of all, due to the increase in the number of foreign visitors to Japan in recent years, we wanted to differentiate ourselves from other applications in the current situation where there are various similar applications.
We also wanted to offer benefits to customers who have already installed and are using My Interpreter Assist that would outweigh the cost.

Satake: I think both of these points are very important for customer satisfaction.

Shibata: We also wanted to propose products that can be used in a variety of situations and applications that each customer wants to use, by adding functions that can flexibly meet niche demands.

Satake: So this upgrade was made with usability as the top priority. Now, please tell us about the improvements compared to the previous version of My Interpreter Assist.

Shibata: Let's start with the layout.

Satake: Compared to the previous version, there are more templates!

Shibata: Yes, I will explain this "canned scenario function" later.

Three functions have been renewed! More Convenient and Easy-to-use!

Satake: We have just confirmed the changes in the interface layout compared to the previous version, but what exactly has been upgraded in terms of functions in this version?

Shibata: I will pick up three major features and explain them.
Canned text function
Canned scenario function
3. Dedicated interface screen
These three are the main points of this upgrade!

Satake: Hmmm...first of all, please explain the first one, the "canned text function!

(Point 1) What is the "Canned Sentence Function"?

Shibata: In each conversational situation, there are always phrases that occur frequently. For example, think of a customer service scene at a restaurant. Beginning with "Irasshaimase" (Welcome!), followed by "Would you like your food here?　The bill is ____ yen. and "The bill is ________ yen.
A canned sentence is a text data set that contains a bilingual translation of these phrases into Japanese and a foreign language by human translation. ₁

Satake: So the bilingual Japanese and foreign language translations of a single sentence are called canned sentences?

Shibata: That's right. You can call up the registered canned sentences by pressing a button or by speaking the sentence into the terminal. Then, the registered translation in the foreign language is read out loud by the voice synthesizer, which acts as an interpreter on the spot.

Satake: It is safe to register canned phrases, from phrases that are expected to be used frequently to phrases that are used in case of emergency.

(Point 2) What is the "Canned Scenario Function"?

Shibata: The second point is the "canned scenario function.

Satake: "Canned scenario function," what kind of function is this?

Shibata: It is a function that allows you to call up a flow of standardized sentences that match the conversational scene at the touch of a button.
In the customer service scene I mentioned earlier, the order in which phrases such as "Welcome" and "Are you ready to eat here?
The "canned scenario" function allows the application to memorize this order as a single scenario and call up one canned phrase after another according to the scenario.

Satake: That is a very useful function! Since the phrases of the conversation are already registered in the "Canned Sentence Function," all you have to do is answer the questions that need to be answered, and the conversation is easily composed.

Shibata: Just by pressing the "Next" button on the screen, the next line of dialogue is read out, so high performance in niche situations can be expected. Of course, it is also possible to swap the order of conversations.

By preparing dialogue scenarios in advance, you can easily obtain answers from the other party.
For example, if you are looking for a lost foreigner, you can ask for his/her name, gender, age, clothing, etc., which will eventually result in a sentence in a foreign language.
The tablet can also be connected to broadcasting equipment to make announcements through the museum's speakers.
In addition, evacuation information can be smoothly conveyed to foreigners who are likely to be vulnerable to disasters in the event of a large-scale earthquake or other emergency.

Satake: It is also simple and easy for anyone to use, as all they have to do is answer the questions to complete the text they want to guide.

(Point 3) What is the "Dedicated Interface Screen"?

Satake: The third point of upgrade, the "dedicated interface screen," refers to the dedicated screen for displaying standardized sentences?

Shibata: Yes, it is. It is the screen below where the female driver apologizes for the delay.

Shibata: We made it so that canned messages are displayed in an easy-to-understand manner on a dedicated screen, rather than on the main list screen. Not only text, but also images and videos can be displayed.

Shibata: Originally, this function was created for inputting answers to canned question-type texts, but you see, even if a question is displayed at the bottom of the list screen, foreign customers who use this application for the first time would not know how to operate it and answer it.

Satake: And if a child gets lost or loses something, they may not have the time to calm down and operate the app.

Shibata: We had such concerns, so we thought that if the questions were displayed on a single screen, and if there were buttons and pull-downs on the same screen to answer the questions, anyone would be able to operate the system intuitively, which is how we came up with the upgraded screen.

Satake: This would certainly make it clearer what to answer and how to operate the system!

Shibata: We also thought it would be easier to convey the message to the recipient if we displayed not only the text but also images and videos that correspond to the content of the guide, so we made it possible to display all canned messages on a dedicated interface screen.

Satake: By showing illustrations and videos appropriate to the situation, it is easier to get the message across as visual information!

Shibata: For example, we can show a video of a receipt coming out with the message "Please pick up the receipt" when paying for a cab.

Satake: I think the specifications have been made very user-friendly, with an emphasis on the user's point of view!
I think there were some aspects that were difficult to complete. Please tell us about those aspects.

Behind-the-scenes development

Satake: Did you make any innovations throughout the development of this project?

Shibata: It is not limited to this time,
How to make it simple to incorporate.
I always try to put myself in the user's shoes so that people who are using the product or service for the first time can use it without difficulty.
I always keep this in mind.

Satake: I'm not very familiar with the operation of machines like this... (laughs) (laugh) I think that simplicity of use is an important point.

Shibata: Yes, we had to be aware of this point because the service is designed for foreign customers. You can also use a smartphone to input information, so you can deal with cases where recognition did not go well.

Satake: I heard that the development was done while maintaining compatibility with the previous version. Were there any difficulties during development?

Shibata: Well, we wanted to not only incorporate new functions, but also to be able to use the functions of the previous version as before, so we had to find a way to make it simple to do both. We went through a lot of trial and error.

Satake: I see! What are the main features of the new version, or what features do you recommend with confidence?

Shibata: All of them! I'd like to answer that, but if I had to narrow it down to one feature, it would be the "dedicated interface screen. I think the ability to respond to questions with buttons and pull-down answers is significant.

Satake: Compared to the previous version, I also found this point very convenient. It is also easier to understand which questions to answer!

Shibata: Yes, I think it can be used in a wide range of situations where many foreign customers use or visit, such as reception at hotels and inns, information broadcasts at department stores and amusement parks, interactions in buses and cabs, tourist information centers and information centers, and medical institutions.

Satake: I heard that there is also a "fill-in-the-blanks" function!

Shibata: Yes, it is a function for inputting canned sentences with blanks.
A standard sentence with a blank space is "The bill is ________ yen. XX is lost. The "○○" part is very important. The "XX" part is very important. Of course, if you don't input the "XX" part, the above canned messages will be useless, right?

Satake: Yes. Then how do you complete the sentences (canned sentences)?

Shibata: Using the canned scenario function I mentioned in the previous article, you simply ask the person the XX part of the question according to the scenario. Conversely, if the person says, "The bill is ____ yen." then you answer the XX part yourself and the sentence is completed.

Satake: I see! This is convenient because I don't have to register many similar broadcasts that differ only in name, time, and location.

Shibata: Also, the possessive is automatically changed depending on what words are entered in the "XX" part.
For example, in a sentence with the blank "XX and his parents," the possessive is automatically changed depending on what word is entered in the "XX" part,
A boy and his parents
A girl and her parents

Satake: Oh~~! I see that the possessive has been changed to correspond to the underlined part.
This makes it grammatically safe to use.

Confidential information? Future Development Plans

Satake: Can you tell us about your plans and plans for future development and version upgrades?

Shibata: I can't give you too many details because it is confidential information (laugh), but we are considering adding more supported languages and developing the software for offline use. We hope to address these two points in the near future.
Also, we are considering support for mirroring operation with multiple terminals (i.e., sharing the same screen on multiple terminals).

Satake: Thank you for the valuable information! I have received a comment from a customer who said, "It would be convenient if I could use it even without a wi-fi connection." We have received a comment from a customer that "It would be convenient if it could be used even without a wi-fi environment. Please do your best to develop it.

Shibata: Yes, I will do my best!

Satake: Lastly, as a developer, what are your passionate thoughts on this real-time voice interpretation application "My Interpreter Assist V2"?

Shibata: The first version of My Interpreter Assist had the three major functions of "voice input," "automatic translation," and "speech synthesis," but we felt it was a little too much.
The new version makes full use of these features and has been reborn as a more practical, higher-end version.
There is a fine line between "usable apps" and "unusable apps. We believe that with just a little ingenuity and imagination, we can make a difference.
We will continue to incorporate feedback from users who have actually used the system.

Satake: Thank you very much. In this interview, we talked to Mr. Shibata, the developer, about the new real-time voice interpretation application "My Interpreter Assist V2.
We also learned about future development plans, and our expectations are high!

If you have any questions or concerns about the real-time voice interpretation application "My Interpreter Assist" introduced here, please contact us. We are also looking forward to hearing from you about actual operation methods and demos!