“Hey Siri, set an alarm for 8:00 am.” “OK Google, call my dad.” In the smartphone era, it’s common to see people talking to their computers and computers assisting users. Computers now play an important role in enriching people’s lives. What kind of user experience awaits when this sort of communication between computer and human expands to the automotive world, especially the world of autonomous driving (AD)? Drivers and passengers will talk to their cars and the cars will respond, making journeys much more comfortable. That’s how we see the future.
Pursuing this aspiration, we’ve developed a new AD interface based on the concept of “Cars That Think and Talk.” The CT3 interface enables cars to think and interact with humans by leveraging advanced artificial intelligence (AI) technologies, including the formidable GPT-3.5/4 large language models (LLM), automatic speech recognition (ASR), and text-to-speech (TTS).
Drivers are used to operating interfaces with physical buttons, switches and displays. With CT3, we can do this operation in a simpler and more flexible way through dialogue, eliminating the need for such physical operations. Drivers and passengers can also enjoy natural interactions with vehicles because wake-word and end-of-speech detection have reduced the response time of the LLM. Avatars will also help to make the communication more intuitive. Information about vehicle status and the driving environment can be monitored in real time via the open-source AD software Autoware*, giving drivers and passengers peace of mind.
CT3 is already being used by our JPNTaxi vehicles in AD tests underway in Tokyo’s Odaiba district. Instructing a CT3-equipped taxi to “Set location X as the destination” is all that’s needed to start the journey. The car sets off without the need to manually input travel details on the in-car display.
In addition to the safety benefits of AD technology, which eliminates the risk of driving fatigue from hours spent concentrating behind the wheel, the introduction of natural interactions with vehicles stand to further enhance the user experience and add another level of enjoyment to autonomous driving journeys.
“We believe CT3 is an interface that has the potential to change AD vehicles from mere means of transportation to a more human-friendly and familiar presence,” said Tomoki Hayashi, an engineer from TIER IV subsidiary Human Dataware Lab. Tomoki participated in the 4th Generative AI Study that was held in August this year and gave a presentation titled, “Building a dialogue agent for autonomous driving cars using ChatGPT”. He is one of the key members involved in the development of CT3. “We want to create an experience where people can enjoy not only conversations, but also things that are automatically suggested and executed by the car as it constantly monitors the user’s condition.”
Makoto Yabuta, an engineer in charge of tools at TIER IV, thinks it’s important to develop the technology with a focus on the user’s perspective. “I think there are many situations in which people feel uneasy about autonomous driving. To solve this problem, it’s necessary for us developers to understand the users, for the users to understand the car, and then, based on mutual understanding, for the users to trust the car. CT3 will help us do that,” he said. “I believe it will be an application that can give people peace of mind, creating a comfortable riding experience through dialogue.”
Yoshihiro Saito, an engineer in charge of architecture at TIER IV, said, “Currently, users can specify where they want to go. In the future, CT3 will be able to select a route automatically based on natural interactions, like ‘I want to stop at a nearby convenience store on the way to my destination.’ I’m very excited to be able to provide a driving experience that is more friendly and personalized with the CT3 user interface.”
Shinpei Kato, founder, CEO and CTO of TIER IV, described another benefit of the new interface, “CT3 being tested by taxi drivers for things like route instructions. But the essential value of this technology is that it will allow drivers and passengers to give directions to cars in situations that are difficult for Autoware to handle. For example, if an AD maneuver to avoid a stationary car directs your vehicle toward the opposite lane, users will be able to give complementary instructions as to whether it is safe to do so. You may have heard the passenger say ‘FIVE’ in the video, but we’ll share what this is in the near future, so stay tuned!”
We aim to continue research and development as well as evaluation and verification of CT3 to make this technology even more enjoyable for users. TIER IV believes this technology will be a major step forward in redefining the relationship between cars and users. Of course, We will share CT3 as our open-source software. We will continue to contribute to improving the safety of autonomous driving and promoting its implementation in society.
*Autoware is a registered trademark of the Autoware Foundation.
Join Our Open-Source Journey
We’re looking for people who share our goal of reimagining autonomous driving with the art of open source software and are passionate about making it happen together.
Media Contact
pr@tier4.jp
Share the post
LinkedIn | X | Facebook | Instagram