The Future of Smart Devices with Natural Language Processing

The Internet of Things and artificial intelligence are deeply connected. IoT systems produce big data, whereas, data is the heart of AI and machine learning. At the same time, as the rapid expansion of connected devices and sensors continues, the role of smart technologies in this space is growing too.

Today, the applications of computer intelligence in IoT products vary. In this article, I’d like to focus on a specific domain of AI – Natural Language Processing.

The Power of Speech Understanding

One of the core concepts of Natural Language Processing is the ability to understand human speech. It would be simply impossible to implement voice control over different systems without NLP.

In IoT, it’s particularly difficult to overestimate the value of speech recognition. Hands-free voice interfaces promise rich benefits to IoT environments. In some cases, it’s just a matter of usability - the more complex a system is, the harder it is to implement a user-friendly mobile or web interface to control it. Voice interface, in turn, is intuitive by its nature and doesn’t require a serious learning curve.

Sometimes, voice interface isn’t just about usability, but also about safety. Imagine a technician who works on 150 ft. high power lines and, instead of manually, gives voice commands to digital tools, or people who can manage devices while driving without using their hands.

In consumer markets, the popularity of voice control is also growing. About 50% of US households use voice to access online content. Therefore, increasing the amount of smart consumer electronics activated by voice becomes a natural step of technological evolution.

Moreover, NLP allows us not only to integrate voice understanding into devices and sensors. Thanks to machine translation capabilities, it enables localization features. And with the level of market globalization we experience today, localization goes even beyond translation and unlocks the benefits of transcreation (creative translation).

Machine translation is priceless for any IoT product with enabled speech recognition, if the product is focused on cross-country distribution. However, translation functionality is not less valuable on its own.

For example, Tokyo-based startup ili created a wearable that can translate simple common phrases for travelers without access to the Internet. Pilot is another connected device that enables translation on the go. Unlike ili, it facilitates a two-way conversation; not only does Pilot understand various languages, but also can synthesize a relevant response in a foreign language.

Image credit: ili

Indispensable Assistants

Speech recognition goes hand in hand with the other NLP concept - question answering. The term is pretty much self-explanatory. Question answering tasks allow us to determine answers to the questions given in a natural language.

Nowadays, more and more devices enabled with speech recognition use question answering to provide feedback to user input. The most common examples are popular home assistants, such as Amazon Alexa, Google Home and Apple HomePod. These devices are both activated and controlled by voice and able to answer various questions. As a result, voice assistants enhance user productivity as they help people get relevant information quickly on the go.

Question answering has an even higher value when it comes to industries such as healthcare. Thomas Jefferson University Hospital has put this idea into practice and in cooperation with IBM Watson IoT created the environment were patients can manage a smart concierge in their rooms using natural speech. The digital concierge is able to answer questions and even adjust environment conditions such as light and temperature based on patients’ preferences.

This example demonstrates how the capabilities of digital assistants go way beyond curiosity and usability as they enable tangible help and contribute to human comfort.

There’s one more NLP concept behind question answering – information retrieval. Using this NLP task, systems can extract relevant information from different text sources such as scientific papers, documents, and feeds. Moreover, machine learning can enhance this functionality and further work on the retrieved information – analyze, determine correlations and patterns, find anomalies fast and efficiently.

In the IoT space, combining NLP and machine learning allows intelligent devices to give relevant answers. The connected car is a truly noteworthy example in this case. Thanks to improvements in NLP and machine learning, the automotive landscape is changing fast and providing drivers with smart navigation, strong safety features and voice controls for cars.

Understanding Emotion

The ability to understand text is a treasure by itself, but human speech is much more complicated than plain text. Usually, when we talk, we enrich our speech with emotion. Using sentiment analysis, also known as emotion AI, devices can detect emotionality and better understand the context.

Sentiment analysis helps us identify, extract and study subjective information such as the speaker’s emotional reaction. For example, IBM Watson API for sentiment analysis allows developers to build the systems able to identify agreeableness, conscientiousness, extraversion, emotional range and openness in natural language.

These capabilities unlock a whole new space for smart devices across industries. First of all, sentiment analysis is a great tool for marketers. We know how to save our words, but emotions are much harder to hide. Analyzing emotional reactions to products, marketers can make data-driven conclusions on their success and failures.

Today, this approach is successfully implemented by Disney Research. The company is planning to use sentiment analysis combined with computer vision to understand how people react to movies.

Image credit: Disney Research

Similar technology paired with NLP could also enhance smart home environments. With sentiment analysis, connected systems could understand user reactions to the news, music or any other service controlled by intelligent home devices.

Customer service can take advantage of sentiment analysis too. It may even eliminate ineffective surveys. Instead, a smart concierge can ask customers a couple of questions about their experience and determine their level of satisfaction automatically.

In general, these features can both create a competitive advantage for businesses and enable personalization of products and services for customers. Moreover, thanks to sentiment analysis and trend monitoring, various connected devices can finally find answers and offer the products and services consumers need and want.

As you see, the Internet of Things connects not only things, it connects technologies. Imagine a world where devices work in tandem with humans, understand their queries, feel their needs and provide relevant responses. At this point, a world like that is only possible with improving artificial intelligence and NLP – the technologies that enable contextual understanding and allow smart devices to actually solve our problems.