Audio Analytics: Vital Technology for Autonomous Vehicles

Artificial Intelligence (AI) and Machine Learning (ML) are projected to play a major role transformation of the Automotive Industry by designing future-state autonomous vehicles enabled by AI and ML. With the advancement of supply chain management, manufacturing operations, mobility services, image and video analytics, audio analytics, next-generation autonomous vehicles are poised to transform the automobile’s consumer perception. As these technologies continue to develop, the autonomous automotive industry is positioned to reach a global market size of nearly 60 billion USD by 2030.

Audio Analytics under Machine Learning in driverless cars consists of Audio classification, NLP, voice/speech, and sound recognition. Voice recognition, in particular, has become an integral part of autonomous vehicle technology providing enhanced control for the driver. Up until this point, in the traditional models of cars, speech recognition was a challenge because of the lack of efficient algorithms, reliable connectivity, and processing power at the edge. Further, in-car cabin noise reduced the performance of the audio analytics, which resulted in false recognition.

Audio analytics in machines has been a subject of constant research. With technological advancement, new products are coming online like Amazon’s Alexa and Apple’s Siri. These systems are rapidly evolving through cloud computing technology, a tactic that other recognition systems lacked previously. 

Recently, various Machine Learning algorithms like kNN (K Nearest Neighbour), SVM (Support Vector Machine), EBT (Ensemble Bagged Trees), Deep Neural Networks (DNN), and Natural Language Processing (NLP) have made Audio Analytics more effective and better positioned to add value to autonomous vehicles.

In audio analytics, data is pre-processed to remove the noise, and then the audio feature will be extracted from the audio data. The audio features such as MFCC (Mel-frequency cepstral coefficient) and statistical features like Kurtosis and Variance are used here. The frequency bands of MFCC are equally spaced on the Mel scale, which is very close to the human auditory system's response.  Finally, the trained model is used for inference, a real-time audio stream is taken from the multiple microphones installed in the car, which is then pre-processed, and the features will be extracted. The extracted feature will be passed to the trained model to correctly recognize the audio, which will help make the right decision in autonomous vehicles.

Data Processing & ML Model Training

With new technologies, end user’s trust is the key point, and NLP is a game-changer to build this trust in autonomous vehicles. NLP allows passengers to control the car using voice commands, such as asking to stop at a restaurant, change the route, stop at the nearest mall, switch on/off lights, open and close the doors, and many more. This makes the passenger experience rich and interactive.

Let's take a look at a few Applications where audio analytics provide benefits to autonomous vehicles.

Emergency Siren Detection

The sound of the siren of any emergency vehicle such as an ambulance, fire truck, or police car can be detected using the various deep learning models and machine learning models like SVM (support vector machine). The supervised learning model - SVM is used for classification and regression analysis. The SVM classification model is trained using huge data of the emergency siren sound and non-emergency sounds. With this model, the system is developed, identifying the siren sound to make appropriate decisions for an autonomous car to avoid any dangerous situation. With this detection system, an autonomous car can decide to pull over and give away for the emergency vehicle to pass.

Engine Sound Anomaly Detection

Automatic early detection of a possible engine failure could be an essential feature for an autonomous car. The car engine makes a certain sound when it works under normal conditions and makes a different sound when it is exhibiting problems. Many machine learning algorithms available among K-means clustering can be used to detect anomalies in engine sound. In k-means clustering, each data point of sound is assigned to the k group of clusters. Assignment of the data point is based on the mean near the centroid of that cluster. In the anomalous engine sound, the data point will fall outside of the normal cluster and be a part of the anomalous cluster. With this model, the health of the engine can constantly be monitored. If there is an anomalous sound event, then an autonomous car can warn the user and help make proper decisions to avoid dangerous situations. This can avoid a complete breakdown of the engine.

Lane Change on Honking

For an autonomous car to work exactly as a human-driven car, it must work effectively in the scenario where it is mandatory to change its lane when the vehicle from behind needs to pass urgently, indicated with honking. Random forest, a machine learning algorithm, will be best suited for this type of classification problem. It is a supervised classification algorithm. As its name suggests, it will create the forest of decision trees and finally merge all the decision trees to accurately classify. A system can be developed using this model, identifying the certain pattern of horn and taking the decision accordingly.

NLP (Natural Language Processing) processes the human language to extract the meaning, which can help make decisions. Rather than just giving commands, the occupant can actually speak to the self-driving car. Suppose you have assigned your autonomous car a name like Adriana, then you can say to your car, “Adriana, take me to my favorite coffee shop.” This is still a simple sentence to understand, but we can also make the autonomous car understand even more complex sentences such as “take me to my favorite coffee shop and before reaching there, stop at Jim’s home and pick him up.” Importantly to note, self-driving vehicles should not obey the owner’s instructions blindly to avoid any dangerous situations, such as dangerous, life-threatening situations. To make effective decisions in dangerous situations, autonomous vehicles need a more powerful NLP which actually interprets what humans have told, and it can echo back the consequences of that.

Thus, machine learning-based audio analytics is attributed to the increasing popularity of autonomous vehicles due to the safety and reliability enhancements. As Machine Learning continues to develop, more and more service-based offerings are becoming available that offer such services as audio analytics, NLP, voice recognition, and more, enhancing passenger experience, on-road safety, and timely engine maintenance automobiles.