What is AutoML? Promises vs. Reality

DataRobot made waves in the machine learning community last week when they raised $54 million in a Series C round, putting the total funding to $111 million. DataRobot is a Boston-based software company working to automate machine learning (AutoML) solutions. Despite the hype surrounding machine learning and artificial intelligence, a lack of quality data scientists and machine learning experts hinder the growth and mass adoption of these technologies. Jeremy Achin, the CEO and Co-Founder of DataRobot, seeks to address the deficit by automating parts of the data science workflow.

Image Credit: Pat Research

This news was met with a mix of overblown optimism and skepticism. Some around the internet predicted that democratization of machine learning was near. On the other hand, an IoT pundit, Stacey Higginbotham, expressed her disbelief: “[This is a] company making a huge assertion, namely that it can automate the development of AI. I don’t really believe this, since the top minds at Google, Facebook, Microsoft, and Baidu are hard at work publishing papers on their incomplete efforts here.”

While I can understand the reasons for such hopes and concerns, DataRobot — and automated machine learning in general — doesn’t promise or claim any of this. To clear up this confusion, we’re taking a look at the current state of AutoML and carefully defining what it is and what it isn’t.

*** For a primer on the differences between AI, ML, and Deep Learning, please read Calum McClelland’s excellent overview ***

What is AutoML?

Before defining AutoML, it’s imperative to distinguish machine learning from data science. As Matthew Mayo from KDnuggets stresses, the difference is not just a matter of semantics. Machine learning, which deals mainly with data modeling (selecting the best algorithm, tuning its parameters, etc.), is part of a larger data science toolkit that encompasses things like data preparation and descriptive analytics, to name a few.

With that in mind, Mayo defines AutoML to be “the automated process of algorithm selection, hyperparameter tuning, iterative modeling, and model assessment.” It is NOT automated data science, nor automated development of artificial intelligence. It is, however, “transforming model building” as DataRobot claims on its website.

Image Credit: DataRobot

Ok, So How is AutoML Useful?

Currently, selecting the “best” algorithm to use per dataset requires a level of intuition or expertise about the data. Data scientists leverage their experience to experiment with different combinations of models and hyperparameter values to achieve the highest accuracy.

AutoML will lessen our dependency on intuition by iteratively trying out an algorithm, scoring its performance, and choosing and refining other models. In other words, it will automate the machine learning process of the data science work flow as we carefully defined above.

DataRobot isn’t alone in making progress in this field. There are other openly available tools such as Auto-sklearn for Python users and AutoWEKA for Weka users. Another tool called TPOT, returns the best performing model with its source code in Python to work with the typical scikit-learn pipeline.

The Future of AutoML

Randy Olson, the lead developer for the TPOT project, expressed his confidence that AutoML will become mainstream and help accelerate the model building process. He was also quick to dismiss the fear of AutoML replacing data scientists, as he stressed that “the purpose of AutoML is to free data scientists from the burden of repetitive and time-consuming tasks (e.g., machine learning pipeline design and hyperparameter optimization).”

And to Stacey’s point about other tech giant’s progress in this field, I would add that AutoML is related, but a separate discipline in the ultimate journey towards better artificial intelligence. The tech giants are more focused on improving their deep learning architecture. One could argue that AutoML can be generalized to help pick out the best deep neural network architecture and hyperparameter tuning, which is a much harder problem than what AutoML solves with non-deep learning networks.

It seems to me that the tech giants’ focus is to embed a sense of memory in these machines to free them from specialized tasks, as shown by recent DeepMind papers (e.g. Elastic Weight Consolidation or Neural Episodic Control). In essence, these efforts highlight a trend towards general AI so that machines can remember what they learned and apply to new situations. If achieved, that could re-define AutoML as it would be the automated process of machines learning. But for now, AutoML takes on a stricter definition, which still has plenty of promise nonetheless.