What is Google Cloud ML Engine?

The cloud and machine learning: two phrases with a lot of hype that few people understand. We're intimately familiar with both here at Leverege, so hopefully this article will shed some light on the two topics.

Before we share what we've learned using Google Cloud ML Engine, we need to do a quick refresher on how machine learning is done in production. There are roughly 4 steps:

1. Identify the problem you want to solve. For instance, you want an algorithm that can classify emails as spam or not, so that your inbox isn’t flooded with junk mail.
2. Find/build a machine learning model that solves the problem. Using the above example, you might try an ML algorithm called ‘Naive Bayes’.
3. Train your ML model. All machine learning models need to be trained before they can be used. You’ll need to feed your ‘Naive Bayes’ model examples of spam/legitimate emails so they can learn how to classify new emails.
4. Deploy your ML model. Now that you have a trained ‘Naive Bayes’ spam filter, use it to filter new emails.

[caption id="attachment_7730" align="aligncenter" width="411"]

Training

With Cloud ML engine, you can train your ML model in the cloud using Google’s distributed network of computers. Instead of just using your laptop to train your model, Google will run your training algorithm on multiple computers to speed up the process. Furthermore, you can configure the types of CPUs/GPUs these computers run on. There are some algorithms that run a lot faster if you use GPUs instead of CPUs.

[caption id="attachment_7731" align="aligncenter" width="764"]

Another benefit we've found of training with Cloud ML Engine is that you don’t have to worry about storing the training data. If you have a million emails to train your spam filter, how are you going to get them on your laptop to train your model? When you train your model using Cloud ML, you can easily store your training data online in a Google Cloud Storage “Bucket”.

[caption id="attachment_7732" align="aligncenter" width="682"]

The steps involved with building a machine learning model in TensorFlow and packaging your code so that Cloud ML Engine can process it are a bit complicated and beyond the scope of this high level overview. You can start learning about the details here.

However, when you’ve built the model and are ready to train, submitting the training job to Google Cloud ML Engine is just a quick shell script command:

gcloud ml-engine jobs submit training $JOB_NAME \      --job-dir $OUTPUT_PATH </span>       --runtime-version 1.2 </span>    --module-name trainer.task </span>    --package-path trainer/ </span>    --region $REGION </span>    --scale-tier STANDARD_1 </span>    -- </span>    --train-files $TRAIN_DATA </span>    --eval-files $EVAL_DATA </span>    --train-steps 1000 </span>    --verbosity DEBUG  </span>    --eval-steps 100

The important part to understand is that trainer.task is the file that contains your TensorFlow application and STANDARD_1 specifies that you want to use multiple computers (distributed computing) to train your model.   

Deployment

One thing you could do is build a backend web server, like www.my-spam-filtering-api.com (not a real website) that takes in emails as POST requests and responds with the ML algorithm’s guess about whether or not the email is spam. But what if your backend web server gets tens of thousand of requests? Will it be able to handle the load or will it break?

Google Cloud ML let's you avoid the complexity of building a scalable machine learning web server by doing it for you. If you put your machine learning algorithm on Google Cloud ML, it can handle predictions for you. Right now, it supports “online predictions” and “batch predictions”.

Use online predictions if you have a couple (i.e. ten) data points that you want to run through your algorithm. For larger datasets (i.e. thousands of points) use batch predictions.            

[caption id="attachment_7733" align="aligncenter" width="690"]

Conclusion