Machine learning in production
Training machine learning models after sorting out the data pipeline and the use case which is a major hurdle in the development phase different problems come into
the picture such as
*offline hosting or online hosting
*performance / latency issues
*batch or online predictions
All these scenarios give rise to independent problems such as when doing regular inferencing we can manage to batch the operations as it is computationally efficient and require GPUs for lower latency. In the case of online predictions where the client can perform an API request to get the predictions, it is easier to Just perform
inference in the CPU and serve.
These problems are important to be articulated before employing a strategy for deployments and have a good understanding of the business outcomes of each decision.
Some outline steps that can be followed are
* Same data pipeline for both training and inference
* Cloud deployments - as they are easier to scale and monitor