Model Deployment, Monitoring, and Continuous Optimization

The journey of every machine learning model reaches a momentous milestone when the model is deployed for public use. However, this accomplishment is merely an intermediary step in what is, in actuality, a dynamic continuous process. The production stage requires the model to operate reliably in an environment populated by users and competing models. Such an operational environment is rife with challenges that demand considerable technical investments using knowledge and techniques that transcend issues of mere performance. Deployment decisions can determine whether the platform will smoothly provide services for months or whether constant firefighting and regime change will compromise reliability and quality of service and lead to eventual abandonment or deprecation of the model. Both the definition of an automated monitoring framework and the establishment of a continuous optimization feedback loop are critical in determining the need for constant firefighting, burnout of scarce technical resources, and the ultimate operational success of the model.