Scalable artificial intelligence infrastructure: building tech stacks for financial institutions

Over the past few years, there has been a growing interest in AI in the financial industry. AI has shown great promise in enhancing existing systems and creating new insights in a wide range of applications, such as algorithm-based trading, fraud detection, risk assessment, and client services. Despite this, most financial institutions are still in the early stage of AI adoption. While they are executing pilot projects, they are facing the challenge of deploying these models due to infrastructure and compliance issues across the full value chain. We propose a full-stack distributed machine learning platform that can enhance the efficiency of researchers and data engineers and provide tools to handle the heavy task of DevOps.

Our AI infrastructure is designed to be multi-purpose: (1) data and model governance, to ensure that all in-house models remain in compliance; (2) performance and cost efficiency, particularly for deep learning inference; (3) scalable orchestration on data-parallel and model-parallel with distributed deep learning for research and DevOps tasks. In recent years, the research community has proposed many system designs as well as software tools and libraries to help with these tasks. We recognize that there are several key design considerations for crafting practical software, especially for the heavily regulated financial services. We discuss our design principles and the design choices. To validate our design, we have successfully deployed our full-stack distributed machine learning platform in one of the largest financial organizations.