TM

Ai-API
 
Bridge Data Science and DevOps
Deliver prediction services Anywhere

Getting Machine Learning models into production is hard. Data Scientists are not experts in building production services and DevOps best practices. The trained models produced by a Data Science team are hard to test and hard to deploy. This often leads to a time consuming and error-prone workflow, where a pickled model or weights file is handed over to a software engineering team.

Ai-API™ is a framework within Zeblok’s Ai-MicroCloud™ for serving, managing and deploying machine learning models. It is aiming to bridge the gap between Data science and DevOps, and enable teams to deliver prediction services in a fast, repeatable and scalable way.  

Ai-API™ makes moving trained ML models to production easy:

  • Package models trained with ML framework and then containerize the model server for production deployment 

  • Deploy anywhere for online API serving endpoints or offline batch inference jobs

  • High-Performance API model server with adaptive micro-batching support

  • Ai-API™ server is able to handle high-volume without crashing, supports multi-model inference, API server Dockerization, Built-in Prometheus metric endpoint, Swagger/Open API endpoint for API Client library generation, serverless endpoint deployment etc. 

  • Central hub for managing models and deployment process via web UI and APIs

  • Supports various ML frameworks including:

Scikit-Learn, PyTorch, TensorFlow 2.0, Keras, FastAI v1 & v2, XGBoost, H2O, ONNX, Gluon and more

  • Supports API input data types including: 

DataframeInput, JsonInput, TfTensorflowInput, ImageInput, FileInput, MultifileInput, StringInput, AnnotatedImageInput and more

  • Supports API output Adapters including: 

BaseOutputAdapter, DefaultOutput, DataframeOutput, TfTensorOutput and JsonOutput

Easy steps to Ai-API Deployment

List of APIs 

Quick view of the APIs that are successfully deployed

Image 2.png

Select Model to Deploy as API

Select Model.png

Select the model that users want to deploy as API

Select NameSpace

Option to select a namespace

Select Name space.png

Select Edge/Data Center

Data center.png

List of data center or Edge locations where model can be deployed

Name the API

Give a name for your deployment

give a name.png

Create API

CLI Interface.png

Click Create button to create the API