What UbiOps delivers more than standard Model Serving Platforms?

Model serving is the process of providing access to production-level models for end-users or applications. Meaning that they will be deployed for internal or external usage . In most cases, such as with UbiOps, they will be available via a REST API. This stage is very hardware intensive and can be expensive to set up. That is why it is sometimes best to offset the cost and required expertise to a specialized company offering a solution. 

In this article, we will compare UbiOps, a Dutch MLOps company, to other companies offering similar solutions. We will discuss what UbiOps does differently and what functionalities it offers which other ML serving providers do not.

What UbiOps does different to other model serving providers

What UbiOps does on top of serving

UbiOps is a platform specialized for the development stage of MLOps. While it primarily handles the model deployment and serving stage, it is also designed to handle Workflow Management/Inference Pipelines and Model (re-training). 

Workflow management/Inference pipelines

UbiOps has several features which are very well suited towards managing complex workflows, dynamic dataflows and inference pipelines. This is well demonstrated by the flexibility of our Pipelines feature.

Pipelines in UbiOps are a workflow management feature that allows you to connect deployments, operators, and sub-pipelines.

Deployments in UbiOps serve your code for data processing by building a container that runs as a microservice. Each deployment has a unique API endpoint for receiving data requests. Typical deployments include algorithms, data aggregation scripts, and trained machine learning models.

Operators are objects that you can incorporate into your pipeline. Some operators enable the addition of complex logic, while others allow for minor data manipulations without the need to create a deployment.

Each pipeline has its own API endpoints, enabling them to be called in a manner similar to deployments. This feature simplifies the management of inference steps, application of conditional operations, and error handling. Additionally, each element within a pipeline scales independently, allowing you to make the most out of your available resources.

Figure 1: A pipeline in UbiOps

Pipelines allow you to perform actions such as multi-model routing, and A/B tests. Multi-model routing is a process of connecting and directing data to different machine learning models based on certain criteria. With UbiOps pipelines, this can be done easily. 

Model (re-)training

Model re-training or training is the process of training a model. This can be either from scratch or at the fine-tuning stage. UbiOps offers several features which make this process achievable on our platform. 

Given their enormous size and billions of parameters, GenAI models are ideal candidates for fine-tuning. This is because of their broad range of knowledge and reasoning abilities, allowing them to easily learn new topics. This process is often significantly more efficient than training a model from scratch, depending on the technique used. However, fine-tuning is hardware-intensive and usually challenging to set up. UbiOps addresses this issue by providing easily accessible hardware clusters that are ready for use. Additionally, UbiOps can be deployed on-premise, allowing you to run it on your own hardware or in your own cloud environment too.

UbiOps offers evaluation capabilities, including custom metrics, letting you see graphically how much your model has improved: 

Figure 2: Evaluation tab in UbiOps

Figure 3: Custom metrics in UbiOps 

Conclusion

To conclude, UbiOps offers several advantages compared to other standard model serving platforms. Specifically, we delved into its workflow management and training features. In general, UbiOps encompasses all necessary features needed to deploy your ML models to production. Alternatively, you would possibly need to couple several solutions together. UbiOps gives you the full package. 

Our website offers guides on mult-imodel routing, reducing inference costs and how to deploy Mistral 7B v0.3. Create an account here!

The post UbiOps vs standard Model Serving Platforms appeared first on UbiOps - AI model serving, orchestration & training.