Seldon Core is an open-source platform designed to deploy machine learning models on Kubernetes. It provides a robust framework for managing, scaling, and monitoring machine learning models in production environments. One of its key features is the ability to perform canary deployments, which allows you to test new versions of a model with a small percentage of traffic before a full rollout.
When a canary deployment is not functioning as expected in Seldon Core, you might observe that the traffic is not being split between the main and canary models. This can result in the canary model not receiving any traffic, making it impossible to test its performance effectively.
The primary issue with a non-functioning canary deployment often lies in the traffic split configuration or the absence of a proper canary definition. In Seldon Core, traffic splitting is managed through the traffic field in the SeldonDeployment resource. If this field is not configured correctly, the traffic will not be routed as intended.
Ensure that the canary model is defined correctly in the SeldonDeployment YAML file. The canary model should be listed under the predictors section with a specified traffic percentage.
Check the traffic field in your SeldonDeployment YAML file. Ensure that the sum of traffic percentages for all models equals 100. For example:
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
name: my-model
spec:
predictors:
- name: main-model
graph:
name: classifier
implementation: SKLEARN_SERVER
modelUri: gs://my-bucket/model
traffic: 80
- name: canary-model
graph:
name: classifier
implementation: SKLEARN_SERVER
modelUri: gs://my-bucket/canary-model
traffic: 20
Make sure that the canary model is properly defined in the predictors section. Each model should have a unique name and a valid model URI.
After verifying and correcting the YAML configuration, apply the changes using the following command:
kubectl apply -f my-seldondeployment.yaml
Use Seldon Core's monitoring tools to ensure that traffic is being split as expected. You can check the logs and metrics to verify that the canary model is receiving the correct percentage of traffic.
For more detailed information on Seldon Core and canary deployments, refer to the following resources:
(Perfect for DevOps & SREs)
(Perfect for DevOps & SREs)
