KFServing and Kubeflow Pipelines
1 like•423 views
The document discusses a Kubeflow Pipelines component for Kubeflow Serving (KFServing) that allows usage of KFServing within Kubeflow Pipelines. The component uses the KFServing Python package and API to deploy InferenceServices and perform canary rollouts. A sample pipeline is shown that uses the component to deploy a TensorFlow model. The document also analyzes the component and discusses passing InferenceService YAML as the most flexible way to deploy models with full customizability.
KFServing and Kubeflow Pipelines
- 1. Kubeflow Pipelines component for KFServing Animesh Singh, Paul Van Eck
- 2. Kubeflow Pipelines •Pre-built components: Just provide params or code snippets (e.g. training code) •Create your own components from code or libraries •Use any runtime, framework, data types •Attach k8s objects - volumes, secrets Containerized implementations of ML Tasks •Specified via Python DSL •Inferred from data dependencies on input/output Specification of the sequence of steps •A “Run” = Pipeline invoked w/ specific parameters •Can be cloned with different parameters Input Parameters •Invoke a single run or create a recurring scheduled pipeline Schedules
- 3. KFServing Component ● Allows usage of KFServing within a Kubeflow pipeline. ● Uses KFServing Python package (v0.5.1) and v1beta1 API. ● Can easily deploy InferenceServices and perform canary rollouts. ● Supports passing in raw InferenceService YAML ● Source Code
- 4. Sample Usage import kfp kfserving_op = kfp.components.load_component_from_url('.../kfserving/component.yaml') @dsl.pipeline( name='KFServing Pipeline', description='Pipeline for KFServing.' ) def kfserving_pipeline(): kfserving_op( action='apply', model_name='tf-sample', model_uri='gs://kfserving-samples/models/tensorflow/flowers', framework='tensorflow', ) kfp.Client().create_run_from_pipeline_func( kfserving_pipeline, arguments={} ) Deploy a TensorFlow model
- 5. Kubeflow Pipelines can train, deploy and serve 5 Open Source Dojo
- 7. Analysis ● Pipeline components rely on passing in specific args to some CLI program. ● The primary support is for predictors with limited customization options. ● KFServing supports many PodSpec fields that aren’t first-class component arguments. ● Transformers and explainers are also not first class.
- 8. Analysis ● Users can still deploy InferenceServices with full customizability by passing the component their YAML definitions. ○ All PodSpec fields. ○ Transformers/Explainers. ○ Specific annotations/labels. ● This is the most flexible format. ○ Perhaps this should be the recommended way? ○ Offers format consistency compared with kubectl/KFServing interactions. isvc_yaml = ''' apiVersion: serving.kubeflow.org/v1beta1 kind: InferenceService metadata: name: torchserve-transformer spec: transformer: containers: - image: kfserving/torchserve-image-transformer:latest name: kfserving-container predictor: nodeSelector: disktype: ssd pytorch: storageUri: gs://torchserve/image_classifier ''' kfserving_op( action='apply', inferenceservice_yaml=isvc_yaml )