Why it matters

KServe is K8s-native serving. Understanding shapes K8s LLM deployment.

Advertisement

The architecture

InferenceService CRD: declarative model.

Predictor + transformer + explainer.

Scale-to-zero via Knative.

KServe stackInferenceServiceK8s CRDPredictorvLLM / TGI / etc.Scale-to-zeroKnativeScale-to-zero saves cost when idle; cold start impacts latency
KServe.
Advertisement

How it works end to end

Predictor: model container (vLLM, TGI).

Scale-to-zero: Knative-based.

Model registry integration.