current position:Home>Istio Troubleshooting: uneven grpc service load

Istio Troubleshooting: uneven grpc service load

2022-06-24 12:43:47imroc

This article excerpts from istio Learning notes

The phenomenon

grpc call , The same client Your request always hits the same server Of pod, Cause uneven load .

analysis

grpc Is based on http2 Long connection of , Multiple requests to reuse the same connection . If not istio, Use only ordinary k8s service, Will not perceive grpc Agreed , Just as tcp To forward , Load balancing at the connection level , No load balancing at the request level . But in istio in , The default will be for grpc Load balancing at the request level , If uneven load is found , It is usually not configured correctly .

Must let grpc Load balancing at the request level , The core is to let istio Correct identification is grpc agreement , Don't configure it to tcp, use tcp Then load balancing can only be performed at the connection level , Request levels can be loaded unevenly .

resolvent

  1. If you want to be exposed ,gateway in protocal To configure GRPC no need TCP, Example :
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: grpc-gw
  namespace: demo
spec:
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  servers:
  - hosts:
    - '*'
    port:
      name: grpc-demo-server
      number: 9000
      protocol: GRPC #  Use here  GRPC  no need  TCP
  1. If you define vs, Need to use http Match without tcp, because grpc stay istio Chinese matching is also used http Field , Example :
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: grpc-svc
  namespace: demo
spec:
  gateways:
  - demo/grpc-gw
  hosts:
  - '*'
  http: #  Use here  http  no need  tcp
  - match:
    - port: 9000
    route:
    - destination:
        host: grpc.demo.svc.cluster.local
        port:
          number: 9000
      weight: 100
  1. Deploying services service Of port name Need to use "grpc-" Opening definition , Give Way istio Be able to correctly identify , Example :
apiVersion: v1
kind: Service
metadata:
  name: grpc
  namespace: demo
spec:
  ports:
  - name: grpc-9000 #  With  grpc-  start 
    port: 9000
    protocol: TCP
    targetPort: 9000
  selector:
    app: grpc
  type: ClusterIP

For more protocol specification methods, please refer to istio Best practices : Explicitly specify the protocol for the service

copyright notice
author[imroc],Please bring the original link to reprint, thank you.
https://en.chowdera.com/2022/175/20210526184507570x.html

Random recommended