Skip to content

Shoot Istio Service Mesh and Cilium

When using Istio service mesh you need to have properly configured shoot cluster. If you have kubeProxy enabled then Istio works out of the box. This documentation describes how to check kubeProxy and how to set Cilium when kubeProxy is disabled.

kubeProxy Configuration

Follow these steps to check if kubeProxy is enabled:

  1. Validate your shoot manifest on keyPath .spec.kubernetes.kubeProxy.enabled by opening in any editor of your choice.

    Or if you have yq installed you can use:

    yq e '.spec.kubernetes.kubeProxy.enabled' <shoot-manifest>.yaml

    output will be true if kubeProxy is enabled or false if kubeProxy is disabled.

    If you already have a cluster deployed, you can check this setting by checking YAML directly on the Garden cluster using kubectl:

    kubectl get shoot <shoot-name> -n <shoot-namespace> -o yaml | less

    or if you want to see just the value you can use jsonpath instead like this:

    kubectl get shoot <shoot-name> -n <shoot-namespace> -o jsonpath='{.spec.kubernetes.kubeProxy.enabled}'

    output will be same as from yq command mentioned above.

  2. If result of a first step is false, please continue with Cilium Configuration section.

  3. If you want to use kubeProxy then you can turn it on by either manually updating shoot manifest before deploying or editing existing shoot cluster by using:

    kubectl edit shoot <shoot-name> -n <shoot-namespace>

    or by using kubectl patch as follows:

     kubectl patch shoot <shoot-name> -n <shoot-namespace> --type=merge --patch='{"spec":{"kubernetes":{"kubeProxy":{"enabled": true}}}}'

    !!! note

      Kubectl patch command can output following warnings:

  - Warning: you should consider disabling the static token kubeconfig, see
    <https://github.com/gardener/gardener/blob/master/docs/usage/shoot_access.md>
  - Warning: you should consider migrating to PodSecurity, see
    <https://github.com/gardener/gardener/blob/master/docs/usage/pod-security.md#migrating-from-podsecuritypolicys-to-podsecurity-admission-controller>

  You can **safely ignore** these warnings.

Cilium Configuration

With kubeProxy Configuration completed successfully we can now add providerConfig to shoot cilium network plugin by either manually updating shoot manifest before deployment or by kubectl patch command. You can check cilium official documentation. Please check both options below:

1. Manually updating shoot manifest

cilium.networking.extensions.gardener.cloud configuration example:

    providerConfig:
      apiVersion: cilium.networking.extensions.gardener.cloud/v1alpha1
      bpfSocketLBHostnsOnly:
        enabled: true
      kind: NetworkConfig
      store: kubernetes

by adding cilium.networking.extensions.gardener.cloud configuration example on .spec.networking as in example below:

apiVersion: core.gardener.cloud/v1beta1
kind: Shoot
metadata: ...
spec:
...
  networking:
    ipFamilies:
    - IPv4
    nodes: CIDR
    pods: CIDR
    services: CIDR
    # paste here configuration example
    type: cilium
  provider:
...

with result as:

apiVersion: core.gardener.cloud/v1beta1
kind: Shoot
metadata: ...
spec:
...
  networking:
    ipFamilies:
    - IPv4
    nodes: CIDR
    pods: CIDR
    services: CIDR
    providerConfig:
      apiVersion: cilium.networking.extensions.gardener.cloud/v1alpha1
      kind: NetworkConfig
      store: kubernetes
      overlay:
        enabled: false
      cni:
        exclusive: false
      tunnel: disabled
      loadBalancingMode: dsr
      bpfSocketLBHostnsOnly:
        enabled: true
    type: cilium
  provider:
...

2. By kubectl patch command

kubectl patch shoot <shoot-name> -n <shoot-namespace> --type=json -p \
'[
  {
    "op": "add",
    "path": "/spec/networking/providerConfig",
    "value": {
      "apiVersion": "cilium.networking.extensions.gardener.cloud/v1alpha1",
      "bpfSocketLBHostnsOnly": {
        "enabled": true
      },
      "kind": "NetworkConfig",
      "store": "kubernetes"
    }
  }
]'

Here's a breakdown of the command:

• --type=json specifies the content type as JSON for the patch operation.

• -p indicates that you're providing a patch.

then validate using:

kubectl get configmaps -n kube-system cilium-config -o yaml | grep bpf-lb-sock-hostns

with output:

bpf-lb-sock-hostns-only: "true"