Create an ASK cluster using the CLI, not yet available in Terraform due to a bug an Azure rest API : https://github.com/Azure/azure-rest-api-specs/issues/22208 Issue on TF side : https://github.com/hashicorp/terraform-provider-azurerm/issues/19300 So the creation of the cluster happened like : important settings below are `--network-plugin none` and `--kube-proxy-config kube-proxy-config.json` ``` echo "{"enabled": false}" > kube-proxy-config.json az aks create -g "alm-aks" -l westeurope -n alm-aks --network-plugin none --os-sku CBLMariner --service-cidr 10.0.0.0/16 --node-count 1 --kubernetes-version 1.25.5 --dns-service-ip 10.0.0.10 --vnet-subnet-id /subscriptions/1afa17a1-cab0-420d-bb09-1aeb8489ea70/resourceGroups/alm-aks/providers/Microsoft.Network/virtualNetworks/aks-nodes/subnets/public --kube-proxy-config kube-proxy-config.json ``` Gateway API : a role-oriented API https://gateway-api.sigs.k8s.io Then we proceed with the installation of the gateway's CRD (if we don't, disable that feature as well in cillium helm values) ``` kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.6.1/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.6.1/config/crd/standard/gateway.networking.k8s.io_gateways.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.6.1/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.6.1/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml ``` Then we can install Cilium with the value file provided here, you just need to change `k8sServiceHost` value. `helm upgrade -i -f cilium-vmm.yaml cilium cilium/cilium --version 1.13.0 --namespace kube-system` For the purpose of the demo and lack of Datadog, we will install jaeger, otel-collector and cert-manager (dependency of both) ``` kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.yaml kubectl create namespace observability kubectl create -f https://github.com/jaegertracing/jaeger-operator/releases/download/v1.42.0/jaeger-operator.yaml -n observability kubectl apply -f jaeger.yaml kubectl apply -f https://github.com/open-telemetry/opentelemetry-operator/releases/latest/download/opentelemetry-operator.yaml kubectl apply -f otelcol-hubble # Enable HTTP visibility for the podinfo app and all of DNS traffic: cat << EOF|kubectl apply -f - --- apiVersion: cilium.io/v2 kind: CiliumClusterwideNetworkPolicy metadata: name: default-allow spec: endpointSelector: {} egress: - toEntities: - cluster - world - toEndpoints: - {} --- apiVersion: cilium.io/v2 kind: CiliumClusterwideNetworkPolicy metadata: name: dns-visibility spec: endpointSelector: {} egress: - toEndpoints: - matchLabels: k8s:io.kubernetes.pod.namespace: kube-system k8s:k8s-app: kube-dns toPorts: - ports: - port: "53" protocol: ANY rules: dns: - matchPattern: "*" - toFQDNs: - matchPattern: "*" - toEndpoints: - {} --- apiVersion: "cilium.io/v2" kind: CiliumClusterwideNetworkPolicy metadata: name: "rule1" spec: endpointSelector: {} ingress: - fromEndpoints: - {}