Monitoring your Azure Container Services (AKS) cluster with Grafana and InfluxDB

With any compute platform, whether it’s on premise, or cloud based, it is essential that we have a capability to monitor how our application is performing on that platform. The same applies with Kubernetes.

With Azure Container Services (AKS), Microsoft’s go to monitoring solution is Log Analytics, which is part of OMS. With this approach, you install a monitoring agent via a Kubernetes Daemonset, so that the monitoring agent runs on every node within the cluster. Now this implementation works well, and will probably satisfy most users requirements, but it’s always nice to have choice in life.

Another approach is to use Heapster. Heapster aggregates monitoring data from each node in the cluster and then writes the monitoring data to a configurable backend store. Heapster runs as a single pod in the cluster, and queries the Kubelet of each node within the cluster for the node’s monitoring data. You’ll find that the Heapster pod and its associated service are already running within the AKS cluster. To find It you can run a Kubelet get pods, but providing the kube-system namespace in the query so:

get pods --namespace=kube-system


One of Heapster’s supported backend stores is InfluxDB. InfluxDB with Grafana as a front end provides us with a rich visualizer for the captured monitoring data

Now setting up InfluxDB and Grafana is straight forward, we just run them as pods on our cluster. Installation instructions can be found here, and the deployment files can be found here

If you decide not to remove the existing Heapster deployment within your cluster, then you’ll at least need to update it to include a sink to InfluxDB, otherwise the InfluxDB k8s database won’t be created, and the monitoring data won’t be persisted. The fragment below shows the relevant entry for the InfluxDB sink.

- --source=kubernetes:https://kubernetes.default
- --sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086

You can now go to the IP address of your Grafana service and see how your cluster is performing.

Comments

  1. Unfortunately this post and the referenced installation instructions both dodge the question of how to best configure persistent storage for the collected data with AKS. If you are using this for real, perhaps you can share the setup? Do you use a normal volume or a persistent volume and what kind of backing store is best in Azure?

    ReplyDelete

  2. It is really a great and useful piece of info. I’m glad that you shared this helpful info with us. Please keep us informed like this. Thank you for sharing.

    Integritas Solutions



    ReplyDelete
  3. I have read this post. collection of post is a nice one..!!..Azure Online Training

    ReplyDelete
  4. This information is impressive; I am inspired with your post. Keep posting like this, This is very useful.Thank you so much. Waiting for more blogs like this.
    Airport management courses in chennai
    airline management courses in chennai
    aircraft maintenance course in chennai
    diploma in airport management in chennai

    ReplyDelete

Post a Comment