An amazing tool for Data Visualization
Use Cases and Deployment Scope
Our organization uses Kibana primarily to visualize and analyze large volumes of logs and performance data generated by our applications and infrastructure.
Kibana is integrated with AWS OpenSearch. We use AWS OpenSearch to store AWS WAF logs. Whenever we identify an issue, we go to the Kibana console and search for various parameters related to our infrastructure that help us in searching the logs quickly and enable us to identify the issues.
Pros
- Real-time Dashboards:
- We use Kibana to create live dashboards that track WAF performance in real-time. We have a dashboard that visualizes our whitelabel partners and the requests they received on various pages. Using these metrics, we identify the origin of the requests and also how many requests were allowed/blocked by our AWS WAF.
- Quick Search functionality: We have used OpenSearch to index WAF logs and hence Kibana gives us a quick search feature over several indexes in real time. We are able to filter logs almost real time against our WAF logs.
- Another feature which is great in Kibana is the alerting and monitoring. We use Kibana to send alerts to our Slack channels that helps us in quickly identifying the issues.
Cons
- Kibana supports both KQL and Lucene Syntax. While this functionality is great, but it is sometimes very confusing for the users to switch between these two.
- I have faced several performance issues with large data sets and dashboards. Kibana takes a lot of time to response when run against a large data set. Also, the visulization is delayed.
- While Kibana is great in alerting in a Slack channel, it is limited to send alerts to a single channel. I have been using Datadog and it allows sending alerts in multiple channels. This is a limitation from Kibana.
Return on Investment
- Kibana helped us improve decision making by the use of various dashboards. We can come up with conclusions about possible attacks by just looking at the visualization dashboards created for security.
- It is integrated with our AWS WAF OpenSearch cluster and thereby providing us with optimum cost efficiency for logging website traffic data. Previously, we used cloudwatch for logging WAF data and it costs a lot while providing less capabilities then Kibana.
- It has improved our incident response time because we are proactively informed about various issues with our infrastructure on Slack channels immediately.
Usability
Alternatives Considered
Datadog, OpsGenie and Grafana
Other Software Used
Datadog, Cloudflare, Slack, Culture Amp, Atlassian Jira, Atlassian Bitbucket, GitLab, AWS Backup, AWS Elastic Beanstalk, Elasticsearch, Amazon Elasticsearch Service
