Visualization in Monitoring

Define Visualization

  • Visualization is the graphical representation of information/data collected from your business infrastructure, aiding in understanding and maintaining application performance.
  • Visual elements include charts, graphs, and maps which provide an accessible way to see and understand trends, outliers, and patterns in data.

Importance of Data Visualization

  • Essential for analyzing massive amounts of information and making informed decisions.
  • Allows operators to quickly spot trends or problems.
  • Informs operators through rapid alerts about significant events.
  • Enables quick and effective communication using visual information.
  • Helps businesses identify factors affecting customer behavior and pinpoint areas for improvement.
  • Provides developers with valuable feedback on application performance.

How Visualization Helps Developers

  • Predict application behavior.
  • Gain real-time operational insight into IT infrastructure and business systems.
  • Make product and service improvements.
  • Find and fix problems quickly.
  • Continue to innovate rapidly and maintain customer satisfaction.
  • Improves insights and accelerates decision-making
  • Eliminate debugging.

Data Visualization and Information Distribution

  • Easily distribute information and share insights with everyone involved.
  • Allows DevOps to achieve greater success rapidly, with fewer mistakes, and reduced operational costs.
  • Round the clock data availability.

Forms of Data Presentation

  • Dashboards: Most common method for visualizing data, often displayed on large panel screens and updating in real-time.

    • Contain indicators for business systems, web performance, network traffic, security, revenue flow, login failures, etc.
    • Use visual elements like charts, graphs, and timelines.
    • Some are customizable, allowing operators to select date ranges and review results in real time.
    • Can be configured to send or display alerts to operators.

  • Alerting

  • Reporting

Common Visualization Types

  • Bar graphs and pie charts
  • Data tables
  • Maps
  • Infographics
  • Dashboards
  • Line charts: Display how variables change over time.
  • Area charts: Display the values of multiple types of data collected at consecutive, equally spaced points in time.
  • Scatter plots: Display relationships between two variables (x and y axis).
  • Tree maps: Show hierarchical data in a nested format, with rectangle sizes proportional to the percentage of the whole.
    • Best used when multiple categories are present and the goal is to compare different parts of the whole.
  • Population pyramids: A graphical illustration to show how a given population is distributed between age groups and gender.

Factors to Consider When Choosing a Visualization Tool

  • Friendly User Interface
  • Ability to connect to external sources for information extraction
  • Scalability: Ability to scale easily
  • Ease of use: Small learning curve to avoid extensive training
  • Artificial Intelligence: AI and machine learning for quickly identifying data patterns (typically in cloud-based solutions)
  • Management: Ability to right size, delete, spot or unspot issues quickly
  • Storytelling: Narrative approach when visual analytics aren't enough

Visualization Tool Options

  • Free open-source apps: Kibana, Elasticsearch
  • Proprietary tools: Sysdig (with visualizations and dashboards), Splunk (enterprise, hosted, and free versions with limited capabilities)
  • Cloud-based solutions: IBM Cloud, Amazon Web Services, Microsoft Azure, Google Cloud
  • On-premise installation

Kibana

  • Open-source web application used with Elasticsearch.
  • Provides a user interface for managing authentication and authorization requests for Elasticsearch
  • Allows visualizing, searching, and analyzing data using an Elasticsearch database.

Splunk

  • Proprietary solution for monitoring, searching, analyzing, and visualizing big data.
  • Combines log analysis and visualization of data collected from web applications, sensors, devices, user-created data, etc.
  • Can analyze structured or semi-structured data with data modeling.

Key Takeaways

  • Visualization is a graphical representation of real-time data displayed as charts, graphs, and timelines.
  • Alerting helps developers quickly spot issues and pinpoint areas for improvement.
  • Operational insight gives DevOps staff a deeper understanding of IT infrastructure and business systems.
  • Choosing the right tool requires considering project needs, requirements, goals, and costs.
  • Kibana (open source) and Splunk (proprietary) are two popular solutions.