Dyantrace 101

application observability - the ability to understand an application health, performance, and behavior from inside out.

application monitoring - shows symptoms, like slow response times, or incresed erorr rates, essentially watching numbers in a dashboard, observability takes it further.

application monitoring tells you what is wrong, observability tells you why it went wrong.

Ex. a smart watch monitoring your heart rate, and alerting you if its to high to go to a hopsital, is reactive. A smart watch observing your sleep over the course of a week, and reccomedning you go to bed ealier is proactive.

Dynatrace positions its self as an all in one, like a smartphone replaces a camera, gps, calculator, etc.

In Dynatrace is combines gathering logs, infrastructure monitoring, network performance, user experience.

In Splunk I am used to having to creating a mental image visualizing distruvuted systems, in order to troubleshoot an error. However dynatrace provices a service map and transaction flow automatically, so I can more quickly identift which service is the root cause.

In Dynatrace it vizualizes how the microservices are connected (service map), and displays how long the data is staying at each microservice (transaction flow). So if there is one microservice that is taking longer, then I can start their to troubleshoot.

service map - live architecture diagram

transaction flow - live path a single request takes through the service map, and how long it stays at each service

Dynatrace Troubleshooting Mindset

Service Map → Problematic Service (What is impacted) → Dependency Analysis (Why it is impacted) → Transaction Trace → Root Cause

Splunk Troubleshooting Mindset

Index → Logs → SPL Search → Correlation (What appears to be related) → Root Cause

Correlation - connecting related pieces of data from different sources to determine cause and effect.

Correlation Example: "Claim submissions are timing out."

index=eims sourcetype=claim_service ERROR

Database connection timeout

index=oracle

Slow query detected

12:01:02 Claim Service error

12:01:01 Oracle query took 8 seconds

Claim Service errors are related to Oracle latency.

Metrics - Measurements we collect from systems and applications over time, like CPU usage, memory consumption, response times, error raters, or how many users are active

Logs - events over time, who did what, when it happened, and what the system reported about it. ex. successful login attempt, failed payment

Traces - the flow of a single request through an application, ex. buy now → 1 sec on web server → 1 sec onmiddleware server, 4sec DB (Dynatrace would voice error to long)

Dynatrace has automated root cause analysis powered by the service map, and transaction trace.

Dependency vs. Correlation

Correlation

Rain started

Traffic got worse

Conclude Rain and traffic are related

Dependency Analysis

Traffic conditions depend on road conditions.

Rain affects road conditions.

OneAgent is like a Universal Forwarder that doesn't just collect logs, it also understands the application, tracks requests between services, collects metrics, discovers dependencies, and sends all of that through ActiveGate, which acts like a routing hub between the monitored systems and the Dynatrace platform.

Grail - Data store, with the purpose of connecting, and analyzing data in real time

Dynatrace Apps - prebuilt or custos apps to do things like visualzae data, analyze user sessions, monitor service performance, etc

Dynatrace SmartScape Topology - the complete automatically discovered topology of applications, services, processes, hosts, containers, databases, and their relationships. Ex. Smartscape is probably closest to having an always-updated architecture diagram, CMDB relationship map, and dependency map all rolled into one.

4 Layers - Host, Processes, Services, and applications

Host - any cloud, virtual, or physical device that helps support an application and running an operating system

OneAgent - monitors the host

Process - it is like a role that fulfills a set of task

Service - it is a specific task contained in a role

Example Process vs. Role - A process is a store employee, and a service is a task that the store employee fulfills like stocking shelves would be a service.

Installing OneAgent on the app server, allows you to monitor DB interactions, but not how the DB is functioning itself, unless you install another OneAgent on the DB, but this can be costly if you don’t install the OneAgent in infra only mode.

Dynatrace Application - an app is only considered an app in Dynatrace if it has frontend functionality, backend only apps aren’t considered apps

Dynatrace Real User Monitoring (RUM) - captures key metrics that reflect how real users experience you app, metrics collected are performance, reliability, and responsivness.

App Perfoemance Index (Apdex) - this is a score that totals from 0 - 1 on how user friendly your app is Ex. Page loads 0 - 3 sec = satisfide user, 3 - 15sec = tolerable, > 15sec = frustruating. The score can be tweaked as well, if for whatetever reason a page taking more then 3 sec is satisfiable.

Auto Injection - When OneAgent is monitoring an app, and you send a request to access an apps webpage, OneAgents adds a JavaScript snippet in order to collect metrics for RUM and calculate an Apdex score

Host Groups - Organize similar groups together like Prod Env host and Dev Env host

Naming Rules - you can change the name of things to be more readble, instead of aws_server_123 you can name it myloginserver

Tags - you can add these to anything pretty much in dynatrace, and then you can filter by tag, so all services related to appA, instead of AppB

Dynatrace leverages its topology service map, to group correlated errors together, to deduce a root cause and avoid alert flooding of downstream systems, that are being impacted by 1 upstream system

Example: DB has a CPU Usage issue, app servers relying on the DB slow down, web servers waiting for the apps also slow down. Dynatrace combinces this into 1 issue/ alert, since it has identified the root cause is the CPU, rather than you getting bombarded with 3 alerts from different sources.

Dynatrace Intelligence (AI engine), is what powers all this automated issue identification, predication, and root cause anlysis

Dashboards are for viewing data in rela time, notebooks are better to capute and view historical data, or historical data over time. Like a stock price every December 31st