Call Mapping Functions and Features Notes

Oliver Kut from the application perspectives team will discuss call mapping functions and features.
This is the first of two sessions; the second is next week.
The session will include a Q&A, and a recording will be available.
The plan is to recap basics, cover application services and endpoints, and provide a high-level view of data processing.
The follow-up session will focus on issue diagnosis and workarounds.

The applications area revolves around the concept of "calls."
Calls are interactions entering or exiting monitored processes.
If an interaction is monitored from both client and server sides (exit and entry spans), they are merged.
If only one side is monitored, the call consists of single-sided information.
Inconsistencies may arise if some calls have both sides of the interaction while others have only one, potentially leading to incorrect service assignments.

Calls contain a wealth of information recorded as "tags."
Tags originate from two sources:
- Tracer: Tags that are embedded within the interaction itself. This provides reliable information, for example, HTTP headers or paths.
- Infrastructure: Tags that related the monitored call to infrastructure entities. To infer this you need to link it up with infrastructure information.
Infrastructure tags connect calls to processes, Kubernetes pods, namespaces, and containers.
Linking to infrastructure data is somewhat unreliable, as it requires matching information from different sources.
Missing infrastructure tags can lead to calls being assigned to unexpected services or applications.

Services, endpoints, and applications are built upon the tags within calls.
They provide a way to group calls for analysis, such as success rates, error rates, and performance.
Assigning calls to services, endpoints, and applications requires mapping rules.
Services, endpoints, and applications, from an application monitoring perspective, are merely groups of calls or labels attached to calls.
A service is not a self-contained entity but a label. Its existence is determined by the presence of calls with that label.

Applications are the initially visible and global concept, although they will be discussed last.
An application is a subset of calls selected by user-defined rules.
Customers define which calls belong to an application to isolate monitoring scopes.
Common use cases include separating monitoring by environment (test vs. production) using conditions like Kubernetes cluster.
Another method is to define applications by the user journey, tracking calls originating from a specific entry point and their downstream services. Calls that can't be properly mapped to a cluster will cause problems with accurately tracking which environment that the call belongs.

Services are global concepts but can be viewed in the context of an application.
The same service can be part of multiple applications.
A service is identified solely by its name.
If two entities map to the same service name, they are treated as the same service for monitoring purposes.
A historical exception exists: an HTTP call to an entity named "discount service" is not considered the same as a database call to an entity also named "discount service."
This can result in the same entity appearing twice in the dependency graph if not merged.

A catalog demo service shows up in three different applications: all services, Robotrop, and RoboChop backend.
The calls for that service are mapped to all three applications. A discount service had all calls going to "all services", robot shop, and robot shop backend, however, a subset of calls were directed to the discount canary build application.

Endpoints are scoped to a specific service.
They represent subgroups of calls processed by the same entity (e.g., different HTTP path templates within an HTTP service).
Two endpoints with the same name in different services are distinct endpoints.
Endpoints are dynamic concepts derived from traces, active only when calls are being received or sent.
Relationships between services, applications, and endpoints are derived from calls and dependent on the viewed time frame.
The same service, "quote of the day," may have 619 endpoints in a one-hour timeframe but only 71 endpoints in a five-minute timeframe, reflecting activity during those intervals.

Service and endpoints are built-in concepts that are auto-detected, without any user configuration.
Services and endpoints are tracked separately for the source and destination of calls.
Each call destination maps to a single service and endpoint, in contrast to applications where a call can belong to many.
A hierarchy of rules is applied to each call, with the first matching rule determining the service and endpoint.

A serviceRuleId internal tag identifies the rule used for service mapping on any particular call.
Installer service name represents internal rules based on environment variables.
A base64 gibberish ID of a custom service mapping rule means that the customer is overriding the built-in rules.
A troubleshooting tab provides quick access to internal diagnostic tags, such as the service mapping rule, resilient service mapping, and infrastructure linking status.

Built-in mapping rules are found in the default service rules YAML file in the backend GitHub repository.
The rules are named and specify conditions for assigning service names.
call database schema is an easy rule that assigns call database schema tag to the service name. Other rules are based on the value of call tags, and use regular expressions.
Tags can be absent, can be equal to specific value, or do fancy regex pattern matching.
The rules are ordered from most specific to most general. The installer service name allows customers to explicitly set the service name.

Customers can define custom rules for service naming.
These rules use the same matching engine but have restrictions in the UI.
Administrators can define a number of tags that, if present, determine the service name.

Some services with zero calls are derived from infrastructure data, going against the service existence rule.
The same mapping rules are applied to infrastructure entities.
If a call links to this infrastructure entity and rules are applied, then this is the service name that should be assigned to the calls.
There may be no calls to the service for various reasons, such as a lack of infrastructure linking or higher-precedence rules overriding the assignment.

Application mappings are always based on custom definitions provided by the customer.
There are is typically some condition evaluated for calling the calls to the application.
Application perspectives allow selecting downstream calls to capture calls from a specific point and downstream services. Note that this is specific to application perspective and is not a typical unified analytics filter.
Tag rule matching may be more relaxed than unified analytics; the former may be case-insensitive while the latter is case-sensitive.

Service mappings are usually based on built-in rules with customizations and are single-endpoint for each call.
Application mappings are always based on custom definitions.
Endpoint mapping works similarly to service mapping but relies on a different, and often less problematic, rule set that is not dependent on infrastructure checks.