Performance, Trend & Impact Analysis

Performance Analysis

The process of evaluating how a product is doing by examining key metrics and asking two questions: How are we doing overall compared to last month, last quarter, last year? Where specifically is performance breaking down?

Trend Analysis

Identifying patterns that persist across time or cohorts — not just a one-month spike. Used to determine whether a metric movement is a signal worth acting on or noise worth monitoring

Performance Analysis Workflow

Pull the data → look for patterns → form a hypothesis → recommend an action → monitor the outcome

Data Sources for Performance Analysis

Loan-level data (every account, origination date, credit tier, state, channel, current status), payment history (who paid, who didn't, how late), bureau data (credit score and attributes at time of origination)

Loan-Level Data

Data on every account including its origination date, credit tier, state, acquisition channel, and current status. The foundation of portfolio performance analysis

Payment History Data

Records of who paid, who didn't, and how late. Used alongside loan-level data to assess delinquency and default trends

Bureau Data

Credit score and attributes captured at time of origination. Used to analyze whether certain score bands are performing better or worse than expected

Segmentation

Slicing portfolio performance by origination period, credit tier, acquisition channel, geography, and underwriting policy version to identify where specifically performance is breaking down

Vintage Analysis

Measuring portfolio performance in different time periods after the loan was granted. Vintage = the month or quarter an account was opened. Used to identify if accounts opened in a particular period are riskier, determine optimal performance windows for scorecards, monitor portfolio risk, and forecast future losses

Deteriorating Vintage Performance

When newer cohorts are defaulting faster than older ones at the same age. Could signal that underwriting was loosened too much or that economic conditions changed

Channel-Specific Degradation

When one acquisition channel produces worse credit quality than others. Signals that marketing targeting is pulling in riskier borrowers

Geographic Concentration

Defaults clustering in specific states. Could be economic, legal, or a collections issue specific to that state

Score Band Drift

Approvals shifting toward lower score tiers over time without a deliberate policy change. A trend worth flagging to Credit Risk

Champion/Challenger Testing

A controlled experiment where two underwriting policies run simultaneously on different segments to compare performance over time. Champion = current policy, Challenger = proposed change. The formal method for measuring initiative impact before full rollout

A/B Testing

Running two versions of a product decision simultaneously on different segments to compare outcomes. Similar in concept to champion/challenger but applied more broadly to product and marketing decisions

Scorecard Analysis

Evaluating how well a credit score or custom model is predicting default. Uses metrics like Gini coefficient and KS statistic to measure model performance

Gini Coefficient

A metric used in scorecard analysis to measure how well a model separates good borrowers from bad ones. Higher Gini = better model discrimination

KS Statistic

Kolmogorov-Smirnov statistic. Used in scorecard analysis to measure the maximum separation between the cumulative distributions of good and bad borrowers. Higher KS = stronger model

Regression to the Mean

The statistical tendency for extreme values to normalize over time. Relevant when interpreting short-term spikes in delinquency — don't overreact to a single bad month before confirming a trend

Loss Forecasting

Projecting future charge-offs based on current delinquency trends and historical loss curves. Used to size credit loss reserves and set forward-looking risk strategy

Hypothesis Formation

After identifying a trend, forming a specific, testable explanation for what is causing it before recommending a policy or product change

Recommendation Structure

Each recommendation needs three things: what the data shows, what you think is causing it, and what specific change you are proposing with measurable success criteria

Initiative Impact Assessment

Measuring whether a specific product or policy change did what you expected. Requires a pre-launch baseline, defined success criteria, post-launch cohort tracking, and attribution analysis. Monitored monthly

Attribution Analysis

Isolating whether a metric movement was caused by your initiative or by external factors like seasonality or macro conditions. Where data scientists earn their keep

Baseline

The documented state of metrics before an initiative launches. Required to measure whether the initiative had any effect

Success Criteria

Specific, measurable targets defined before an initiative launches. What specific movement in what metric over what time period constitutes success. Written into the BRD

Impact Analysis

The structured process for understanding how proposed changes ripple across people, processes, and systems. Identifies who and what will be affected, where potential breakdowns may occur, and which hidden costs could emerge

Risk Assessment vs Impact Analysis

Risk assessment asks "what could go wrong?" whereas impact analysis asks "what happens when we do this?" — a forward-looking evaluation of a specific proposed change

Business Impact Analysis (BIA)

Examines operational and financial consequences when critical functions fail. Identifies essential processes, recovery time objectives, and financial losses

7 Steps of Impact Analysis

1. Define the change initiative 2. Map all affected stakeholders and systems 3. Identify dependencies and ripple effects 4. Assess impact severity and likelihood 5. Create multiple scenario models 6. Document analysis results and actions 7. Establish ongoing impact monitoring

Step 1 Impact Analysis

Clearly define the scope, objectives, and boundaries of the initiative. Document what is changing, the timeline, and success criteria. Validate with stakeholders before analysis begins

Step 2 Impact Analysis

Identify every team, individual, and system the change will affect. Map direct impacts (obvious) and indirect effects (require careful tracing). Internal teams, external partners, and end customers

Step 3 Impact Analysis

Trace connections between stakeholders and systems to reveal cascading impacts. Dependency mapping helps identify bottlenecks that might otherwise go unnoticed. Examine both upstream and downstream outputs

Step 4 Impact Analysis

Quantify consequences by severity (minor inconvenience vs. critical failure), likelihood (rare vs. certain), and timeline (immediate vs. long-term)

Step 5 Impact Analysis

Construct multiple scenarios: best-case (seamless adoption), worst-case (major disruption), most-likely (realistic middle ground)

Step 6 Impact Analysis

Present findings in structured documents including identified impacts, proposed mitigation strategies, and resource requirements for each stakeholder group

Step 7 Impact Analysis

Set up monitoring after implementation to track actual effects against predictions. Define KPIs to measure adoption and health

Dependency Mapping

Tracing connections between systems and processes to reveal cascading impacts. System A changes → Process B is disrupted → Team C is delayed. Helps identify bottlenecks before they occur

Scenario Modeling

Planning for multiple possible outcomes of a change: best-case, worst-case, and most-likely. Rarely does a single scenario match reality

Approval Rate

Percentage of applicants who get approved. Tighter underwriting = lower approval rate = less growth but fewer defaults. Monitored daily

Take Rate

Of people who are approved, how many actually activate and draw on the line. Low take rate signals a product or marketing problem, not a risk problem

First Payment Default (FPD)

Borrower defaults before making a single payment. The clearest and earliest signal of underwriting failure. Monitored daily

Delinquency Rate

Percentage of borrowers 30, 60, or 90 days past due. Leading indicator before charge-offs. Monitored weekly

Draw Activity

How frequently active customers are drawing on their credit line. Monitored weekly as a product engagement and revenue signal

Fee Revenue

Revenue generated from cash advance fees and carried balance fees. Monitored weekly

Net Charge-Off Rate (NCO)

Charge-offs minus recoveries as a percentage of outstanding balance. The headline risk metric. Monitored monthly

Charge-Off Rate

The percentage of outstanding balances written off as uncollectable. A lagging indicator — delinquency rates lead it

Credit Line Utilization

How much of their available credit limit customers are actually using. High utilization can signal financial stress

Return Rate / Repeat Borrowing

How often customers draw again after repaying. Key metric for understanding product engagement and lifetime value on a revolving line of credit

Lifetime Value (LTV)

Total revenue a customer generates over their relationship with the product. Important for evaluating how much to spend on acquisition

Customer Acquisition Cost (CAC)

The total cost to acquire a new customer. Compared against lifetime value to assess whether acquisition spending is sustainable

Days Sales Outstanding (DSO)

Average number of days to collect payment after a credit sale. Used in real-time monitoring of collection efficiency

Bad Debt Ratio

The proportion of receivables written off as uncollectable. A key credit risk KPI tracked in performance reporting

Collections Effectiveness Index

Measures how effectively a company collects on its outstanding receivables. A performance metric for the collections function

Probability of Default

Used to assess the likelihood of a borrower defaulting on their obligations within a specified timeframe. An input into both underwriting decisions and loss forecasting

Monitoring Cadence

The schedule on which different metrics are reviewed. Daily: application volume, approval rate, FPD. Weekly: delinquency trends, draw activity, fee revenue. Monthly: vintage analysis, charge-off rate, initiative impact. Quarterly: portfolio-level trends, policy change outcomes, forward-looking projections

Daily Monitoring

High-sensitivity metrics: application volume, approval rate, FPD flags. Usually automated alerts if something crosses a threshold

Weekly Monitoring

Delinquency trends, draw activity, fee revenue. Usually a standing team meeting with a dashboard walkthrough

Monthly Monitoring

Full vintage analysis, charge-off rate, initiative impact assessment. Feeds into bank partner reporting

Quarterly Monitoring

Strategic performance review covering portfolio-level trends, policy change outcomes, and forward-looking projections. Becomes a formal presentation for senior leadership or bank partners

Power BI

Microsoft's business intelligence tool used to build performance dashboards. Likely Elevate's tool of choice given their Microsoft ecosystem (Teams, SharePoint, Office 365)

Tableau

A leading business intelligence tool used to build performance dashboards tracking delinquency, charge-offs, and approval rates by segment. Most common BI tool in mid-to-large financial services

SQL

Used to query loan-level data directly from the database for performance analysis. The baseline data pulling tool for PMs working in risk-heavy environments

Python (Pandas)

Used for more complex performance analysis, data cleaning, and automation. PMs who can write basic Pandas scripts are rare and valuable in this context

Excel

Used for ad-hoc analysis when a dashboard metric moves unexpectedly. Pivot tables, VLOOKUP/XLOOKUP, conditional formatting, and basic financial formulas are the core skills needed

Dashboard

A visual display of key performance metrics updated on a regular cadence. The PM opens this daily, consumes the data, and flags anomalies for deeper investigation

Automated Alerts

Threshold-based notifications that fire when a daily metric crosses a defined boundary. The first line of defense in performance monitoring

Performance Review

A standing meeting cadence where dashboard metrics are walked through with the team. Weekly for operational metrics, monthly for risk metrics, quarterly for strategic review

PM Role in Performance Analysis

Less about running the analysis and more about owning the narrative — knowing what the numbers mean, communicating anomalies quickly, and connecting metric movements to specific product decisions

Credit Quality Ratings

Assessments of borrower or portfolio creditworthiness used alongside quantitative metrics to evaluate overall risk exposure

Market Environment Monitoring

Tracking whether the macro environment is improving or declining as context for interpreting portfolio performance trends

Employment and Cash Flow Trends

Monitoring whether borrowers' employment and cash flow stability is improving or declining compared to similar borrowers. A forward-looking credit risk signal

Debt-to-Income Trend Monitoring

Tracking how a borrower's or cohort's DTI ratio changes over time compared to historical norms. A signal of increasing or decreasing repayment stress