HEART Framework
HEART complements business and technical metrics with user-centered evidence. Define the goal, signal and metric before interpreting a change.
-
-
Holdback Group
A randomized holdback can estimate a policy’s continuing effect when assignment, follow-up, treatment history, and the outcome window support the comparison.
-
Data Product Adoption
A product can serve one team and retain useful caches or snapshots. Measure use and reduced duplicate work without treating either count as proof of value.
-
Content Certification
Self-service makes it easy to add dashboards, and nothing assigns anyone to remove them. Certification exists to say which ones somebody stands behind.
-
BI Capacity
A model that fits at rest can exceed memory during a full refresh. Size for processing and concurrent queries, and monitor both reports and the shared pool.
-
Row-Level Security
One person duplicates a workbook, forgets the filter, and the report renders perfectly for everyone. That is the failure mode to design against.
-
Metric Change Management
A new definition can change a historical query while an old report stays fixed. Preserve the versions and decide which figures to restate.
-
Ratio Metric
An average daily rate weights days equally; a pooled session rate weights sessions. Define the question and compatible components before aggregating.
-
Semantic Layer
The same filter copied into seventeen dashboards is seventeen definitions. A semantic layer exists so there is one.
-
Statistical Sampling
A larger convenience sample is a more precise measurement of the wrong group. Size does not repair how the rows were selected.
-
Practical Significance
A big enough sample makes a 0.1% difference significant. Whether 0.1% is worth the engineering is a different question, decided in advance.
-
Survivorship Bias
A current-status filter can hide former members. Recovering a historical population depends on retained membership history and explicit missing-data checks.
-
Multiple Comparisons
Slicing a metric until something stands out is a search procedure. Selected findings need error control that accounts for the search.
-
Decision Question
“What result would change your mind?” is the most valuable question at intake and the least often asked, because the answer is sometimes none.
-
Prescriptive Analytics
A system that recommends without a declared objective has one anyway — chosen by whoever wrote the code, and invisible to everyone else.
-
Descriptive and Diagnostic Analytics
Breaking a decline down by region and channel produces candidates, not causes. The gap between the two is the hardest step in analytics.
-
Direct Lake
Direct Lake reads Delta tables in OneLake instead of importing them, so a refresh updates only metadata. Guardrails, fallback, and security decide the rest.
-
Cross-Selling
Using 2.40 versus 1.00 points per added product gives $400,000 versus $120,000 in the illustrative campaign.
-
Upselling
Reaching a customer costs $3 and acquiring one costs $320, which cuts both ways.
-
Customer Segmentation
Useful segments connect customer differences to actions. Learn how to define membership, choose methods and evaluate stability and business value.
-
Valuation Metric
Four companies at exactly Rule of 40: three-year cumulative EBITDA ranges from -$165m to +$116m.
-
D2C (Direct-to-Consumer)
At one order per customer and a $60 CAC, the example loses $0.42 per customer.
-
LTV:CAC Ratio
The same $700 net LTV reads as 5.60 against media spend and 2.15 against a fully loaded CAC.
-
Net LTV (sometimes called Contribution LTV)
Specify which costs are subtracted, avoid double counting, and distinguish lifetime contribution from acquisition payback and company profit.
-
Gross LTV
Calculate lifetime revenue consistently, separate observed revenue from forecasts, and understand why equal gross LTV can produce unequal contributions.
-
Predictive LTV (pLTV)
Forecast customer value with a clear cutoff and horizon, choose a model for the purchasing process, and validate the decision it supports.
-
Cohort-Based LTV
Build cohort value from period totals, use a fixed customer denominator, and make the forecast horizon and unobserved tail explicit.
-
Customer Lifetime
Derive the reciprocal-churn formula, distinguish total from remaining life, and account for customer mix and right-censored observations.
-
Gross Margin
Calculate gross margin, interpret changes in price, costs and mix, and choose the right margin for break-even and customer-economics decisions.
-
Fully Loaded CAC
An overall CAC of $90.00 can sit on segment CACs of $33, $196 and $2,000, and describe none of them.
-
Organic CAC (Customer Acquisition Cost)
When organic CAC carries the content team’s salaries and paid CAC leaves out its own team’s, the paid-to-organic comparison is asymmetric, as in this example.
-
Paid CAC (Customer Acquisition Cost)
A normal-approximation power calculation gives about 0.83 power for a user-level holdout with 5,000 users in each group; a geographic design needs cluster-level planning.
-
Channel-Specific CAC (Customer Acquisition Cost)
Platform-claimed conversions total 3,130 against 1,700 real customers, so every channel CAC undercuts the blended one.
-
Blended CAC (Customer Acquisition Cost)
Customer-weighted channel CACs reconcile with blended CAC only when cost scope and customer attribution partition the same totals.
-
Session Length
A hypothetical checkout comparison combines 39% shorter sessions with 47% more revenue; the figures alone do not identify the cause.
-
Revenue per User (RPU / ARPU)
Define the revenue and user population first, then distinguish better monetization from changes in mix, retention and the denominator.
-
Customer Churn
The same 5% of customers is 0.1% of revenue or 40.6%, depending on which 5% leaves.
-
Retention
Retention depends on the starting population, return event and time window. Revenue can grow even while the number of retained customers falls.
-
Model KPIs (Key Performance Indicators)
Precision falls from 0.6200 to 0.0880 and recall rises from 0.0287 to 0.8155, with the model unchanged.
-
Lagging Indicators
AUC uncertainty depends on positive and negative counts and paired-score dependence, not just the AUC values.
-
Leading Indicators
At a million rows a shift of 0.02 standard deviations returns a p-value of 1e-28.
-
Cohort
An aggregate metric mixes behavior with composition. Cohorts hold the entry period fixed so the two can be told apart.
-
ROI (Return on Investment)
A 225% return on $8,000 makes $18,000; a 40% return on $600,000 makes $240,000.
-
Conversion Rate (CR)
Conversions divided by something. Which something decides what the metric means, how it moves, and whether its error bars are honest.
-
Cost-Per-Click (CPC) Models
With a fixed competing rank and slot, doubling predicted CTR halves CPC in the simplified auction.
-
LTV (Customer Lifetime Value)
LTV depends on what value includes, when payments arrive and how retention is forecast. Define these choices before comparing customers or acquisition costs.
-
CAC (Customer Acquisition Cost)
Lifetime divided by LTV:CAC gives a simple payback shortcut under constant gross profit; actual cohort recovery also depends on retention.
-
CTR (Click-Through Rate)
The same creative shows 3.000% at the top slot and 0.540% at position five.
-
Segment Action
Learn why a segmentation pays off only when each actionable group gets a purposeful response, and which two groups look identical but are not.
-
RFM Analysis
Learn how RFM ranks customers using only purchase behavior, what its core assumption is, and where that assumption stops holding.