Edilitics | Data to Decisions

Methodology Notes

Inspect the exact statistical method, sample size, and test result behind every AskEdi response, written by the system, never by the AI.

Methodology Notes show the statistical work behind an AskEdi response in plain language: the sample size, the method chosen, and the real test result. Unlike the response text, which the AI writes to answer your question, the methodology note is generated directly by Edilitics, so it cannot misstate a number, a p-value, or a method.


Why This Exists

An AskEdi response tells you what the data shows. A methodology note tells you how AskEdi knows. The two are generated separately: the response is the AI's narrative, written to be readable and specific to your question. The note is a factual account of the computation, written by the system itself, independent of the AI's phrasing.

This matters because it gives you two independent ways to check the same answer. If the response says a trend is reliable, the methodology note states the actual p-value and sample size that made it reliable. If a section reports a correlation, the note names the exact statistical test used and its result.


Opening Methodology Notes

Every response with an underlying statistical computation has a sparkle icon in the action row beneath it. Click it to open the notes for that response.

The panel shows one note per statistical section in the response. A Root Cause Analysis response, for example, shows separate notes for the trend, the contribution ranking, the correlation, and the anomaly scan.

A response with no underlying statistical computation, such as a plain narrative or a chart with no trend or comparison behind it, does not show the sparkle icon.


What a Methodology Note Contains

The exact content depends on which response type generated it, but every note follows the same principle: state what was measured, how, and with what result.

  • Sample size and the model chosen for the trend, and why
  • The observed trajectory in the actual data, including net slope and net change
  • The statistical reliability of the trend: the fit quality (R²), p-value, and noise threshold
  • How many segments were ranked in the contribution analysis, and whether the difference was tested for significance (ANOVA/t-test)
  • Which secondary metric was identified as the strongest correlate, and whether that relationship is statistically significant
  • How many data points were checked for anomalies, and the most extreme deviation found (in standard deviations σ)

A typical note for the trend section reads close to this:

Trend & forecast analysis:
  • Scope: Evaluated 10 consecutive time periods across the analysis window
  • Model selected: Linear trend regression - straight-line fit applied across the window
  • Observed trajectory: Movement is down (total net change: -6.20, net slope β = -0.62 from first to last period)
  • Statistical reliability: Unreliable trend (R² = 0.1500, p = 0.28) - model explains only 15% of variance; movement cannot be distinguished from random variation
  • Sample size and the projection method chosen, and why
  • The real observed trajectory from the first to the last period, alongside the fitted trend
  • Whether the trend is statistically reliable, and therefore whether a forecast number is shown at all
  • For a category breakdown, how many categories showed a statistically reliable trend

A typical note reads close to this:

Trend & forecast analysis:
  • Scope: Evaluated 11 consecutive time periods across the analysis window
  • Model selected: Linear trend regression - straight-line fit applied across the window
  • Statistical reliability: Unreliable trend (R² = 0.0700, p = 0.44) - model explains only 7% of variance; movement cannot be distinguished from random variation
  • Next-period projection: Suppressed - trend did not clear the statistical reliability threshold (p < 0.05); projection withheld to prevent false precision
  • The real baseline value the scenario was compared against
  • The exact sensitivity used for every driver, and its source: measured from your own data, taken from a column description, or a general assumption
  • When measured from your data, the fit quality (R²), sample size (N), and significance behind that measurement
  • The full sensitivity range (the milder and more severe bounds) with real numbers
  • The rollback guardrail threshold
  • An explicit note that drivers are modeled independently, with cross-driver interaction boundaries noted

A typical note reads close to this:

What-if scenario simulation:
  • Baseline value: 29.98 (latest recorded actual)
  • Driver 'marketing_spend': +10.0% scenario shift (sensitivity coefficient β = 0.5, a coefficient you explicitly specified for this re-run)
  • Projected outcome: 31.48 (+5.0% net change vs baseline)
  • Sensitivity range: From 30.73 (2.5%) under milder conditions, down to 32.22 (7.5%) under severe conditions
  • Rollback guardrail: Action threshold set at 31.58
  • Model assumption: Modeled as independent drivers acting multiplicatively; cross-driver interaction is not modeled
  • The table dimensions and sample size the test was run on
  • The real test statistic (χ²) and its significance result
  • The practical effect size (Cramer's V), with a plain-language strength label (weak, moderate, or strong)
  • A reliability caveat when the underlying data is too sparse for the significance result to be fully trustworthy (Cochran cell count threshold)
  • For a composition shift question, how many categories were tested, how many showed a significant shift, and how many could not be tested at all

A typical note reads close to this:

Categorical association test (Chi-Square Test of Independence):
  • Contingency dimensions: 2 × 2 distinct categories (10,000 rows evaluated, 1 degrees of freedom)
  • Test statistic: χ² = 646.83
  • Statistical result: Statistically significant association (p < 0.001)
  • Practical effect size: Cramer's V = 0.25 (moderate association) - quantifies real-world association strength independently of sample size
  • Test validity: Reliable - all contingency cells satisfy the minimum expected count threshold (E_ij ≥ 5)

Frequently Asked Questions


Next Steps

Need help? Email support@edilitics.com with your workspace, job ID, and context. We reply within one business day.

Last updated on

On this page