Edilitics | Data to Decisions

Methodology Notes

Inspect the exact statistical method, sample size, and test result behind every AskEdi response, written by the system, never by the AI.

Methodology Notes show the statistical work behind an AskEdi response in plain language: the sample size, the method chosen, and the real test result. Unlike the response text, which the AI writes to answer your question, the methodology note is generated directly by Edilitics, so it cannot misstate a number, a p-value, or a method.


Why This Exists

An AskEdi response tells you what the data shows. A methodology note tells you how AskEdi knows. The two are generated separately: the response is the AI's narrative, written to be readable and specific to your question. The note is a factual account of the computation, written by the system itself, independent of the AI's phrasing.

This matters because it gives you two independent ways to check the same answer. If the response says a trend is reliable, the methodology note states the actual p-value and sample size that made it reliable. If a section reports a correlation, the note names the exact statistical test used and its result.


Opening Methodology Notes

Every response with an underlying statistical computation has a sparkle icon in the action row beneath it. Click it to open the notes for that response.

The panel shows one note per statistical section in the response. A Root Cause Analysis response, for example, shows separate notes for the trend, the contribution ranking, the correlation, and the anomaly scan.

A response with no underlying statistical computation, such as a plain narrative or a chart with no trend or comparison behind it, does not show the sparkle icon.


What a Methodology Note Contains

The exact content depends on which response type generated it, but every note follows the same principle: state what was measured, how, and with what result.

  • Sample size and the method chosen for the trend, and why
  • The observed movement in the actual data, not just the direction of a fitted line
  • The statistical reliability of the trend: the fit quality and whether it is distinguishable from normal variation
  • How many segments were ranked in the contribution analysis, and whether the difference between them was tested for significance
  • Which secondary metric was identified as the strongest correlate, and whether that relationship is statistically significant
  • How many data points were checked for anomalies, and the most extreme deviation found, even when nothing crossed the anomaly threshold

A typical note for the trend section reads close to this:

Trend & forecast analysis:
  • Data points analyzed: 10 time periods
  • Method: linear trend (regression), no repeating seasonal cycle detected
  • Observed movement: down 6.2 percentage points from the first to the last period
  • Statistical reliability: the trend explains 15% of the variation (p=0.28), NOT statistically significant
  • Confidence rating: low
  • Sample size and the projection method chosen, and why
  • The real observed movement from the first to the last period, alongside the fitted trend
  • Whether the trend is statistically reliable, and therefore whether a forecast number is shown at all
  • For a category breakdown, how many categories showed a statistically reliable trend

A typical note reads close to this:

Trend & forecast analysis:
  • Data points analyzed: 11 time periods
  • Method: linear trend, no seasonal cycle detected
  • Statistical reliability: the trend explains only 7% of the variation (p=0.44), NOT statistically significant
  • Next-period projection: suppressed, the trend is not statistically reliable, so projecting forward
    would give false precision to a noisy signal
  • The real baseline value the scenario was compared against
  • The exact sensitivity used for every driver, and its source: measured from your own data, taken from a column description, or a general assumption
  • When measured from your data, the fit quality and sample size behind that measurement
  • The full sensitivity range (the milder and more severe bounds) with real numbers
  • The rollback threshold
  • An explicit note that drivers are modeled independently, with no interaction effect between them assumed

A typical note reads close to this:

What-if simulation:
  • Baseline (latest actual): 29.98
  • Driver 'marketing_spend': +10.0% change, sensitivity coefficient 0.5 (a coefficient you explicitly
    specified for this re-run)
  • Projected result: 31.48 (+5.0% vs baseline)
  • Sensitivity range: 30.73 (2.5%) under milder assumptions, up to 32.22 (7.5%) under more severe ones
  • Rollback threshold: 31.58
  • Note: drivers are modelled as acting independently, real-world interaction between them is not captured
  • The table dimensions and sample size the test was run on
  • The real test statistic and its significance result
  • The effect size, with a plain-language strength label (weak, moderate, or strong)
  • A reliability caveat when the underlying data is too sparse for the significance result to be fully trustworthy
  • For a composition shift question, how many categories were tested, how many showed a significant shift, and how many could not be tested at all

A typical note reads close to this:

Categorical association test (chi-square test of independence):
  • Table dimensions: 2 x 2 distinct category values, 10,000 rows used
  • Chi2 statistic: 646.83
  • Result: statistically significant (p<0.001)
  • Effect size (Cramer's V): 0.25 (moderate association), measures practical strength
    independently of sample size
  • Reliability: all contingency table cells meet the expected count threshold, p-value is trustworthy

Frequently Asked Questions


Next Steps

Need help? Email support@edilitics.com with your workspace, job ID, and context. We reply within one business day.

Last updated on

On this page