Category Comparison
How AskEdi tests whether two categories are really related, or whether a category mix genuinely shifted, using real statistical tests instead of a visual read.
Category Comparison is AskEdi's response to questions that test a relationship between two categorical fields, or whether a category's mix genuinely changed between two periods. Neither question has a numeric metric to explain, so AskEdi runs a dedicated statistical test rather than the regression and contribution math used for a "why did this number move" question.
When Category Comparison Activates
AskEdi classifies the intent of your question automatically. There are two distinct shapes, both routed to Category Comparison.
Asks whether two categorical fields are related to each other:
- "Is shipment direction associated with on time delivery outcome?"
- "Does weather condition vary by city?"
- "Is region associated with churn status?"
Asks whether a category's mix changed between two periods:
- "Did the customer mix shift toward enterprise?"
- "Did the proportion of mobile orders change?"
A question that mentions a category only as a filter on a numeric metric, for example "why did revenue drop in the West region," is answered by Root Cause Analysis instead, the metric being explained decides the response type, not whether a category is mentioned.
What Data This Needs
Category Comparison needs genuinely categorical columns, not just any column with repeated values:
- A small number of distinct values on each column. Both columns in an association question need to be truly categorical, region, status, or type, not a high-cardinality field like an order ID that happens to repeat.
- At least two values in each category. A column already filtered down to a single value has nothing to test against.
- A usable date column, for a composition shift question. Comparing a category's mix between two periods needs a real date or timestamp column with enough range to define two distinct periods to compare.
AskEdi checks that both categorical columns genuinely have a small number of distinct values before answering, and for a composition shift question, that there is a usable date column with two clear periods to compare. If either check fails, AskEdi explains why and suggests what to ask instead. No credit is consumed for that response.
What a Category Comparison Response Includes
Every Category Comparison response is four sections, shorter than Root Cause Analysis because it tests exactly one relationship, association or shift, never both at once.
1. Observed Pattern
States what is being compared, the raw shape of the data, and why the comparison matters in plain terms.
2. Statistical Test Result
The real computed result, never a visual estimate.
States whether the two categories are statistically associated, the significance result, and the effect size together, never significance alone. Effect size is described as weak, moderate, or strong.
With enough data, even a very weak relationship can be statistically significant. AskEdi always cites the effect size alongside significance, so a real-but-tiny association is not overstated as a meaningful finding.
Renders as a table, one row per category value, comparing the share each category held in the earlier period against the later period:
| Category | Earlier period | Later period | Change | Significant |
|---|
3. Category Breakdown
Names the specific category values driving the result, not just the aggregate statistic. For an association question, this describes which combination of categories appears over or under represented. For a composition shift question, this highlights the category with the largest real change.
4. Decision & Impact Summary
The closing section renders as a highlighted card, not plain text. It contains three labeled lines:
| Line | What it contains |
|---|---|
| Recommended action | A concrete next step grounded in the specific category values identified in the breakdown above. |
| Evidence | The real test result and effect size (association) or the real percentage-point shift (composition) that supports the recommendation. |
| Confidence | High, medium, or low. Cites the real significance and effect-size result, the AI does not grade its own confidence. |
What This Looks Like
A shortened example, for a question like "is shipment direction associated with on time delivery outcome?":
Observed Pattern This analysis compares shipment direction in 2 distinct categories against on time delivery outcome in 2 distinct categories.
Statistical Test Result Yes, there is a statistically significant association. A p-value of less than 0.001 means this result is very unlikely to have occurred by chance. The effect size of 0.25 falls in the moderate range.
Category Breakdown One shipment direction is systematically more concentrated in on time outcomes than the other, though both directions still contain a mix of results.
Decision & Impact Summary (highlighted card) Recommended action: Manage forward and return shipments as separate service-performance cohorts. Confidence: medium.
The real response is longer and grounded in your own column names and numbers. This shows the shape, not the content.
Verifying the Numbers Behind the Answer
The contingency table, the significance test, and the effect size are all computed server-side before the response is written. To inspect the work directly:
- Analysis view: shows the exact query that ran against your data source.
- Methodology Notes: shows the real test statistic, the sample size, and the effect size with a plain-language strength label, independent of what the response text says.
Frequently Asked Questions
Next Steps
Root Cause Analysis
Explain why a numeric metric moved instead of testing a relationship between two categories.
Charts
Visualize a category breakdown directly instead of testing it statistically.
Methodology Notes
Inspect the exact test statistic, sample size, and effect size behind every comparison.
AskEdi
Back to the AskEdi overview: how a chat works, response types, and how to get started.
Need help? Email support@edilitics.com with your workspace, job ID, and context. We reply within one business day.
Last updated on