Sampling

Sampling reduces your dataset to a subset of rows using one of three strategies: simple random sampling by percentage, systematic sampling by fixed interval, or stratified sampling by category group. The original dataset is replaced by the sample. Every operation downstream works on the sampled result.

When to Use Sampling

• Speed up development. Work on 10% of a large dataset while building and testing transformations.
• Create train/test splits. Sample a percentage of rows for model training before exporting.
• Audit a structured subset. Use systematic sampling to review every Nth transaction without bias.
• Ensure category representation. Use stratified sampling to guarantee rows from each region or product tier.
• Reduce export volume. Sample before exporting to keep output files to a manageable size.

Sample Dataset

The examples in this doc use the Edilitics sample orders dataset. Download it to follow along in your own workspace.

Relevant columns for Sampling examples:

Sampling Methods

How to Apply Sampling

Simple Random Sampling

Selects a random percentage of rows from the full dataset. Every row has equal probability of being selected.

Fields:

Before and After (Sample Size 60%, 500-row dataset):

Input: 500 rows

Output: approximately 300 rows (random selection, order not preserved)

Systematic Sampling

Selects every Nth row starting from an offset position. N is computed automatically as total rows divided by the requested sample size.

Fields:

Before and After (Sample Size 50, 500-row dataset, Offset 3):

Selection Frequency = 500 / 50 = 10. Rows selected: 3, 13, 23, 33 ... (every 10th row starting at position 3).

Input: 500 rows

Output: 50 rows

Stratified Sampling

Divides the dataset by unique values in a categorical column and samples from each subgroup. Two distribution modes: Proportionate and Disproportionate.

Fields:

Proportionate maintains original group ratios. If region has 40% North America and 20% Europe, the output sample preserves that 2:1 ratio.

Disproportionate takes an equal fixed count from every group regardless of size. Useful when smaller groups need representation equal to larger ones.

Before and After (Stratified, Disproportionate, Group By Column region, 10 from each subgroup):

Input: 500 rows across 5 regions

Output: 50 rows (10 per region)

Code Equivalent

-- Simple random sample (PostgreSQL)
SELECT *
FROM orders
ORDER BY RANDOM()
LIMIT 300;

-- Systematic sample: every 10th row (DuckDB)
SELECT *
FROM (
  SELECT *, ROW_NUMBER() OVER () AS rn FROM orders
)
WHERE (rn - 3) % 10 = 0;

-- Stratified sample: 10 rows per region (PostgreSQL)
SELECT *
FROM (
  SELECT *,
    ROW_NUMBER() OVER (PARTITION BY region ORDER BY RANDOM()) AS rn
  FROM orders
) t
WHERE rn <= 10;

import polars as pl

# Simple random sample: 60% of rows
df_simple = df.sample(fraction=0.6, seed=42, with_replacement=False)

# Systematic sample: every 10th row starting at offset 3
df_systematic = df[list(range(3, df.shape[0], 10))]

# Stratified sample: proportionate, 300 total rows
ratios = dict(
    df["region"]
    .value_counts()
    .with_columns(pl.col("count") / df.shape[0])
    .iter_rows()
)
df_stratified = pl.concat([
    df.filter(pl.col("region") == region).sample(n=int(ratio * 300))
    for region, ratio in ratios.items()
])

# Stratified sample: disproportionate, 10 rows per region
df_disproportionate = pl.concat([
    df.filter(pl.col("region") == region).sample(n=10)
    for region in df["region"].unique()
])

Operation Reference

Frequently Asked Questions

Next Steps

Need help? Email support@edilitics.com with your workspace, job ID, and context. We reply within one business day.