Aggregation pipelines

Build aggregations stage by stage like a Compass pipeline — every stage shows its own result — and switch to JSON or ES|QL at any time.

Elasticsearch aggregations are powerful but hard to read: a query, buckets nested in buckets, and pipeline aggregations pointing at metrics by path. The Aggregations tab turns that tree into a flat list of simple stages, shows what each stage returns as you build it, and writes the JSON for you.

Open it from any Index view — it sits next to Query.

How stages work

Stack stages top to bottom with + Add stage or the quick chips under the list. The important rule:

Each Group by opens a new level. Every stage after it runs inside each bucket of that group.

So Filter → Group by city → Metrics means “for the matching documents, per city, calculate…”. Add another Group by month and the next stages run per city and month. The Group by card says so: Each next stage runs inside every city bucket.

Stages that read metrics — Keep only, Sort & limit, Running total and Change over time — attach to the level where those metrics are defined. Keep only avg_price > 1,500,000 placed after a Group by month still filters cities, because avg_price is a per-city metric; the card shows applies to city buckets.

Stage types

Group by options in detail:

• Values — field, number of buckets (default 10), order by Document count, Key or any metric defined inside the group, ascending or descending, and whether documents without the field are skipped or collected in a (missing) bucket.
• Date histogram — every minute, hour, day, week, month, quarter or year, or a fixed interval like 12h.
• Number / Date ranges — from, to and an optional name per range. Date ranges accept date math such as now-1M/M.
• Named filters — one condition per named bucket.
• Multi-field — combine several fields (values, histogram or date histogram) into one bucket key, with a page size.

Field pickers

Field pickers search as you type and show each field’s type. Only fields that can be aggregated are offered; text fields are greyed out with a hint to use their .keyword sub-field. Numeric operations only list numeric fields. A ⚠ marks fields mapped with different types in different indices behind an alias.

Working with stages

Every stage card has:

• a kind menu to turn it into another stage type;
• </> JSON — edit just this stage’s part of the request; shapes the form can’t show become a Custom JSON stage;
• Disable / Enable — a disabled stage is left out of the request and later stages re-validate;
• ⋯ — Collapse, Move up / down, Duplicate, Delete.

Mistakes are shown in red on the card — Needs a Group by above it, No metric named “avg_prise” at this level, Histogram needs a numeric field — and the rest of the pipeline still runs.

Live previews

With Live preview on, each stage runs its own small request — the pipeline up to and including that stage — and shows the result next to the form:

Previews are kept light so they never strain the cluster: at most 10 buckets per group, 3 sample documents and 3 top hits, a 10-second timeout, two requests at a time, cancelled when you keep typing (400 ms debounce), and cached for a minute. The real values are used when you Run.

Fast (sampled) previews large indices on a random sample of about 100,000 documents (random_sampler, Elasticsearch 8.2+); sampled numbers are marked ≈. On OpenSearch and older Elasticsearch it uses sampler instead. Indices under 200,000 documents always get exact previews.

Turn Live preview off to update previews only when you press Run.

The flow bar

Above the stages, the flow bar sums up the pipeline: Index 12.4M docs → 1 Filter 48,213 → 2 Group by city 10 buckets → 3 Metrics ×3 → 4 By month 12 / city → 5 Keep 7 of 10 → 6 Top 5 rows. Click a chip to jump to its stage; the stage you’re editing is highlighted.

Run, profile and results

• ▶ Run (⌘↵) executes the full pipeline without preview limits. The results pane shows one row per bucket combination — e.g. one row per city and month — as a Table or JSON, and Export results saves the rows as CSV or NDJSON.
• Profile runs with profile: true and shows the time each stage took on its card (⏱ 3.2 ms).
• Copy as cURL copies the request (credentials are never included).

The JSON view

Switch to JSON (⌘⇧J cycles views) to see the generated request. The gutter shows which stage each line comes from, and a side panel explains how each stage maps to Elasticsearch.

You can edit the JSON or paste any existing aggregation request: after a one-second pause (or when you click away) kabanos turns it back into stages. Anything that doesn’t map to a form — a second sibling bucket aggregation, an unknown aggregation type, a script — is kept as a Custom JSON stage; nothing is ever dropped. A notice tells you what was kept as custom.

The ES|QL view

On Elasticsearch 8.11 and later, ES|QL shows the same pipeline as a piped ES|QL query — one line per stage, with stage numbers in the gutter:

FROM listings-v7
| WHERE status == "active" AND price <= 2000000
| STATS avg_price = AVG(price), median_price = MEDIAN(price), doc_count = COUNT(*)
        BY city.name
| SORT doc_count DESC
| LIMIT 10
| WHERE avg_price > 1500000
| SORT avg_price DESC
| LIMIT 5

ES|QL is flatter than aggregations, so kabanos never silently changes the meaning: every stage it can’t translate exactly gets a Note — for example a nested group (can’t be expressed in a single ES|QL STATS), top documents, running totals, raw filters or “contains text” (which becomes a case-sensitive LIKE). Use Copy, Open in workspace or ▶ Run ES|QL to try it.

The ES|QL view is read-only and hidden on OpenSearch.

Saving and reusing

• Save stores the pipeline in your query library with a ∑ icon. It’s searchable like any query; clicking it reopens the pipeline in the Aggregations tab.
• ⋯ → Save as new…, Open in workspace (as a normal request block) and Copy request JSON.
• In a routine, a saved pipeline step exposes its flattened rows: steps.<id>.rows[0].avg_price.
• Your unsaved pipeline for each index is kept between launches.

Shortcuts