ABS Data API Skill

Query live ABS datasets, return data + citations, optional tables/charts/reports.

Bundled Resources

File	Purpose
INLINECODE0	Metadata cache manager — refresh catalog, search all 1,200+ dataflows, generate structured metadata
INLINECODE1

NL → dataset mapper — curated lookup + fuzzy fallback + ambiguity detection | | scripts/abs_query.py | Query engine — fetches data, formats output, summary/report/describe modes | | scripts/test_presets.py | Preset validation — tests all presets against live API, pass/fail summary | | presets.json | 20 validated preset queries for common indicators | | metadata.overrides.json | Manual overrides for discontinued datasets and nicer labels | | references/dataset-catalog.md | ~55 curated datasets with IDs, versions, notes (human reference) | | references/api-guide.md | ABS API URL patterns, response structure, example queries | | references/sdmx-patterns.md | Dimension codes (REGION, TSEST, FREQ, MEASURE) per dataset |

---

Quick Start

CODEBLOCK0

---

Workflow

Step 1 — Identify the dataset

1. 1. Check references/dataset-catalog.md for the dataflow ID and version
2. If not found, run python3 scripts/abs_search.py "<user query>" for fuzzy match + ambiguity hints
3. If still not found, run python3 scripts/abs_cache.py search "<term>" (searches all 1,200+ dataflows)

Step 2 — Determine dimension key

1. 1. Check presets.json — if a preset exists, use it directly
2. Read references/sdmx-patterns.md for common dimension codes
3. For an unfamiliar dataset, fetch its structure:

CODEBLOCK1

Step 3 — Query the data

CODEBLOCK2

Step 4 — Format and deliver

• - Default text format includes citation. Use --format table for markdown tables.
• For charts, requires matplotlib; gracefully falls back if not installed.
• Use --summary latest for quick briefs with change context.
• Use --report macro-snapshot for a full multi-indicator briefing.
• Always include the citation line in any response to the user.

---

Presets (20 validated)

Common indicator queries are bundled in presets.json. All validated live March 2026.

CODEBLOCK3

Key presets: cpi-annual-change, unemployment-rate, participation-rate, employment-level,
underemployment-rate, labour-force-size, gdp-annual-change, wage-annual-change,
population-national, dwelling-prices-mean, trade-balance, goods-exports, goods-imports,
household-spending-change.

---

Output Formats

Flag	Output
(default)	Human-readable text with friendly labels + citation
INLINECODE33

Markdown table with friendly labels and rendered periods |
| --format csv | CSV with raw codes + citation comment |
| --format json | JSON with raw codes + *_label fields + TIME_PERIOD_rendered |
| --chart | PNG chart with dataset title, subtitle, latest-point annotation |
| --summary latest | Latest value + previous + absolute/percentage-point deltas + textual summary |
| --report macro-snapshot | Compact multi-indicator macro briefing (7 key economic indicators) |
| --citation-style analyst | Analyst-style source footnote block |
| --flat-view | AllDimensions format (wider; may be large) |

---

Period Rendering

All output modes now render periods in human-readable format:

• - 2026-01 → January 2026
• INLINECODE44 → December quarter 2025
• INLINECODE45 → March quarter 2025
• Ranges: INLINECODE46

This applies to table headers, text output, citations, chart labels, and summary/report output.

---

JSON Output with Labels

INLINECODE47 returns both raw dimension codes and friendly *_label fields:

CODEBLOCK4

Backward compatible — raw codes are preserved.

---

Ambiguity Detection

INLINECODE49 classifies ambiguity when multiple datasets match:

• - frequency — monthly vs quarterly
• geography — national vs state vs SA2/LGA
• measure — index vs % change vs level
• series — original vs seasonally adjusted
• dataset — distinct series cover the same topic

Prints clarifying questions to help the user or agent narrow the query.

---

Cache and Metadata

Command	Description
INLINECODE50	Fetch all dataflows from ABS, save to INLINECODE51
INLINECODE52

Generate metadata.generated.json from presets + catalog + overrides |
| abs_cache.py status | Show cache age, dataflow count, structure count, metadata status |
| abs_cache.py search <term> | Search across all cached dataflows |
| abs_cache.py structure <ID> [VER] | Fetch and cache DSD for a specific dataflow |

Runtime metadata priority: metadata.generated.json > catalog.json > dataset-catalog.md.
Override quirks (discontinued datasets, nicer labels) in metadata.overrides.json.

---

Validation

CODEBLOCK5

---

Ambiguity Rules

• - Multiple matching datasets: prefer the most specific. E.g. for "inflation", CPI beats CPI_M beats PPI.
• No dimension key provided: use all — the API will return everything; then filter. If the response is large (>100 observations), the tool warns you.
• Version unknown: look up from generated metadata, then catalog; try 1.0.0 as last resort.
• User asks for "latest": always add --latest flag (uses lastNObservations=1).
• Census data requested: redirect to the census-database skill; this skill handles ABS time-series only.
• Chart requested but matplotlib missing: output text/table format and note how to install matplotlib.
• Retail Trade (RT) requested: DISCONTINUED after June 2025. Use HSI_M or BUSINESS_TURNOVER instead.
• RPPI requested: note the API only has data to ~2021-Q4. Use RES_DWELL_ST for current dwelling prices.

---

Citation Format

All responses include a citation:

Source: Australian Bureau of Statistics, <Full Dataset Name> (Cat. <catalogue-number>; dataset <ID>; v<version>). <human-readable-period>. Retrieved via ABS Data API: <url>.

Example:

Source: Australian Bureau of Statistics, Consumer Price Index (Cat. 6401.0; dataset CPI; v2.0.0). January 2026. Retrieved via ABS Data API: https://data.api.abs.gov.au/rest/data/ABS,CPI,2.0.0/.

---

What's New in v1.0.2

1. Metadata Generation

• - gen-metadata command: Builds unified metadata from presets + live catalog + manual overrides
• Auto-refresh: Generated metadata automatically updates when older than 24 hours
• Ensures all datasets are findable and correctly labeled, even as ABS API evolves

CODEBLOCK6

2. Smart Ambiguity Detection

• - Classifies ambiguity when multiple datasets match a user query (frequency, geography, measure, series, dataset)
• Provides clarifying questions grouped by intent (prices, wages, employment, housing, etc.)
• Flags discontinued datasets with replacement suggestions (e.g., RT → HSI_M)
• Uses curated intent groups + ambiguity tags to guide disambiguation

CODEBLOCK7

3. Summary Mode with Change Context

• - --summary latest: Shows latest value + previous + absolute deltas + brief summary
• Automatically detects rates/growth measures and uses percentage-point notation instead of misleading relative % changes

- Example: Unemployment rate rises from 4.0% to 4.3% → "change of +0.3 percentage points" (NOT "+7.5% relative change") - Applies to: unemployment, participation, inflation rates, growth measures

• - Ideal for quick briefings and executive summaries

CODEBLOCK8

4. Macro-Snapshot Report

• - --report macro-snapshot: Single-command economic briefing covering 7 key indicators
• Fetches CPI, unemployment, participation, employment, GDP growth, wage growth, household spending
• All with change context and period rendering
• Perfect for media snippets or executive briefings

CODEBLOCK9

5. Percentage-Point Delta Fix

• - Smart detection: Automatically recognizes rates and growth measures via keyword matching
• Applies percentage-point notation to avoid confusion with relative % changes
• Examples:

- Unemployment: 4.0% → 4.3% = +0.3 percentage points (not +7.5%) - CPI: 3.5% → 3.2% = -0.3 percentage points - Wage growth: 4.1% → 4.0% = -0.1 percentage points

• - Applies to all output modes: text, table, JSON, summary

6. Metadata Overrides (metadata.overrides.json)

• - Discontinued datasets (RT → HSI_M, RPPI stale warning)
• Friendly names for complex dataset IDs
• Replacement hints with explanations
• Easy to extend for future dataset changes

The query engine appends this automatically. Do not strip it from tool output.

---

Changelog

v1.0.2 (March 2026)

New Features:

• - ✨ Metadata generation (gen-metadata command) — builds unified metadata from presets + catalog + overrides with auto-refresh
• ✨ Smart ambiguity detection — classifies multiple matches by type (frequency, geography, measure, series, dataset) and provides grouped clarifying questions
• ✨ Summary mode with change context (--summary latest) — shows latest + previous + absolute deltas + brief summary
• ✨ Macro-snapshot report (--report macro-snapshot) — single-command economic briefing covering 7 key indicators
• ✨ Percentage-point delta fix — rates/growth measures automatically use pp notation instead of misleading relative % changes
• ✨ Intent grouping — curated entries now include intent_group and ambiguity_tags for smarter disambiguation

Improvements:

• - Discontinued dataset detection (RT → HSI_M, RPPI stale warning)
• Better metadata overrides system for dataset quirks
• Enhanced search with ambiguity classification
• All output modes now respect percentage-point notation where applicable

Affected Scripts:

• - abs_cache.py — added gen-metadata command and generate_metadata() function
• INLINECODE91 — added ambiguity detection, intent grouping, and clarifying questions
• INLINECODE92 — added --summary latest, --report macro-snapshot, percentage-point delta detection
• INLINECODE95 — new file for manual dataset overrides

v1.0.1 (Previous)

• - Base preset system with 20 validated queries
• Curated dataset catalog and SDMX dimension references
• Cache refresh and fuzzy search capabilities