Production agentic AI for Indian accounting

Six-layer hallucination defence

Standard prompt-engineering tricks (RAG, temp=0, schema validation) miss too many cases when answers cost real money. We layered them:

1. Retrieval grounding — every tax claim must come from an indexed government regulation chunk; the agent refuses if no chunk above similarity 0.5 is found
2. Schema-constrained output — Zod → JSON Schema enforced via Anthropic structured outputs; the model cannot return gst_rate: "eighteen percent"
3. Referential probes — every tool call’s factual claims (HSN code, GST rate, GSTIN format) checked against a master table BEFORE execution; failure → self-correct loop
4. Idempotent writes — every mutating tool carries a client-supplied idempotency_key; double-firing is a silent no-op
5. Append-only audit log — hash-chained audit_log table with chain-of-thought capture; replayable for any conversation
6. Bounded cost ceiling — per-business monthly cap (default ₹500); model router downgrades Opus → Sonnet → Haiku as budget tightens; hard step-cap on agent loops

Hybrid retrieval pipeline

Pure vector retrieval misses queries with rare proper nouns (Section 269ST, specific HSN codes). Pure BM25 misses paraphrases. We combine both with Reciprocal Rank Fusion + Voyage rerank.

User query
   │
   ├──► OpenAI text-embedding-3-small (1536-d)
   │        │
   │        ▼
   │    pgvector HNSW    →    top-30 vector candidates
   │
   └──► PG tsvector ts_rank →   top-30 BM25 candidates
                  │
                  ▼
              RRF fusion (k=60)
                  │
                  ▼
            Voyage rerank-2.5-lite
                  │
                  ▼
            top-5 to LLM with citations

Why the hybrid: pure vector retrieval misses queries with rare proper nouns (Section 269ST, a specific HSN code); pure BM25 misses paraphrases. Combining both with RRF + rerank is what puts the right regulation chunk in front of the model before it answers — or the agent refuses.

Watch the agent work

A recorded playback of the tool-use loop — the agent reasons, calls real registered tools one at a time, grounds its claim in cited regulation, and stops for your approval before anything posts.

The stack, opening up

One piece is already open to any developer today; the reusable primitives behind it are being packaged next.

Benchmarking Indian-tax agents

There’s no good public yardstick for how well an AI agent handles Indian tax & compliance. We’re building one — spanning closed-form factual recall, multi-hop reasoning, and tool-using execution — because “sounds right” isn’t good enough when answers move money. More to share as it firms up.

Building this together

OnGravy is led by Pratik Revankar. Built since early 2025; now live, with a design-partner program for chartered accountants.

If you are a CA interested in the design-partner program, or an engineer interested in working on production agentic AI: hello@ongravy.com.