A chain doesn’t hand you application state, it hands you a stream of events with a footnote attached: this stream can be revised. Ponder is one good vehicle for turning that stream into state you can actually query and trust, but the tool is secondary to the problem. This post is mostly about the problem.
The problem#
A block your service already processed can stop being canonical. An RPC node can tell you two different things about the same height five minutes apart. An application built to read “current state” directly off the chain on every request has no principled way to handle any of that; it just re-derives an answer each time and hopes the chain agrees with itself in the meantime.
The actual engineering problem isn’t “how do I read events off a contract.” It’s: how do I turn a stream of events that can be reorganized, replayed, and re-observed into application state I can trust enough to build a product on. Everything below, backfills, reorg handling, crash recovery, reindexing, is really one problem viewed from different angles: keeping derived state honest about the event history it’s built from.
Raw events vs derived state#
Two different things live in an indexer, and conflating them is where most confusion starts.
transfer_event: the durable historical record. One row per canonicalTransferlog in the indexed history. Rows don’t change shape once written; a reorg is the one thing that adds or removes them.account: a derived projection. Today’s balance per address, computed from the event history, not observed directly from the chain.
The second table exists because recomputing “current balance” by replaying the full transfer history on every read doesn’t scale. You materialize it once and keep it incrementally up to date instead. That relationship, projection derived from durable events, is where the interesting properties come from:
- Backfill is possible because
accountcan be thrown away and rebuilt by replaying every row intransfer_eventin order: it’s a deterministic function of that history and nothing else. - Deterministic replay matters because recovery can cause historical events to be processed again after their previous effects have been rolled back first. Given the same event and the same preceding database state, the handler needs to produce the same result. That’s a related property to idempotency, but not the same one: idempotency is about safely reapplying an operation on top of unknown prior state; deterministic replay is about Ponder rolling state back to a known point before reprocessing, so the handler never has to reason about “did this already run.”
- Reorg rollback is tractable because there’s no special “undo”: if a block turns out not to be canonical, you roll the event history back to before that block and replay forward again, and the projection lands wherever the corrected history says it should.
None of that guarantees the projection is complete, though. That depends entirely on how much of the real history you actually observed, which is the subject of its own section below.
Building the indexer with Ponder#
A Ponder project has three parts: a config file declaring which contracts and chains to index, a schema file declaring the Postgres tables, and indexing functions that run once per matching event.
Config: declare the contract and the chain it lives on:
// ponder.config.ts
import { createConfig } from "ponder";
import { erc20Abi } from "./abis/erc20Abi";
export default createConfig({
chains: {
mainnet: {
id: 1,
rpc: process.env.PONDER_RPC_URL_1,
},
},
contracts: {
Token: {
abi: erc20Abi,
chain: "mainnet",
address: "0xYourTokenAddressHere",
startBlock: 18000000,
},
},
});Schema: one table for the durable event history, one for the derived projection:
// ponder.schema.ts
import { index, onchainTable } from "ponder";
export const account = onchainTable("account", (t) => ({
address: t.hex().primaryKey(),
balance: t.bigint().notNull().default(0n),
}));
export const transferEvent = onchainTable(
"transfer_event",
(t) => ({
id: t.text().primaryKey(),
from: t.hex().notNull(),
to: t.hex().notNull(),
amount: t.bigint().notNull(),
blockNumber: t.bigint().notNull(),
}),
(table) => ({
blockNumberIdx: index().on(table.blockNumber),
}),
);Indexing function: runs once per Transfer log, appends to the history, and updates the projection:
// src/index.ts
import { ponder } from "ponder:registry";
import { account, transferEvent } from "../ponder.schema";
const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000";
ponder.on("Token:Transfer", async ({ event, context }) => {
const { from, to, value } = event.args;
await context.db.insert(transferEvent).values({
id: event.id,
from,
to,
amount: value,
blockNumber: event.block.number,
});
if (to !== ZERO_ADDRESS) {
await context.db
.insert(account)
.values({ address: to, balance: value })
.onConflictDoUpdate((row) => ({ balance: row.balance + value }));
}
if (from !== ZERO_ADDRESS) {
await context.db
.insert(account)
.values({ address: from, balance: -value })
.onConflictDoUpdate((row) => ({ balance: row.balance - value }));
}
});event.id is Ponder’s own globally unique identifier for the event, so there’s no need to build one by hand from the transaction hash and log index. The zero address gets excluded on both sides: from == 0x0 is a mint, to == 0x0 is a burn, and neither makes the zero address a real holder worth a row in account. The -value seed on the sender side is deliberate, not a bug: if a sender has no prior row, that’s a real address transacting outside your observation window, and a negative balance is the honest signal of that, not something to paper over. The next section is about exactly when and why that happens.
That’s the core of the indexer, not a complete deployable system: it still needs a real ABI, a real token address, RPC and database configuration, and dependencies installed. What it does demonstrate completely is the durable-event/derived-projection split from the previous section, in actual code.
A start block is part of your data model#
startBlock looks like a config knob. It’s actually a claim about how much history your projection can see, and that claim shapes what the data means.
Start indexing at the token’s deployment block, and you’ve observed every transfer that ever happened. account.balance is a full reconstruction: replay the complete transfer_event history and you get every address’s real balance, because there’s nothing before your observation window that could be missing.
Start later, and transfer_event is still correct: every row in it accurately reflects a real event you observed. It’s no longer complete. An address that already held a balance before your startBlock shows up in account reflecting only what it did after that point, not its real balance. It can go negative: an address that, from your indexer’s point of view, only ever sends, because it received its tokens before you started watching, looks like it’s spending money it never had.
That distinction is worth sitting with, because it’s easy to blur: the indexer can be entirely internally consistent, every event stored once, every projection update correctly derived from the events it saw, while still producing application state that’s semantically incomplete, because the events it saw were never the whole story. Correctness of the mechanism and completeness of the data are different properties. A startBlock picked for convenience (“whenever we started this project”) rather than for correctness (the contract’s real deployment block, or a point where you independently know the true starting balances) trades one for the other without saying so.
Reorg handling#
This is where the durable-events-plus-projection split earns its keep. Ponder maintains its own transaction log, independent of any table you declare, recording every write your indexing functions make. When its sync engine detects a reorg, the chain no longer agreeing with what it reported before at some height, it walks that log back to the common ancestor, the last block both the old and new chain agree on, discards the recorded changes after that point, and reprocesses the new canonical chain by re-running your indexing functions against it.
The handler above has no reorg logic in it, because none belongs there. The mechanism for “what happens when the chain changes its mind” lives in Ponder’s sync engine, not in application code, which is exactly what makes it safe to write indexing functions as if the chain were append-only, even though it isn’t.
One thing worth being precise about: this rollback runs against Ponder’s own internal log, not against whatever indexes you declare on your own tables. The blockNumberIdx on transfer_event above is for the queries you’d actually run, “transfers in this block range”, not for reorg performance; reorg recovery works identically without it.
Crash recovery and deterministic replay#
Reorg recovery and crash recovery look similar, both roll something back and reprocess, but they answer different questions.
A reorg means the chain itself changed: some of what you indexed is no longer canonical, so Ponder rolls back to the common ancestor and processes the replacement canonical history. A crash is different: Ponder does not try to preserve the partially indexed unfinalized region. Ponder tracks a content hash of your config, schema, and indexing-function code, called the build_id, to recognize whether a restart is the same app resuming or something new. On recovery with the same build_id, it rolls back unfinalized changes and resumes from finalized state. That’s deliberately broader than a reorg rollback, which only needs to unwind history after the common ancestor.
Both recovery paths lean on the same property: indexing functions are meant to be deterministic. The same event against the same prior state produces the same result every time, which is what makes “discard some writes and reprocess” a safe recovery strategy instead of a guess. A handler that depended on wall-clock time, external randomness, or state outside the event and the database would break that guarantee, and neither recovery path would be trustworthy anymore.
What happens when indexing logic changes#
Changing your schema or your indexing-function code doesn’t resume the current build, it starts a new one. A different build_id means the old derived state might not match what the new code would have produced from the same history, so Ponder backfills fresh rather than resuming.
“Fresh backfill” doesn’t mean “refetch everything from your RPC provider,” and that distinction is the actual point here. Ponder caches the raw chain data it fetches, logs, blocks, transactions, independently of build_id, and that cache persists across restarts and rebuilds. A logic change replays your new handlers against data Ponder already has; it doesn’t re-download it. Recomputing derived state and refetching raw data are two different costs, and keeping them separate is what makes iterating on indexing logic against a large history practical instead of something you avoid. That separation is useful well beyond blockchain indexing: keeping source data reusable while treating projections as disposable makes changes to application logic much cheaper.
Ponder vs The Graph, and when self-hosted indexing makes sense#
I’ve written before about the other end of this trade-off: indexing with The Graph. Both solve the same underlying problem, turning chain events into queryable state, with a different shape:
- The Graph: chain → mapping functions → a subgraph’s own entity store → GraphQL API, executed by a network of indexers you don’t operate.
- Ponder: chain → TypeScript indexing functions → Postgres tables you define → SQL, GraphQL, or direct access from the rest of your backend, executed by a service you do operate.
This isn’t “one has a fixed shape and the other doesn’t”; a subgraph defines its own schema too. What actually differs is who runs the indexing and where the result lives. Ponder is worth the operational cost when the indexed state needs to live in the same database as the rest of your backend, when other services already read and write Postgres and a second, differently-shaped data source to reconcile against is real cost, or when the indexing logic needs to be tightly coupled to other backend code rather than sitting behind a separate GraphQL boundary. A hosted or decentralized indexer is worth it when what you actually want is to operate less infrastructure, and a GraphQL API in front of someone else’s indexing pipeline is enough.
Takeaway#
The tool underneath this post could change; the properties it needs don’t. Turning a chain’s event stream into application state worth trusting means separating source events from derived projections, being explicit about how much history the system has observed, and treating reorgs, crashes, and logic changes as normal operating conditions rather than exceptional cases. Ponder provides one practical way to implement that model on EVM chains; the underlying design principles carry well beyond Ponder.