Zinder: rebuilding the Zcash indexer around its own protocol

General
1

Zinder is a Zcash indexer with its own query protocol, a drop-in lightwalletd-compatible runtime, and separately deployable ingest, wallet, and explorer components. If your wallet already speaks lightwalletd, adoption can be an endpoint substitution rather than a rewrite.

I was trying to get Zcash products working end-to-end: a node, the services that interpret the chain, and something a user can actually use. That path, and the products I built to exercise it, are in Building Zcash products end-to-end. This post is the indexer that path forced into existence.

I wanted a more performant indexer because the existing ones kept getting in the way of those products. Most indexers in this ecosystem are shaped by one of two premises: they mirror how lightwalletd works, or their architecture is organized around the wallet SDK they were built to serve (like zcashd). In both cases, the consumer’s protocol ceases to be an interface and becomes the indexer’s internal architecture, and every subsequent decision inherits that coupling.

I wanted an indexer with its own protocol as the spine, with supporting the existing ecosystem as a module of equal quality, not the premise the whole system is built around.

A protocol of its own, compatibility as a module

Wallet and application clients talk to a native gRPC contract, WalletQuery, served by the zinder-query runtime, with a remote-first Rust SDK (zinder-client) on top. It names things the lightwalletd protocol never named: typed errors with stable reason codes, explicit capability discovery, and resumable chain events. Reads are pinned to a single chain epoch (ChainEpoch), so a single response can never mix artifacts from competing tips or from a half-committed update.

Lightwalletd compatibility lives where it belongs: in zinder-compat-lightwalletd, a separate runtime that translates the vendored CompactTxStreamer protocol onto the same serving layer. The vendored protocol is pinned, and CI runs a live parity suite that compares responses against the reference Go lightwalletd, so any compatibility break will result in a failing test.

Modularity: run what your use case needs

Most indexers bundle ingestion, indexing, and serving into one all-or-nothing process. A wallet-only Zinder deployment never pays for explorer indexes, and explorer state is never a prerequisite for wallet sync. Zinder splits ingest, projection, and serving into independently deployable runtimes:

  • zinder-ingest is the only process that writes canonical chain state and owns the live mempool.
  • zinder-projector is the only process that writes wallet projection state.
  • zinder-query and zinder-compat-lightwalletd are serving runtimes; they do not write storage.
  • An optional explorer plane (zinder-explorer, its materialized views, and a REST adapter compatible with CipherScan) adds aggregates for block-explorer workloads.

You enable the planes your product needs, and each advertised capability is derived from what the deployment is actually composed of, not claimed by a config flag.

Storage: the noisiest reader cannot stall the writer

Zinder can serve lightwalletd wallets, native SDK applications, and explorers from a single deployment because those workloads do not share a writable handle with the ingestion. Canonical ingestion is an ordered, block-local write stream; wallet serving needs cross-block address history and unspent outputs; explorers need expensive aggregates. When all three share a single foreground commit, the writer maintains indexes that are not chain-truth, and a demanding explorer workload can slow the writer down.

Each store has exactly one primary writer. Every reader opens its own RocksDB secondary on the same host, catches up explicitly, and checks it is looking at the right store before it admits traffic. A serving response is bound to one consistent snapshot of chain state (one admitted read pair at one exact fence), so in-flight requests keep that view while newer epochs publish. Query load never touches a primary, and no reader ever repairs or mutates storage.

The ecosystem it serves

Zinder reads from Zebra, via JSON-RPC and, when available, Zebra’s indexer gRPC stream. Releases are certified against current Zebra, including Ironwood (NU6.3) decoding and live regtest plus lightwalletd parity against Zebra 6.2.2.

On the consumer side, I validated both Zallet and ZODL on Testnet against the same Zinder deployment. Zallet talks native WalletQuery; ZODL talks the lightwalletd compatibility endpoint through the trusted-TLS route. That Testnet work covers fresh-wallet, restore, sync, send, mempool, mining, continuous-follow, and restart. It is not a public-operator or reorg claim.

The native Zallet backend is the open zcash/zallet#701 PR, with a replayable Regtest harness in zcash/integration-tests so reviewers can reproduce the contract without a public network. Official packaging of that backend is still tracked separately.

What running real wallets uncovered

Building the indexer was not enough. Pointing real wallets at it found bugs that only show up at the boundary, and those fixes belong in the wallet and SDK repositories.

Zinder pins every read to one chain epoch and treats mempool streams as fallible. Zallet did not, so the native backend work also produced wallet-side fixes: reacquire expired chain views, propagate mempool stream failures, and own admitted runtime tasks through startup (the last of those is already merged). Those PRs exist because the indexer contract is stricter than the assumptions the wallet previously made.

The same loop produced the Ironwood wallet and tree-storage fixes from the earlier product work, now merged in librustzcash and incrementalmerkletree so other clients inherit them:

What operators get today, and the single-host limit

Zinder is built for the jobs operators actually run: configure by environment, gate on readiness, cap memory, and verify the supply chain. Configuration is environment-variable driven and validated before anything opens or binds. Every runtime exposes health, readiness, and metrics endpoints, and readiness means exact agreement with the writer’s fence, not merely an open socket. RocksDB memory is bounded per role, so storage growth cannot make memory unbounded.

Releases ship container images for amd64 and arm64, reproducible Linux binaries, supply-chain provenance (SLSA), and software bills of materials (SBOMs), with deployment runbooks for VMs and PaaS targets.

The supported topology today is rocksdb-single-host: primaries and secondaries share one host filesystem. RocksDB secondary mode is not a cross-host replication protocol, and Zinder does not pretend otherwise.

PostgreSQL is the path off that host

The goal is a PostgreSQL storage topology alongside RocksDB, so operators can scale reads with replicas, achieve high availability at the database layer, and distribute load across hosts rather than vertically on a single machine.

Today there is no deployable PostgreSQL backend. What exists is a diagnostic replay arm in the benchmark harness, a design for the horizontal production topology, and early tracer work. The storage architecture was built backend-neutral from the start (domain-shaped contracts with engine-specific modules), so a second engine is a complete topology implementation rather than a generic database adapter. When it is a supported deployment mode, it will be announced as one.

Where community input can help

  • Wallet teams: which native WalletQuery capabilities would make you drop the lightwalletd path entirely?
  • Operators: is single-host RocksDB sufficient for your workload, or is horizontal read scaling the blocker?
  • Explorer and analytics builders: which aggregates would you want as materialized views?

The source, documentation, and runbooks are at github.com/ZcashFoundation/zinder, and I would especially like to hear from teams running wallet infrastructure at scale

Alpha Notice: Zinder remains in active evolution. A tagged release is a milestone, not a stability guarantee. Deployment to production is a deliberate choice of risk: pin your versions, anticipate breaking migrations, and verify every user-facing artifact yourself.

Acknowledgments

The Zcash Foundation is now funding this indexer. I am grateful they saw potential in this work and chose to support it :heart:

9 Likes
2

super cool! is the zcash foundation funding via governance or is this something incubated internally?

indexers are critical and very cool that it’s built around its own protocol

3

It’s being incubated internally.

1 Like
4

love to see it!