Grant Application - Zcash Sapling Parameters Availability Hardening

Project Summary

Some Zcash wallets and SDKs depend on downloading the Sapling spend and output proving parameter files at runtime. Public issues show that when downloads are slow, blocked, interrupted, or return incorrect bytes, users can be unable to complete shielded actions. This project adds multi-origin, hash-verified fallback in zcash/librustzcash, ships adversarial regression tests, and provides two additional optional HTTPS origins with automated integrity verification

Project Description

This project hardens one of the most failure-sensitive dependencies in Zcash’s shielded user experience: access to the Sapling spend and output proving parameter files that some wallets and SDK flows download at runtime. When these downloads are slow, blocked, interrupted, or return incorrect bytes, users can be unable to complete shielded actions at the moment they need to, even if the chain and other infrastructure are functioning normally. This creates avoidable reliability failures that are difficult to diagnose, costly for wallet teams to support, and damaging to confidence in using ZEC for real payments.

The goal is to remove single-origin fragility from parameter distribution while keeping the integrity model unchanged. We will implement multi-origin downloads with deterministic fallback in zcash/librustzcash, and we will keep strict hash verification as the acceptance boundary, so availability improves without introducing new trust assumptions. We will also add adversarial regression tests that simulate timeouts, partial downloads, and incorrect content to ensure fallback remains correct over time and does not silently regress.

In parallel, we will provide two additional optional HTTPS origins for the same parameter files and ship an automated integrity verification workflow that continuously checks each origin against expected hashes and produces a public pass/fail record. These origins are additive and removable; the core deliverable is the upstream fallback logic and test coverage that downstream projects inherit as they update dependencies. Success means wallets and SDKs have a safer, more reliable path to obtain the proving parameters they need, and parameter availability stops being a single point of failure for shielded spending.

Proposed Problem

Zcash shielded payments can fail for a frustrating reason that is not obvious to users: some wallets and SDK flows still need to fetch Sapling proving parameter files at runtime, and when that download fails, the user can be blocked from completing a shielded action right when they are trying to pay or shield funds.

Proposed Solution

This project fixes the problem at the shared dependency layer so every downstream wallet/SDK benefits without building custom workarounds. The core change is simple: parameter downloads become multi-origin, hash-verified, and protected by regression tests, so a single host outage or a bad response can’t easily turn into a user-visible “can’t spend” incident.

  • Implement upstream multi-origin fallback in zcash/librustzcash
    Replace the single-origin download behavior with an ordered list of HTTPS origins and deterministic fallback with bounded timeouts and retries. Integrity does not change: each file is accepted only if it matches the expected hash. This directly addresses the single-point-of-failure concern and matches the upstream request for multi-host fallback that still works under adversarial behavior.
  • Add adversarial regression tests so the fix can’t quietly regress
    Introduce tests that simulate real failure modes: timeouts, 5xx errors, partial downloads, and “wrong bytes served.” The tests enforce two guarantees:
    1. incorrect files are never accepted
    2. downloads succeed when any later origin provides correct bytes
      These tests make the behavior durable, not a one-off patch.
  • Provide two optional independent origins and continuously verify them
    Host the same parameter files on two additional HTTPS origins under separate hosting and domain control, and publish an automated verification workflow that periodically downloads from each origin and checks hashes. These origins are additive and removable; the primary deliverable remains the upstream fallback logic and tests.
  • Make adoption easy without making it a dependency
    Submit at least one downstream PR that bumps the dependency and adds CI coverage to demonstrate the intended adoption path, while keeping completion criteria tied to upstream merge and test coverage, not downstream release schedules.

The result is a safer, more reliable parameter retrieval path that reduces “blocked at first spend” failures, while keeping the security boundary unchanged and making the improvement sustainable through upstream tests.

Solution Format

The solution will be delivered as open source software changes plus optional static hosting and automated verification for additional download origins.

Deliverables

  • 1) Upstream code (software)
    Merged PR(s) to zcash/librustzcash implementing multi-origin parameter downloads with deterministic fallback, bounded timeouts/retries, and strict hash verification.
  • 2) Adversarial regression tests (software)
    Merged test suite in librustzcash that simulates origin failures (timeouts, 5xx, partial responses, wrong bytes) and enforces that incorrect files are never accepted and fallback succeeds when a later origin is correct.
  • 3) Two optional additional HTTPS origins (infrastructure)
    Two independently controlled, read-only HTTPS locations hosting the Sapling spend/output parameter files.
  • 4) Automated integrity verification (automation)
    A scheduled workflow that downloads from each configured origin and verifies hashes, producing a public pass/fail record and a visible failure signal when an origin is unavailable or incorrect.
  • 5) Downstream enablement PRs (software integration)
    At least one downstream PR submitted (wallet SDK repo) that bumps the dependency and adds CI coverage for parameter download/fallback behavior.
  • 6) Short documentation (documentation)
    Concise docs describing guarantees, origin override, and how to interpret verification failures.

Dependencies

Technical dependencies

  • The current zcash/librustzcash parameter download implementation and its expected hash set for the Sapling spend/output parameter files (used to keep the integrity model unchanged).
  • A CI environment for running the new adversarial tests and the scheduled integrity verification workflow.

Service and resource dependencies

  • Two read-only HTTPS hosting locations for the optional additional origins (domain + object storage/CDN as needed).
  • Minimal bandwidth budget for periodic verification downloads.

Collaboration dependencies

  • Upstream review and merge coordination with librustzcash maintainers for the downloader change and test suite.
  • Optional coordination with at least one downstream wallet SDK maintainer to submit a dependency-bump PR and add CI coverage (submission is a deliverable, but downstream merge is not required for completion).

Technical Approach

We will implement this as a small, upstream-first change in zcash/librustzcash that improves availability of the Sapling spend/output parameter downloads while keeping the integrity model unchanged. The work is limited to the download path and its tests.

1) Multi-origin download and deterministic fallback

  • Define an ordered list of HTTPS origins for the Sapling parameter files in the zcash_proofs download code path.
  • Attempt origins in order. For each origin, use a fixed timeout and a small, fixed retry count so behavior is predictable.
  • If an origin fails due to timeout, HTTP error, partial response, or verification failure, move to the next origin.

2) Strict integrity checks and safe file handling

  • Download to a temporary file.
  • Verify the file matches the expected hash used by the existing implementation (and apply any existing size checks if present).
  • Only after verification succeeds, atomically rename the file into the final location.
  • If verification fails, discard the temporary file and try the next origin. No unverified bytes are retained as “good enough.”

3) Adversarial regression tests in CI

  • Add tests that run against local HTTP servers designed to simulate specific failure cases:
    • timeouts and 5xx responses
    • interrupted or partial downloads
    • wrong bytes (including “correct length, wrong content”)
  • Tests enforce two concrete guarantees:
    • incorrect files are never accepted
    • download succeeds when any later origin serves the correct bytes

4) Optional additional origins with automated integrity verification

  • Host the same parameter files at two additional read-only HTTPS locations under separate hosting and domain control.
  • Add a scheduled GitHub Actions workflow that:
    • downloads each file from each configured origin
    • verifies the expected hash
    • publishes a public pass/fail result and a visible failure signal when an origin is unavailable or incorrect
      These origins are optional and removable; the upstream fallback logic remains the primary deliverable.

5) Downstream enablement (non-blocking)

  • Submit at least one downstream PR (wallet SDK repo) that bumps the dependency version and adds a CI job that exercises the download path under a simulated “primary origin fails” condition. This demonstrates the adoption path without requiring downstream merges for project completion.

Upstream Merge Opportunities

Which upstream repositories we plan to modify

  • Primary: zcash/librustzcash
    This is the main target because it contains the Sapling parameter download logic used by downstream wallets and SDKs.
  • Optional (only if maintainers request it): zcash/zcash
    A small, strictly scoped clarification only if there is a remaining script or message that implies a single required download origin.

What changes we plan to make

  • In zcash/librustzcash:
    1. Add an ordered list of HTTPS origins for Sapling spend/output parameter downloads with deterministic fallback.
    2. Keep the integrity model unchanged by enforcing expected-hash verification before accepting files (download to a temp file, verify, then atomic rename).
    3. Add adversarial regression tests that simulate timeouts, HTTP 5xx, partial downloads, and wrong bytes, ensuring incorrect files are never accepted and fallback succeeds when a later origin is correct.
    4. Add short documentation describing the guarantees and how an operator can add an internal mirror safely if needed.
  • In zcash/zcash (optional):
    • A minimal clarification to a specific script or message, only if maintainers agree it is necessary.

How these changes benefit the wider Zcash ecosystem if merged upstream

  • Downstream wallets and SDKs inherit verified multi-origin fallback without building custom workarounds.
  • Fewer user-facing failures where shielded actions are blocked due to parameter download issues.
  • Regression tests keep the behavior correct as upstream code evolves.

Coordination needed with upstream maintainers

  • librustzcash maintainers: confirm the preferred way to define the origin list, acceptable timeout/retry defaults, and the test approach.
  • If the optional zcash/zcash change is pursued, confirm the exact file and scope with maintainers before submitting a PR.

Coordination will happen through GitHub issues and PR review. A design note will be posted early to reduce rework.

Timeline considerations for potential upstream merges

  • Weeks 1–2: Post the design note and test plan; open an initial PR for early feedback.
  • Weeks 3–5: Submit the main implementation PR and iterate through review.
  • Weeks 5–6: Add adversarial tests and land the merge, subject to review timelines.
  • Weeks 6–8: Stand up the two optional additional origins and the automated verification workflow, and submit at least one downstream enablement PR.

Hardware/Software Costs (USD)

$0

Hardware/Software Justification

n/a

Service Costs (USD)

$1,200

Service Costs Justification

Covers the minimal external costs needed for the two optional additional HTTPS origins and continuous verification, including:

  • Domain registration or DNS management (if required)
  • Object storage and bandwidth for hosting the static Sapling parameter files
  • Bandwidth for periodic verification downloads

Compensation Costs (USD)

$28,800

Compensation Costs Justification

Covers engineering time to deliver the core upstream work and keep it stable:

  • Implement multi-origin fallback in zcash/librustzcash with strict hash verification and safe file handling
  • Add adversarial regression tests and CI coverage
  • Set up automated integrity verification for all origins
  • Write concise operator/developer documentation
  • Submit at least one downstream enablement PR (dependency bump + CI coverage)
  • Six months of critical-fix support limited to the downloader, tests, and verification workflow

Total Budget (USD)

$30,000

Previous Funding

Yes

Previous Funding Details

No response

Other Funding Sources

No

Other Funding Sources Details

No response

Implementation Risks

  • Upstream review and merge timing
    The main deliverables must be merged into zcash/librustzcash. Review timelines can affect when the changes land.
  • Fallback correctness
    The implementation must handle timeouts, HTTP errors, partial downloads, and wrong bytes while never accepting an invalid file. This requires careful logic and strong regression tests.
  • Origin stability
    Optional additional origins may change behavior or experience outages. Verification and documentation must account for this so operators know what to expect.
  • Downstream update lag
    Downstream projects benefit after they bump dependencies. We will submit enablement PRs, but downstream merges are not fully under the project’s control.

Potential Side Effects

  • More download attempts in some failure cases
    If the first origin is unavailable or returns incorrect bytes, the downloader will try additional origins, which can increase download time in those cases.
  • Misunderstanding about trust
    Some users or operators might assume that adding more origins changes the trust model. In reality, files are still accepted only after hash verification, but this needs to be communicated clearly.
  • Operational dependency perception
    Even though additional origins are optional, some operators may come to rely on a specific mirror. If that mirror later becomes unavailable, they may need to switch to another origin.

Success Metrics

  • Upstream adoption
    • Multi-origin, hash-verified fallback is merged into zcash/librustzcash.
    • Adversarial fallback tests are merged and running in CI.
  • Correctness under failures
    • Tests demonstrate that timeouts, HTTP errors, partial downloads, and wrong bytes never result in accepting an invalid file.
    • Tests demonstrate that downloads succeed when any later origin serves the correct bytes.
  • Origin verification
    • Two optional additional HTTPS origins are online and serve the correct parameter files.
    • An automated verification workflow produces a public pass/fail record and flags any origin that becomes unavailable or serves incorrect bytes.
  • Ecosystem enablement
    • At least one downstream enablement PR is submitted and CI-green (dependency bump plus a minimal CI check that exercises the download path).

Startup Funding (USD)

n/a

Startup Funding Justification

n/a

Milestone Details

  • Milestone: 1
    Amount (USD): 6000
    Expected Completion Date: 2026-02-14
    User Stories:

    • “As a librustzcash maintainer, I want a precise design for multi-origin Sapling parameter downloads, so that I can review and approve the change without ambiguity.”
    • “As a wallet/SDK integrator, I want clear guarantees about hash verification and fallback behavior, so that availability improves without changing trust assumptions.”
      Deliverables:
    • Design note posted in the upstream issue/PR defining: the ordered origin list, per-origin timeout/retry policy, and verification invariants.
    • Test plan defining adversarial cases (timeouts, HTTP 5xx, partial downloads, wrong bytes) and the expected outcomes.
    • Initial PR scaffold in zcash/librustzcash to start review early (structure and test harness layout).
      Acceptance Criteria:
    • Upstream maintainers confirm in the PR discussion that the design matches the intended scope: multiple origins, deterministic fallback, and hash verification as the integrity boundary.
  • Milestone: 2
    Amount (USD): 14000
    Expected Completion Date: 2026-03-07
    User Stories:

    • “As a wallet/SDK integrator, I want parameter downloads to continue when one origin is unavailable or misbehaving, so that shielded actions are less likely to be blocked by a single host failure.”
    • “As an upstream maintainer, I want incorrect bytes to be rejected deterministically, so that fallback does not weaken integrity.”
      Deliverables:
    • Merged PR in zcash/librustzcash implementing multi-origin downloads with deterministic fallback for the Sapling spend/output parameter files.
    • Safe file handling: temporary file download, hash verification, then atomic rename into the final location.
    • Merged adversarial regression tests that simulate timeouts, HTTP 5xx, partial responses, and wrong bytes, proving incorrect files are never accepted and fallback succeeds when a later origin is correct.
      Acceptance Criteria:
    • CI passes with the new tests enabled.
    • Maintainers can run the tests and confirm: incorrect bytes are rejected and the downloader succeeds when a later origin serves correct bytes.
  • Milestone: 3
    Amount (USD): 7000
    Expected Completion Date: 2026-03-21
    User Stories:

    • “As an operator, I want additional independent origins available, so parameter availability is not tied to a single domain.”
    • “As a maintainer, I want continuous integrity verification, so origin problems are detected and visible.”
      Deliverables:
    • Two optional, read-only HTTPS origins hosting the Sapling spend/output parameter files.
    • Automated verification workflow that downloads from each configured origin and verifies expected hashes, producing a public pass/fail record and a visible failure signal on failure.
    • Short runbook describing how to remove or rotate an origin without changing integrity assumptions.
      Acceptance Criteria:
    • Verification passes for all configured origins and produces a visible failure signal when an origin is unavailable or serves incorrect bytes.
    • The runbook steps are sufficient for an operator to remove an origin and confirm fallback still works with remaining origins.
  • Milestone: 4
    Amount (USD): 3000
    Expected Completion Date: 2026-03-28
    User Stories:

    • “As an upstream maintainer, I want concise documentation for override behavior and troubleshooting, so the change is easy to support.”
    • “As a downstream SDK maintainer, I want a ready-to-review update, so adopting the fix is low effort.”
      Deliverables:
    • Concise upstream documentation describing guarantees, origin override mechanism, and troubleshooting steps.
    • At least one downstream enablement PR submitted and CI-green (dependency bump plus a minimal CI check that exercises the download path).
    • Start of a six-month critical fixes window limited to the downloader, tests, and verification workflow.
      Acceptance Criteria:
    • Upstream maintainers confirm the documentation matches the implemented behavior.
    • A downstream enablement PR exists and passes CI, demonstrating an adoption path.
  1. Wallets can easily choose to embed the parameters in their binary as an app resource
  2. 30k usd and 8 weeks of work to add two download hosts is rather excessive

Hi @hanh , thank you for your feedback Two clarifications:

  1. Embedding Sapling parameters in a wallet binary is an option for some wallets, but it does not resolve the ecosystem-wide reliability issue.
    Different consumers of the shared Rust path (wallet SDKs, tooling, and CI environments) still rely on downloading the Sapling spend/output parameter files at runtime, and we have public examples of failures in the wild (HTTP errors and invalid parameter files) as well as an upstream maintainer issue calling out the single-origin dependency and requesting multi-host fallback that remains correct under adversarial behavior.
    Our goal is to fix this once at the shared-library layer so downstream projects do not each need to implement and maintain their own bespoke packaging or fallback behavior.

  2. This proposal is not “$30k to add two download hosts.” The hosts are optional and low-cost; the core work is upstream correctness and regression protection.
    The primary deliverables are:

  • multi-origin, deterministic fallback in zcash/librustzcash with bounded timeouts/retries
  • strict hash verification and safe file handling (download to temp file, verify, atomic rename)
  • adversarial regression tests (timeouts, partial downloads, wrong bytes) so the behavior cannot silently regress

The two additional origins are additive and removable and do not change the trust model because files are accepted only after matching expected hashes. Hosting is a small line item; most of the budget is engineering time to land the upstream change and tests.

If the committee still feels $30k/8 weeks is high, we can reduce scope while keeping the main value:

  • Option A ($20k–$22k): upstream fallback + adversarial tests only (no additional origins)
  • Option B (~$25k): upstream fallback + tests + one additional origin + verification workflow

Either way, the objective stays the same which is to reduce the chance that parameter download problems turn into user-visible “blocked shielded action” failures, by addressing the single-origin fragility with upstream code and durable tests. Happy to hear from you again, thank you!

The librustzcash library already checks for downloaded sizes and hashes. If a wallet dev feels that there is a risk of failure, they can host their own copy or include it in their app.

Thanks @hanh We agree on the baseline:

  • librustzcash already verifies Sapling parameter downloads by size and hash, so integrity is not the gap.
  • Wallets can choose to bundle the parameters or host their own copy.

What this proposal addresses is the part that those points do not solve: availability and failure handling in the shared path.

  1. Hash/size checks only help after bytes arrive. They don’t prevent or mitigate real-world failures like HTTP errors, blocked access, timeouts, partial downloads, or “wrong bytes served.” We have public examples of these failures causing user-visible problems (HTTP 403 downloads and invalid parameter files).

  2. “Each wallet can host or embed” pushes a recurring burden to every downstream. If each wallet/SDK/tool implements its own workaround, we get duplicated effort, inconsistent behavior, and no shared regression protection. The upstream request here is explicitly for multiple independently controlled hosts and fallback that stays correct even under adversarial conditions, while keeping hashes as the integrity boundary.

  3. This is not “$30k to add two hosts.” The optional additional origins are a small, additive availability improvement. The core deliverables are upstream and testable:

  • deterministic multi-origin fallback in librustzcash
  • strict verification unchanged (still accept only expected hashes)
  • adversarial regression tests so fallback can’t silently regress

If the committee prefers, we can also reduce scope to code & tests only (no extra origins) and still achieve the primary goal which is to reduce the chance that parameter download issues become user-visible “blocked shielded action” failures by fixing availability behavior once, upstream, for all downstream consumers.

Thank you for submitting your proposal. Following a thorough review by the ZCG and a period for community feedback on the forum, the committee has decided not to move forward with this proposal.

We sincerely appreciate the time and effort you invested in your application and encourage you to stay involved and continue contributing to the Zcash community. Further details will be available in the meeting minutes to be posted later this week.