Grant Application - Secure-by-Default Zebra ↔ lightwalletd: Add RPC Cookie-File Authentication Support

[Crackdevs]

Project Summary

This project adds RPC cookie-file authentication support to zcash/lightwalletd so it can authenticate to Zebra’s JSON-RPC backend without requiring operators to disable cookie authentication or rely on long-lived static RPC credentials. It also ships a CI integration test and updated deployment docs/examples so the authenticated Zebra ↔ lightwalletd setup is straightforward to run and maintain.

Project Description

Zebra includes a documented setup for running zcash/lightwalletd using Zebra’s JSON-RPC interface. Today, that setup requires a security tradeoff: when Zebra’s RPC server is enabled, Zebra uses cookie-based RPC authentication by default, but the current Zebra ↔ lightwalletd documentation and example configurations instruct operators to disable cookie authentication because lightwalletd does not support it. In parallel, lightwalletd documentation requires configuring rpcuser/rpcpassword because it “doesn’t work with RPC cookies,” which pushes operators toward static credentials even for local-only RPC use.

This project closes that integration gap by implementing cookie-file authentication in lightwalletd (the same username:password cookie-file mechanism used by zcashd-style RPC setups), and packaging the change so it is immediately usable by operators and hard to regress.

What will be delivered

• Cookie-file auth in lightwalletd: add a new option (e.g., --rpc-cookie-file <path>) that reads the RPC cookie file and uses it for HTTP Basic authentication to the JSON-RPC backend.
• Deterministic refresh behavior: on an authentication failure (HTTP 401/403), re-read the cookie file and retry once; fail fast with clear errors if the cookie file is missing or unreadable at startup.
• Regression-proof CI integration test: a CI job that starts Zebra with RPC enabled (cookie auth enabled), mounts the cookie file into lightwalletd, starts lightwalletd using the cookie file, and verifies lightwalletd is healthy by successfully serving a standard gRPC health check call (e.g., GetLightdInfo).
• Docs + reference deployments updated: update the Zebra ↔ lightwalletd docs and example configs (Docker Compose, plus a minimal Kubernetes snippet) so the recommended setup uses cookie auth enabled. The legacy “disable cookie auth” workaround is kept only as a clearly labeled fallback until the cookie-auth-capable lightwalletd release is available.

Project goals

• Enable Zebra ↔ lightwalletd deployments that keep RPC authentication enabled (no “disable auth to make it work” requirement).
• Reduce reliance on static RPC passwords by supporting the cookie-file mechanism that is already used in Zcash RPC setups.
• Make adoption immediate via runnable examples, and keep it durable via CI that detects regressions early.
• Keep scope tightly upstreamable: no protocol/consensus changes, no new hosted infrastructure, and no TLS redesign only the missing authentication method plus the tests/docs needed for reliable adoption.

Proposed Problem

When Zebra is used as the backend for zcash/lightwalletd, the currently documented “works today” setup requires weakening RPC authentication.

• When Zebra’s RPC server is enabled, Zebra uses cookie-based RPC authentication by default, but Zebra’s lightwalletd guide states that lightwalletd does not support cookie authentication, so operators must disable cookie authentication for compatibility.
• In parallel, lightwalletd’s deployment docs require setting rpcuser and rpcpassword because it “doesn’t work with RPC cookies,” which pushes operators toward long-lived static credentials as the default configuration.

This creates a practical security and operations problem: to run a common Zebra ↔ lightwalletd deployment, operators are steered toward either turning off cookie-based authentication or relying on static RPC passwords, instead of using the cookie-file mechanism designed for safer local RPC authentication.

Proposed Solution

This project eliminates the Zebra ↔ lightwalletd authentication gap by adding RPC cookie-file authentication support to zcash/lightwalletd, then making that secure configuration the default recommended deployment path.

1. Add cookie-file auth to lightwalletd
   Implement a new option (e.g., --rpc-cookie-file <path>) that reads the RPC cookie file (username:password, typically __cookie__:<random>) and uses it for HTTP Basic authentication when connecting to Zebra’s JSON-RPC backend. Existing rpcuser/rpcpassword configuration remains supported as a fallback when a cookie file is not provided.
2. Handle cookie rotation deterministically
   On RPC authentication failure (HTTP 401/403), lightwalletd will re-read the cookie file and retry once, then surface a clear error if authentication still fails. This supports real deployments where the cookie may change, without introducing retries that hide misconfiguration.
3. Ship a regression-proof CI integration test
   Add a CI job that starts Zebra with RPC enabled (cookie auth enabled), mounts Zebra’s cookie file into lightwalletd, starts lightwalletd with --rpc-cookie-file, and verifies lightwalletd is operational by successfully serving a standard gRPC call (e.g., GetLightdInfo). This ensures the secure path stays working over time.
4. Update docs and reference deployments for immediate adoption
   Update Zebra ↔ lightwalletd documentation and example configs (Docker Compose, plus a minimal Kubernetes snippet) to use cookie-file auth instead of disabling authentication. The “disable cookie auth” workaround is kept only as a clearly labeled fallback until a lightwalletd release containing this feature is available.

Solution Format

1. Software (upstream PR to zcash/lightwalletd)

• Add RPC cookie-file authentication support (e.g., --rpc-cookie-file <path> / config equivalent).
• Implement deterministic behavior for missing/unreadable cookie file and a single re-read + retry on HTTP 401/403.

2. Automated testing (CI)

• A CI integration test that starts Zebra with RPC enabled (cookie auth enabled), starts lightwalletd using the mounted cookie file, and verifies lightwalletd is healthy via a standard gRPC call (e.g., GetLightdInfo).

3. Documentation

• Update lightwalletd docs to document cookie-file auth as a supported alternative to rpcuser/rpcpassword.
• Update Zebra ↔ lightwalletd documentation to recommend cookie-file auth and remove “disable cookie auth” as the default path (kept only as a clearly labeled legacy fallback until a lightwalletd release includes the feature).

4. Reference deployments

• Updated Docker Compose example for Zebra + lightwalletd using a read-only cookie-file mount.
• A minimal Kubernetes snippet showing the same cookie-file mount pattern.

Dependencies

• Upstream repository access and review: Maintainer review and merge in zcash/lightwalletd for the cookie-auth feature, tests, and docs.
• Documentation/example updates: Maintainer review for updates to Zebra ↔ lightwalletd documentation and the Zebra Docker Compose example (or equivalent location where the example is maintained).
• CI resources: GitHub Actions (or the project’s existing CI) to run the new integration test that starts Zebra + lightwalletd and performs the gRPC health check.
• No external services required: No hosted infrastructure, third-party APIs, or paid services are required to deliver or operate the solution.

Technical Approach

Technical Approach

Work will be delivered as upstream-ready changes to zcash/lightwalletd (Go), backed by unit tests and an end-to-end integration test, plus documentation and reference deployment updates.

1) Add cookie-file auth support in lightwalletd (config + CLI)

• Implement support for specifying an RPC cookie file using the existing Zcash RPC convention:
  • CLI: --rpccookiefile <path> (matching the style of existing --rpcuser/--rpcpassword flags)
  • Config file: support rpccookiefile=<path> in the zcash.conf file that lightwalletd already reads.
• Define explicit precedence to avoid ambiguous operator configs:
  • If rpccookiefile is set, use cookie-file auth.
  • Else fall back to existing rpcuser/rpcpassword.
  • If both are provided, fail fast with a clear configuration error.

2) Cookie parsing + secure handling (no secret leakage)

• Read the cookie file and parse the first line as username:password.
• Use those credentials for HTTP Basic authentication to the JSON-RPC backend.
• Ensure no secrets (cookie contents, derived credentials, full auth headers) are ever written to logs, including error paths.

3) Deterministic refresh behavior (bounded, not “magic”)

• On RPC authentication failures (401/403):
  • re-read the cookie file
  • retry the same request once
  • if it still fails, return a clear error (no loops or silent retries).
• If the cookie file is missing/unreadable at startup, fail with a clear error describing the expected path and required setup.

4) Testing strategy (unit + integration, regression-proof)

Unit tests (lightwalletd repo)

• Test cookie parsing (username:password, whitespace/newline handling, invalid formats).
• Test precedence rules (cookie file vs rpcuser/rpcpassword).
• Test refresh logic using a mocked RPC endpoint that returns 401 once, then succeeds after a cookie re-read.

Integration test (end-to-end)

• Extend an existing CI path that already validates Zebra ↔ lightwalletd interoperability (rather than inventing a new heavyweight sync-based test).
• Run Zebra with RPC enabled and cookie auth enabled, mount the cookie file into the lightwalletd environment, configure lightwalletd with --rpccookiefile, and assert a lightweight gRPC call succeeds (e.g., GetLightdInfo or an equivalent call used in the existing integration suite) to confirm the stack works end-to-end with authentication enabled.

5) Deployment examples + docs (make secure config the recommended config)

• Update the Zebra ↔ lightwalletd Docker Compose example to:
  • remove ENABLE_COOKIE_AUTH=false
  • mount Zebra’s cookie file (or cookie directory) read-only into lightwalletd
  • pass --rpccookiefile (or set rpccookiefile= in zcash.conf) for lightwalletd.
• Update documentation in both places:
  • Zebra docs: recommended path uses cookie-file auth; “disable cookie auth” kept only as a clearly labeled fallback until the cookie-auth-capable lightwalletd release is available.
  • lightwalletd docs: document cookie-file auth as a supported alternative to rpcuser/rpcpassword.

6) Tools / methodology

• Language: Go (lightwalletd)
• CI: GitHub Actions + container-based integration test (leveraging existing interoperability CI where possible)
• Quality gates: unit test coverage for parsing/precedence/refresh + integration test coverage for Zebra↔lightwalletd cookie-auth compatibility.

Upstream Merge Opportunities

Upstream repositories to modify

• zcash/lightwalletd (primary): implement RPC cookie-file authentication + tests.
• ZcashFoundation/zebra (secondary): update the Zebra↔lightwalletd Docker Compose example and related guidance to stop recommending “disable cookie auth” once cookie support is available in lightwalletd.

Planned changes

• In zcash/lightwalletd:
  • Add cookie-file RPC auth support (--rpccookiefile <path> and rpccookiefile=<path> in zcash.conf).
  • Define precedence: cookie file overrides rpcuser/rpcpassword; fail fast if both are set.
  • Add bounded refresh: re-read cookie + single retry on 401/403.
  • Add unit tests and a CI integration test proving Zebra (RPC enabled, cookie auth enabled) works with lightwalletd using the cookie file.
• In ZcashFoundation/zebra (examples/docs):
  • Update docker-compose.lwd.yml and any referenced setup text to mount the cookie file read-only into lightwalletd instead of setting ZEBRA_RPC__ENABLE_COOKIE_AUTH=false.
  • Keep the “disable cookie auth” workaround only as a clearly labeled fallback until a lightwalletd release includes cookie support, then remove it.

Ecosystem benefit if merged upstream

• Removes the need to disable RPC authentication for Zebra-backed lightwalletd deployments.
• Reduces reliance on long-lived static RPC credentials.
• Adds CI coverage to prevent regressions in the secure deployment path.

Coordination needed

• Open/track an issue in zcash/lightwalletd and submit an early draft PR to confirm flag/config naming, precedence rules, and the test approach.
• Coordinate with Zebra maintainers/docs owners on when to flip the example and docs to the cookie-based setup (ideally aligned with a lightwalletd release containing the feature).

Timeline / merge considerations

• Weeks 1–2: implementation + unit tests + draft PR.
• Weeks 3–4: CI integration test + review iteration.
• Weeks 5–6: docs/example updates + final merge-ready PRs.
• If merge/release timing slips, deliverables remain merge-ready PRs with CI proof; the docs/example switch can be synced to the first lightwalletd release that includes cookie support.

Hardware/Software Costs (USD)

none

Hardware/Software Justification

none

Service Costs (USD)

none

Service Costs Justification

none

Compensation Costs (USD)

$18,000

Compensation Costs Justification

Compensation covers engineering time to implement cookie-file RPC authentication in zcash/lightwalletd, add unit and CI integration tests, update deployment examples (Docker Compose + minimal Kubernetes snippet), update documentation, and iterate through upstream review.

• Estimated effort: ~160–190 hours total (single engineer, blended rate implied by total), roughly:
• Implementation (cookie-file auth + precedence + safe error handling): 55–70h
• Bounded refresh behavior + unit tests: 25–35h
• CI integration test (Zebra RPC enabled + cookie auth + gRPC health check): 30–40h
• Docs + reference deployments (Compose + K8s snippet) + example updates: 20–30h
• Upstream review iterations, fixes, and merge support: 10–15h

Total Budget (USD)

$18,000

Previous Funding

No

Previous Funding Details

No response

Other Funding Sources

No

Other Funding Sources Details

No response

Implementation Risks

• Cookie file path and container mounting pitfalls
  In Docker/Kubernetes deployments, the most common failure mode is mis-mounting the Zebra cookie file/directory or using the wrong path.
  Mitigation: ship a working reference Docker Compose update and a minimal Kubernetes snippet that mounts the cookie file read-only into lightwalletd and documents the expected path and permissions.
• Cookie rotation / timing edge cases
  If the cookie changes while lightwalletd is running, authentication can fail unless lightwalletd refreshes it.
  Mitigation: implement deterministic refresh: on 401/403, re-read the cookie file and retry once; surface clear errors if it still fails.
• Maintainer review and merge latency
  Even correct changes can take time to review and land upstream.
  Mitigation: open a tracking issue and submit a draft PR early (Week 1) to confirm flag/config naming and behavior; keep PRs small and include CI proof to reduce review burden.
• Integration test flakiness or excessive runtime
  End-to-end tests can be slow or flaky if they depend on heavy sync behavior.
  Mitigation: keep the CI check lightweight (container startup + authenticated RPC connectivity + GetLightdInfo gRPC call) and avoid full chain sync requirements.
• Accidental secret exposure in logs
  Any auth feature increases the risk of logging sensitive material during debugging.
  Mitigation: explicitly avoid logging cookie contents/auth headers; add tests or review checks to ensure secrets are not emitted in errors.

Potential Side Effects

• Misconfiguration could prevent lightwalletd from starting or connecting
  If an operator supplies the wrong cookie-file path or mounts it incorrectly, lightwalletd will fail authentication (or fail fast if the file is unreadable). This is a usability risk during rollout.
  Mitigation: clear, actionable error messages plus reference Docker/K8s examples showing the correct mount and path.
• Slight increase in filesystem reads on auth failure
  The refresh behavior re-reads the cookie file on 401/403 and retries once. This adds a small amount of I/O only when authentication fails, not during normal operation.
• Behavioral change for deployments that mix auth methods
  If an operator accidentally sets both rpccookiefile and rpcuser/rpcpassword, the proposed “fail fast” rule will force them to choose one method.
  Mitigation: document precedence rules clearly and provide a straightforward migration note.
• Docs/example changes may need to be staged with a release
  Switching the recommended Zebra↔lightwalletd example from “disable cookie auth” to “use cookie file” should be aligned with a lightwalletd release that includes the new feature; otherwise users following docs could hit errors.
  Mitigation: keep the legacy workaround clearly labeled until the released version is available, then flip the default recommendation.

Success Metrics

• Secure-by-default deployment path available
  Zebra can run with RPC enabled and cookie authentication enabled, and lightwalletd can authenticate using the cookie file (no requirement to disable cookie auth for compatibility).
• Regression protection in CI
  A CI integration test exists that brings up Zebra (RPC enabled, cookie auth enabled) + lightwalletd (cookie-file auth configured) and verifies a successful gRPC call (e.g., GetLightdInfo). The test fails if cookie-auth compatibility regresses.
• Documentation and examples updated for adoption
  Zebra↔lightwalletd docs and reference deployment examples (Docker Compose, minimal Kubernetes snippet) document cookie-file auth as the recommended path and no longer present “disable cookie auth” as the default configuration (kept only as a clearly labeled legacy fallback until the feature is released).
• Upstream deliverables are merge-ready or merged
  The lightwalletd implementation PR and the associated docs/example PRs are merged upstream, or are review-complete and explicitly approved/merge-ready with CI passing.

Startup Funding (USD)

none

Startup Funding Justification

none

• Milestone: 1
  Amount (USD): 2500
  Expected Completion Date: 2026-02-22
  User Stories:

  • “As a lightwalletd maintainer, I want a clear proposal for RPC cookie-file auth (flags/config, precedence, and error behavior), so that I can approve the approach early.”
  • “As a Zebra↔lightwalletd operator, I want an explicit configuration plan (what to mount, what flag to set), so that I can adopt the secure setup without trial-and-error.”
    Deliverables:
  • “Tracking issue (or PR description) in zcash/lightwalletd defining: --rpccookiefile / rpccookiefile=, precedence vs rpcuser/rpcpassword, bounded refresh behavior, and the CI test plan.”
  • “Draft PR in zcash/lightwalletd wiring the new option end-to-end (flag/config parsed and passed through), with placeholder implementation notes where final logic will land.”
    Acceptance Criteria: Maintainers can review and comment on the interface/behavior, and the draft PR builds successfully and is structured for straightforward review (no ambiguity about config/precedence or intended test coverage).

• Milestone: 2
  Amount (USD): 4500
  Expected Completion Date: 2026-03-01
  User Stories:

  • “As an operator, I want lightwalletd to authenticate to the JSON-RPC backend using a cookie file, so that I don’t need long-lived static RPC credentials.”
  • “As a maintainer, I want unambiguous auth precedence rules, so that misconfigured deployments fail clearly rather than behaving unpredictably.”
    Deliverables:
  • “Implement cookie-file RPC auth in zcash/lightwalletd: read cookie file as username:password and use it for HTTP Basic auth to the RPC backend.”
  • “Precedence rules implemented: cookie file overrides rpcuser/rpcpassword; if both are set, fail fast with a clear error.”
  • “Unit tests for cookie parsing + precedence (valid formats, invalid formats, whitespace/newlines).”
    Acceptance Criteria: Unit tests pass, and a maintainer/operator can validate (locally or via CI logs) that lightwalletd successfully authenticates to an RPC backend using --rpccookiefile.

• Milestone: 3
  Amount (USD): 3500
  Expected Completion Date: 2026-03-08
  User Stories:

  • “As an operator, I want lightwalletd to handle cookie changes safely, so that a rotated cookie does not permanently break RPC auth.”
  • “As a maintainer, I want bounded behavior, so that failures are visible and not hidden by repeated retries.”
    Deliverables:
  • “Implement bounded refresh: on HTTP 401/403, re-read the cookie file and retry once (no loops).”
  • “Unit tests for refresh behavior using a mocked RPC endpoint (e.g., fail once with 401, then succeed after cookie re-read).”
  • “Clear, non-secret error messages for missing/unreadable cookie file at startup.”
    Acceptance Criteria: Tests demonstrate the single retry behavior works as specified, does not loop, and does not log secret material.

• Milestone: 4
  Amount (USD): 4500
  Expected Completion Date: 2026-03-15
  User Stories:

  • “As an operator, I want confidence the secure Zebra↔lightwalletd setup keeps working across upgrades, so that I don’t need the ‘disable auth’ workaround.”
  • “As a maintainer, I want an automated regression test, so that cookie-auth interoperability can’t silently break later.”
    Deliverables:
  • “CI integration test (GitHub Actions): start Zebra with RPC enabled (cookie auth enabled), mount Zebra’s cookie file into lightwalletd, start lightwalletd using --rpccookiefile, and verify lightwalletd is live via a lightweight gRPC call (e.g., GetLightdInfo using grpcurl).”
  • “CI logs/artifacts sufficient to debug failures (container logs + grpcurl output).”
    Acceptance Criteria: The CI job passes reliably without requiring a full chain sync, and it fails if cookie-auth interoperability regresses.

• Milestone: 5
  Amount (USD): 2000
  Expected Completion Date: 2026-03-22
  User Stories:

  • “As an operator, I want runnable deployment examples, so that I can enable cookie auth correctly without guesswork.”
  • “As a documentation reader, I want the recommended setup to keep authentication enabled, so that I’m not guided toward an insecure workaround.”
    Deliverables:
  • “Update Zebra↔lightwalletd Docker Compose example to remove ENABLE_COOKIE_AUTH=false and instead mount the cookie file read-only into lightwalletd + set --rpccookiefile (or equivalent config input).”
  • “Update lightwalletd docs to document cookie-file auth as supported alongside rpcuser/rpcpassword.”
  • “Add a minimal Kubernetes snippet showing the same read-only cookie-file mount pattern.”
  • “Update Zebra↔lightwalletd guidance to recommend cookie-file auth; keep ‘disable cookie auth’ only as a clearly labeled legacy fallback until a lightwalletd release includes the feature.”
    Acceptance Criteria: A maintainer/operator can follow the updated docs/examples to run Zebra↔lightwalletd with authentication enabled using cookie-file auth, without relying on the insecure workaround.

• Milestone: 6
  Amount (USD): 1000
  Expected Completion Date: 2026-03-29
  User Stories:

  • “As a maintainer, I want merge-ready PRs with CI green and feedback addressed, so that the change can land cleanly upstream.”
  • “As an operator, I want final documentation and proof-of-work links, so that adoption is low-risk.”
    Deliverables:
  • “Address review feedback and finalize PR(s) as merged or explicitly approved/merge-ready with CI passing.”
  • “Update tracking issue(s) with final links (PRs, CI runs, docs/example updates) and a short ‘how to use cookie-file auth’ note.”
    Acceptance Criteria: PR(s) are merged upstream or explicitly approved/merge-ready with CI green, and the docs/examples consistently point to the cookie-auth-enabled path (with any legacy fallback clearly labeled).

[jenkin]

Yes please! Right now I’m using a weird command override in z3 docker compose to inject credentials from cookie file to lwd config template. We really need a more robust solution until zaino will be production ready. Thanks!

[ZCG]

Thank you for your submission. After consideration from ZCG and sufficient time for the community to provide feedback on the forum, the committee has decided to reject this proposal.

The committee appreciates your grant submission efforts and encourages you to continue as an active member of the Zcash community going forward!