Zaino, The Zallet Release

Zingo Labs is requesting funding in the form of a ZCG grant for the current, and recent work we’ve been doing to finalize the zaino codebase, for the Zallet Release.

As we worked through the “respec grant work” it became clear that we needed to redesign and update several parts of the codebase to better leverage tools and techniques that, in some cases weren’t available during the original implementation, and in other cases weren’t used in favor of expedient code completion.

A good example of the the former is transaction deserialization code that we had to implement in house because alternatives weren’t available. It’s now that the case that community deserialization tools are available, so we’re able to deprecate our bespoke implementation in favor of the community supported standard tool. Of course, we immediately began working on these improvements, so some of the work is already complete.

Once the Zallet team began working with Zaino, they were able to produce extensive feedback on places where Zaino was not meeting their requirements.

We’re working at a rapid clip to address all of the known issues resulting from each of these factors (thanks to @pacu for setting up this board). Our intention is complete all of these tasks within the next two weeks, with Zallet blockers prioritized and addressed first.

Here’s the grant details, as published at:

https://github.com/ZcashCommunityGrants/zcashcommunitygrants/issues/87

Requested Grant Amount (USD)

$69,811.20

Category

Infrastructure

Project Lead

Name: Za
Role: Zingo Labs Coordinator
Background: Za has been with Zingo since its inception
Responsibilities: Za coordinates intra-, and inter-, operation of the Labs.

Additional Team Members

We disclose personal details to the ZCG and ZCG operations staff, with the consent of the recipients, for operational purposes.

Project Summary

We will upgrade Zaino to a production ready state. This will provide consumers e.g. (Zallet, ZecRocks, Zingolib) with the resilient system they need to access Zcash Blockchain Indices.

Project Description

Zaino is a pure Rust indexer. It provides consumers with indices of chain state for efficient operation.

Proposed Problem

Zaino has been built to meet a dynamic set of external constraints a challenge it has repeatedly risen to. Now that it is ready for integration with the wider Z3 system, we need to complete three complimentary tasks:

(1) fix observed integration issues

(2) Refactor internal patterns, to eliminate identifiable bug-classes (e.g. unexpected runtime states, that can be eliminated at compile time)

(3) triage new issues dynamically

Proposed Solution

We recognized the need for a more mature type system to back Zaino interfaces, concurrently with the completion of those interfaces. In other words, now that we have a functioning system we know exactly how to refine and stabilize the core to maximally leverage best practices and patterns.

We believe that this iteration and adaptation has been, in many ways, optimal. We needed the Proof Of Concept scaffold in place, to know exactly how to best upgrade the core abstractions.

Solution Format

Two core consumers of Zaino are in production, and Zaino development accelerates:

  • Zallet developers call Zaino’s API, and get a useful response in:
    • the happy path state
    • a failure state
  • Zainod is used by a lightclient to support lightclient
    • send
    • receive
  • Zainod infrastructure providers
    • use the new configuration
  • Zaino developers use the new strongly typed configuration system in test
  • Zaino development produces weekly updates tracking

Dependencies

We coordinate closely with the ECC, the ZF, and SL.

Technical Approach

We’ll continue using the agile development process we have pursued so far, but we are now leveraging github planning tools, including time to completion estimates and project view workflow tracking.

Upstream Merge Opportunities

We contribute to librustzcash, zebra, and other zcash related projects.

Hardware/Software Costs (USD)

0

Hardware/Software Justification

N/A

Service Costs (USD)

0

Service Costs Justification

N/A

Compensation Costs (USD)

$69,120

Compensation Costs Justification

We use a base rate of $120/hour for experienced specialized developer pay.

Line Items:

developer days of retroactive work completed: 30
prospective days of scheduled work: 42
total developer days: 72

Total developer pay: $69,120.00
Grant administration pay = Total Developer Pay * 0.01 = $691.20

Total: $69,120

Total Budget (USD)

$69,120

Previous Funding

Yes

Previous Funding Details

We are a ZCG infrastructure project. The Zaino project has been entirely funded by ZCG grants.

Other Funding Sources

No

Other Funding Sources Details

No response

Implementation Risks

Upon completion the indexer will be implemented in Rust, making the stack more monolithic.

Potential Side Effects

Success of the project may accelerate the depreciation of the USD relative to ZEC the superior currency.

Success Metrics

Zallet and Zainod receive succinct and useful data using Zaino under varying circumstances including states that are not the ideal steady state for the Zcash blockchain distributed network.

Startup Funding (USD)

$29,491.20

Startup Funding Justification

This pays for completed development work, primarily in the design of the new zaino type system.
This pays for grant conceptualization, drafting, revision, publication, and maintenance.

Milestone Details

Deliverable Details Tracked As “Time Allocated” Issues Here

- Milestone: 1
  Amount (USD): $40,320
  Expected Completion Date: 2025-09-26
  User Stories:
    - As a Zainod provisioner, I want to deploy zainod to support light clients.
    - As a Zallet developer, I want to use the Zaino API interface with the blockchain.
  Deliverables:
    - Zainod launches as a Rust binary, with a single cargo invocation against the correct repo commit (of zaino).
    - The zainod launching commit is trivial (for a competent docker user) to integrate into the provided Dockerfile, to produce a provisionable image/container
    - The Zaino API provides the complete set of interfaces necessary for Zcashd-deprecating Zallet operations.
  Acceptance Criteria:
    - A Zcash provisioning specialist launches a ZainoD instance, and it integrates with the scenarios/systems that the provisioner has created.
    - A Zallet developer is able to invoke the necessary Zaino APIs and received well formed responses in both "Ok" and "Err" cases.
9 Likes

Writing Good Code is Not Enough!

Requesting: $69 000

Solution Format

Two core consumers of Zaino are in production, and Zaino development accelerates:

Zallet developers call Zaino’s API, and get a useful response in:
    the happy path state
    a failure state
Zainod is used by a lightclient to support lightclient
    send
    receive
Zainod infrastructure providers
    use the new configuration
Zaino developers use the new strongly typed configuration system in test
Zaino development produces weekly updates tracking

No where are resources, time of key individuals to develop documentation, demo videos, workshops in order to increase the usage of the software solutions your team is developing. The Zcash ecosystem has a long, long list of projects writing code with no to little usage.

Everyone understands timelines are tight, work loads are heavy. But if no can follow how can we get new users to start from nothing.

Please Update, Please Document

ZingoLabs Fan

4 Likes

@zancas ZCG voted async to approve this proposal. Congratulations!

To keep the community informed, ZCG requests that you provide monthly updates via the forum in this thread.

Please check your forum inbox for a direct message from FPF with important next steps, including a link to the Milestone Payment Request Form and your unique validation code for submitting payment requests.

10 Likes

This is great news!!! We are hard at work.

6 Likes

Hiya @kworks thanks for your support.

We take forum feedback very seriously.

We’re heads down on Zaino at the moment (see this grant thread) which will produce a new version of zainod shortly.

In the meantime, our zingo-cli offering is relatively low priority at the moment but to disentangle dependencies with zaino, we have done some work that updated its interface a bit.

If you try out zingo-cli and notice any issues please let us know!

We’re continuing work on this project in parallel with work on the continuation grant.

We are wrapping up the code quality upgrade, and update to zebrad 3.0.0 / NU6.1 support.

For a more detailed view please checkout our project tracker:

2 Likes

Can it do rotating transparent addresses? If so, I could pledge 5k or something.

Hey Zcashers! We’ve extensively rearchitected zaino in order to fulfill the requirements of this grant.

We are concurrently rewriting two layers of abstraction, and extending the test framework.

Testing

In order to prove correctness in real-world stress scenarios with extensive reorgs, as might be expected in some attacks, we built a test framework which has detected multiple bugs that will be fixed prior to release.

Modification of ChainIndex Sync

The ChainIndex Sync algorithm incorrectly synced non-finalized state. This is corrected here.

Complete Deprecation of Local Cache, Zainod now uses ChainIndex

Previously Zainod used data structure that was:

  1. only suitable for use after the Sapling Epoch
  2. vulnerable to a state corruption bug
  3. unable to account for non-best chain blocks

These issues are fixed by the deprecation of local_cache in favor of the ChainIndex.

It’s been a bumpy ride, but we’re closing in on major bugfixes that will finally allow us to start leveraging the upsides of zaino!

7 Likes

Hi folks! It has been too long! I apologize for the delay in posting this update.

The work we mentioned in the previous post is still in flight. We aspired to publish the resulting release today, but as it happens we are still resolving issues revealed by our extensive and improving tests.

If the remaining issues are resolved, we expect to release next week!

3 Likes