Zcashd Deprecation updates

Ecosystem Updates
1

Hello fellow Zebras!

As you may be aware of Zcashd deprecation is one of the top priorities of the Zcash development teams. This thread attempts to keep the community updated in a succinct and easy to grasp way.

Outreach (@pili @decentralistdan @pacu):

We are conducting an Ecosystem Outreach to have first-hand insights on Zcashd use cases and other important input for Zcash developers. ZF, ECC and ZCG (with Pacu appointed resource) will be leading this important

  • Status: initiated

Zaino:

formerly known as Zingo Indexer, Zaino, is product of a lot of architecture design, planning and collaboration between ZingoLabs, ZF, ECC and ZCG which shaped a grant that will let Zingo to expand its team and contribute a key piece of infrastructure that will not only accelerate Zcashd Deprecation but also evolving the existing Zcash infrastructure into a more modern, maintainable and versatile shape that can not only accommodate current use cases, but be ready to adopt the challenges of the Zodlers of tomorrow. While you will be able to follow milestone updates on the corresponding thread, here you will find smaller incremental updates.

  • Status: on track

Librustzcash

LRZ is going through a huge overhaul! Although it has been supporting a lot of recent functionality on Zcashd and the Mobile SDKs, it will continue to grow to support the wallet that will ultimately replace the “Zcashd wallet”.

In-Memory backend

Chainsafe is doing this development as part of their Javascript Zcash SDK grant!

  • Status: on track
    • they are pretty close to implementing this backend.
    • we will meet shortly to polish how upstream integration LRZ will play out since there are significant and important changes they’ll be contributing back to that codebase.
    • Kudos to Eric and @Wollum-ChainSafe

SQLITE test framework migration

this is necessary in order to make the tests reusable with other wallet backends

Zcashd wallet’s successor

Agree on overal architecture

This is an ongoing discussion that will shape the incoming years of the Zcash wallet ecosystem

  • Status: on track

Zebra

No introduction needed!

Cookie auth for RPC methods

kudos @oxarbitrage

  • Status: In review!

Note: if I’m missing anything, please comment below! if you wish to actively be part of this development effort, send me a DM (or comment below if you can’t)

Have a Nice Weekend y’all!

References:

9 Likes
2

The Zenith RPC server will be released as part of our third milestone, allowing programmatic access to the Zenith wallet for Zebra.

4 Likes
3

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

  • Status: on track

We have collected and prioritized partners for the outreach. We divided the different contacts in three tiers (1, 2, 3) and already reached out to tiers 1 and 2. We started to receive responses for the outreach which will be an essential input for Zcash developers. We are focusing on knowing which are the RPCs used by those who run Zcashd nodes among other contextual information that helps us know more about Zcashd infrastructure usage.

For privacy reasons there’s a lot of information that can’t be publicly shared with you all. I’m sure that you will understand. :slightly_smiling_face:

Zaino:

  • Status: on track
    Zaino’s Github repo is pretty loaded with issues. Each one of them is labeled with their corresponding milestone among other informative tags.

I created a kanban board for Zaino. It’s outside ZingoLabs’ org to save ZEC. Otherwise it would require Zingo to pay github for an extra user for me and we can momentarily use this one perfectly fine.

This board only has milestones 1 and 2 (for now) in order not to clog the board just for the sake of it.

Zaino devs are currently working on milestone 1.

In-Memory backend wrap up

  • Status: On track

Eric and Willem from ChainSafe, Kris and Str4d from ECC and myself met last monday to talk about how to merge ChainSafe’s work on librustzcash back to upstream. There’s a lot of boiler plate code Eric and Willem had to create to convert some Rust data structures into Javascript objects. That code isn’t “useless” but it’s in an odd place in terms of a valuable general-purpose contribution to upstream, whereas there’s other portions of the In-Memory wallet code what we Do want merged back upstream.

While ChainSafe has gone the extra mile to contribute upstream and make their contributions be useful to the wide ecosystem of Zcash developers, we still have to figure out this last bit, which is a non-trivial effort on their side.

This work will enable one of the critical path branches of the Zcashd deprecation DAG so we are particularly interested on this being completed along with the release of their great library WebZ.js :tada:

Zcashd wallet’s successor

Agree on overall architecture

This is an ongoing discussion that will shape the incoming years of the Zcash wallet ecosystem

  • Status: on track (no changes since last week)

Zebra

No introduction needed!

@pili just published ZF’s engineering update! Go read it

There’s work related to implementing a Regtest mode that is equivalent to the one provided by Zcashd.

Zcashd deprecation RPC analysis

Yesterday ECC, ZF, Zingo and ZF folks met in the weekly Zcashd Deprecation sync meeting and we discussed

There are some outstanding questions on how to proceed on already deprecated methods on Zcashd that are being revealed to be highly used by current users.

[Action items]

  • If you have ideas, please post them below!
  • If you run Zcash infrastructure and have input about this, reach out!

Block Explorer Support

There’s a lot of work needed to be able to support block explorers. Overall, the major point to bring up is that Zaino will have a key role here, given that Explorers will have to point to it to get the information related to the blockchain. This has many benefits. Zaino is easier to scale horizontally than nodes. Also, nodes don’t learn about transactions queried by the block explorer.

A list of RPCs related to Block Explorers is being curated and maintained by Pili. (Kudos!)

7 Likes
4

Regarding block explorer support, we have a working explorer on Zebra 1.9.0 that allows to search by transaction hash and block height.

What other feature should we look at implementing next?

6 Likes
5

It would be really useful to show the last mined blocks.

3 Likes
6

as a list of hashes for the last 10 blocks?

7

i cant think of much atm.

but just list of all mined blocks on the site and txns in mempool for ex.

wait if i click on any block there i see only 1 txn. is dat normal or not?

8

Exactly. Usually when I use a block explorer I want to check something on the recently mined blocks, and it’s a easy way to start navigating.

2 Likes
9

It is for testnet, there’s not much activity so it’s usually just the coinbase Tx.

2 Likes
10

ah ok. makes sense. :joy:

11

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

  • Status: on track

We are following up on tier 1 contacts and starting to reach out to block explorers to get a list of RPC requirements in terms of existing RPC methods they use and points of improvement.

Zaino

  • status: on track

Deliverable 1.1 is complete and 1.2 has been started! wohoooo!!! kudos to @idky137 and the rest of Zingo Labs!

Also they started work on the testing framework required for comparing functionality between lightwalletd-zaino and zcashd-zebra and also they are brewing the changes required for using zebra as zainos backend

In-Memory backend wrap up

  • Status: On track

As you might have read in this forum post WebZ.js is almost ready and ChainSafe is working with ZCG on a grant extension to contribute upstream to librustzcash in a way that’s fruitful for both ends.

As you can appreciate below, crossing out these tasks clears a path of the DAG that is super important to our Goal

image
image1920Ă—1291 208 KB

Zcashd deprecation RPC analysis

  • no changes

There are some outstanding questions on how to proceed on already deprecated methods on Zcashd that are being revealed to be highly used by current users.

Others

I was blocked on some tasks so I grabbed a task from the DAG which is in review. Thanks Daria-Emma for your thorough feedback

[Action items]

  • If you have ideas, please post them below!
  • If you run Zcash infrastructure and have input about this, reach out!
4 Likes
12

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

  • Status: on track

We continue to follow-up with tier 1 contacts and getting data on their usage of existing Zcashd APIs. Also, we are talking with NEW partners that want to support Zcash to incorporate new use cases that we don’t know of. We are in conversations with important infrastructure player of the Avalanche Ecosystem to know what we need to get Zcash accessible in Avalanche block explorers. I also connected with a chain metrics provider that will help us have a better API for having better metrics when it comes to TPS, Blocktime, TX Volume and others. This kind of metrics are used by service providers like exchanges to forecast their operations.

Zaino

  • status: on track

Zingo folks keep progressing on their planned milestones This PR is landing work to wrap up 66% to milestone 1.3

We are currently working collectively between ZF, ECC and Zingo devs to decide where some RPCs belong (Zebra or Zaino) depending on their scope, use and latency requirements.

In-Memory backend wrap up

  • status: on track
    ChainSafe and ECC engineers are still collaborating to get their contributions upstream with support of ZCG.

Zcashd Wallet replacement

Zebra

@pili published a great engineering update on this post

[Action items]

  • If you have ideas, please post them below!
  • If you run Zcash infrastructure and have input about this, reach out!
5 Likes
13

This Report corresponds to progress up to November 1st.

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

  • Status: on track

We continue to contact, nudge and follow up with organizations and companies running Zcashd nodes.

Zaino (ZingoLabs)

  • status: on track

  • Deliverable 1.3: Code has now landed in Zaino, there is a PR in zebra that needs to land, they have reviewed it and are currently fixing an issue with their CI and will merge once they have sorted it.

  • Deliverable 1.4: About to be completed. Zingo engineers have realised that there is reason for them to integrate zebra regtest mode into their test framework now (this was originally going to be completed in milestone 2). Oscar is working on this now, this is the last work required for this deliverable.

  • Deliverable 1.5: In progress. Around 50% of all the gRPC services of this deliverable are implemented.

  • Deliverables 1.3, 1.6: There is still an ongoing conversation with @nuttycom about a couple of the Services that Zingo wanted to deprecate. They are close to obtain necessary input to finalize Deprecation plans for 1.3 and once that happens 1.6 can be finished.

In-Memory backend wrap up

  • status: on track

ChainSafe and ECC engineers are still collaborating to get their contributions upstream with support of ZCG.

Zcashd Wallet replacement

4 Likes
14

November 8th

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

  • Status: on track

We continue to reach out to folks out there. Also Pacu proposed to start merging another Outreach to this one to ZCG. We are seeing that “optional-KYC” CEXs like Stealthex are increasing their confirmation times. This was well reported by @dismad and @zksquirrel on ZecHub but it seems that it can get worse at times. A user reported to pacu that their wait was 124 confs for a large sum of ZEC. Given that these kinds of CEXs are part of the “Tier 3” contacts it would be useful to add extra questions to their feedback request in terms of confirmation times to make ZEC users’ lives easier. “Two hours is too much to wait” - user said…

Zaino (ZingoLabs)

  • status: on track

Zingo continues to advance in development of Zaino RPCs. They are close to wrapping up Milestone 1. And also adding extra Regtest features to Zebra

In-Memory backend wrap up

  • status: Grant under evaluation

Chainsafe presented a follow-up grant for their In-memory wallet work upstream contribution.

Collaboration with ECC Engineers is required

Links:

Zcashd Wallet CLI

  • Status: on track :warning:

This is scheduled to be completed by the end of January! This timeline is TIGHT! So please check out the call to action below.

All hands eyes on deck! We need more reviewers!

Librustzcash is getting a lot of contributions! That’s good! But also a bottleneck for our dear maintainers Daira-Emma, Kris and Str4d.

So for that ECC folks will also invite Zcash developers from all teams to a “Librustzcash walkthrough” like the one ZF engineers did with the Zebra codebase so developers interested in helping on this can learn about the invariants and other important context of the Zcash Rust Crates (aka librustzcash) and provide insightful reviews that go beyond the formal aspects of language and style that seasoned Rust devs like them can already provide.

Happy weekend to you all!

5 Likes
15

Great idea!

4 Likes
16

November 15th

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

* Status: on track

We have received further feedback on RPC usage. This has been tougher than expected. As it has been our experience with other Outreach efforts, the closer we are to the target deprecation date the more urgency is perceived. We have the methodical task of Nudging everyone every week. On the bright side, “Tier 3” partners have opened the kitchen door to know how these exchanges use blockchain service providers and who they are, so we are focusing on those as well.

We pour information to this public spreadsheet. If you are running Zcashd infrastructure and rely on RPC methods check it out and contact us through here or the R&D discord

Zaino (ZingoLabs)

* status: on track

ZECDEV Zaino Board
Zingo is about to wrap up milestone 1 in the next day or two!

According to their engineers “We now have the full set of Lightwallet gRPC Services implemented in Zaino. And an (almost) complete set of RPC tests in zcash-local-net (with the exception of orchard subtree roots. This needs to be a mainnet test, we are currently reviewing this code).”. Isn’t that awesome?

Documentation on its way:
1.2 - Explicitly mark and broadcast list of lightwallet gRPCs to be deprecated
Agree on LightWallet gRPC Services to be Deprecated.
Agree on transparent tx service for wallets
1.6 - Outline of the Indexer design, contextualised by implementation

Also work on Upgrade Zaino-testutils to use zcash-local-net. Is being done which will enable Zingo folks to start milestone 2 work once milestone 1 is finished.

In-Memory backend wrap up

- status: Grant approved

@zcashgrants approved the follow-up grant!

Also, @danforbes from the SNAP Team at ChainSafe posted this update a few days back. WOOT. ChainSafe’s involvement with Zcash has brought a lot of improvements to our stack and also bonus points because they are all great people to work with!

Zcashd Wallet CLI

  • Status: on track :warning:

This is scheduled to be completed by the end of January! This timeline is TIGHT!

ECC engineers are tracking work of this on this repository

School of Rock Reviewers

One of the major bottlenecks of Zcash development is PR review of core contributions. In order to keep the high standards Zcash has been always renowned and characterized for we need to be able to augment the throughput of the review pipeline by adding more qualified reviewers. That’s a process that takes time and a learning curve for new reviewers and also requires time and attention from current reviewers because they’ll need to supervise it all and help new reviewers learn the common criteria, how to make good judgment on coding decisions, etc.

@str4d, @nuttycom and @daira will present a Librustzcash walkthrough in the coming weeks over a Zoom Webinar that @decentralistdan will facilitate and record. Details need to be defined but stay tuned!

11 Likes