Zcashd Deprecation updates

Ecosystem Updates
1

Hello fellow Zebras!

As you may be aware of Zcashd deprecation is one of the top priorities of the Zcash development teams. This thread attempts to keep the community updated in a succinct and easy to grasp way.

Outreach (@pili @decentralistdan @pacu):

We are conducting an Ecosystem Outreach to have first-hand insights on Zcashd use cases and other important input for Zcash developers. ZF, ECC and ZCG (with Pacu appointed resource) will be leading this important

  • Status: initiated

Zaino:

formerly known as Zingo Indexer, Zaino, is product of a lot of architecture design, planning and collaboration between ZingoLabs, ZF, ECC and ZCG which shaped a grant that will let Zingo to expand its team and contribute a key piece of infrastructure that will not only accelerate Zcashd Deprecation but also evolving the existing Zcash infrastructure into a more modern, maintainable and versatile shape that can not only accommodate current use cases, but be ready to adopt the challenges of the Zodlers of tomorrow. While you will be able to follow milestone updates on the corresponding thread, here you will find smaller incremental updates.

  • Status: on track

Librustzcash

LRZ is going through a huge overhaul! Although it has been supporting a lot of recent functionality on Zcashd and the Mobile SDKs, it will continue to grow to support the wallet that will ultimately replace the “Zcashd wallet”.

In-Memory backend

Chainsafe is doing this development as part of their Javascript Zcash SDK grant!

  • Status: on track
    • they are pretty close to implementing this backend.
    • we will meet shortly to polish how upstream integration LRZ will play out since there are significant and important changes they’ll be contributing back to that codebase.
    • Kudos to Eric and @Wollum-ChainSafe

SQLITE test framework migration

this is necessary in order to make the tests reusable with other wallet backends

Zcashd wallet’s successor

Agree on overal architecture

This is an ongoing discussion that will shape the incoming years of the Zcash wallet ecosystem

  • Status: on track

Zebra

No introduction needed!

Cookie auth for RPC methods

kudos @oxarbitrage

  • Status: In review!

Note: if I’m missing anything, please comment below! if you wish to actively be part of this development effort, send me a DM (or comment below if you can’t)

Have a Nice Weekend y’all!

References:

10 Likes
2

The Zenith RPC server will be released as part of our third milestone, allowing programmatic access to the Zenith wallet for Zebra.

5 Likes
3

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

  • Status: on track

We have collected and prioritized partners for the outreach. We divided the different contacts in three tiers (1, 2, 3) and already reached out to tiers 1 and 2. We started to receive responses for the outreach which will be an essential input for Zcash developers. We are focusing on knowing which are the RPCs used by those who run Zcashd nodes among other contextual information that helps us know more about Zcashd infrastructure usage.

For privacy reasons there’s a lot of information that can’t be publicly shared with you all. I’m sure that you will understand. :slightly_smiling_face:

Zaino:

  • Status: on track
    Zaino’s Github repo is pretty loaded with issues. Each one of them is labeled with their corresponding milestone among other informative tags.

I created a kanban board for Zaino. It’s outside ZingoLabs’ org to save ZEC. Otherwise it would require Zingo to pay github for an extra user for me and we can momentarily use this one perfectly fine.

This board only has milestones 1 and 2 (for now) in order not to clog the board just for the sake of it.

Zaino devs are currently working on milestone 1.

In-Memory backend wrap up

  • Status: On track

Eric and Willem from ChainSafe, Kris and Str4d from ECC and myself met last monday to talk about how to merge ChainSafe’s work on librustzcash back to upstream. There’s a lot of boiler plate code Eric and Willem had to create to convert some Rust data structures into Javascript objects. That code isn’t “useless” but it’s in an odd place in terms of a valuable general-purpose contribution to upstream, whereas there’s other portions of the In-Memory wallet code what we Do want merged back upstream.

While ChainSafe has gone the extra mile to contribute upstream and make their contributions be useful to the wide ecosystem of Zcash developers, we still have to figure out this last bit, which is a non-trivial effort on their side.

This work will enable one of the critical path branches of the Zcashd deprecation DAG so we are particularly interested on this being completed along with the release of their great library WebZ.js :tada:

Zcashd wallet’s successor

Agree on overall architecture

This is an ongoing discussion that will shape the incoming years of the Zcash wallet ecosystem

  • Status: on track (no changes since last week)

Zebra

No introduction needed!

@pili just published ZF’s engineering update! Go read it

There’s work related to implementing a Regtest mode that is equivalent to the one provided by Zcashd.

Zcashd deprecation RPC analysis

Yesterday ECC, ZF, Zingo and ZF folks met in the weekly Zcashd Deprecation sync meeting and we discussed

There are some outstanding questions on how to proceed on already deprecated methods on Zcashd that are being revealed to be highly used by current users.

[Action items]

  • If you have ideas, please post them below!
  • If you run Zcash infrastructure and have input about this, reach out!

Block Explorer Support

There’s a lot of work needed to be able to support block explorers. Overall, the major point to bring up is that Zaino will have a key role here, given that Explorers will have to point to it to get the information related to the blockchain. This has many benefits. Zaino is easier to scale horizontally than nodes. Also, nodes don’t learn about transactions queried by the block explorer.

A list of RPCs related to Block Explorers is being curated and maintained by Pili. (Kudos!)

7 Likes
4

Regarding block explorer support, we have a working explorer on Zebra 1.9.0 that allows to search by transaction hash and block height.

What other feature should we look at implementing next?

6 Likes
5

It would be really useful to show the last mined blocks.

4 Likes
6

as a list of hashes for the last 10 blocks?

7

i cant think of much atm.

but just list of all mined blocks on the site and txns in mempool for ex.

wait if i click on any block there i see only 1 txn. is dat normal or not?

8

Exactly. Usually when I use a block explorer I want to check something on the recently mined blocks, and it’s a easy way to start navigating.

2 Likes
9

It is for testnet, there’s not much activity so it’s usually just the coinbase Tx.

2 Likes
10

ah ok. makes sense. :joy:

11

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

  • Status: on track

We are following up on tier 1 contacts and starting to reach out to block explorers to get a list of RPC requirements in terms of existing RPC methods they use and points of improvement.

Zaino

  • status: on track

Deliverable 1.1 is complete and 1.2 has been started! wohoooo!!! kudos to @idky137 and the rest of Zingo Labs!

Also they started work on the testing framework required for comparing functionality between lightwalletd-zaino and zcashd-zebra and also they are brewing the changes required for using zebra as zainos backend

In-Memory backend wrap up

  • Status: On track

As you might have read in this forum post WebZ.js is almost ready and ChainSafe is working with ZCG on a grant extension to contribute upstream to librustzcash in a way that’s fruitful for both ends.

As you can appreciate below, crossing out these tasks clears a path of the DAG that is super important to our Goal

image
image1920Ă—1291 208 KB

Zcashd deprecation RPC analysis

  • no changes

There are some outstanding questions on how to proceed on already deprecated methods on Zcashd that are being revealed to be highly used by current users.

Others

I was blocked on some tasks so I grabbed a task from the DAG which is in review. Thanks Daria-Emma for your thorough feedback

[Action items]

  • If you have ideas, please post them below!
  • If you run Zcash infrastructure and have input about this, reach out!
4 Likes
12

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

  • Status: on track

We continue to follow-up with tier 1 contacts and getting data on their usage of existing Zcashd APIs. Also, we are talking with NEW partners that want to support Zcash to incorporate new use cases that we don’t know of. We are in conversations with important infrastructure player of the Avalanche Ecosystem to know what we need to get Zcash accessible in Avalanche block explorers. I also connected with a chain metrics provider that will help us have a better API for having better metrics when it comes to TPS, Blocktime, TX Volume and others. This kind of metrics are used by service providers like exchanges to forecast their operations.

Zaino

  • status: on track

Zingo folks keep progressing on their planned milestones This PR is landing work to wrap up 66% to milestone 1.3

We are currently working collectively between ZF, ECC and Zingo devs to decide where some RPCs belong (Zebra or Zaino) depending on their scope, use and latency requirements.

In-Memory backend wrap up

  • status: on track
    ChainSafe and ECC engineers are still collaborating to get their contributions upstream with support of ZCG.

Zcashd Wallet replacement

Zebra

@pili published a great engineering update on this post

[Action items]

  • If you have ideas, please post them below!
  • If you run Zcash infrastructure and have input about this, reach out!
5 Likes
13

This Report corresponds to progress up to November 1st.

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

  • Status: on track

We continue to contact, nudge and follow up with organizations and companies running Zcashd nodes.

Zaino (ZingoLabs)

  • status: on track

  • Deliverable 1.3: Code has now landed in Zaino, there is a PR in zebra that needs to land, they have reviewed it and are currently fixing an issue with their CI and will merge once they have sorted it.

  • Deliverable 1.4: About to be completed. Zingo engineers have realised that there is reason for them to integrate zebra regtest mode into their test framework now (this was originally going to be completed in milestone 2). Oscar is working on this now, this is the last work required for this deliverable.

  • Deliverable 1.5: In progress. Around 50% of all the gRPC services of this deliverable are implemented.

  • Deliverables 1.3, 1.6: There is still an ongoing conversation with @nuttycom about a couple of the Services that Zingo wanted to deprecate. They are close to obtain necessary input to finalize Deprecation plans for 1.3 and once that happens 1.6 can be finished.

In-Memory backend wrap up

  • status: on track

ChainSafe and ECC engineers are still collaborating to get their contributions upstream with support of ZCG.

Zcashd Wallet replacement

4 Likes
14

November 8th

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

  • Status: on track

We continue to reach out to folks out there. Also Pacu proposed to start merging another Outreach to this one to ZCG. We are seeing that “optional-KYC” CEXs like Stealthex are increasing their confirmation times. This was well reported by @dismad and @zksquirrel on ZecHub but it seems that it can get worse at times. A user reported to pacu that their wait was 124 confs for a large sum of ZEC. Given that these kinds of CEXs are part of the “Tier 3” contacts it would be useful to add extra questions to their feedback request in terms of confirmation times to make ZEC users’ lives easier. “Two hours is too much to wait” - user said…

Zaino (ZingoLabs)

  • status: on track

Zingo continues to advance in development of Zaino RPCs. They are close to wrapping up Milestone 1. And also adding extra Regtest features to Zebra

In-Memory backend wrap up

  • status: Grant under evaluation

Chainsafe presented a follow-up grant for their In-memory wallet work upstream contribution.

Collaboration with ECC Engineers is required

Links:

Zcashd Wallet CLI

  • Status: on track :warning:

This is scheduled to be completed by the end of January! This timeline is TIGHT! So please check out the call to action below.

All hands eyes on deck! We need more reviewers!

Librustzcash is getting a lot of contributions! That’s good! But also a bottleneck for our dear maintainers Daira-Emma, Kris and Str4d.

So for that ECC folks will also invite Zcash developers from all teams to a “Librustzcash walkthrough” like the one ZF engineers did with the Zebra codebase so developers interested in helping on this can learn about the invariants and other important context of the Zcash Rust Crates (aka librustzcash) and provide insightful reviews that go beyond the formal aspects of language and style that seasoned Rust devs like them can already provide.

Happy weekend to you all!

5 Likes
15

Great idea!

4 Likes
16

November 15th

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

* Status: on track

We have received further feedback on RPC usage. This has been tougher than expected. As it has been our experience with other Outreach efforts, the closer we are to the target deprecation date the more urgency is perceived. We have the methodical task of Nudging everyone every week. On the bright side, “Tier 3” partners have opened the kitchen door to know how these exchanges use blockchain service providers and who they are, so we are focusing on those as well.

We pour information to this public spreadsheet. If you are running Zcashd infrastructure and rely on RPC methods check it out and contact us through here or the R&D discord

Zaino (ZingoLabs)

* status: on track

ZECDEV Zaino Board
Zingo is about to wrap up milestone 1 in the next day or two!

According to their engineers “We now have the full set of Lightwallet gRPC Services implemented in Zaino. And an (almost) complete set of RPC tests in zcash-local-net (with the exception of orchard subtree roots. This needs to be a mainnet test, we are currently reviewing this code).”. Isn’t that awesome?

Documentation on its way:
1.2 - Explicitly mark and broadcast list of lightwallet gRPCs to be deprecated
Agree on LightWallet gRPC Services to be Deprecated.
Agree on transparent tx service for wallets
1.6 - Outline of the Indexer design, contextualised by implementation

Also work on Upgrade Zaino-testutils to use zcash-local-net. Is being done which will enable Zingo folks to start milestone 2 work once milestone 1 is finished.

In-Memory backend wrap up

- status: Grant approved

@zcashgrants approved the follow-up grant!

Also, @danforbes from the SNAP Team at ChainSafe posted this update a few days back. WOOT. ChainSafe’s involvement with Zcash has brought a lot of improvements to our stack and also bonus points because they are all great people to work with!

Zcashd Wallet CLI

  • Status: on track :warning:

This is scheduled to be completed by the end of January! This timeline is TIGHT!

ECC engineers are tracking work of this on this repository

School of Rock Reviewers

One of the major bottlenecks of Zcash development is PR review of core contributions. In order to keep the high standards Zcash has been always renowned and characterized for we need to be able to augment the throughput of the review pipeline by adding more qualified reviewers. That’s a process that takes time and a learning curve for new reviewers and also requires time and attention from current reviewers because they’ll need to supervise it all and help new reviewers learn the common criteria, how to make good judgment on coding decisions, etc.

@str4d, @nuttycom and @daira will present a Librustzcash walkthrough in the coming weeks over a Zoom Webinar that @decentralistdan will facilitate and record. Details need to be defined but stay tuned!

11 Likes
17

November 22nd

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Outreach (@pili @decentralistdan @pacu):

* Status: on track

We have received further feedback on RPC usage. Decred DEX responded with insightful thoughts!

“I’ll take this opportunity to respectfully offer some constructive complaints:

Estimating the number of nActionsOrchard required for ZIP-317 tx fees is seemingly impossible, so we have to use some shaky logic to get fee estimations for the user.

The way the fromaddress parameter works in the z_sendmany RPC is odd and causes us to have to do some dancing. It would be great if we could just specify an account number, since that’s almost what it reduces to internally anyway.

As mentioned above, block median time is a critical component of script logic for HTLCs, but the RPC doesn’t provide it anywhere, so we have to constantly iterate block headers and calculate it on the fly. I would have loved to see zcash/zcash#6005 or similar implemented, but there’s probably no point now.

The RPC is, somewhat understandably, a little discombobulated as a result of its historical development, esp. in regards to the implementation of accounts and ZIP-317. I look forward to seeing some of these oddities ironed out in whatever interface we migrate to.”

It’s feedback like this that will make our tooling better. Thank you Decred folks!

Note: We pour information to this public spreadsheet. If you are running Zcashd infrastructure and rely on RPC methods check it out and contact us through here or the R&D discord

Zebra

The Zebra team has been working on RPC method development and also on NU6 tasks.

Zaino (ZingoLabs)

  • status: on track

Zaino Project Board

Milestone 1 completed!
Zingo folks are preparing the release and starting work of Milestone 2 which will include performance updates by integrating to the ReadState service of Zebra.

Let’s congratulate the Zingoistas on this awesome achievement.

In-Memory backend wrap up

- status: on track

Willem from Chainsafe started work on this upstream collaboration and Quote

“I started work on this grant to work on librustzcash and the memory wallet backend. For ease of review I’m going to split changes into as many small PRs as possible and here is the first one https://github.com/zcash/librustzcash/pull/1627”

Let’s chip in with our reviews. Keep them coming!

Zcashd Wallet CLI

  • Status: on track :warning:

This is scheduled to be completed by the end of January! This timeline is TIGHT!

tracking work on this repository.

ECC core engineers are working on Keystone integration items and solving issues with address book encryption issues. This also contributes to changes needed to make the sqlite-test wallet crate public. .

School of Rock Reviewers

  • Status: scheduling

Given that we have holidays in between and people on time-off we are still this. I’ll keep you posted!

Wallet Export format

  • Status: exploring collaboration

image
image2606Ă—472 89.4 KB

We are exploring a collaboration with a group of developers that is very experienced in these kinds of standardized formats. “Gordian Developers” from Blockchain Commons and different wallet developers of the Zcash ecosystem are collaborating to complete this line of the Zcash deprecation DAG. A conversation is starting here on this github issue.

I’m always looking around to see who would like to contribute to DAG work that’s pending and I’d like to give a special recognition to @zancas who is always teaming up with me on this ideas and helping me out to bring them to fruition.

This week has been a very productive one. NU6 activates in less than a day! Next week there are two holidays in the US. Some folks will be catching their breath and enjoying well deserved time with their families and friends.

Happy thanksgiving to those who celebrate. I’d like to take this opportunity to thank the Zcash community and everyone at ZCG, ZF, ECC, ZingoLabs, RedDev, Shielded Labs, Zondax, ZECHub, LCWG, EDGE, Unstoppable, Ywallet, ChainSafe, NightHawk, zec.rocks and many others I’m thankful to work with everyday.

Ps: I got a hold of a turkey recipe, apparently another specialty of the Red Bridge team that I’d like to try so… I’ll probably be cooking some turkey as well

8 Likes
18

November 29th

Hello fellow Zebras! Another week, another update on Zcashd deprecation!

Happy NU6 Deployment to you all!!

This has been a mixed week! between people taking time off with their families for thanksgiving and this tremendous milestone over the last weekend. I hope all those who celebrated it had a great time with their loved ones.

On Saturday we had our Network Upgrade 6 monitoring call. Every NU, Zcash developers get into a conference call where everyone is focused on watching the blocks go by while observing server logs of the Zebra and Zcashd servers devs run on their own to look for any anomalies and double-triple-check every finding with their colleagues. Some also play to see if they get their transaction into the NU block! I’m always so attentive to running several wallets at the same time that I always forget to send mine! How does that work?
When the NU is about to happen, the nodes evict all transactions of the prior NU out of the mempool. This means that from a given block on, all the transactions have to be created with the new consensus rules. That’s how the “get into NU block” game originates from.

Only the sender will know if the transaction got in time for the NU6 block or not. That’s the beauty of Shielded ZEC.

NU6 is the last Network Upgrade that Zcashd will receive. This is a huge milestone. Everything went pretty good. But, there are still some problems with some block explorers and wallets that did not follow the published updates or couldn’t get their builds in time for App Store, Google Play publication.

Status of wallet NU6 support

Wallet Ready on NU6 Same day or 1 day Today (Nov 27)
Edge :white_check_mark:
Exodus :x: :warning: (FIX IN PROGRESS)
eZcash :white_check_mark:
NightHawk :x: :white_check_mark:
Ledger Transparent Zcash :x: :warning: (FIX IN PROGRESS)
Ledger Shielded Zcash :x: :white_check_mark:
Unstoppable :x: :white_check_mark:
Trezor :white_check_mark: :white_check_mark:
Trust :x: :warning: (FIX IN PROGRESS)
Ywallet :x: :white_check_mark:
Zingo :x: :white_check_mark:
Zashi :white_check_mark:

I have been actively communicating with all wallet teams to assist them in what they might need. Sometimes it was just sending the corresponding ZIPs their way and others connecting them with core developers for further troubleshooting. We will keep you posted on the latest developments. We thank every team involved in fixing these road bumps and prioritizing their backlog to make NU6 a success.

Outreach (@pili @decentralistdan @pacu):

* Status: on track

NU6 upgrade took most of the time for the Deprecation Outreach. We’ve been attending to the urgent matters of Zcashers finding issues. But, along with the NU6 contacts we’ve been piggybacking the Zcashd outreach as well. So we hope to see more feedback coming in from this past week.

Zebra

The Zebra team has been working on RPC method development and also on NU6 tasks.
This is the in-flight pull requests sorted by latest updated

Important bug fixes on RPC return values and also important refactoring in critical RPCs such as getblock which is used by all known clients.

Zaino (ZingoLabs)

  • status: on track

Project Board
Work on Milestone 2 has already started. Zaino devs are focusing on hooking up to the ReadStateService of Zebrad which will bring performance improvements on the existing features of milestone 1.

In-Memory backend wrap up

- status: on track

Willem from Chainsafe started work on this upstream collaboration and Kris Nuttycombe from ECC has provided feedback on his review Add serde derives to primitive types behind feature flag by willemolding · Pull Request #1627 · zcash/librustzcash · GitHub

Let’s chip in with our reviews. Keep them coming!

Zcashd Wallet CLI

  • Status: on track :warning:

This is scheduled to be completed by the end of January! This timeline is TIGHT!

tracking work on this repository.

ECC core engineers are working on Keystone integration items and solving issues with address book encryption issues. This also contributes to changes needed to make the sqlite-test wallet crate public. .

Last week some issues started be In-Progress

School of Rock Reviewers

  • Status: scheduling

Given that we have holidays in between and people on time-off we are still this. I’ll keep you posted!

Wallet Export format

  • Status: Proposal in progress

As I mentioned in our last post, we are exploring a collaboration with a group of developers that is very experienced in these kinds of standardized formats. A proposal is being worked on by a development team and there will be a feasibility discussion next Thursday December 5th in the Zcash deprecation call.

Have a nice weekend!

13 Likes
19

thank you pacu for doing the most important work so the users can also keep transacting with Zcash on various wallets

6 Likes
20

Thank you! It’s a team effort! I’m just the visible face. I’m sure there are things we can improve for NU7, we will work on that. My head is looking towards ZSAs, V6 transactions and how we can make a such a game-changing event have the process and momentum it deserves.

So far we’ve spotted a few anti-patterns in the code of some transparent wallets that we must be aware of. It’s something really hard to do in terms for human communication because we want these wallet developers to have their best disposition to ammend the problem, but at the same time we need to point out that sometimes it’s because of their own doing. For example, some wallets just write the Zcash protocol constants inside some file and leave it there instead of using something that is more dynamically adjustable. We had the awkward moment of pointing that out as “the fault” while appealing to their good will to fix the issue. Fortunately I’ve had @daira and @nuttycom by my side that can deliver such “news” with a very good educational and collaborative approach and navigate those shaky waters and difficult conversations.

Some contacts have been facilitated by @aquietinvestor, other hard to reach wallet contact was delivered by @Cacho and so on.

All hands on deck! Thanks for the support!

6 Likes