All ECC teams focused on wallet performance

Hi everyone,

Firstly, I want to thank everyone for their patience during this time. The team is working through the issues to get mobile wallets working to the standard users expect. These issues are complex, but our team is confident in the improvements that have been made and those that are in progress both for the wallet SDKs and the zcashd node.

Here are updates from our engineering teams:

  • Continued the back-end work required to support the next release of the IOS and Android SDKs which will include the first phase of an improved sync capability

  • Fixed a number of intermittent out-of-memory issues that were introduced by the batch scanning optimizations which will be included in zcashd release 5.3.0.

  • Built a number of dashboards analyzing historical transaction data which we are using in our analysis of the proposed fee change mechanism in ZIP 317.

@Daira also provided an update in another thread: Zcash's current performance issues are frustrating - #60 by daira

Thank you all for your patience during this time. Please reach out with any questions.

Ian

12 Likes

Just compiled latest zcashd master, Huge improvement. Excited :zebra:

6 Likes

The blockchain size is quickly approaching 100GB

https://bitinfocharts.com/zcash/

Isn’t it already well over 100GB?

The Zcash Block Explorer, funded by Zcash Community Grants, shows 116.05GB and counting

The Zcash Block Explorer reports size on disk.

2 Likes

Hi everyone, Ian from ECC here to provide another update.

ECC recently got together for its planning week to discuss key objectives for the upcoming term. The team was able to develop a high-level strategy (both short and long term), and also reaffirm our immediate objectives regarding syncing issues with our wallet SDKs.

ECC’s No. 1 priority at the moment is updating the Zcash protocol and our wallet SDKs to help third-party providers fix wallet syncing issues. An increased on-chain transaction load has resulted in unacceptable sync times for many everyday users, and ecosystem partners are also affected. We are focused — across teams — on alleviating these issues as soon as possible, so Zcash users have an experience they expect and deserve. Our immediate objectives are centered on working with third-party wallets that are built on the ECC SDKs (Edge, Nighthawk and Unstoppable) to:

  • Ensure users can spend their funds (funds that are already synced when they open their wallet)
  • Ensure users can receive and spend new incoming funds at a rate of a month’s worth of transactions in one hour
  • Implement updates which provide users clarity on syncing progress
  • Ensure none of those wallets are impacted by frequent crashes or inconsistent behavior (such as failing to display some already synced transactions), nor do they require work-around behaviors due to the ECC SDK

We have also released zcashd 5.3.0. The main focus of 5.3.0 is to reduce concurrent memory utilization during scanning among other memory and performance related optimisations.

We will provide updates around our short- to long-term strategies in the coming weeks. We will also provide an update on future releases and wallet syncing methods we’re reviewing for our SDKs, when those are available.

Thanks again,
Ian

12 Likes

Does this include fixing the issues with ledger?

When will the release be available from the repository?

If you’re on Debian/Ubuntu, you can build directly from source as documented here: Building Zcashd & Zcash-cli on Debian/Ubuntu — Zcash Documentation 5.2.0 documentation

Just makes sure you select the correct release branch:

 git clone https://github.com/zcash/zcash.git
 cd zcash/
 git checkout v5.3.0
 ./zcutil/fetch-params.sh
1 Like

Not that I’m aware of. Ledger doesn’t depend on ECC mobile SDK or improvements made to shielded transactions on zcashd as Ledger doesn’t support shielded transactions.

I have personally been affected by issues with Ledger since the NU5. I understand that this can be frustrating as I also couldn’t access ZEC stored with Ledger hardware wallet. My suggestion is to reach out to Ledger directly and explain your situation through https://support.ledger.com/hc/en-us/categories/4404369571601-Support

Reminder: Never share you seed/recovery phrase with anyone especially with those that claim to be from official supports team

If there’s any lesson to be learned from this experience, never store too much of your money in places you might lose access to.

1 Like

There was a patch applied to ledger desktop live about a month ago but I’m not sure if applies here https://mobile.twitter.com/Ledger_Support/status/1575467661600980996?s=20&t=lKH7crpTv5gVzNfK3AJdNw

Do you mean it will take 12h to sync a year’s worth of data?

We are targeting Monday for the binaries to be available.

2 Likes

This is our minimum target for sync improvements. The way our DAGSync algorithm works, part of the capability is to make recently received funds spendable as soon as possible even before a full sync has occurred.

Thanks for the info. Were you able to resolve your issue with ledger?

2 Likes

No, not yet. I’m still waiting for their response but they have told me they’re working on it.

1 Like

I wish there was more information I could offer, ZerkaPGM#0875 posted on the Discord a few days ago about a ledger issue specifically about the code 0x6f01 which points to a technical issue e.g. damaged, check connections, try different computer. I couldn’t help him much either and ended up digging down a rabbit hole of an old issue of that fault that was speculated to have been caused possibly by too many apps accessing the device though was eventually recreated by passing empty input but this was patched years ago. I then stumbled on to these most recent patches which I assume apply to v5.x.x but idk.

Hey everyone, providing another update on the work ECC is doing here. Tl;dr is that faster sync for SDKs will be released in phases, zcashd 5.3.0 was released, and we’ve completed our analysis for ZIP 317.

Update:

  • Our #1 priority remains alleviating issues with slow sync times for wallets built on our mobile SDKs, primarily Edge, Unstoppable, and NightHawk.
  • A faster sync capability for our SDKs is being released in a couple of phases, the first of which lays the groundwork for DAGSync and adds support for Unified Addresses (UAs). The iOS version is currently in internal testing with Android following in about a week.
  • ECC released zcashd 5.3.0 to reduce concurrent memory utilization during scanning among other memory and performance related optimisations for full nodes.
  • We’ve completed our analysis for ZIP 317 which will make fees fair and equitable and consistent with chain usage. We are working on the necessary changes to zcashd at the moment and working with our ecosystem partners to gather additional feedback and discuss broader implementation efforts.
  • We’re continuing to work on recursion for Halo 2 which will form the basis for some additional performance and scalability optimizations in the medium-term.
  • We are currently working on additional optimizations for zcashd which will further reduce memory utilization and block propagation times.

Thanks again for your time and patience during this time. If there are any questions, please post them below and our team will answer them. You can also reach out to me directly.

Ian

15 Likes

Hi everyone, another ECC update on the work we’re doing. Tl;dr is we’ve identified and fixed the last known bugs that were blocking the first phase of SDK releases, we’re continuing to make good progress on DAGsync, and we’re prepping both zcashd and mobile wallets for ZIP 317.

Update:

  • Squashed the last known librustzcash bugs that were blocking wallet release and completed Android & IOS SDK integration.
  • Made good progress on the groundwork for DAGSync in the mobile SDKs.
  • Completed the implementation of zcashd optimizations expected to save memory and to reduce orphan rates for miners. We’re testing now and will included the updates in zcashd release 5.3.1.
  • Working on abstractions to support ZIP 317 (Proportional Transfer Fee Mechanism) in the mobile wallets.
  • Nearing completion on making zcashd have a single way of constructing transactions, rather than 4 different approaches. This will also support ZIP 317 changes.
  • Continuing to make progress on recursion for Halo 2, a fundamental building block for future scalability solutions

Thanks again for your time and patience during this time. If there are any questions, please post them below and our team will answer them. You can also reach out to me directly.

Ian

14 Likes

Exciting stuff!

What does the ECC team think about putting a EIP-1559 style non-discretionary, algorithmic fee mechanism in place as one of the next top priorities? In my mind, there’s some big things to work on - retiring old pools, posterity fund, POS, fee mechanisms. It seems like 1559/posterity fund (or perhaps Sustainability Fund since posterity is a fancier word less people might recognize and brings up images of “old” or “aged” while sustainability brings up images of “green” or “solar punk” in my mind :wink:) might be good to do in one fell swoop. The earlier we institute a posterity fund, the more ZEC will start accumulating in it for rainy days, and the also the sooner a future proof, non-discretionary fee mechanism is put in place, the less developers would have to later tinker with different fee mechanisms, sooner we could get an organic revenue source for posterity fund, etc.

As an aside, it would be very interesting to do a model of how much txn fees there would be under a 1559 style thing given current and past network loads - similar to how a comparison was done with current fees versus 317.

I remember @Souptacular said there was a certain economist who did some great modeling for 1559 for ETH - what was their name Hudson?

3 Likes