Zebra 1.0.0 Stable Release

[Dodger]

We’ve just announced the first stable release of Zebra.

Zebra has been audited by NCC Group, who found no major issues (and only one medium issue), and it currently has the following functionality:

• it joins the Zcash peer-to-peer network, validates the blockchain, maintains a mempool, and gossips new transactions and blocks
• it can act as a back-end to lightwalletd
• it can generate block templates for miners

I invite everyone who reads this to download and run Zebra, and let us know how it performs. As with any major software engineering project (the core Zebra repository contains nearly 100,000 lines of Rust code, with thousands more in dependencies), there will inevitably be bugs and issues so please report them on Github.

I want to give a huge shout out to everyone who has contributed to Zebra’s progress to date:

• the ZF team (including former team members), for whom this has been a long, difficult journey to reach this milestone
• the ECC team, who have provided much-needed technical help, support and advice, as well as moral support
• the many community members who have downloaded and tested Zebra throughout its development, reported bugs and issues, and contributed to the Zebra codebase.

It’s also important to acknowledge that Zebra relies on a bunch of libraries developed and maintained by ECC.

I also want to thank the Zcash community for their support, and in particular for their decision to fund ZF through the Dev Fund. Without a reliable and stable source of funding, it would not have been possible for ZF to create Zebra.

While this is a major milestone for Zebra (and ZF), there is more to do before Zebra can fulfill all the use cases that the zcashd node is currently used for.

In the short term, our next priority is to upgrade Zebra to support the changes that ECC are adding to zcashd and lightwalletd to allow light wallets to implement the DAGsync algorithm.

Beyond that, while we have our own ideas about what functionality we could add to Zebra next, we want to hear from the Zcash community—especially people who currently run a zcashd node—so please complete this poll, and tell us below what functionality you think we should add to Zebra next!

• I do not run a Zcash node
• I run a Zcash node, and Zebra fulfills my use case
• I run a Zcash node, and Zebra does not fulfill my use case (please post below to explain why not)

[dconnolly]

Super proud to have our 1.0.0 release out in the world

[ebfull]

Great job on the release!

I’m looking forward to contributing to Zebra development.

[aiyadt]

Congratulations to the Zcash Foundation on the successful stable release of Zebra 1.0.0
It’s truly a remarkable achievement by the ZF team of engineers to have developed Zebra from the ground up.

I have been running and exploring Zebra release candidates over the past year and reviewing its light resource usage when deployed on cloud-based infrastructure. Not only does this bode well for efficiency and performance, but it also means significant savings on infrastructure costs.

We at Nighthawk Apps are thrilled to hear about this development, as it aligns perfectly with our roadmap. We have planned for one mainnet lightwalletd node to be powered by Zebra, and we eagerly anticipate making it public. The robustness and reliability of Zebra will undoubtedly contribute to a seamless and secure experience for our users.

[earthrise]

Congrats!!

Are there plans to publish the audit report? It sounds like it would be a fun read!

[lexaleth]

Excellent performance. I can’t imagine all the journey to achieve this achievement, but it sure was worth all the effort

[ZecProphet]

Awesome news! Zcash devs rock!

[Dodger]

@earthrise

Are there plans to publish the audit report? It sounds like it would be a fun read!

Definitely yes. We’re just awaiting final review and sign-off.

[nathan-at-least]

Huge congrats!

Zebrad engineering is top-notch and we’re extremely fortunate to have that team in Zcash!

The step from one node implementation to >1 is a huge boon to Zcash decentralization and resilience. This is a major milestone for Zcash easily on par with major network upgrades.

I’m elated. <3

[kayabaNerve]

Congratulations on the accomplishment! I understand how much work this was, and how much effort went into it.

Are comparative performance of the following available?

1. Time to sync the full chain, without partial sig/proof verification/any checkpointing
2. Time to intake a new block
3. Peak mem usage during both 1 and 2

I also look forward to the audit being released.

[dismad]

Rough snapshot: I’m about a day into syncing (main drive an ssd)

[Autotunafish]

Yep, 1.7m in 1 day and then a few days leading up to just past 1.9m and then it’ll pick up more speed coming out the spam. It does have 8 logic threads so that’ll definitely help over having just 6 or 4

[dismad]

[miodragpop]

When benchmarking zebrad against zcashd, keep in mind that zebra doesn’t maintain local wallet db (someone correct me if I’m wrong). Fair comparison would be zebrad vs. zcashd built with --disable-wallet configure flag. Anyway, I expect “C++ hellscape” zcashd to win in both resource usage and speed.

[zkcroc]

Is there a decent guide for a non technical person to follow to set up a node?

[Autotunafish]

Let me know if you have any more questions

https://zebra.zfnd.org/user/requirements.html

Zebrad for Linux Tutorial #1

[james_katz]

Specs:

CPU: AMD Ryzen 3 3200G
RAM: 16gb
OS: Linux Mint 21 vanessa 5.15.0-41-generic x86_64

Roughly 14 to 15 hours on and off (12 hours the first day, ~3 hours the second day) got to 82% (block 1753639)

Unfortunately I had to stop the sync because I realized I don’t have enough disk space left

I’ll buy a new ssd from china and will try again.

My experience so far:

• It’s quite CPU intensive.
• No crashes.
• UI is not that friendly, but I was informed there’s a progress-bar option that makes the UI prettier.
• Really easy to install (I installed with cargo install --locked --git https://github.com/ZcashFoundation/zebra --tag v1.0.0 zebrad command)

[teor]

@kayabaNerve

Congratulations on the accomplishment! I understand how much work this was, and how much effort went into it.

Are comparative performance of the following available?

Good question! We haven’t done a lot of optimisation on Zebra yet, or set up many performance tests. But I’ll link to what we have so far.

@kayabaNerve

1. Time to sync the full chain, without partial sig/proof verification/any checkpointing

Zebra takes about 2-4 days to fully sync the chain, depending on CPU and network latency. Here’s one of our weekly scheduled full sync tests which took 2 days and 4 hours:

2023-06-18T15:22:32.445047Z  INFO {net="Main"}: zebrad::components::sync::progress: finished initial sync to chain tip, using gossiped blocks sync_percent=100.000% current_height=Height(2125997) network_upgrade=Nu5 remaining_sync_blocks=1 time_since_last_state_block=0s
test full_sync_mainnet ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 40 filtered out; finished in 184268.60s

https://github.com/ZcashFoundation/zebra/actions/runs/5289726397/jobs/9600566501#step:8:20240

We don’t measure memory usage during this sync test yet.

Network upgrade checkpoints are required by the Zcash specification. Zebra implements additional checkpoints for security, correctness, and performance.

Zebra requires a checkpoint shortly after Canopy, because it lets us skip implementing some legacy consensus rules.

Using checkpoints for older blocks is more secure than full verification, because transaction data is verified using block header commitments, and then block hashes are verified against the social consensus chain. (The chain we all roughly agree on.)

Is there a specific reason you want to turn off checkpoints? Or is it just to compare performance using the same operations?

@kayabaNerve

2. Time to intake a new block
3. Peak mem usage during both 1 and 2

It’s tricky to measure the download and verification times for new blocks, because Zebra does most block verification in parallel. This includes batch verification of most proofs and signatures.

Here’s my local instance verifying 3 blocks within about 600ms.
(I don’t have the times when the download or verification was started, Zebra doesn’t log them.)

2023-06-19T01:07:23.300274Z INFO {zebrad="23cb29a" net="Main"}:{peer=Out("v4redacted:8233")}:msg_as_req{msg="inv"}:inbound:download_and_verify{hash=0000000000de8cbc262e4bd4adf0bc1252bf0439b24f34022d3dcb07ac010c8d}: zebrad::components::inbound::downloads: downloaded and verified gossiped block height=Height(2126465)
2023-06-19T01:07:23.584663Z INFO {zebrad="23cb29a" net="Main"}:{peer=Out("v4redacted:8836")}:msg_as_req{msg="inv"}:inbound:download_and_verify{hash=0000000001c5a640d4e2e7ed0bbe41cf2bdd63c7475bf810883f668e318cead8}: zebrad::components::inbound::downloads: downloaded and verified gossiped block height=Height(2126466)
2023-06-19T01:07:23.934685Z INFO {zebrad="23cb29a" net="Main"}:{peer=Out("v4redacted:8233")}:msg_as_req{msg="inv"}:inbound:download_and_verify{hash=00000000012ab71e7292bb0844439a6ccf44f36b6568463a890f7a67bc405ff1}: zebrad::components::inbound::downloads: downloaded and verified gossiped block height=Height(2126467)

In my synced instance, I see memory usage around 10 GB. But my machine has a lot of memory, so Zebra’s database likely has a very large memory cache. On a machine with less memory, it should be around 4-8 GB.

I’m also running Zebra with prometheus metrics, which takes extra memory.

[teor]

Also, Zebra takes about 6-7 hours to fully sync lightwalletd:

{"app":"lightwalletd","level":"info","msg":"Waiting for block: 2126326","time":"2023-06-18T22:23:46Z"}
2023-06-18T22:24:38.273710Z  INFO {net="Main"}: zebrad::components::sync::progress: finished initial sync to chain tip, using gossiped blocks sync_percent=100.000% current_height=Height(2126325) network_upgrade=Nu5 remaining_sync_blocks=0 time_since_last_state_block=0s
test lightwalletd_full_sync ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 40 filtered out; finished in 24364.56s

https://github.com/ZcashFoundation/zebra/actions/runs/5289726397/jobs/9602805363#step:8:9617

[Dodger]

@Dodger

Beyond that, while we have our own ideas about what functionality we could add to Zebra next , we want to hear from the Zcash community—especially people who currently run a zcashd node—so please complete this poll, and tell us below what functionality you think we should add to Zebra next!

I want to bump this request for feedback, ideas or suggestions for what functionality should we add to Zebra next?