Application for Retroactive Funding for Pepper-Sync (second attempt)

Zingo Labs produced pepper-sync (details in the proposal), we’re asking for retroactive funding for this Zcash enhancing, previously unpaid work.

Please see our proposal here!

I look forward to conversations with the community. I hope to get feedback from @peacemonger @hanh @NealJayu @decentralistdan @thowar2 and @outgoing.doze early on, as they have offered particularly useful criticisms in the past, and I very much appreciate their insights.

Thanks for your thoughtful attention!

13 Likes

Aye. Can’t promise that I will participate to that vote, but if I do that’s what I’ll do.

1 Like

I am trying to use pepper-sync in zingo-cli.
when Re-org is detected, panic happens:

INFO pepper_sync::sync: Re-org detected.

thread ‘tokio-runtime-worker’ panicked at /usr/src/zingolib/pepper-sync/src/sync.rs:1284:10:
should be non-empty in this scope

I would love to use zingo-cli with pepper-sync as a server-side wallet, it fits my use cases very well, but these intermittent panics prevent me from deploying it in production.

I am willing to spend time helping diagnose and fix the issue. Would you be able to prioritize investigating this, or point me to the best place to coordinate on a fix? I have not had much success filing standard GitHub issues to report similar problems. Thank you

5 Likes

We’ll look into this right away!

3 Likes

There are 3 parts to this grant:

  1. Documentation, tutorial, training, video, outreach: This is Zingo Labs En Español and is covered by the ZCG community grant, isn’t it?
  2. Zingo-pc & Zingo-mobile: that’s maintenance of the Zingo app (except for the integration of pepper sync)
  3. And, pepper sync itself.

Pepper sync is only used by your own applications, ie, it is not part of any independent application. I actually don’t see why you didn’t choose an existing synchronization method and built your own, as it turns out to be slower. Is it for diversification?

No, ZingoLabs en Español (ZLE) has never received a grant from ZCG.

1 Like

As mentioned here the ZExCavator work we were asking to be retroactively funded, which is subsequent to the ZeWIF covered work is in progress but not complete, and is therefore not eligible for retroactive funding.

I will edit the ask to remove the reference, and leave this message by way of explanation.

Our keysweeper work is in progress, and it was our intention to complete it in time for this retroactive funding application, but we reallocated our effort to support a Cross Link prototype mobile application first. The grant application should have been updated to track the shift in priorities and completed work.

1 Like

Hi all I wanted to point out the @ZecHub is currently testing :right_arrow_curving_down:

which uses Zingo’s Zingolib for the backend. We have been working with the Zingo devs and support has been amazing! Here is our open sourced code which we invite everyone to disect and find issues with – this platform is for the community so dont hold back :grin:

Client view

Admin view

Try it out!

We invite everyone to test with their testnet addresses and let us know about any issues/concerns/bugs! We recommend zingo, zkool, or zcash-devtool to generate testnet addresses!

Here is how they look:

13 Likes

this is great! I hope we can take this to production. It really solves a problem we’ve been having where there’s no incentivized on-ramp for contributors that helps teams to get contributions that are a priority for their roadmap which can’t be left to good will of pro-bono contributors but that aren’t substantial enough for a grant of their own.

@ZCG :eyes:

9 Likes

The Zingo team developed a mobile wallet for our Crosslink testnet, which we’ve used in multiple workshops over the past several months. The wallet uses Pepper Sync and allows participants to interact with the network and see how things work in practice, including staking, unstaking, and redelegation.

They did a terrific job on the wallet’s design and user experience. It’s intuitive, easy to set up, syncs reliably, and works well in a workshop environment where people need to get up and running quickly. I’ve received positive feedback from attendees on the overall experience.

The team has been easy to collaborate with, and we value the work they’ve done to support our workshops.

11 Likes

Nice one. During Setup Required, all attempted zcash addresses kept failing with error Please enter a valid Zcash shielded address.

I’m not sure which setup you are referring to exactly but, given the context, it sounds like the wallet network was not configured properly i.e. its still on a mainnet server and/or account.

I have tried all the addressses all fail with the same error

”Please enter a valid address”

You can paste the address into a block explorer (or another wallet) to verify that the address you’re pasting is a valid address. If so, the explorer will show you the receivers. https://mainnet.zcashexplorer.app/

If the website enforces a ‘z’ address i.e. Sapling and your ua doesn’t have one, then perhaps it could give this error. In which case, you just get another wallet easy peasy.

Please use testnet addresses right now! Here is quick demo on one way to get one:

You can also you the latest Zingo! :right_arrow_curving_down:

2 Likes

My experience with switching nets on zingo is that after you switch, you need to create a new account that starts in testnet mode yes.

Ok, as promised, my full review of the Pepper-Sync grant.

Due to the amount requested ($461,000.00) a higher bar of diligence is required.

The grant summary states:

In December of 2024, slow sync was a critical usability issue blocking Zcash adoption.

As described below, and posted on the Zcash Forums Zingo Labs produced a novel sync implementation that’s competitive with the fastest alternatives in raw performance, and was the fastest spend-before-sync implementation in production when it was completed.

In addition to the direct benefit to the community provided by a new sync implementation, there has been a salutary effect on other sync providers.

A few problems here:

  • All Zcash wallets are still slow to sync. The problem remains unsolved for the ecosystem.
  • Zingo has produced a somewhat faster syncing method, which has not fully solved the problem as syncing is still slow
  • Zingo claims competition increased due to the presence of pepper-sync. However syncing is still slow the the problem has been largely sidestepped via optimistic UI.
  • Syncing has not blocked this round of adoption. Syncing and scale in general remains a problem for Zcash, and there are other proposals in the works to solve this altogether.

So, I find it hard to accept the claim that Pepper-Sync gets to claim much, if any, credit towards solving an unsolved problem or increasing user adoption.


Justification: 2 full-time developers for 15 months at $96/hour.

I find it hard to believe that 2 full time senior engineers spent 15 months on a syncing library alone. If that is the case, I would consider it a cost overrun. I think its far more likely that much of that “full time” effort was spent on other projects at Zingo Labs.


Proof of traction

The algorithm is already deployed in four Zingo Labs applications:
zingo-cli , zingo-pc , zexcavator , and zingo-mobile .

Important to note that the grant is not for the above libraries, but that pepper-sync is used in those libraries. Let me check that.

Product Notes Result
zexcavator On MacOS Installs and runs on system with Rust already installed. (not user friendly for normal users that don’t have rust libraries). Both 12 and 24 word mnemonics run in to “Sync Error” and go into a restart cycle. Syncing progress is incredibly slow, probably 100x slower than most Zcash apps. Not sure that pepper-sync is solving anything here. Fail
zingo-cli Wallet loaded but UI was not clear if it was syncing or not. Could not find a way to display sync status. Transaction history empty which means sync was either unsuccessful or incomplete. Fail
zingo-pc On MacOS, the app itself is very sluggish and difficult to use. If pepper-sync is faster I wouldnt notice because I spend most of my time waiting for basic UI items to load on the screen. Pass (slower than benchmark)
zingo-mobile UI worked quickly vs. desktop. Syncing took several minutues. Pass (slower than benchmark)
Zkool As a test, I loaded the same wallet I used on the zingo products into Zkool with the same birthday height of 2905000. Zkool synced history within 30s. All the Zingo products took several minutes or did not complete during my review session. Benchmark

Non-Zingo Users

Summary: Some to the zingo products did not work. The ones that were functional were slower at both syncing and general usage than other Zcash wallets. Based on this, am not sure that pepper-sync has added any additional utility to Zingo users or Zcash. 3rd party users did not mention pepper-sync as a useful feature.


Conflict of Interest Disclosure

Proposer left this empty.

As I have recently discovered, there is a web of familial and economic interests surrounding the applicant which I believe should be fully disclosed in this application. A non-exclusive list includes: previous dev reward recipient, shareholdings or debts related to previous dev reward or grant recipients, familial ties to previous dev reward recipients.

Additionally the Zingo Lab suite of products seems to be receiving multiple grants from multiple sources for multiple products, which makes it difficult to track any “double reimbursements”.

Given the above, I am of the opinion that Zingo Labs should disclose extensive documentation around the various grants received and for what. As well all other conflicts of interest, while not disqualifying, should be disclosed. For me, lack of disclosure is on it’s own, disqualifying.


Overall Summary

The proposal is $461,000.00 for the pepper-sync code, and nothing else. I find the library itself has not demonstrated a large value add to the ecosystem. I find the claimed cost (2x senior engineers for 15 months) to be an exaggeration or wasteful. As a voter, I do not think this proposal is a net benefit to the Zcash ecosystem, so I would vote AGAINST funding.

Hi @thowar2 I appreciate your diligence, and clear language.

The community benefits from skeptical inquiry and robust argument.

We’re working on a nuanced point-by-point response.

Really really really really really excited about the keysweeper functionality and eager to see that work get funded! :face_holding_back_tears:

4 Likes

Hi @thowar2 you mentioned this:

I left this blank because I am not aware of any conflict of interest. It seems like you are? Please explain your concern so we can better understand it.

Per the fact that I am Zooko’s brother please see this disclosure from 4 years ago which accurately predicted that I would still be his brother moving forward.

Corollary: Zooko’s mother is my mother.

There is absolutely no conflict of interest in being Zooko’s brother and applying for a retroactive grant.

Since I believe that Zcash is beneficial, I actively encourage my family to be involved with it I predict this will also continue.

8 Likes