Zcash UniFFI library addenda (resubmission)

Introduction

This is a resubmission from this forum post.

In the previous grant proposal responding to this RFP, we proposed to add a layer to Librustzcash that translates the main Zcash library methods, written in Rust, into four other different languages by utilising Mozilla’s UniFFI library. That work has now been finished, and the end result of having Librustzcash be usable in other languages has been accomplished. This avoids the need for double work from developers working with Ruby, Python, Swift, or Kotlin.

Most of the core functionalities needed to work with a small light client or a wallet were exposed (assuming features like synchronisation and broadcasting are already present). A solid amount of documentation is publicly available on Github in form of READMEs and wikis.

In this grant proposal, we want to take our efforts with UniFFI two steps further, by:

  1. increasing the integration between the UniFFI-supported library and the original library,
  2. creating a demonstration codebase showcasing how to use the library on Android and iOS

The librustzcash codebase is an evergrowing part of the Zcash ecosystem, and is destined to grow with it. Similarly, the uniffi-supported version of it will need to grow in tandem. To anticipate this growth, we will work on making our library as easy to upgrade and maintain as possible. We should aim at a low maintenance and development burden for the future maintainer of the, as well as making sure the maintenance work does not block or hinder the core librustzcash development and workflow. This will be very intricate, coordination-heavy work, but will save a lot of time from the future maintainers of both libraries.

Code demonstrations and tutorials are often considered purely optional when it comes to a libraries implementation, often being replaced by things like thorough static analysis and automated testing. While those are also critically important, when it comes time for developers to use the codebases, testing and static analysis are often inadequate ways to learn. This is why in the original project we included tutorials for a developer of any of the UniFFI languages to start utilising the library for their project.

Now, we would like to go a step further and include an extensive demo codebase of the most used functionalities for each major mobile ecosystem. This would allow a developer to have a “quick start” repository that they can clone and get things up and running for a Zcash-based project in just a matter of minutes.

Project roadmap and milestones

We will split the process into two milestones of equal value. Each milestone will be submitted consecutively after completion of the previous one.

Milestone 1: Librustzcash library integration

To start the integration work, we need to collaborate with the librustzcash library developers to discuss the best way of conducting a deeper integration of the UniFFI library with the main library. Some parts of this milestone may be done parallel to other tasks. Some approaches we will test are:

  • CI-based: Setup the CI to react by keyword, and implement a soft block on pull requests, i.e. that warns as a standalone on the pre-push checks, warning that a feature was implemented but did not include its FFI version.
  • Rust-based: implement some kind of “directive” or “trigger” during compilation that warns when a feature on one side changed and the other did not.
  • Manual intervention: the developer has to check whether this breaks the UniFFI part on a case-by-case basis, so we should implement a tool to aid this process.

Milestone 2: UniFFI demo codebase

To demonstrate the usage of the library created in our previous endeavour, we will provide sample applications in form of codebase for both Android and iOS platforms, written in Kotlin and Swift, respectively, and with minimal UI. The demo app will have the ability to:

  • Show balance at an address
  • Build a transaction
  • Explore a transaction, searching by the transaction id
  • Broadcast a transaction after connecting to a node
  • Encode and decode addresses

This codebase will provide multiple useful additions to the project such as:

  • help with debugging eventual issues a developer might have on their app.
  • our examples will show some exemplary values for encodings/decodings, so the developer will have a reference to “correct” constants (e.g. encoded addresses, transaction bytecode generation) for their testing.
  • it will show some common design patterns that the developer would have to find otherwise on some previous app, i.e. it will show some common use cases for the library.
  • it will be retroactively useful to see whether our app is truly usable - if we see that some elements of the API do not work well as design, we may change that.

It is important to remind the reader, at this step, that since Librustzcash does not include synchronisation capabilities, the UniFFI library cannot implement such features either. Thus to write, for instance, a complete wallet interacting with the mainnet and broadcasting a transaction, there has to be an additional component present taking care of the synchronisation part.

Resumes

Development team

  • Eloy López Peñamaría (Github) is a Cloud Software Engineer with extensive experience in Rust and Go, focusing on observability and resilience, with more than ten years of building backends for cloud environments. He likes to collaborate in blockchain-related projects and distributed high-traffic systems. Always attentive to details and documentation. He is the main member of the team who built the Zcash UniFFI library.
  • Tomasz Piotrowski (GitHub, LinkedIn) is a Rust Engineer at Eiger with extensive software development experience. Since joining Eiger he has worked exclusively with Zcashd and Zebra codebases and supported Eiger’s efforts in the Zcash ecosystem.

Advisory team

  • Luca Campobasso (GitHub, LinkedIn) is a Software Engineer at Eiger. He’s been working on research and development for several projects within the company helping in identifying new research opportunities and taking those ideas to full fledged implementations (e.g. the Zcash UniFFI library). He spends a good amount of his time reading the Zcash protocol, the details of the cryptographic operations and primitives involved, the ZIPs implemented so far and the proposed ones. Additionally, he has experience writing academic research documents.

About us

The promise of web3 is to upgrade the very foundations of our society – from money, finance, and governance to media, gaming, and science. To deliver on that promise, decentralised technologies are to be integrated into the everyday experiences of billions of people. For engineering, this is a mountain of a challenge.

Eiger was founded to develop infrastructure for web3 mass adoption. We help technology companies improve and integrate the core technologies of web3 to meet the climbing demands for scale and performance. We currently employ 30+ senior web3 engineers across the globe and work with some of the most ambitious organisations in the industry, including Forte, Aleo, and XRP Labs, to name a few. Eiger is part of Equilibrium Group, a blockchain powerhouse founded in 2018, composed of three entities:

  • Eiger – high-value add engineering services
  • Equilibrium Labs – applied research, proprietary products, and investing in early-stage ventures
  • Membrane Finance – infrastructure and services for EUROe, a euro-backed stablecoin

Budgeting

Total Funds Requested: $96 000

  • Milestone 1: Librustzcash integration - 1 month, 2 engineers (8 FTEs)
  • Milestone 2: UniFFI demo codebase - 1 month, 2 engineers (8 FTEs)

Total FTEs: 16 - One FTE is 40 hours.

The standard grants-work rate at Eiger for an engineer is 150 $/hr.

The above quantities result in a total budget ask of

150 $/hour x 40 hours x 16 = $ 96 000

5 Likes

What was changed since the last submission?

Hey Hanh, the actual work scope has mostly been the same as before, and the focus has been on working on the feedback we received from the ZCG. We believe that the proposal should now better represent the value we want to add to the original grant, and clear up some earlier misunderstandings.

I looked at the library and the docs. It’s quite informative and clear. Looking good! However, I still think that without synchronization a user will not be able to make a working app. I remember you suggest using an external source for the wallet data but in my opinion, it doesn’t represent a practical situation.
If I was looking for sample code, I think your existing documentation works fine.

3 Likes

Building on what @hanh said, is there any chance this project could also provide a unified (pun intended) synchronization library? I think we’re headed towards DAGSync becoming the optimal syncing algorithm for wallets, at least for the time being with the current protocol, and something like it is necessary for building basic Zcash functionality.

We currently have many syncing implementations spread across multiple community projects, and it’s a security-critical piece. As I’ve said previously, I think in my time working with Zcash I’ve audited 5+ independent syncing implementations. I’d like to hear from wallet devs (including ECC) what it would take for us all to start using a single implementation of syncing. If we can figure out how to do that, we get some security wins (much less code to audit), and future performance improvements can be pushed out to all wallets.

I personally wouldn’t block accepting this grant on that, I think it’s valuable in its own right, but I would fully support any effort that unifies syncing implementations across all the projects in our ecosystem.

4 Likes

Totally agree! A unified synching implementation would simplify development and reduce the amount of code to audit.

1 Like

@MeerKatDev, At the most recent meeting, the @ZcashGrants Committee voted to approve this grant. Please provide updates after completion of every milestone. Also, the grants committee has asked for a video demo upon completion of the second milestone.

4 Likes

Hello everyone - we have some great news.

At the Zcon4 we had the opportunity to consult with the ECC developers regarding the next steps of this project, and reached the conclusion that there are enough functionalities in librustzcash for us to now go forward with including sync capabilities in the UniFFI library.

Apologies for the delays and confusion in this matter; we wanted to make sure the currently existing synchronization features were working properly and were valuable enough to be translated with UniFFI.

We would like to add a third milestone to our grant above:

Milestone 3: UniFFI synchronization

In order to interact with the blockchain, we might need to know what is the last block and keep track of all notes, and to do this we need to include synchronization capabilities to the UniFFI library.

In order to showcase the integration, we will integrate this code into the second milestone which, in addition to all the features listed, will be able to read the chain and show real-time information about last block height and address status.

We will also look into broadcasting a transaction without the need of connecting to an external node, as previewed for Milestone 2.

We will use the Zcash Android SDK (and its iOS version) as reference for features needed for synchronization, trying to parallel their features.

Budget Update

Additional Funds Requested

  • Milestone 3: UniFFI synchronization - 1 month, 2 engineers (8 FTEs)

Total FTEs: 8 - One FTE is 40 hours.

The standard grants-work rate at Eiger for an engineer is 150 $/hr.

The above quantities result in an (added) total budget ask of

150 $/hour x 40 hours x 8 = $ 48 000

3 Likes

@MeerKatDev & Zcash Community, I am happy to relay that the @ZcashGrants Committee has voted to approve the UnniFFI Library Addenda resubmission proposal at the most recent meeting.

3 Likes

Hello,
the first milestone is available for review, on the repository.
We also produced a video of the tool in action.

5 Likes

Hi, just a quick question. Are there projects that are using this library at this time? That would help understand the usage better.

Hi @refinance! Thank you for asking, and sorry for the delay. So, at this time, there is no central repository of people using the library. I guess you could use Github metrics to track usage. A reason for this is that at this point in time the library misses some fundamental features. The endgame of this grant project is to make the library complete enough feature-wise for a wallet to be built without additional native tools.

2 Likes

Hello,

the last two milestones are ready for review. We decided to submit them together as Milestone 3 was necessary to complete Milestone 2. The relevant repositories are below:

As requested by the ZCG

we produced two videos showcasing the action of both demo apps from Milestone 2:

Milestone 3 added many more methods to the library, and improved on some features. A complete changelog is not available, but it’s safe to say that the vast majority if not all librustzcash structs and functions commonly used were translated. All methods needed for creating and operating a Zcash wallet are included. Some new methods, not present in librustzcash, were added for convenience.

A how-to for the methods is given in form of documentation, extended wiki and tests on the main repository, as well as comments and code on the demo mobile apps.

4 Likes