Zephyr report March’22
In the last few months (Dec’21, Jan, Feb) our team had been working on adopting wallet core of Aditya’с fantastic Zecwallet Lite for usage with the Zephyr wallet. This task turned out to be very different from the work that we have done for the earlier milestone in 2021. This time we had to shift our focus from high-level tasks like:
- produce UI designs and high-fidelity wireframes for the Zephyr wallet
- create PoC that a Zcash wallet can live within the sandbox of Chrome/Firefox extension
to low-level stuff:
- Rust
- C-libraries
- Wasm bindings
- gRPC / RPC protocols
- protobuf & formats
- async runtimes
- Transport layer with Envoy <> lightwalletd
In 2022 our team switched from JavaScript and discussing UI/UX, to the tasks related to low-level code. This new journey turned out to be full of technical challenges, some of which were fun and some of which hair pulling moments. But we’re very excited about our humble advances and very happy to be part of this technological adventure of making Zcash browser wallet.
Our last year’s status report was very visual (as it contained a lot of UI screenshots). This status report instead of much more technical. Thus in order to explain what we worked on, I am required to provide a high-level view of the crypto wallet architecture.
CLI Wallet
We can start our architectural refresher with the simplest crypto wallet there is - a CLI wallet. It takes input directly from user via command line arguments
A:\> wallet send -to t123456 -value 42zec
The CLI wallet is compiled to native code (x86 instructions) and has full access to the Operating System and underlying primitives (eg. network sockets).
Here’s a diagram which represents CLI wallet architecture at glance.
- user interacts with CLI wallet via command line: issuing a Rescan command
wallet rescan - CLI commands are eventually handled by the module called
LightClientwhich encapsulates most of the operational logic - As
LightClientneeds to communicate with thelightwalletdserver, it goes through number of layers:- RPC module which is based on Tokio runtime
- Operating System which provides network sockets for communication
- The last (orange) step of the diagram is TCP/IP connection to the
lightwalledserver to do RPC overlightwalletdprotocol
Electron based Desktop wallet
A more user friendly wallet can be created using Electron framework. Electron based Desktop applications can use JavaScript/HTML to code User Interface. However thanks to the Electron “Magic” any Electron app still has full access to the Operating System. This is possible because Electron contains (wraps around) fully functional NodeJS runtime. This means that all the OS primitives (eg. network sockets) are available via NodeJS or NAPI.
Having access to the operating system means that architectural diagram of an Electron based wallet would be conceptually similar to the architectural diagram of a CLI app. There will be few small changes though:
- There can be a rich interactive UI
- UI logic is written using JS/HTML
- UI code can communicate with NodeJS runtime via Electron built in mechanisms
- NodeJS runtime can execute native code (eg. C-libraries or Rust-libraries) via Node’s NAPI (the green rectangle)
You can spot some differences in the upper part of the diagram, but if you look at the bottom parts of the diagram, you will see that everything from zecwalletlite-lib, “Operating System” and “lightwalletd” components are presented in exactly same way as on the CLI App diagram and are using native code (eg. x86).
Web based wallet
The architecture of a web-wallet has very significant differences vs CLI or Desktop wallets. Zephyr, which is a wallet delivered to the user as a Chrome/Firefox extension is a subset of a web wallet.
So let’s have a look at the architecture diagram to and spot those significant differences.
The biggest changes are represented by the two large red rectangles. They represent “bridges” (between JS and WASM world) which are sandwiching zecwalletlite-lib code.
You can notice that this diagram does not have anymore any native code blocks (all “x86” icons are replaced by “JS” or “WA”-WebAssembly icons). All the code is either JS or pure WASM.
There is no access to the underlying Operating System anymore. And that means that there is no access to networking primitives like sockets neither.
Without access to the OS and sockets we would need to find an alternative way to perform RPC calls, as RPC and networking mechanisms from CLI Wallet or Electron Desktop App Wallet do not apply anymore. Our web-stack can only perform HTTP requests via fetch()/XHR browser API. As our host is browser, not OS.
To bridge HTTP requests and the lightwalletd server and protocol, an Envoy proxy is introduced. Also our wallet-core, executed as WASM module is has to use wasm-->js bridge
to ask browser to perform RPC calls destined at lightwalletd. This results in slightly cumbersome multi-step architecture with lots of intermediaries and moving parts. And that’s where most of our work and research effort was put in the last few months.
Progress report
Now that you have had a chance to get introduced to the architecture of the wallet, let’s have a glimpse into some of the technical things we’ve been working on.
Our main focus was adopting zecwallet-cli/lib code to work in WASM environment (as per diagram #3).
The fantastic codebase of Aditya’s from zecwallet-lite#wasm (https://github.com/adityapk00/zecwallet-lite/tree/wasm) was used as a reference of the changes required to be ported to zecwallet-cli/lib .
However the latest versions of zecwallet-cli/lib have moved forward significantly since the #wasm branch (eg. Aditya added Blaze sync implementation).
Our intention was to be able to maintain a fork of zecwallet-cli/lib code which is WASM compilable and can accept upstream changes from Aditya’s commits.
The challenges and tasks we had to tackle:
- bridge the core wallet code (wasm-core) with the gRPC within JavaScript host
- implementation of async operations within wasm code and brining in alternative async runtime into our code (wasm-compatible instead of Tokio)
- sorting out usage of c-library within the zecwallet-lib core (when dealing with c-library secp256k1, we tried making it work as WASM module using emscripten, but eventually realized that Aditya’s approach from 2021 of using pure-Rust implementation of libsecp256k1 is a much simpler and cleaner)
- Preparing internal documentation and research notes re: gRPC, protobuf, wasm<>js interop, Envoy RPC, etc
- Fixing issues with Envoy proxy. (there were subtle differences in gRPC generated code produced by different versions of
protobuf-codegen); fixing timing, handling empty payloads, CORS
Future roadmap
In order to finish our current milestone, in the next few weeks we will continue working on:
- removing Rust’s Tokio dependency from wallet code (Tokio async runtime does not work in WASM)
- porting Blaze sync code for WASM (mostly removing Tokio runtime dependencies)
- wasm—>js bridge related work (handling gRPC calls)
- debugging RPC calls and Envoy Proxy interaction
