### Terms and Conditions
- [X] I agree to the [Grant Agreement](https://9ba4718…c-5c73-47c3-a024-4fc4e5278803.usrfiles.com/ugd/9ba471_f81ef4e4b5f040038350270590eb2e42.pdf) terms if funded
- [X] I agree to [Provide KYC information](https://9ba4718c-5c73-47c3-a024-4fc4e5278803.usrfiles.com/ugd/9ba471_7d9e73d16b584a61bae92282b208efc4.pdf) if funded above $50,000 USD
- [X] I agree to disclose conflicts of interest
- [X] I agree to adhere to the [Code of Conduct](https://forum.zcashcommunity.com/t/zcg-code-of-conduct/41787) and [Communication Guidelines](https://forum.zcashcommunity.com/t/zcg-communication-guidelines/44284)
- [X] I agree to post request details on the [Community Forum](https://forum.zcashcommunity.com/c/grants/33)
- [X] I understand it is my responsibility to post a link to this issue on the [Zcash Community Forums](https://forum.zcashcommunity.com/c/grants/33) after this application has been submitted so the community can give input. I understand this is required in order for ZCG to discuss and vote on this grant application.
### Application Owners (@Octocat, @Octocat1)
@ChristopherA, @shannona
### Organization Name
Blockchain Commons with Zingo Labs
### How did you learn about Zcash Community Grants
Long familiarity with the blockchain ecosystem, and discussion with some Zcash ecosystem principals.
### Requested Grant Amount (USD)
$121,200
### Category
Wallets
### Project Lead
```project-lead.yaml
Name: Christopher Allen
Role: Chief Architect
Background: Christopher Allen is the coauthor of the TLS and DID standards and is currently working with the IETF to advance dCBOR and the Envelope data format for standardization. Recognized as a “Top 100 Influencer in Identity,” he was Principal Architect at Blockstream, Vice President at Blackphone, and CTO of Certicom. Today, he is the Executive Director of Blockchain Commons. For this project, Christopher is coordinating between Blockchain Commons and Zingo Labs to offer a solution that reflects both wide knowledge of the cryptocurrency ecosystem and specific details of Zcash needs.
Responsibilities: Oversee project & provide technical vision
```
### Additional Team Members
```team-members.yaml
- Name: Wolf McNally
Role: Lead Researcher & Designer
Background: Wolf McNally is Blockchain Commons' lead researcher and engineer, who developed most of Blockchain Commons' specifications, such as Animated QRs, Uniform Resources (URs), and Sharded Secret Key Reconstruction (SSKR), and also designed LifeHash and the upcoming Provenance Mark specification.
Responsibilities: Design of extensible interchange format, engineering of libraries for import and export.
- Name: Darío Paz
Role: Developer
Background: Darío Paz is a software engineer at Zingo Labs who previously founded Mayland Labs, a startup that explored the intersection between design, virtual spaces and digital assets. He has worked on MoonyApp, developing a signal-trading platform, and has over three years of experience working on EVM-based blockchains.
Responsibilities: Survey of wallets, testing of Blockchain Commons format & libraries, design of real-world ZExCavator tool to recover content from old zecwallets.
- Name: Shannon Appelcline
Role: Technical Writer
Background: Shannon Appelcline is an award-winning nonfiction author and technical writer who has written for numerous blockchain and DeFi companies and who has described most of Blockchain Commons' specifications.
Responsibilities: Organization & recording of wallet survey, authorship of various best practices.
```
### Project Summary
To support the deprecation of zcashd, we will create an extensible wallet interchange format for Zcash that supports historic wallet.dat, such as that found in zcashd, and leaves rooms for expansions and improvements in the future.
### Project Description
The goal of the project is to create an extensible interchange format that may be used immediately to extract content from zcashd and other legacy wallets and longer term to allow users to move between different wallets without worrying about losing information or even assets.
This will be done in four phases:
1. Wallet Survey
2. Wallet Interchange Specification Design
3. Rust Import & Export Libraries
4. Legacy Wallet CLI Tool
The specifics of this are described in the Solution Format section, but ultimately the results of this project will be that: wallet developers can easily produce importers, exporters, and sweep tools to allow transition between wallets; and users can even salvage old wallet content themselves using a CLI tool.
### Proposed Problem
The zcashd reference client is being deprecated, and this means that funds must be exported from the client and converted to other wallets. The fact that there are now at least four popular major wallets and numerous lightclient wallets, each with their own formats for metadata about funds and transactions, makes this transition challenging as does the number of Zcash address formats (transparent, sprout, sapling, and orchard) and the proliferation of key generation formats (master keys, seeds for HD keys, and BIP-39 compatible seeds for HD keys).
This reflects a longer standing problem in the Zcash community: any type of migration between wallets is very difficult (see for example https://forum.zcashcommunity.com/t/shifting-from-zeclite-to-zashi/47756). It's been covered by numerous GitHub Issues, including zcash #6873 (https://github.com/zcash/zcash/issues/6873), zips #821 (https://github.com/zcash/zips/issues/821), zips #964 (https://github.com/zcash/zips/issues/964), and librustzcash #1552 (https://github.com/zcash/librustzcash/issues/1552).
The zcashd deprecation therefore offers a major opportunity: we can create an interoperable wallet interchange format that not only serves the main purpose of exporting data from zcashd in a standardized manner, but that also supports the interoperability of the whole Zcash wallet ecosystem going forward, ensuring that users always have their choice of wallet and are never locked into a single provider out of fear of losing past transactions, addresses, seeds, or keys.
This helps to preserve the fairness and openness that are among Zcash's core values.
### Proposed Solution
Blockchain Commons and Zingo Labs will: collect and survey existing wallets (a wide-ranging survey, with a focus on zcashd and zecwallet); design an interoperable and extensible wallet interchange format that can preserve and secure data from zcashd and other wallets including legacy zecwallets; create Rust crates that import data into and export data from that interchange format; and produce a ZExCavator tool that recovers buried ZEC from old zecwallets.
The primary users of this solution will be current and future wallet developers. Zingo Labs has already agreed to work with us and we've also reached out to Andrew Arnott and other members of the Zcash ecosystem. We expect to reach out to all of the other major wallet developers, including ECC (Zashi) and YWallet.
As wallets adopt the extensible wallet interchange format, wallets customers will become users too, and they should see big wins in the ability to make their own choices on which wallets to use.
### Solution Format
This project is laid out in four phases:
1. WALLET SURVEY. Examine current wallets to enumerate all stored information on seeds, keys, addresses, and transactions including labels, notes, unmined transactions, and other wallet-specific data, with highlighting of information that is not legacy and that is stored by at least two different wallets, and therefore the highest priority for inclusion in an extensible wallet format. The goal here is to be thorough in recording all off-chain data stored in wallets (and perhaps even on-chain data that is replicated in wallets).
* Deliverable #1.1: One or more meetings for wallet developers (Month 1).
* Deliverable #1.2: Final report on all surveyed data, which at a minimum includes coverage of zcashd and zecwallet but is expected to cover a wide diversity of wallets, including highlight of data suggested for inclusion in a extensible wallet interchange format (Month 1).
2. WALLET INTERCHANGE SPECIFICATION DESIGN. Create a format that will allow the export of zcashd and zecwallet data as well as data from other major Zcash wallets and future expansions.
* Deliverable #2.1: Interchange design document, describing a format focused on well-known data that is used by two or more major wallets (including zcashd, zecwallet/zingo, zashi, and eZcash). The goal of this document is to provide what's needed to export data from popular wallets and for wallet companies to import any exported data (Month 2).
* Deliverable #2.2: A developer's how-to document describing the use of Envelope attachments for data not included in basic interchange format.
* Deliverable #2.3: An example implentation using the Envelope format for storage of non-standard zcash wallet data, based on user testing of deliverables #2.1 & #2.2 by Zingo Labs (Month 2).
3. RUST IMPORT & EXPORT LIBRARIES. Release an open-source reference importer and exporter Rust library for the new extensible wallet interchange format.
* Deliverable #3.1: A Rust crate to import a wallet into an intermediate in-memory representation, using API inputs (Month 3).
* Deliverable #3.2: A second Rust crate to convert that intermediate representation to the serialized Gordian Envelope format as an export. (Month 3).
* By dividing the import/export library into two crates, we give developers the ability to convert into other serialized formats if they prefer.
* Alongside the two crates, an integration test will ensure the validity of the libraries using a real-world wallet format and will be published as part of the open-source release but isn't intended as a user-facing tool.
* Deliverable #3.3: A Rust abscissa CLI for testing the import of a zecwallet-specific format (Month 3).
* Deliverable #3.4: A best practices document on importing & exporting data.
4. LEGACY WALLET CLI TOOL. Expand the Rust abscissa CLI into a single-purpose tool that extracts legacy wallet.dat files from ZecWallet Fullnode and zcashd via the ZeWiF format, syncs the wallet, and exports the updated wallet via ZeWiF for compatible consumers. This will provide an important community utility, as well as a proof of concept of the advantages offered by the standard ZeWiF format.
* Deliverable #4.1: Full legacy wallet CLI tool (ZExCavator).
The extensible interchange format, the Rust crates, and the ZExCavator wallet tool together comprise the technical heart of this project. The two best practices documents are meant to supplement that.
The phase 2 document, on using Envelope attachments, will lay out which data was incorporated into the core interchange format, which was delegated to attachments, and the general reasons for doing so (as best-practices support for future extensions). This will include discussions on legacy vs. modern formats and how they're supported differently to ensure the interchange format isn't saddled with technical debt from the start. The document will also provide guidance on attachment formatting and usage as well as how to access an archival copy of the original wallet data.
The phase 3 document then talks about best practices for importing and exporting data. This will discuss ways that legacy data can be imported or converted (primarily through sweeps from legacy keys) and also what an importer should do if it is unable or unwilling to import all data.
### Dependencies
This project is already a collaboration between Blockchain Commons and Zingo Labs. We have also begun work with other principals working in the Zcash ecosystem. Still more collaboration will be required to achieve the best success. We feel that it's especially important to get buy-in from ECC because of their central position in maintaining librustzcash and zcashd.
We believe we're on the path to success here, both because we've already begun these discussions and because Blockchain Commons has worked with a variety of wallet developers before in the larger blockchain ecosystem, leading to the successful deployment of formats such as Animated QRs, Uniform Resources, Sharded Secret Key Reconstruction, and Envelope itself.
The other major dependency is making sure that we can properly support the the zcashd deprecation plan. The current goal is for zcashd to close out in April 2025, which is obviously very near. To support this, we'd like to be able to have the format fully available for use and testing by the start of April. To meet that deadline, our initial three months of work would need to be January, February, and March (with the final legacy wallet tool from Zingo Labs to follow in April). An end-of-the-year decision on this project would allow us to fulfill that timeline. Otherwise, things will get pushed back, which would endanger the April deprecation date.
### Technical Approach
This project will take advantage of the extant Gordian Envelope technology, which is a data storage and exchange format intended for sensitive and confidential data, exactly like this. Gordian Envelope (see https://developer.blockchaincommons.com/envelope/) is privacy-focused and supports encryption and elision without destroying signatures. It can even be used over untrusted or unreliable transports, all of which should be to the benefit to wallet interchange (though the best benefits might await a later stage than the more pragmatic need to get wallets input and output, which is the heart of this proposal).
With those technologies the approach should be fairly simple:
* Survey existing wallets and collected wallet.dat files & talk with wallet designers
* Spreadsheet all possible data elements
* Highlight all repeated, non-legacy data (used by at least two wallets)
* Register CBOR tags for highlighted data
* Write specification for Envelope/CBOR-based extensible wallet interchange format
* Write how to incorporate other data into Envelopes as "Envelope Attachments"
* Write Rust crate that supports import of data into intermediate in-memory format.
* Write Rust crate that supports export of in-memory format to serialized Gordian Envelope format.
* Write test suite to read a specific wallet format and call other Rust crates to convert it into Gordian Envelope format.
* Write full reference app for important zecwallet format.
We expect the extensible interchange format to encode data in three ways:
1. Widely used data that we expect will be used well into the future will be encoded to explicit specifications with CBOR tagging.
2. Less common data or data types that have been deprecated will be encoded as "attachments", which are essentially "blobs of data", with the format referenced in conjunction with a specific vendor. This allows preservation of data without creating technical debt for the data format for little-used or older content.
3. The entirety of the original wallet file can also be preserved, probably in a compressed, encrypted form with metadata revealing where it came from. This allows its review in the future if anything is determined to be missing.
Because Gordian Envelope supports encryption, data that is stored (such as Zcash keys, addresses, seeds, and of course the archived data) can be encrypted with a symmetric key, protecting it on disk or in transport. Because of the standardization of this encryption method, this data remains fully interoperable. An exporting wallet could even sign the data and that signature would remain valid after the encryption, if proof was desired of the origin and validity of the seeds, keys, archival, and other data. This is all a crucial improvement over the use of older formats such as JSON, which do not natively support encryption, and so do not offer it in a standardized way.
Future Possibilities
This proposal does not include long-term support of the new extensible interchange format. We think that upgrades, bug fixes, improvements, training workshops, and more comprehensive documentation on the format might all be desirable. We're happy to work with wallet companies on their individual needs or to write a future proposal for longer term support if needed.
We also think there's an opportunity to make it *easier* to manage encryption of ZeWIF content during storage and transmission. (We think that a password-to-encryption-key method is likely the optimal way to manage easy encryption with less concern of loss.) However, our goal in this proposal is to focus on creating a forward-looking interchange format that can quickly support the zcashd deprecation, so we haven't extended into this parallel area of research. This is another place where we are happy to write a new proposal in the future to improve this support as usage of the format matures.
Finally, we generally believe that there is value in more universal reference applications that can help designers and developers to test out formats, to ensure compatibility, and to see best practices. Blockchain Commons has written an iOS app called Gordian Seed Tool and a few CLI apps, including seedtool and keytool, that offer such support for Bitcoin, Ethereum, and Tezos. We'd like to bring these over to Zcash, but such a project could be better specced out once the extensible interchange format is finalized, so that it can be fully integrated.
We hope to advance some of these possibilities in the future.
### Upstream Merge Opportunities
The extensible interface format will be merged primarily into wallet systems.
### Hardware/Software Costs (USD)
$0
### Hardware/Software Justification
N/A
### Service Costs (USD)
$0
### Service Costs Justification
N/A
### Compensation Costs (USD)
$121,200
### Compensation Costs Justification
This budget is based on rates for Zingo Labs contractors ($90/hr) and Blockchain Commons' contractors ($125/hr for our technical writer, $150/hr for our lead engineer & researcher).
It breaks down as follows:
Phase 1:
* Research into Zcash address derivations & wallet file formats as well as review of current Zcash-related Rust crates (3 BC research days: $3,600)
* Comprehensive survey of zcashd, zec/zingo, and eZcash wallets (20 ZL days: $14,400)
* Management & recording of wallet meeting, overview & coordination of survey & writing of final report (6 BC writing days: $6,000)
Phase 2:
* Interchange specification (15 BC research days: $18,000)
* Attachment best practices (3 BC writing days: $3,000)
* Review of howto and example implementation (20 ZL days: $14,400)
Phase 3:
* Wallet importer & exporter library & test suite (25 BC engineering days: $30,000)
* Importing & Exporting best practices (3 BC writing days: $3,000)
* POC ZeWiF consumer & CLI-app importer (20 ZL days: $14,400)
Phase 4:
* Zcash Community Utility ZExCavator (20 ZL days: $14,400)
### Total Budget (USD)
$121,200
### Previous Funding
No
### Previous Funding Details
_No response_
### Other Funding Sources
No
### Other Funding Sources Details
_No response_
### Implementation Risks
The biggest challenges will be adoption, and adoption that's rapid enough to meet those deadlines. We would like all of the major wallet developers to create true interoperability by incorporating the extensible wallet interchange format in advance of the zcashd deprecation date, but we're well aware that might not be sufficient time depending on each team's priorities.
The other major issue is the fact that older wallets, including zcashd, may have keys not based on seeds or keys based on seeds that were not originally BIP-39 compatible. For many modern wallets, this will require either an adjustment to how they store keys or else a sweep function to import master-key-based funds. We've built a similar sweeptool-cli for Bitcoin (https://github.com/BlockchainCommons/sweeptool-cli) and can highlight it as a reference. We'll also discuss in our best practices ways to accomodate these older keys, in order to reduce Zcash's technical debt without losing funds.
The biggest unknown threat is not knowing what data might be in ancient wallets. We're hoping things are pretty clean because the Zcash ecosystem has had just more than eight years to really create differentiation within their wallets, but it's still an unknown possibility.
### Potential Side Effects
Any standardization implicitly creates limits. By saying how things should be done, you might be limiting the innovation of doing new things in new ways. We'll do our best to prevent this through our planned survey (which should catch all the major ways things are being done currently) and through our best practices for use of Envelope attachments (which will describe how to incorporate data other than that which is explictly included in the wallet interchange standard).
### Success Metrics
Our major metric for success will be how many wallets have incorporated the format. We'll report that out a month or two after zcashd is deprecated and at the one year mark, by which time we expect everyone who's planned to incorporate the new format has.
The other success metric will be if users find it easier to move among wallets and if otherwise lost funds are recovered.
### Startup Funding (USD)
0
### Startup Funding Justification
N/A.
### Milestone Details
```milestones.yaml
- Milestone: 1
Amount (USD): 24,000
Expected Completion Date: 2025-01-31
Deliverables: Survey of Zcash Wallets & Final Report.
- Milestone: 2
Amount (USD): 35,400
Expected Completion Date: 2025-02-28
Deliverables: Interchange specification; Best practices for incorporating legacy or single-use wallet data as Envelope attachments; Example implementation of interchange specification.
- Milestone: 3
Amount (USD): 47,400
Expected Completion Date: 2025-03-31
Deliverables: Import Rust crate; Gordian Envelope Export Rust crate; test suite (or CLI or crate); Best practices doc for importing & exporting; Rust abscissa CLI to test import of a zecwallet-specific format.
- Milestone: 4
Amount (USD): 14,400
Expected Completion Date: 2025-04-30
Deliverables: ZExCavator Zcash recovery tool.
```
### Supporting Documents
_No response_