Grant Proposal: Zcash Extensible Wallet Interchange Format (ZeWiF)

[BC-Shannon]

We held our first meeting with first report out on the survey today.

Info on the meeting is here, including video, slides & key points:

ZeWIF Meeting, 1/24/25

Discussion of the survey we’re doing and the meeting is here:

ZeWIF: First Meeting & First Results

We held out first meeting on the Zcash Extensible Wallet Interchange Format (ZeWIF) today and also presented the first results of our survey, with the intent of getting feedback and expertise before we close out the survey and our related spreadsheet of data at the end of this month. The initial survey is at zcash-wallet-formats/README.md at master · dorianvp/zcash-wallet-formats · GitHub. It currently includes a look at the Zcashd, Zecwallet, Zingo, eZcash, Zashi, and YWallet wallets. The st…

[HHIROSHIMA]

Hello Hanh,

I would appreciate it if you could contribute your expertise to Zcash more often in the future. Especially with regard to projects that are submitted for funding before they are decided.

In this particular case, I would have been interested to know how much time it would have taken you to develop exactly what Zingolabs suggested and, more importantly, what you think the total cost would have been. Let’s leave aside the point of the whole thing since you’ve already gone into it.

I appreciate your work, keep it up.

Ps: By the way, I watched your video on YouTube. I’m impressed with your skills. But I’m not sure how complete the whole thing was after the 1-2 hours it took you to do it.

[BC-Shannon]

We’ve closed out phase one of the project. Here’s our final report on wallet contents:

ZeWIF Wallet Report

This was built on our wallet survey & a wallet contents spreadsheet:

GitHub - zingolabs/zcash-wallet-formats: Zcash Wallet Survey

Zcash Wallet Content

This was of course all in process to lay out the data we need to create a specification for the wallet interchange format, which is going to be the work of phase 2, this month.

Our other deliverable for phase one was our meeting, which we held and reported on a week and a half ago:

ZeWIF Meeting, 1/24/25

[BC-Shannon]

Hi Folks,

We have had a solid month of work on the ZeWIF project in February and remain on target to deliver a complete toolset (per milestones 2 + 3) at the end of March, in advance of the zcashd retirement. But, our model for the task has shifted, which is pushing back what we’d initially identified as the intermediary milestone 2.

Our initial plan was a top-down approach: we were going to produce the ZeWIF specification in February, then produce libraries to support writing and reading ZeWIF in March. However, as our engineer has dived into the project, he’s come to the conclusion that beginning with the coding is a superior methodology due to the complexity and breadth of the Zcash wallet ecosystem, with the specification then to be written based on the pragmatic, real-world needs of the wallets as they currently exist.

As a result, we’ve been able to release an initial version of “zmigrate”, the Rust crate that we’ll be offered to support conversion to and from ZeWIF. It currently supports reading zcashd files and parsing most keys:

GitHub - BlockchainCommons/zmigrate: ZCash wallet migration framework

A video describing the zmigrate work is here:

Zcash ZeWIF Demo: February 2025

The keys we’re parsing are described here:

src/zcashd/KEYS.md

We are looking for the following support to help make zmigrate successful:

1. If you have a zcashd wallet, download and run zmigrate.

2. File Issues and/or Create PRs based on your use of zmigrate. Tell us if it successfully parsed your entire wallet.dat, or if there were values that it missed.

3. If you have a zeroed-out wallet from any wallet app (particularly wallet apps other than zcashd) that you are willing to donate to help us continue to test wallet formats and improve ZeWIF, please let us know ASAP.

More pragmatically, this has obviously interfered with our ability to meet the second milestones as originally planned.

Ideally, we’d like to be able to submit our published “zmigrate” repo and video as fulfillment of our second milestone, or some portion thereof. We’d then meet the rest of the milestone 2+3 requirements at the end of March. We could get by without this, but it’d be helpful, so please let us know if there’s an option here.

Otherwise, we just need an OK to mush our original milestones 2+3 together, with both the specification and libraries likely to arrive at the end of March—and of course let us know if there are any issues or concerns with our previously discrete milestones being completed together at the end of March.

Thanks very much for the understanding and support. Progress has been strong, just in a different order than originally intended, and we wanted to make sure the ZCG community was aware of that and to see if accommodations could be made for it.

Shannon

[aquietinvestor]

@BC-Shannon

Otherwise, we just need an OK to mush our original milestones 2+3 together, with both the specification and libraries likely to arrive at the end of March—and of course let us know if there are any issues or concerns with our previously discrete milestones being completed together at the end of March.

ZCG approves this milestone amendment.

[BC-Shannon]

Thanks, folks.

[BC-Shannon]

Last Friday, we held another ZeWIF meeting as we continue to work on the abstraction at the core of the specification.

ZeWIF Meeting #3: 3/14/25

Comments continue to be welcome!

[ChristopherA]

ZeWIF Status Update (3/26/25)

Blockchain Commons and Zingo Labs have made strong progress on the ZeWIF project in March, and have in fact dived deeper into some areas than expected, but are also facing some challenges with our scheduled milestones. Here’s more on all of that …

The main focus of work continues to be on the zmigrate tool (GitHub - BlockchainCommons/zmigrate: ZCash wallet migration framework), in the crate of the same name. This tool can now import data from zcashd using the functionality in the zewif-zcashd crate, which then abstracts that data into the ZeWIF in-memory abstractions in the zewif crate. Any developer can now build on this central tool: they create their own import and export crates, link them to the zewif crate, and support import and export from their own wallets. (All of these elements are still very much subject to refinement and testing, which we are relying on the Zcash community to contribute.)

We’ve talked with the developer who will be using this at ECC and of course we have the internal support of Zingo Labs, and we believe that what we have meets the needs of the April 1st zcashd deprecation, which of course was the most immediate need. In fact, we expect ECC to be working with the tool this week and Zingo Labs will be producing their first-cut zecwallet importer as part of our current milestones.

The main puzzle piece missing from the project at this time is the ability to import and export with the Gordian Envelope-based serialization format, which allows the storage of the in-memory data. It remains a top priority as we know some, perhaps most, wallets will use the file as an intermediary, but we think there are likely a couple of weeks work to close this out. (Our priority to date was instead advancing zmigrate and associated crates to their current state, where they successfully import zcashd data and other developers can start hooking into them for their own input and output wallet formats.)

Completing the Gordian Envelope work is going to represent a time overrun from our original proposal, and it’s the result of a number of factors.

1. Complexity in translating zcashd. In collaboration with the Zcash community, we established the baseline abstractions for the zewif crate. However, it turns out that the mismatch between the information actually stored by zcashd and the zewif abstrations was high, and the coding to perform this translation turned out to be much more in-depth than planned. (See here for our most recent meeting on the subject, two weeks ago.)

2. Intermediary Revision. Though we originally imagined the Gordian-Envelope file output format as the intermediary for all interchange, we increasingly see the in-memory ZeWIF binary representation as the core interchange and that some wallets might directly use that and never go to the Envelope-based text format. As such, the zmigrate tool and the zewif crate have received more attention and definition. (Developers might either use the tool or the zewif crate.)

3. Creating Clear Separation of Responsibilities. To maximize accessibility to developers, we have refactored the original zmigrate crate into four crates:

• The core zmigrate crate, which vends the command line tool.
• The zewif crate contains the universal parsing framework and in-memory abstractions. This crate also directly supports the reading and writing of the zewif interchange format, which is based on Gordian Envelope.
• Several front-end back-end libraries that support the parsing of wallet formats into zewif abstractions, and translating those abstractions to their format. zewif-zcashd is where most of our work has gone, in translating the zcashd format to the zewif abstractions. zewif-zingo only reads the Zingo file format but does not yet translate it to zewif abstractions. zewif-zecwallet mentioned in the diagram does not yet exist, but is intended to continue the pattern of third party-developed front/backend crates.

4. Deprecation Deadlines. Our original proposal focused on the April deprecation deadline for zcashd as a hard requirement rather than more organically laying out a project and deriving the time needed from that. As the work proved to be in excess of the time, we’ve figured out how to support the needs of the deprecation while still leaving this one major element unfinished.

Beyond this, we always knew there would be desire for longer term support, particularly for revisions of ZeWIF as it comes into use. We can now see this is something that’s going to immediately be needed, because the developers that are starting to use our interchange format today are going to want that support over the next few weeks. (In fact, ECC has requested that we delay the release of the zewif crates specifically to allow time for integration and revision.)

We hope that @ZCG can extend our original grant for the work still needed to close out the Gordian Envelope output-format work on the original grant, and also that they can boost the grant to enable support for other developers in April.

To do so, we’d update the current Milestone 3 as follows, with our expectation being that we’ll have a full report out to close this on April 1 or April 2:

• Zmigrate & Zewif crates for in-memory ZeWIF, available at GitHub
• Cargo docs for ZeWIF Rust API
• test suite (zewif-zcashd crate) for zcashd
• test suite for a zecwallet-specific format.
• First draft: Best practices doc for importing & exporting
• First draft: Best practices for incorporating legacy or single-use wallet data as Envelope attachments

We’d then request a new milestone 3.5 for April 30th with the following deliverables:

• ZeWIF-Envelope Import/Export Crate
• Full ZeWIF Serialization Specification
• Final versions of Docs

This would require the additional Blockchain Commons work:

• ZeWIF Envelope Input & Output + Final Specification + Engineering Support for other developers (22 BC research days [e.g., April]: $26,400)
• Finalization of Docs, Meeting Support (2 BC writing days: $2,000)

Milestone 3.5: $28,400

This would not affect Milestone 4, which is work that Zingo Labs would be doing in parallel to the new Blockchain Commons 3.5 work.

We are of course committed to finishing the work as specified in the original grant proposal, but as we said we hope that ZCG can support the extra time that’s been required, plus allow us to provide better follow-up for the community over the course of the rest of April.

Please let us know if we have approval for this, and if so we’ll submit the revised third milestone on the 1st or 2nd then the new 3.5 milestone around the 30th.

Thank you!

Christopher Allen
Blockchain Commons

[zancas]

It’s important to understand that the work that the Blockchain Commons did in their zmigrate family of crates was (a) unanticipated and (b) is in some ways (to Rust developers) more useful than the as yet unfinished work.

Obviously as a corecipient of this grant I am quite biased, but I believe this work is very useful and we should fund their ask.

[aquietinvestor]

Hi @ChristopherA. Thanks for the detailed post. We have a meeting on Friday and will review your request then. We’ll follow up with you afterwards.

[daira]

This project is making good progress and I thoroughly support it continuing as requested without any disruption to the grant.

[aquietinvestor]

We met this morning to discuss your request. We’re okay with extending the timeline for the current grant in order to finalize the deliverables, and we’re also supportive of the request for additional funding of $28,500 for a new Milestone 3.5.

Please work with @FPF to update your grant proposal, and we’ll process the amendment next week.

[nuttycom]

I’m late to the party, but thank you for extending this grant; this is vital work and I’m currently in the process of integrating it into the zallet wallet; it would be a significant waste to shortchange this work just as it’s starting to bear fruit.

[BC-Shannon]

Updated grant proposal is now available here:

Grant Application - Zcash Extensible Wallet Interchange Format (ZeWiF)

If you need any different format, @FPF, @ZCG, just let me know. Thank you!

[BC-Shannon]

Here’s the report-out on the updated Milestone #3.

Zmigrate & ZeWIF Crates. The ZeWIF (in-memory representation) and Zmigrate (CLI utility) crates have been released. They currently support full parsing of zcashd files (but without decryption of encrypted data) and Zingo files. Additional details are in the Zmigrate repo.

ZeWIF: GitHub - BlockchainCommons/zewif: ZeWIF: The Zcash Wallet Interchange Format
Zmigrate: GitHub - BlockchainCommons/zmigrate: ZCash wallet migration framework

These are fully accessible by developers who want to start experimenting with importing from their wallet to ZeWIF or exporting from ZeWIF to their wallet. We extend to continue advancing them over the course of April and would love your feedback on what else is needed.

ZCashd Test Suite. The Zcashd test suite was the core that we built ZeWIF from. The adapter can now be found in zewif-zcashd.

ZeWIF-Zcashd: GitHub - BlockchainCommons/zewif-zcashd: ZeWIF adapter for zcashd wallets

Zecwallet Test Suite. Zingo Labs created the first third-party test of our work with a Zecwallet conversion. We’re still coordinating the integration, but the full work & testing can be found in a PR.

ZeWIF-ZecWallet PR: Add ZWL support by dorianvp · Pull Request #3 · BlockchainCommons/zmigrate · GitHub

ZeWIF Documentation. The files in the repos have all been extensively documented to detail the ZeWIF API and data structures and offer usage examples. These docs are currently available as parts of all the src files and will be extracted into Cargo docs upon release of the crates.

Best Practices. The first draft of the best-practices for migration doc has been released. Any comments or thoughts on additional best practices are welcome. (There are also a few holes to fill as the whole specification settles out over the course of April.)

Best Practices: zmigrate/docs/bestpractices.md at master · BlockchainCommons/zmigrate · GitHub

Attachments. Similarly, the attachments draft includes a strong overview of Gordian Envelope and its attachments, with the final few HOW-TOs awaiting the hood being shut on the ZeWIF crate. If you’d like additional clarification or documentation on anything here, just let me know.

Attachments: zmigrate/docs/attachments.md at master · BlockchainCommons/zmigrate · GitHub

That’s it for the moment. Thanks for the continue support and faith in this project.

[BC-Shannon]

Hi Folks at @FPF, @ZCG,

I wanted to check if you were able at your Friday meeting to OK the amendment to the grant proposal for the additional funding for this month’s work to close things out. Thank you!

[ZCG]

@BC-Shannon apologies for not responding to your first message. It was approved in the last ZCG meeting: Zcash Community Grants (ZCG) Meeting Minutes 3/31/25. You can request the payout via the usual process when you’re ready.

[BC-Shannon]

Great! Thank you for the support!

[BC-Shannon]

Wolf has been making good progress on the updated milestone #3.5. The main goal here is to produce the Gordian Envelope file format to allow easy interchange of data between machines.

Since the ZeWIF in-memory format was already closed out (in current draft) with milestone #3, the Envelope format just requires round-tripping from the in-memory format to the file format and back. Wolf is attacking the data structures one at a time. Once everything round trips, writing out to the file, then reading back into the file, with the data structures looking the same, we’re set. At that point, all wallet developers need to do is read their data into the in-memory format or convert the in-memory format back to their wallet, all using the API docs already in the source files.

Here’s a sneak peak at what the Envelope file format looks like. (This is “Envelope notation” so it presents everything as human-readable as possible, with the actual CBOR representation one layer lower.)

45839946 [
    'isA': "SaplingOutputDescription"
    "commitment": Bytes(32)
    "enc_ciphertext": Bytes(580)
    "ephemeral_key": Bytes(32)
    "memo": Bytes(64)
    "note_commitment_tree_position": 522950011
    "witness": Bytes(32) [
        'isA': "SaplingAnchorWitness"
        "witness": [Bytes(32), null] [
            'isA': "IncrementalMerkleTree"
            'isA': "SaplingWitness"
            "cursor": [Bytes(32), null] [
                'isA': "IncrementalMerkleTree"
                "right": Bytes(32)
            ]
            "filled": [Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32)]
        ]
    ]
]
1106195738 [
    'isA': "SaplingOutputDescription"
    "commitment": Bytes(32)
    "enc_ciphertext": Bytes(580)
    "ephemeral_key": Bytes(32)
    "note_commitment_tree_position": 597373669
    "witness": Bytes(32) [
        'isA': "SaplingAnchorWitness"
        "witness": [Bytes(32), null] [
            'isA': "IncrementalMerkleTree"
            'isA': "SaplingWitness"
            "filled": [Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32), Bytes(32)]
            "left": Bytes(32)
            "right": Bytes(32)
        ]
    ]
]

Obviously, this is for Sapling output descriptors, but there will be similar outputs/storage for everything.

As you can see, Wolf has also settled on our internal type system (using isA) to define the various structures, basically because of their quantity and the specificity of many of them for Zcash. (Our other options were CBOR tags and our own Known Value tags, but he settled on this one as the best for extensibility rather than burdening the structure with the need to define everything in an external registry.)

Some upcoming notes:

1. We’re basically past any technical risks implicit to the project at this point, and just grinding through the final iterations of the work.

2. I think we’re going to have new releases of the zewif and other repos, with this work, toward the end of this week.

3. We will be holding a demo on April 16th, likely at 11am PT of using zmigrate and round-tripping data through the file format. I need to double-check the exact time with the principals and then I’ll post the specifics. You’re welcome if you want to joing, but we’ll also have a video available afterward.

4. Wolf and I will still be closing out documentation at that point, but in the couple of weeks that close out the month, we’ll be very happy to work with wallet developers to make sure ZeWIF is suiting your needs, so I encourage wallet folks to start looking at what we have so far so that you can take advantage of Wolf’s availability for the last couple of weeks of the month.

As a reminder, here’s the zewif repo that’s the heart of the project: GitHub - BlockchainCommons/zewif: ZeWIF: The Zcash Wallet Interchange Format

Here’s the zmigrate repo that makes use of that for a CLI: GitHub - BlockchainCommons/zmigrate: ZCash wallet migration framework

EDITED: Demo will be on April 16th. Full details to follow.