[Grant Update] Zcash Shielded Assets Monthly Updates

Dear Zcash community,

We are excited to share that we have concluded the milestone on ZSA Protocol Specifications!

This is the fruit of working with the skilled devs from ECC and ZF, resulting in comments, iterations, joint discussions, fixes and re-implementations of key insights.

We have now opened new pull requests for the ZIPs of the Transfer and Issuance Mechanisms that include specification details, the PR can be found here: ZIPs 226 & 227 - ZSA Protocol: Transfer, Issuance and Burn by PaulLaux · Pull Request #680 · zcash/zips · GitHub

With this update, we are submitting the material corresponding to milestone #7 of the ZSA proposal and grant - ZSA Protocol Specification (previously called milestone 5).

Some of the results of these discussions with ECC and ZF are:

  • We have discussed and addressed comments made in the earlier pull requests (viz. https://github.com/zcash/zips/pull/649 and https://github.com/zcash/zips/pull/628) and brought the documents even more in line with the structure and language of other Zcash Improvement Proposals.
  • We have separated out the rationale for our choices from the exact changes we propose, with a view to making it simpler for implementers and reviewers to read what has changed in the specification.
  • We have also added even more references to the protocol specification sections and ZIPs where it would help connect our ZIPs with the existing background material.
  • We have also made certain changes that help make ZSAs easier to use going forward, even if there is a move to a new (post-Orchard) anonymity pool. The most significant of those are:
    • The key structure for Issuance is now constructed in line with (but independent of) the Orchard and Sapling key structures in ZIP 32.
    • The asset identifiers are now independent of the protocol, and we have a separate step that converts the identifier into a format relevant for each protocol.

We look forward to hearing your comments on these proposals on this pull request!

Best,
The ZSA Team

27 Likes

Does each shielded asset have their own separate anonymity set, or do they share? And if they do share, do they also share with ZEC?

2 Likes

Yes, all shielded assets and Orchard ZEC will eventually share the same anonymity set. I say eventually because it’s obvious that all notes created prior to ZSA adoption are Orchard ZEC, but once all wallets adopt the new transaction format, it won’t be possible to tell whether you’re spending Orchard ZEC or any other asset. (Sapling and Sprout still have separate anonymity sets, of course.)

14 Likes

This is great, I gave the ZIPs a quick review and it looks solid to me!

7 Likes

This might actually mitigate the effect the transition to proof of stake will have on the shielded pool then.

2 Likes

Dear Zcash community,

We are excited to share an update on our efforts with the Zcash Shielded Assets (ZSA) project! We have been working hard towards getting our protocol ready for an external audit, and thus getting it on the path for integration with the NU6 network upgrade. There is still work to be done on this front, but we have made a significant amount of progress in the last few months, and are now in a position to mark the completion of Milestones #10 (Audit Preparation) and Milestone #11 (Preparation for Integration towards Deployment) of our grant proposal. In this post, we want to highlight some of our progress towards these milestones, that is also due to the comments and reviews of the members of the ECC and ZF teams and their regular syncs with our team.

Milestone 10: Audit Preparation

  • Clarification of UTF-8 usage in asset descriptions:

    Our discussions resulted in the consensus that the asset description string should be recommended to be encoded in UTF-8 according to Unicode 15.0.0 or later, without making it a mandatory requirement (which would require a consensus rule, and possible future encoding in a circuit).

  • Padding of input notes:

    Our initial design padded notes such that the numbers of Spends and Outputs were equal per Asset Identifier. A more efficient design, that we have incorporated now, is that padding can be applied to the overall number of notes.

  • Issuance Replay Vulnerability:

    Identified a potential replay vulnerability if we allow transactions containing only one bundle, viz. the issuance bundle. In that case, the SIGHASH will be created using only the issue bundle information, and hence a duplicated bundle will have the same issuance signature, which means a replay attack becomes possible. We solved this by requiring at least one input aside from the issuance bundle. This guarantees a change of the SIGHASH between transactions since inputs cannot be reused.

  • Choosing finalize at the block boundary instead of the transaction level:

    Our initial protocol was ambiguous about whether, if an Asset is finalized in one transaction in a block, it can no longer be issued in the same block. The existing Zcash principle is to allow shielded transactions to be freely rearranged within a block. Therefore, we made it explicit that the finalization becomes active at the block boundary, and no new Assets can then be issued from a subsequent block onwards.

Milestone 11: Preparation for Integration towards Deployment

  • Adjusting Asset Identifiers for improved future compatibility:

    Changed the Asset Identifier definition to the pair (ik, asset_desc) and derived the Asset Base for each shielded protocol using a hash into the curve. This has improved future compatibility than the original definition of the Asset Identifier as the point on the curve, since it allows the Asset Identifier to be fixed for every future protocol. Only the hashing to a point of the curve will vary from protocol to protocol. This would allow for easier movement between protocols in the future.

  • Updating Issuance key structure for improved future compatibility:

    This allows issuance keys to be picked independently of the Orchard keyspace. We now follow the mechanism of ZIP 32, which allows for future expansion of the issuance keyspace into a tree structure if it is deemed necessary.

  • Rust version compatibility:

    Our implementation currently uses Rust 1.61, and we have discussed with the ECC team that upgrading to beyond Rust 1.64 at the moment could cause issues with building on Android. We are keeping an eye on the status as this issue is resolved.

  • Updating the txid_digest structure to align with ZIP 244:

    We have ensured, in consultation with the ZIP editors and other stakeholders, that our updates and additions to the transaction digest conform with the existing ZIP 244 structure.

  • Using a flag byte with masking instead of a single bit for finalization:

    Having a full byte with masked bits allows for better serialization and deserialization, and therefore we have changed our design to use this improvement.

We thank the community for their reviews and comments on our ZIPs and implementation so far, as this is helping us roll out an even more robust protocol and implementation. We’re excited about the potential of Zcash Shielded Assets, and how close we are to making it a reality. As always, we welcome your comments and feedback, either on this forum or as comments on our pull requests (viz. PR#680 and PR#667) as we enter these final stages of fine-tuning our design.

Stay tuned for more updates!

Best,
The ZSA Team.

21 Likes

Great Progress and an excellent summary.
Are each of the ZSA team and ECC tracking Rust version compatibility to this project as their own risk entries? (or is this risk tracking isolated within the ZSA team’s work)
Can you share additional detail about the nature of the build problems that could manifest based on Rust 1.64 or later?

Hi @noamchom ,

The ECC is definitely tracking the compatibility of the Rust version for their own changes to their crates (orchard, librustzcash and so on). I don’t think the ZSA changes need the MSRV to be >1.61 at the moment, but we also want to keep track of how the issues resolve so that our code merges into the zcash repos as effortlessly as possible when the time comes.

As for the specific issues, there were a few messages on the Discord channel. To list the issues from there:

Hope this helps!

7 Likes

Dear Zcash Community,

Right before we meet at Zcon4, we wanted to share with you the updates on our progress with the ZSA Project. Our main focus has been working on the implementation of the transfer circuit (milestone #6) and the client wallet (milestone #9), as well as getting to the final design on the ZIPs.

Implementation

We are happy to update that we have largely integrated the ZSA circuit into the Orchard code! We have also added a number of tests, both positive and negative, to confirm the implementation works as expected. We hope to update you with the completion of milestone #6 soon :slight_smile:

On the client side, we have been working on adding and integrating RPC calls for wallet info, issuance and transfer of assets.

The overview of the updates is as follows:

  • We have rebased our repositories on top of Orchard 0.5, zcash_primitives 0.12, zcash_note_encryption 0.4 and a recent version of halo2, something that is necessary to ensure that our additions are well in sync with the rest of the code.
  • The issuance key derivation has been updated to a new mechanism, similar to ZIP 32.
  • We have added support for serialization of the burn mechanism data in librustzcash.
  • We have updated the Python test suite with further unit/integration tests.
  • We have updated the way split notes are handled in the orchard crate.
  • In the halo2 repository, we have added / exposed the necessary functions to support the new circuit functionality.
  • We now have a complete un-optimized circuit implementation for ZSA.
  • The zcashd client has been updated to add support for issuance and wallet info RPC.

Our changes can be tracked at the orchard:#372 PR, and the zsa1 branches of the librustzcash, halo2, zcash-test-vectors, and zcash repositories. Please let us know if you have any comments, and stay tuned for more updates!

ZIPs

We have recently completed going through all the pending comments on the old pull requests for the ZSA ZIPs (namely, PR#628 and PR#649). We have moved to PR#680 for further comments at this stage of the discussion, and we are working on addressing comments there. Most of our recent changes have been towards increasing consistency with existing ZIPs and reducing ambiguities. Some of the changes are as follows:

  • Updates to the txid_digest definition: Some changes were required to the Orchard component of the digest due to the increased size of the notes, and we have also added a section for the issuance digest.
  • Updates to the Action Description: Once again, the note changes necessitated updates to this data structure, which we have now detailed.
  • The derivation of the nullifier for the Split Notes has been updated, based on the discussion with ECC here and here.

The updates to the transfer circuit have also now been included in the ZIPs, and we have also shared a draft document with all the details of these changes and how they fit into the context of the entire circuit. We will include this in the ZIPs as a PDF document once it is finalized.

We have also brought in additional eyes to go over our ZIPs from top to bottom and suggest improvements, so that our document has the highest quality.

Best,
The ZSA Team.

21 Likes

Dear Zcash Community,

It was great to meet and interact with the community at Zcon4! We had a lot of fruitful discussions, and we were also happy to present our progress with the ZSA and Asset Swap projects. The links to our talks at the event are linked below, please do let us know what you think!

We are very excited to update that we have integrated the ZSA circuit into the Orchard code, and are now submitting the implementation of the transfer circuit, corresponding to Milestone #6 of our grant! Simultaneously, we have been working on the zcashd client, and have the RPC interface up and running.

Transfer Circuit Implementation

A summary of our changes are as follows:

Orchard

  • We performed updates to the value commitment and note commitment equations, as well as added support for split notes as detailed in the spec. There were some iterations over the generation of the nullifier of the split note in order to achieve security.
  • We added a few new witnesses to the circuit, namely the asset witness for the Asset Base, the is_native_asset boolean for specifying privately whether we are dealing with ZEC or ZSAs, the split_flag boolean for whether we are dealing with split notes or not, and the psi_nf witness for the random nullifier when we are working with split notes.
  • We added the enable_zsa boolean to allow for pausing of the ZSA functionality in case of any vulnerabilities (along the lines of the enableSpends and enableOutputs flags in Orchard).

The complete set of changes along with their context is documented here.

Link to code that corresponds to the circuit changes. This blob is included within the larger PR#372 that we have on the orchard repository.

Halo2

The changes here largely add functionality for use in the Orchard ZSA circuit.

  • We added methods to perform Variable Base Signed Scalar Multiplication, since this support was needed in the value commitment calculation in the Orchard ZSA circuit (as opposed to the original fixed/variable base scalar multiplication that was sufficient for Orchard).
  • We added functionality for performing the note commitment in constant time after the changes necessary for ZSA.
  • We exposed functions for various requirements, such as to create curve points from constants, for the nullifier computation in the circuit;

Link to Halo2 PR#769.

Node Wallet State Implementation

We are also nearing completion for this milestone — we presented an early version of the RPC interface for the zcashd client at Zcon4, in the ZSA Interaction Workshop! We will share more details about our work in this direction in a subsequent post, so stay tuned!

Next in line is our implementation of the updates to the Fees ZIP (see ZIPs PR#667 for the specification). We are also working on adapting our work for the zebra client. As always, we look forward to hearing your comments on the relevant pull requests and the forum! We also encourage you to try out the demo as shown on stage in the ZSA Interaction Workshop at Zcon4, and let us know how you like it :slightly_smiling_face:

Best,

The ZSA Team.

20 Likes

can you add a timeline? where are you on timeline for completion?

We are on track for ZSAs to be included in the next network upgrade (viz. NU6)! :smiley:

8 Likes

Is there an NU6 delivery timeline?
Is it correspondent to the Halving?
It seems JXG (and myself) are wondering about approximate calendar estimates

3 Likes

Is going to happen with the POS transition

really looking for a date.

Very good!

Just to give an update on this, zcashd is currently on clang 15 and Rust 1.69.0. librustzcash is on MSRV 1.65.0. So that’s no longer a blocker.

1 Like

I can’t make any strong predictions, but the progress on ZSAs is significantly ahead of that on PoS for Zcash, and neither NU6 nor ZSAs are blocked by PoS.

6 Likes

Dear Zcash community,

Hope you are all doing well! We’ve been working hard on both the Zcash Shielded Assets implementation and integration, and the protocol design for building Asset Swaps over the Orchard-ZSA protocol. We’re excited to share with all of you the progress we’ve made in the recent weeks!

We will split our update into two sections, one for each grant that is in progress:

Zcash Shielded Assets

The major areas we were working on are:

Circuit Optimizations

We had shared in the previous post that we had completed the integration of the Orchard-ZSA circuit into the code. This was a first step, with various possible places for optimization. Building on that,

  • We first performed a study to understand which optimizations are relevant and applicable.
  • We optimized the short-range check on 4 and 5 bits.
  • We optimized Sinsemilla hash computation for ZEC vs ZSA: the common prefix inside the circuit is now computed only once.

This entire process occurred with the advice and guidance of the ECC core team. We have integrated these changes into the QED-it/orchard and QED-it/halo2 repositories.

The full details and layouts of the circuits, and the nitty-gritty of how these optimizations have improved the various proof parameters, can be found in this document.

Change of the issuance signature scheme to BIP 340 Schnorr over secp256k1

  • After careful consideration and advice from internal and external contributors, we decided, in the case of the issuance authorization signature, to move to the Schnorr signature scheme over the secp256k1 curve, as described in BIP 340. This change was done to maximize compatibility with existing hardware/software wallets that already support BIP340.
  • The changes have been incorporated into ZIP 227 (ZIP 227: Issuance of Zcash Shielded Assets)
  • The implementation changes for the orchard crate are currently in progress.

Definition of Transaction v6

Since the ZSA protocol is expected to be integrated as part of the NU6 network upgrade, we began to define and implement the transaction structure for this upgrade so that it is compatible with Orchard-ZSA.

  • The expected structure has been included as part of the newly reserved ZIP 230.
  • Of course, this transaction structure is not final, and could change as per the requirements of NU6.

Asset Swaps and beyond

This grant covers the design and implementation of Asset Swaps, and other considerations for the integration of ZSAs into the NU6 upgrade

Asset Swaps

We have presented an initial suggestion for the Asset Swap mechanism. Our goal while coming up with this design has been to minimize the changes needed over and above the Orchard-ZSA protocol.

  • We have published a call for comments on the Asset Swaps design. We have created GitHub Issue #736 in order to discuss this design, so we would love to hear your feedback there!
  • The technical details are in this Google doc.

Testing Framework for the Zebra node

We had presented an E2E demo of the Orchard-ZSA work for zcashd at Zcon4. We are now in the process of porting the necessary changes to the Zebra node.

  • Note that unlike zcashd, the Zebra node does not have the ability to generate transactions.
  • Therefore, in order to support ZSAs, we need the ability to create and verify v6 transactions. To do so, we are creating a testing framework that will allow us to generate v6 issuance and transfer transactions.
  • This framework will use the Orchard-ZSA crate, librustzcash, along with additional new code for managing keys and notes.

The Zebra node

  • As part of the move to Zebra, we are in the early stages of adding TX v6 support for the Zebra node.
  • As it stands, new code related to Orchard-ZSA is being conditionally compiled into the node by using a rust feature flag. This will help in the future with gradually integrating the changes (which touch many different parts of the project) instead of pushing one huge pull request into the code base.

We look forward to hearing what you have to say about these designs and updates! Please feel free to join the discussion.

Best,

The QEDIT Team.

15 Likes

Dear Zcash community,

We have exciting news that we would love to share with all of you! As you know, there was a recent strategic change of moving from zcashd to zebra for the core protocol for Zcash. For the ZSA project, this means we are now working closely with both the Zcash Foundation engineering team and the ECC team to make this transition a success.

As we mentioned in a recent Arborist call, to deliver ZSA along with NU6, we have proactively started to work on zebra:

  1. QEDIT is re-implementing the full ZSA functionality in zebra (and stopping zcashd).
  2. QEDIT is developing non-ZSA transaction testing functionality that is currently missing in zebra.

For this, we had to update the project milestones as follows:

  1. We have finished ZSA on zcashd (grant 1 / milestone #9):
    1. Our milestone was “Node Wallet State Implementation on Zcashd” (Link)
    2. We delivered this milestone for zcashd and used it for the ZSA demo at Zcon4 (Link)
    3. There was little work left to merge the code, but we stopped working on zcashd.
  2. We are implementing ZSA for Zebra using funds from grant 2 / milestone #2:
    1. This milestone is now “ZSA Transaction Testing on Zebra”
    2. The original deliverable was “Testnet ZSA Block Explorer support” (Link)
    3. We have dropped this for now and are instead replicating milestone #9 above, but for zebra.
  3. The Fees implementation (grant 1 / milestone #8) is contingent on building missing pieces in Zebra:
    1. We are updating this milestone to be the delivery of all the non-ZSA transaction functionality development.
    2. The affected deliverable is the “Fees Structure Implementation PR” (Link)

We are working within the amounts that have been approved by the ZCG in our grants in order to meet the moving target of getting ZSAs out into the ecosystem rather than increasing the funding.

Best,
The QEDIT Team.

14 Likes