ECC R&D and Engineering Flight Plan

Hi all. We posted a written version of the flight plan Nathan presented on the livestream. We’d welcome your thoughts!

4 Likes

Thanks @nathan-at-least, and everyone who supported him, for preparing this plan! It’s solid, well-thought-out, well-presented and the short-term objectives are extremely compelling. I particularly like the multi-scale approach, the rationale given for every objective, and the explicit success indicators.

Reading through this, I have some comments/suggestions/questions:


While Horizon 1 is beautifully worked out, Horizons 2 and 3 are very blurry. In particular, the discussion of Horizon 2 (which starts just 6 months from now) is mostly about what other teams will do, and says little about ECC’s objectives and tasks during these 18 months. Especially with the Dev Fund question on the table, I think more clarity is needed on what ECC will be up to in the upcoming years.


Where do user-defined assets and programmability fit into the picture? They’re clearly excluded from Horizon 1, but are they envisioned to happen later on? When does ECC even start researching them in earnest?


It would have been helpful to get some quantitative indication of the quantity of effort each objective requires, especially as one balances these objectives against other potential ones.

Likewise, it would be helpful to explicitly call our risk factors, such as “research required” or external dependencies.


On Mobile wallet SDK improvements, it’s unclear what’s the scope of the mobile wallet SDK, and what functionality still needs to be developed.

I love the plan to develop a functional “dogfood” shielded ZEC wallet! We learned so much from the Zcon1 prototype, and it feels that doing a real mainnet wallet is going to be huge, both in stress-testing the SDK functionality and in getting feedback on shielded tech usability.


On Viewing key support: yay!

What about the per-transaction payment disclosures (as we used to have in Sprout, in addition to the per-private-key viewing keys)?


On Community-driven protocol development, you say:

  • Zcash Foundation and at least two of the four other participating development organizations publicly endorse the new process

I’m curious, who are “the four other participating development organizations”?


  • ECC publishes a new version of the Halo paper with optimizations and performance benchmarks

How about finishing the Halo paper by making clear security claims, and providing security proofs? And adding missing features like zero knowledge?

To the execution success criteria, I would add “the Halo paper is accepted for publication in a high-tier peer-reviewed academic venue”.

Also, what about research and communication on how to use Halo in Zcash? We’ve seen how Halo can be used to verify PoW (which is a relatively easy application which doesn’t even need ZK). But AFAIK nothing’s been said about how Halo would be used to verify transactions. Because Halo not succinct in the efficient-verifier sense, it’s not a drop-in replacement zk-SNARK; rather, it would need some new application-layer protocols to exploit its amortized verification using recursive composition.

Similarly, without pertinent application-level protocols, we don’t know if Halo is helpful for achieving scalability. It sure feels it is, because we know from Coda that recursive compositions helps many aspects of scalability, but more concrete planning is needed here.


Also on Scalability 2021, this strategic success indicator worries me:

  • Zcash’s Scalability 2021 effort is referenced in one or more high-quality industry reports about scalability

Being referenced in a report is not a good goal to set for the engineering flight plan. To put it bluntly, if all that’s done for Scalability 2021 by the end of Horizon 1 is what you set out to do as the execution success criteria, i.e., you never get around to designing the application-level-protocol connection between Halo and scalable Zcash, then no one should believe you’ve made any progress towards Scalability 2021. So if some reporter or industry evaluator is nonetheless fooled into thinking otherwise, I don’t consider it an indication of success for the engineering flight plan.


Well that’s all for now… Thanks again for the deep thinking and excellent presentation, and for pushing the bar on technical plan transparency in our ecosystem. Looking forward to the ongoing discussion!

3 Likes

Hi @tromer, thanks for the detailed response! It helps us improve and refine our plans. There’s a lot here and I’ll strive to respond to each point as well as I can in the moment.

It may not be clear, and I think some of this was lost during editing, but I intended the “Three Horizons” section to not describe ECC’s plans. Instead it is intended to provide our belief about a plausible state of the world with respect for Zcash in those time ranges. This may contribute to it sounding like “it’s a bunch of work others will do” because others doing a lot is a key aspect of our belief about a successful Zcash trajectory.

If we move onto our concrete Objectives and Key Results, that represents active goals for ECC.

With that in mind, how can we improve (a) the “visions” (ie. predictions of the future)?

-and how can we improve (b) the plans ECC has during those time horizons?

I’m curious to hear your responses or other on these forums. Here are my answers:

For (a), improving our plans around predictions:

  • Keep adding more specific detail so that the predictions are falsifiable and we can learn over time our accuracy at prediction.
  • More importantly, if a prediction is shown to be inaccurate, as soon as we learn that, we should adjust our plans accordingly.
  • If other Zcash contributors have substantially different visions or predictions, the sooner that’s discussed publicly the better. This is a way for ECC to document our beliefs so that people understand our rationales.

For (b), I would say it’s always easier to add new goals, especially with shorter time horizons, at a later time than it is to pre-commit to long range goals and then shift them in-situ.

Let’s drill into your next question as a case study:

It’s a bit difficult for me to figure out the best way for me to respond to this.

What I would prefer moving forward is for you, or others, to advocate for whichever vision you seem to want to advocate for here. To me, it sounds like you have these beliefs just based on the quote immediately above:

  • User-defined assets and programmability should be considered together.
  • They are excluded from horizon 1.
  • ECC will start researching them in earnest but hasn’t announced when.

I disagree with every one of those assumptions. (Also, I may be misinterpreting you, and even if I wasn’t I would definitely want to hear your thinking behind them.)

I have a call to action for the whole Zcash community which is for contributors to spell out their own visions for where Zcash could go, especially if it diverges from ECC.

Ok, with that in mind, I hear you that you want more details specifically about ECC’s plans, so:

On programmability: We are working on programmability in Horizon 1! It’s a feature called Whitelisted Transparent Programs. (I kind of wish we had a different name, perhaps “Zcash extensions”.) One way I think of this feature is akin to Ethereum’s precompiles. They are like “hard-coded smart contracts”, and I believe their expressiveness is broader than Bitcoin Script (although the details aren’t well specified enough yet to know).

The reason we’re starting our foray into programmability with this “hard-coded / whitelisted” appoach is to that we have an explicit and clear understanding of the programmability use cases. Also, by keeping the functionality fixed and only changeable with network upgrades, it’s easier to reason about consensus guarantees, scalability, and privacy. In the future if we have a set of several WTP extensions, with known usage/value, we can focus on how to add privacy or scalability while meeting those specific uses.

A key belief of mine is that by restricting the scope of Zcash to ZEC without adding too much complexity like general purpose smart contracts, it will simplify the problem of layer one scalability.

When thinking about programmability, I think it’s especially important to understand existing and potential use cases. We already have Bitcoin Script in t-addrs, and here are some interesting empirical questions about use case:

  • What use cases are enabled by Bitcoin Script?
  • What use cases are seen actively deployed which rely on Bitcoin Script? In Zcash? In Bitcoin? Elsewhere?
  • Of the actively deployed use cases, how much “economic activity” relies on those use cases?
  • How do all of the above questions apply to Ethereum’s smart contract design, or any other similar programmability?

I don’t have solid answers for much of this, but @str4d has done some excellent work on empirically investigating Script usage in Zcash, which you can see here.

Now turning my attention to user defined assets:

It’s clear to me there is something important with user defined assets. What is less clear to me is which parts of that are good for ZEC. Two anecdotal data points:

  • Ethereum + ERC20 interface has lead to Ethereum accruing many tokens atop its platform.
  • Omni has provided something like user defined assets for ages atop Bitcoin.

It’s not clear to me how these platforms support the “native asset” utility or value. The Omni token is worthless. Meanwhile Tether relied on Omni to carry billions of dollars-worth-of-usdt for quite some time.

Meanwhile, we also see tokens migrating off of their “birth” platform, such as Tether volume on Ethereum surpassing volume on Omni (BTC) as per tether’s transparency page, or several projects migrating off of Ethereum.

Counterbalancing those concerns, all such user-defined-assets-on-Zcash could benefit from having a conjoined privacy set, so this may be a unique value proposition that the previous examples lacked.

All this said, from my point of view user defined assets is a potentially valuable feature that I do not see as a given for Zcash’s future. If we see a compelling vision or supporting data or anecdotes about how this feature could benefit Zcash and ZEC, that may change.

I should also note that we already have a key piece of user defined assets (the shielded transfer circuit) designed and implemented in a prototype. Additionally, I believe for many issuance use cases, WTPs will provide a useful building block to deploy a user defined asset implementation with a relatively low amount of effort.

These are two excellent ideas and I’ll strive to include those in our strategic update next quarter.

Quantity of effort delineated in an implicit/nonexplicit way: we believe the goals will be achieved in the horizons given assuming ECC product engineering is focused at almost full capacity with our current team. I agree that breaking down effort estimates per goal could be helpful. (However, I caution that such estimates drop off in accuracy as the time horizons grow larger.)

There is one part of our framework that does identify external dependencies, because we distinguish between “Execution Success Criteria” and “Strategic Success Indicators” where the former are defined to exclude external dependencies. However, I agree that a general assessment of risk factors would be an improvement.

At a very high level, the SDK needs improvement in these areas: t-addr support (and interaction with z-addrs), iOS support, and “production readiness” hardening (better rollback protection, some security improvements, easier deployment).

Exactly! We wanted to continue that kind of full-stack learning that helped us see where some parts of the design are challenged or incomplete when it comes to full end-to-end deployment. We’ll be using this dogfood wallet with small invite only groups for usability testing and also potentially to prototype new end-user facing features.

We do intend to include payment disclosures, maybe in Horizon 1 but likely in Horizon 2. They were not mentioned in the blog post because I expect for most readers they don’t distinguish Viewing Key Support from Payment Disclosures.

If you are reading along and didn’t realize these were different: Viewing Key’s allow the holder to see incoming and outgoing transaction details for an associated address. Payment Disclosures reveal information about a specific transfer only.

I apologize that this got muddled during late editing. An earlier draft (which was way too long) mentioned that as of today there are at least four other organizations involved in Zcash protocol R&D outside of ECC + Zfnd: Keep, Summa, Parity, and Bolt Labs. So in this context, the strategic success criteria is that at least two of those organizations endorse the new process.

I also apologize for a bit of editing loss here: the “Effort”, “Execution Success Criteria”, and “Strategic Success Indicators” are all scoped (by definition of the framework) to one quarter. So this is the effort we believe we can successful execute this quarter. In other words, we don’t think we can finish the paper this quarter.

Again, two details of this framework that may not have come through:

  • execution success cannot rely on external dependencies. It would be a “strategic success indicator” to be published in a peer reviewed journal. (This is just a quibble, and I agree that should be a strategic success we aim for.)
  • these fields are scoped to one quarter and we don’t believe the paper will be complete in a quarter.

This is definitely important future work for Scalability 2021 and Halo in future quarters, and definitely the kind of success criteria we’ll probably use in the 6-24 month range.

Agreed. Again, during this quarter we are only focused on developing Halo, without much progress on general scalability. It remains future work to research how scalability designs could leverage Halo (or other recent research results).

I think you’re right, so maybe we should alter or drop that strategic success indicator.

Thanks for the thoughtful responses! I hope this response helps clarify any confusion you had, and also that it helps us agree on what ways ECC can improve our strategic planning in future quarters. We intend to iterate on this framework each quarter, so when you see our next one, let us know if we’re on the right track.

6 Likes