Hi all. We posted a written version of the flight plan Nathan presented on the livestream. We’d welcome your thoughts!
4 Likes
Thanks @nathan-at-least, and everyone who supported him, for preparing this plan! It’s solid, well-thought-out, well-presented and the short-term objectives are extremely compelling. I particularly like the multi-scale approach, the rationale given for every objective, and the explicit success indicators.
Reading through this, I have some comments/suggestions/questions:
While Horizon 1 is beautifully worked out, Horizons 2 and 3 are very blurry. In particular, the discussion of Horizon 2 (which starts just 6 months from now) is mostly about what other teams will do, and says little about ECC’s objectives and tasks during these 18 months. Especially with the Dev Fund question on the table, I think more clarity is needed on what ECC will be up to in the upcoming years.
Where do user-defined assets and programmability fit into the picture? They’re clearly excluded from Horizon 1, but are they envisioned to happen later on? When does ECC even start researching them in earnest?
It would have been helpful to get some quantitative indication of the quantity of effort each objective requires, especially as one balances these objectives against other potential ones.
Likewise, it would be helpful to explicitly call our risk factors, such as “research required” or external dependencies.
On Mobile wallet SDK improvements, it’s unclear what’s the scope of the mobile wallet SDK, and what functionality still needs to be developed.
I love the plan to develop a functional “dogfood” shielded ZEC wallet! We learned so much from the Zcon1 prototype, and it feels that doing a real mainnet wallet is going to be huge, both in stress-testing the SDK functionality and in getting feedback on shielded tech usability.
On Viewing key support: yay!
What about the per-transaction payment disclosures (as we used to have in Sprout, in addition to the per-private-key viewing keys)?
On Community-driven protocol development, you say:
- Zcash Foundation and at least two of the four other participating development organizations publicly endorse the new process
I’m curious, who are “the four other participating development organizations”?
- ECC publishes a new version of the Halo paper with optimizations and performance benchmarks
How about finishing the Halo paper by making clear security claims, and providing security proofs? And adding missing features like zero knowledge?
To the execution success criteria, I would add “the Halo paper is accepted for publication in a high-tier peer-reviewed academic venue”.
Also, what about research and communication on how to use Halo in Zcash? We’ve seen how Halo can be used to verify PoW (which is a relatively easy application which doesn’t even need ZK). But AFAIK nothing’s been said about how Halo would be used to verify transactions. Because Halo not succinct in the efficient-verifier sense, it’s not a drop-in replacement zk-SNARK; rather, it would need some new application-layer protocols to exploit its amortized verification using recursive composition.
Similarly, without pertinent application-level protocols, we don’t know if Halo is helpful for achieving scalability. It sure feels it is, because we know from Coda that recursive compositions helps many aspects of scalability, but more concrete planning is needed here.
Also on Scalability 2021, this strategic success indicator worries me:
- Zcash’s Scalability 2021 effort is referenced in one or more high-quality industry reports about scalability
Being referenced in a report is not a good goal to set for the engineering flight plan. To put it bluntly, if all that’s done for Scalability 2021 by the end of Horizon 1 is what you set out to do as the execution success criteria, i.e., you never get around to designing the application-level-protocol connection between Halo and scalable Zcash, then no one should believe you’ve made any progress towards Scalability 2021. So if some reporter or industry evaluator is nonetheless fooled into thinking otherwise, I don’t consider it an indication of success for the engineering flight plan.
Well that’s all for now… Thanks again for the deep thinking and excellent presentation, and for pushing the bar on technical plan transparency in our ecosystem. Looking forward to the ongoing discussion!
4 Likes
Hi @tromer, thanks for the detailed response! It helps us improve and refine our plans. There’s a lot here and I’ll strive to respond to each point as well as I can in the moment.
It may not be clear, and I think some of this was lost during editing, but I intended the “Three Horizons” section to not describe ECC’s plans. Instead it is intended to provide our belief about a plausible state of the world with respect for Zcash in those time ranges. This may contribute to it sounding like “it’s a bunch of work others will do” because others doing a lot is a key aspect of our belief about a successful Zcash trajectory.
If we move onto our concrete Objectives and Key Results, that represents active goals for ECC.
With that in mind, how can we improve (a) the “visions” (ie. predictions of the future)?
-and how can we improve (b) the plans ECC has during those time horizons?
I’m curious to hear your responses or other on these forums. Here are my answers:
For (a), improving our plans around predictions:
- Keep adding more specific detail so that the predictions are falsifiable and we can learn over time our accuracy at prediction.
- More importantly, if a prediction is shown to be inaccurate, as soon as we learn that, we should adjust our plans accordingly.
- If other Zcash contributors have substantially different visions or predictions, the sooner that’s discussed publicly the better. This is a way for ECC to document our beliefs so that people understand our rationales.
For (b), I would say it’s always easier to add new goals, especially with shorter time horizons, at a later time than it is to pre-commit to long range goals and then shift them in-situ.
Let’s drill into your next question as a case study:
It’s a bit difficult for me to figure out the best way for me to respond to this.
What I would prefer moving forward is for you, or others, to advocate for whichever vision you seem to want to advocate for here. To me, it sounds like you have these beliefs just based on the quote immediately above:
- User-defined assets and programmability should be considered together.
- They are excluded from horizon 1.
- ECC will start researching them in earnest but hasn’t announced when.
I disagree with every one of those assumptions. (Also, I may be misinterpreting you, and even if I wasn’t I would definitely want to hear your thinking behind them.)
I have a call to action for the whole Zcash community which is for contributors to spell out their own visions for where Zcash could go, especially if it diverges from ECC.
Ok, with that in mind, I hear you that you want more details specifically about ECC’s plans, so:
On programmability: We are working on programmability in Horizon 1! It’s a feature called Whitelisted Transparent Programs. (I kind of wish we had a different name, perhaps “Zcash extensions”.) One way I think of this feature is akin to Ethereum’s precompiles. They are like “hard-coded smart contracts”, and I believe their expressiveness is broader than Bitcoin Script (although the details aren’t well specified enough yet to know).
The reason we’re starting our foray into programmability with this “hard-coded / whitelisted” appoach is to that we have an explicit and clear understanding of the programmability use cases. Also, by keeping the functionality fixed and only changeable with network upgrades, it’s easier to reason about consensus guarantees, scalability, and privacy. In the future if we have a set of several WTP extensions, with known usage/value, we can focus on how to add privacy or scalability while meeting those specific uses.
A key belief of mine is that by restricting the scope of Zcash to ZEC without adding too much complexity like general purpose smart contracts, it will simplify the problem of layer one scalability.
When thinking about programmability, I think it’s especially important to understand existing and potential use cases. We already have Bitcoin Script in t-addrs, and here are some interesting empirical questions about use case:
- What use cases are enabled by Bitcoin Script?
- What use cases are seen actively deployed which rely on Bitcoin Script? In Zcash? In Bitcoin? Elsewhere?
- Of the actively deployed use cases, how much “economic activity” relies on those use cases?
- How do all of the above questions apply to Ethereum’s smart contract design, or any other similar programmability?
I don’t have solid answers for much of this, but @str4d has done some excellent work on empirically investigating Script usage in Zcash, which you can see here.
Now turning my attention to user defined assets:
It’s clear to me there is something important with user defined assets. What is less clear to me is which parts of that are good for ZEC. Two anecdotal data points:
- Ethereum + ERC20 interface has lead to Ethereum accruing many tokens atop its platform.
- Omni has provided something like user defined assets for ages atop Bitcoin.
It’s not clear to me how these platforms support the “native asset” utility or value. The Omni token is worthless. Meanwhile Tether relied on Omni to carry billions of dollars-worth-of-usdt for quite some time.
Meanwhile, we also see tokens migrating off of their “birth” platform, such as Tether volume on Ethereum surpassing volume on Omni (BTC) as per tether’s transparency page, or several projects migrating off of Ethereum.
Counterbalancing those concerns, all such user-defined-assets-on-Zcash could benefit from having a conjoined privacy set, so this may be a unique value proposition that the previous examples lacked.
All this said, from my point of view user defined assets is a potentially valuable feature that I do not see as a given for Zcash’s future. If we see a compelling vision or supporting data or anecdotes about how this feature could benefit Zcash and ZEC, that may change.
I should also note that we already have a key piece of user defined assets (the shielded transfer circuit) designed and implemented in a prototype. Additionally, I believe for many issuance use cases, WTPs will provide a useful building block to deploy a user defined asset implementation with a relatively low amount of effort.
These are two excellent ideas and I’ll strive to include those in our strategic update next quarter.
Quantity of effort delineated in an implicit/nonexplicit way: we believe the goals will be achieved in the horizons given assuming ECC product engineering is focused at almost full capacity with our current team. I agree that breaking down effort estimates per goal could be helpful. (However, I caution that such estimates drop off in accuracy as the time horizons grow larger.)
There is one part of our framework that does identify external dependencies, because we distinguish between “Execution Success Criteria” and “Strategic Success Indicators” where the former are defined to exclude external dependencies. However, I agree that a general assessment of risk factors would be an improvement.
At a very high level, the SDK needs improvement in these areas: t-addr support (and interaction with z-addrs), iOS support, and “production readiness” hardening (better rollback protection, some security improvements, easier deployment).
Exactly! We wanted to continue that kind of full-stack learning that helped us see where some parts of the design are challenged or incomplete when it comes to full end-to-end deployment. We’ll be using this dogfood wallet with small invite only groups for usability testing and also potentially to prototype new end-user facing features.
We do intend to include payment disclosures, maybe in Horizon 1 but likely in Horizon 2. They were not mentioned in the blog post because I expect for most readers they don’t distinguish Viewing Key Support from Payment Disclosures.
If you are reading along and didn’t realize these were different: Viewing Key’s allow the holder to see incoming and outgoing transaction details for an associated address. Payment Disclosures reveal information about a specific transfer only.
I apologize that this got muddled during late editing. An earlier draft (which was way too long) mentioned that as of today there are at least four other organizations involved in Zcash protocol R&D outside of ECC + Zfnd: Keep, Summa, Parity, and Bolt Labs. So in this context, the strategic success criteria is that at least two of those organizations endorse the new process.
I also apologize for a bit of editing loss here: the “Effort”, “Execution Success Criteria”, and “Strategic Success Indicators” are all scoped (by definition of the framework) to one quarter. So this is the effort we believe we can successful execute this quarter. In other words, we don’t think we can finish the paper this quarter.
[Edit by @Daira: but in fact we did. It’s available as an ePrint, and it includes clear security claims and proofs, with zero knowledge support.]
Again, two details of this framework that may not have come through:
- execution success cannot rely on external dependencies. It would be a “strategic success indicator” to be published in a peer reviewed journal. (This is just a quibble, and I agree that should be a strategic success we aim for.)
- these fields are scoped to one quarter and we don’t believe the paper will be complete in a quarter.
This is definitely important future work for Scalability 2021 and Halo in future quarters, and definitely the kind of success criteria we’ll probably use in the 6-24 month range.
Agreed. Again, during this quarter we are only focused on developing Halo, without much progress on general scalability. It remains future work to research how scalability designs could leverage Halo (or other recent research results).
I think you’re right, so maybe we should alter or drop that strategic success indicator.
Thanks for the thoughtful responses! I hope this response helps clarify any confusion you had, and also that it helps us agree on what ways ECC can improve our strategic planning in future quarters. We intend to iterate on this framework each quarter, so when you see our next one, let us know if we’re on the right track.
7 Likes
Nathan, thanks for the well-thought out response! Also, sorry for the delayed response in answering. (I was prioritizing my attention towards the Dev Fund discussion…)
Longer-term planning
I feel there’s some discrepancy between the intent, content and positioning of the document.
It’s suggestively titled “ECC R&D and Engineering Flight Plan for 2020” and supposedly extends 4 years out, but substantial detail is given only for “Horizon 1” which ends in May 2020, and moreover
So it seems like the flight plan runs out of waypoints in a few months…
In particular, it doesn’t answer the question on everybody’s mind, “what would ECC actually do using a four-year Dev Fund”, though to be fair, you didn’t claim to answer that here.
[Regarding] our concrete Objectives and Key Results, that represents active goals for ECC […] how can we improve (b) the plans ECC has during those time horizons? […]
I would say it’s always easier to add new goals, especially with shorter time horizons, at a later time than it is to pre-commit to long range goals and then shift them in-situ.
Of course it’s easier to not plan ahead, but it’s probably not optimal. I definitely want to hear more about the goals that ECC set for itself in the upcoming years.
That’s necessary in order to understand how the visions you’ve outlined for Zcash are supposed to come to fruition. Of course things may change as we learn more, and it would be foolish to irrevocably commit to a fine-grained plan. None the less: on the “nominal” execution path, if all goes as intended, what does ECC do? Which part of the vision ECC will not do, and it’s up to others to show up and do that work? What dependencies are there between these?
This is crucial to scheduling, understanding if the goals are even achievable in terms of resources and dependencies, and (with the Dev Fund in mind) seeing whether ECC’s plans reasonably match its expected level of Dev Fund funding.
Moreover, the prospective non-ECC participants need to know what tasks are “up for grabs”, and so do the folks at ZF who are encouraging and funding grant proposals.
User-defined assets and programmability
I agree that these are separate questions, but there are many technical and resource-contention relations between them, so yes, it seems best to consider them together.
They are excluded from horizon 1.
[…]
We are working on programmability in Horizon 1! It’s a feature called Whitelisted Transparent Programs
I’m asking about adding shielded programmability, not about WTP’s restricted transparent programmability. (And the latter will, hopefully, be eventually obsoleted by deprecation of t-addresses…)
But I understand and like what you said about studying concrete use cases of scripting. This will inform the decision and design of shielded programmability.
Above you also said many wise and true things about user-defined assets. I agree with the concerns,
but the bottom line seems to be: for now ECC is not seriously working on user-defined assets, or even studying the needs and potential design. So user-defined assets are just not happening for Zcash, unless and until ECC changes its mind, or someone else implements them and convinces ECC to merge it despite your reservations.
Frankly I find this very disappointing. Privacy-preserving user-defined assets have a huge potential demand and impact, and are very much within ECC’s core focus and capabilities. It’s especially frustrating when so much attention is given to the opposite direction (“wrapped ZEC” on non-private chains) which, in a sense, weakens the privacy of using ZEC.
Wallet SDK
At a very high level, the SDK needs improvement in these areas: t-addr support (and interaction with z-addrs), iOS support, and “production readiness” hardening (better rollback protection, some security improvements, easier deployment).
So in terms of functionality and API, the wallet SDK is considered feature-complete, other than t-address?
I was imagining a long list of feature requests and plans for important missing functional functionality, but maybe it’s farther along than I think?
Payment disclosures
We do intend to include payment disclosures, maybe in Horizon 1 but likely in Horizon 2. They were not mentioned in the blog post because I expect for most readers they don’t distinguish Viewing Key Support from Payment Disclosures.
OK… Frankly I’m concerned here, because the conflation and confusion seems to run deeper than a simplified blog post narrative. As a cryptographer, I’ve been trying to keep track of ECC’s plans and GitHub issues concerning the various selective disclosure features, and there have been repeated omissions of payment disclosure and of cryptographic definitional+proving+auditing+communicating work (thanks, @nathan-at-least, for your help resolving some of those!). I’m also seeing no discussion of how users will conduct common workflows that in t-address land rely on sending txids, and in the z-address equivalent would require payment disclosures. So to me, the omission from the flight plan is yet another signal that much of the selective disclosure work is underestimated and underprioritized. Am I just reading too much into it? Are there more detailed discussions elsewhere?
Halo
Also, what about research and communication on how to use Halo in Zcash? We’ve seen how Halo can be used to verify PoW (which is a relatively easy application which doesn’t even need ZK). But AFAIK nothing’s been said about how Halo would be used to verify transactions . Because Halo not succinct in the efficient-verifier sense, it’s not a drop-in replacement zk-SNARK; rather, it would need some new application-layer protocols to exploit its amortized verification using recursive composition.
This is definitely important future work for Scalability 2021 and Halo in future quarters, and definitely the kind of success criteria we’ll probably use in the 6-24 month range.
The above “how to use Halo in Zcash” questions are presumably a crucial part of the Halo plan. I dare say: they’re the only thing that makes Halo at all relevant to ECC’s flight plan. So if your flight plan methodology has no place to track (or even mention) this work and execution risk, then perhaps it’s indicative of a flaw in the methodology?
Stepping back
@nathan-at-least, thanks again for your thoughtful answers (including the ones I haven’t mentioned here because they’re completely satisfying to me). I understand that the flight plan is itself an experiment-in-progress, like everything we do! And I think the perspective you’ve shared, and planning you’ve made, already qualify this as a success-in-progress. I look forward to future iterations!
4 Likes
ECC’s @steven-ecc has posted the ECC Engineering flight plan: Mid-horizon update.
Steven, are your horizon timeframes defined with respect to the same starting point as the original flight plan (22 Nov 2019), or counting a new from now (3 Mar 2020)?
Also, with respect to the original flight plan, @nathan-at-least wrote above:
“Execution Success Criteria”, and “Strategic Success Indicators” are all scoped (by definition of the framework) to one quarter. So this is the effort we believe we can successful execute this quarter.
With that quarter having passed, can you please speak of the outcomes of these Execution Success Criteria and Strategic Success Indicators?
Are there explicit criteria and indicators for the current quarter?
5 Likes
I join the question , I would also like to know the fate of the fifth point, which does not participate in the updated plan (Ethereum protocol improvements for Zcash](ECC R&D and Engineering Flight Plan for 2020 - Electric Coin Company)
As I understand it, the cash network must now be prepared , because everything is ready in eth, but why is there no coverage of plans and ideas for implementing such an important development of the network?
1 Like
Have you received feedback on your questions?
Do you have any information on the integration of zec in DeFi?
2 Likes