ECC not requiring User-Defined Asset precursor support in Orchard

Technology
1

ECC’s recently-announced draft Orchard requirements, which guide its design of the proposed protocol upgrade, include this explicit non-requirement:

NonR2. User-Defined Asset Precursor Support

Non-Requirement: The protocol does not require precursor support for a future User-Defined Assets feature.

Rationale - Technical Strategy: Getting precursor support right requires certainty about a subset of UDA requirements, and blocking Pollard on clarifying future UDA requirements introduces more deployment & execution risk.

To recall, UDA (User-Defined Asset) support means letting transactions on the Zcash blockchain convey various distinct assets, beyond than just ZEC. For example, we would have “shielded BTC”, shielded USDC", etc., all benefiting from Zcash’s privacy.

By analogy, consider ERC-20 tokens, and things like wrapped Bitcoin represented on the Ethereum blockchain. Except that wrapping assets on Ethereum is done for the smart-contract functionality or interoperability, while UDAs on Zcash would be used to attain privacy, fungibility and censorship-resistance of a multitude of assets for a potentially huge userbase. And analogously to Ethereum, UDA transactions on Zcash would still be paying their fees in ZEC.

Strikingly, we already know how to add UDA support to the existing Zcash Sapling protocol! ECC has already figured it out and even prototyped it! We could have added it with a network upgrade that’s much simpler than Orchard.

But the above non-requirement means rejecting or delaying UDAs for potentially a very long time. It’s not just wasting this opportunity to deploy UDAs. It’s also refusing to plan ahead and design for the addition. In particular, it means being fine with the possibility that adding User-Defined Assets will require another incompatible major upgrade, with another address format change and another pool migration, even if such disruption can be easily avoid by “precursor” support.

My opinion: this is a grave mistake. Even if we can’t usably deploy UDAs right away with Orchard deployment, we should prepare the ground for UDA support in the protocol. This is an important investment with potentially huge payoff (namely avoiding the above disruption and its associated costs and delays).

If there are specific downsides to planning ahead for UDAs and including whatever precursor support is needed in Orchard to minimize disruption, then these downsides should be openly discussed. And let’s see if the community can help in mitigating them.

Let’s discuss?

Edit: This topic was originally titled “ECC rejecting User-Defined Asset precursor support in Orchard”. Changed after the intended semantics and technical facts became clearer.

10 Likes
2

I’m disappointed that you went with a particularly-inflammatory topic title, while omitting the leading paragraph in the non-requirements section:

This section documents potential requirements that are explicitly not required, and their implementation is up to the product and engineering teams best judgement.

To be crystal clear, “non-requirement” means “this is not required”; it does not mean “we will not do this” or “this is rejected”.

Speaking for myself: I’ve been thinking about how to include UDAs in Zcash for a long time, and I’m confident that even if we omitted UDAs from NU5, adding them later would not require any changes to the address format or a pool migration.

The only concern I have is about how notes commit to their asset types; I think it might be necessary to include an asset type field in the note structure from the start (so that we don’t have to handle two types of commitments in the same global commitment tree). That does require deciding two things now:

  • How an asset type is used. At this point I’m reasonably confident it can be treated as opaque bytes inside the circuit.
  • How an asset type is represented. It would be convenient for the asset type to be a field element, but that places some restrictions on how it might be generated, vs it just being an opaque byte string.

But even that might not be necessary; I believe @daira has thoughts on how both commitment structures could be handled inside the circuit.

10 Likes
3

I agree! Why would anyone set this priority without discussing with the community?

My personal perspective is that Halo is a great improvement but by itself it doesn’t add any new features that end-users care about. Whereas UDAs are a thing that some members of the community have expressed support for. It would be incredible to have a private stablecoin on Zcash, or have Zcash be a pathway for other assets to use with fees paid in ZEC.

As a user I would much rather see ECC take a little extra time to plan for a version of Zcash with additional features the community is excited about, rather than race out a version of Zcash that adds no new features at all when Sapling already works pretty well. I hope ECC will consider discussing this with the community that provided them with such generous funding!

10 Likes
4

Thanks for the clarification @str4d. A thing I’ve noticed lately is that ECC has largely withdrawn from the rest of the technical community and no longer communicates the details of what it’s doing, or the decision process that leads to its requirements [1]. In that environment it’s surprising to see you be upset about a misunderstanding. Of course people aren’t going to understand your notes when you aren’t discussing them!

Come back to the forums at least and discuss what’s going on here. A one hour arborist call every few weeks is not a good process. This community is funding ECC generously and deserves more information.

[1] As an example, the Foundation recently held a technical meeting to discuss the Halo upgrade and not one ECC engineer showed up. What was that about?

8 Likes
5

@str4d, I know that you and @daira and @ebfull have been thinking and working on this, I understand your technical points, and I fully believe that if this was up to your discretion, we would have had excellent UDA support.

But I read this non-requirement as saying that your technical ability to execute on this will not be allowed to be expressed in the actual protocol, and will be squashed by ECC management just like your Sapling UDA work.

More verbosely: the rejection of UDAs has been the consistent party line of ECC’s management for a long time, including recent forum posts that raised objections which seem absurd to me (and did not bother to answer the criticism). This official non-requirement is just the latest and clearest facet of the hostility to UDA. The way I read this is: ECC management does not want to you to work on any of technical points you mentioned within the scope of the Orchard design, and does not want to pay any costs (however small) in protocol design, implementation or evaluation, to facilitate future UDA support.

The “inflammatory topic title” reflects this understanding based on all available evidence. I hope to be proven wrong.

6 Likes
6

The design we discussed with Sean back before Sapling was to add another base to the Pedersen commitment and have the asset ID be a field element. That means all existing Sapling/Orchard field elements could have default ID “0” with no changes. Of course the thinking (all those years ago!) was that we’d be adding UDA support to Sapling. But now we’re discussing an entire new commitment structure (new elliptic curves) and we should be able to make UDA support a part of that upgrade from the start! Except this requirement kind of makes that difficult. Arg.

A 256-bit field element could encode a 255-bit hash. And it seems like a 1-bit zero prepended hash of the minting transaction is a great way to label asset IDs.

2 Likes
7

If that is indeed the case (ie: that ECC simply does not want to have their core team spend thier time/effort to support this feature) is there anything ZF or ZOMG could do to help offset those time and expense costs? Something along the lines of supporting a developer or team to work with the ECC on the parts that would differ from thier planned work?

1 Like
8

The gave an explanation here, that they needed more time to collect info to respond to the questions raised on the previous call:

I’m not sure if those answers are part of this requirements update or if still need to wait til the next call to get those responses

1 Like
9

ECC has largely withdrawn from the rest of the technical community and no longer communicates the details of what it’s doing
…
Come back to the forums at least and discuss what’s going on here.

I speak only for myself when I say I am a bit offended by this framing.

We’ve never met yet I look up to you. When I started at ECC, all I knew was you were the mind behind the compact block model for light clients that @str4d, @gtank and I were trying to get to work on mobile. Today, that work underpins every light client from Zbay and Zecwallet to Nighthawk and Unstoppable. I thank @Matthewdgreen for that.

At ECC, everything I do is public. Sometimes, painfully so. All of my code is open-source, even before I feel it is polished enough to be viewed (imagine having your favorite paper be fully public the entire time you draft it). Even my sprint boards are in public on Github. I proactively make efforts to communicate and connect with the technical community in numerous ways that would take too long to list. Beyond that, I’m available on multiple discords, multiple Keybase channels, Telegram channels, Slack, the forums, community calls, newsletters, I even reactivated my Twitter, where I hadn’t posted since 2015. Additionally, I just had a 1-on-1 call with a member of the “technical community” on Friday (our 3rd this month) and that resulted in a PR that I reviewed a bit after midnight. Now, I’m up passed 2am catching up on the Forums. On the weekend.

Meanwhile, my friends who work at high-profile mobile companies have over 30 Android developers on one app. I’m an Android development team of one. And none of this is even mentioning my excellent co-workers and all the hard work they do and their efforts to communicate details.

Given all this context, when I read your message I felt deflated. A recent post suggests you’ve been away for 7 months. Perhaps you haven’t seen all the things that have occurred. After all, 2020 was a crazy year!

So I’ll try to take your sentiment, along with @tromer’s “inflammatory topic title” with a grain of salt (EDIT: the title has been updated. Thanks that really helps!). I get that you two have concerns and they’re more focused on Orchard than wallets.

However, I’m just going to be painfully honest and say that when I work this hard and consistently watch the organization I work for get dumped on by scientists I look up to . . . it stinks. It makes these forums feel toxic, like they’re doing more harm than good.

It feels counterproductive and makes me want to disengage.

(I almost deleted this post twice before pressing submit)

13 Likes
10

It’s seems like a recursive argument though. Scientists that you look up to might not be active but if you tag them, they would respond. I’m not proposing every technical conversation to happen here but it’s a judgement call for ECC to figure whether to do technical discussion on forums or on github (con: non-tech community can’t pitch in for non-tech decisions).

I think you know this wasn’t about the work you did. In fact, others know the work you did was pretty much in public, & it seemed to turn out well! Have you ever thought why the organization you work gets dumped on? Everyone loves all the work ECC devs do. This is a question for ECC leadership (specifically in the context of UDAs).

I don’t think it makes sense for me to take part in this conversation as I’m not a tech community member.

And I’m sorry for any ECC devs that experience similar feeling like you about ECC & community relationship.

Two facts:

  1. We love innovative work done by ECC devs.
  2. Community feels it is not looped in, while ECC makes strategical & important technical decisions.
3 Likes
11

I’m not criticizing their absence. Rather, I’m pointing to that as one of the potential reasons for me to feel less offended.

My main point is that their words carry a lot of weight and contribute to a toxic environment when framed so harshly.

2 Likes
12

Don’t we have mods to take down offensive comments? Toxic environment towards who exactly? I haven’t seen toxic comments towards @str4d @daira @ebfull & others.

Who is going to benefit from creating toxic community?

We are all here for privacy & we spend time in our capacity to enable others to bring private transactions to everyone.

That said, ECC should respond if the Matt & Eran’s understanding is incorrect.

1 Like
13

Can’t imagine why devs would want to put on their hazmat gear and wade into a toxic swamp.

We’re better than that and should try harder, we value them and their time is precious.

10 Likes
14

If your position “do not interfere with doing what we want” coincides with their position, this does not mean that employees who are doing work to achieve the goal that the community defines from their words are not obliged to contact this very community, in other words, who said that their time is precious, for whom is it dear if they cannot even find out what wishes of the community need to be embodied in their product? The time will come and you will also disagree, and since there is no real mechanism of interaction, you cannot do anything but leave. Good luck with blind trust.

15

I wholeheartedly believe the devs do what they do to progress this tech and for the benefit of the project.

There are no bad intentions, actions are in good faith.

It’s fine to disagree and discuss, but enough with the crappy attitudes and insinuations.

4 Likes
16

I don’t know why you say that there’s just a bog, but I don’t see anything wrong with someone asking direct questions to which, as you can see, no one ever gives direct answers, so believe one thing and how can we check? I believe that this situation was not born from scratch and that that is, we just do not know and apparently will not know, but this is already bad for everyone, because as I wrote above there is no mechanism that would allow the community to influence the project.
Regarding what the developers are doing, over the past 2.5 years, as a user and a zec holder, I will not see anything positive from their fruitful work, the last is an improvement in protected evidence for mobile transfers, but a lot of time has passed since then and there are still no massive mobile transfers, which means Is it that efforts for 2.5 years went in the wrong direction since such an important update has radically brought nothing?
You can name something in zcash that changed the situation, I do not, only one perspective in the future and underestimation in the present, the question is why everyone is happy with this, is there a plan, why is it a secret? For real users there have been no changes for a long time, there are small improvements, I do not argue, but why do we lose users while increasing the number of exchanges, why there is no user growth, this is the question to which I do not receive an answer, well, the trust in the project is nowhere lower. Yes, there is a circle of people who trust, but at the same time they do not depend on the project, so the trust of such people is nothing, there are no companies that stake on the project, which means that no one risks money or reputation, and this speaks volumes.

17

@gmale, I’m very sorry that you and @str4d were offended by what I wrote, because my intent was exactly opposite: to support @str4d’s et al.'s efforts in the face of ECC management that is doggedly pushing against this excellent work (at least as communicated in its formal priorities, requirements, flight plans and various public conversations).

When I see @str4d et al. doing amazing work on UDA, that’s so effective and high-quality that it’s already deployed by Tezos, but then ECC management squelches it by secretly deciding that even “precursor support” for this work is explicitly out of scope for the forthcoming network upgrade, meaning Zcash users won’t get UDAs for at least a year… well yes, color my forum post title inflamed.

BTW, I felt very similarly when ECC management failed to support the early mobile wallet work. Very glad that we’re way past that.

Now, I expect that you jumped at the “secretly deciding” wording above. It was public, wasn’t it?! And this brings me to the other recurring point. Let me diverge from the original topic (is there a different forum topic to move this to?), and dwell on that.

ECC’s effective (non-)transparency

ECC lives in a Where’s Waldo world, where much is in view, but little is visible. ECC uses and generates an enormous amount of public documents, spread over many pages over many accounts over many of services and websites. This is fine and admirable. But practically, it means that mere mortals, whose full-time job isn’t tracking these documents, can’t keep track of what’s important. The community can’t effectively discover important new discussions and decisions, without timely and specific cues. So for effective transparency, ECC has to accept the onus of providing such cues and help.

The good news is that this problem is relatively easy to solve. The hard work of creating the materials, and of sticking your neck out on making them public, is already done! In my opinion, all ECC needs to do is spend a bit more time in helping the community figure out where to look, and creating true public conversations about the most important things. My suggestions:

  1. ECC, please create a clear webpage pointing to all the public resources you use for planning, tracking and development. Explain how they fit together and what’s in each. Keep it up to date.
    Lithmus test: where do I go to find out what are the most important technical and strategic decisions currently underway, that I can provide input for?
    ECC folks, this stuff may be obvious to you, but it’s literally your job to keep track of it. Even I, who have closely collaborated with ECC, have been unable to track the evolution of your planning and tracking resources; and last time I asked, I was told that some crucial “overlay” parts that express priorities and links aren’t even public yet and you’re working on that.

  2. ECC, please be clear and explicit about major planning processes and decisions.
    For example, having explicitly-defined requirements for Orchard is excellent. Furthermore, it’s good that @elenita mentioned it in her January 29, 2021 - weekly forum update post…
    But it was buried in that forum post under the irrelevant "Security’ heading, in a cryptically-labelled “Added requirements to ZIP 224 PRD”, linking to a confusingly-rendered Pull Request, where you have to click on “Files Changed” and read all the way to line 168 to see any mention of UDA.
    So yes, the plans were on display.
    IMHO, when ECC made this major decision to not ensure that a major network upgrade is compatible with a widely-requested feature, the right thing to do was to state so clearly and prominently in, say, a dedicated forum post.
    No, strike that. When ECC was going to decide that question, it should have posted prominently and asked for opinions. And this leads me to:

  3. ECC, here’s a handy keyboard shotcut: Shift+/ produces the symbol ?, which is a punctuation mark commonly used to convey so-called “questions”. “Asking questions” is a form of communication where you seek information from the counterparty to a conversation. By using the keyboard shortcut, you can convey that you wish to receive such information, that you welcome and respect your counterparty’s view, and that your intend to take into consideration the information and needs that your countrparty expresses.
    A downside of “questions” is that receiving and responding to your counterparty’s answers may entail further effort. This can get especially burdensome if your counterparty is also interested in understanding your own view, and may thus ask some questions in return.
    However, this burden should be balanced against the alternative, which is doing whatever you feel like without asking the people affected. This creates the risk of doing the wrong thing, and even worse: dealing with “inflammatory” forum post topics when people find out.

Frustratingly, none of this is new. On numerous analogous occasions, I’ve said similar things to every single person at ECC who has the power to address this communication issue. Some progress has been made, such as @elenita’s excellent updates, but not nearly enough.

6 Likes
18

BTW, note that 18 comments into this topic, we’ve had plenty of indignation, and also some great glimpses from @str4d on what we could do if UDA (precursor) support did become an Orchard requirement, but zero discussion of the actual question at hand: should it?

Correction: @Matthewdgreen did answer to the point, just before we got derailed by consternation.

3 Likes
19

Adding UDAs does not require an address change or new pool. That’s part of the reason why I personally didn’t push back too hard on it being a non-requirement in the Orchard design: there literally is nothing we need to do in this upgrade to help support it later. I’m not even aware of anything we could get wrong, given the current draft Orchard design, that would make it more difficult to add.

I’m sorry we didn’t communicate that well.

@str4d is correct that a “non-requirement” does not mean that it is being discouraged or prohibited. It means precisely, and only, “not a requirement”. Actually I think there’s quite some enthusiasm for UDAs at ECC, modulo answering some economic questions that are being actively worked on.

18 Likes
20

Well this completely changes the picture. Thank you for the very helpful factual clarification, @daira!

In this light, why not add an explicit requirement to “make it possible to add UDAs in the future without address format changes or creating a new pool”, just to make sure things are clear and do not inadvertently regress?

8 Likes