ZIP: Reduce default Shielded Transaction fee to 1000 zats

@secparam and @aiyadt there’s a lot of back and forth here!

I think we can arrive at a conclusion you’re both on board with, and I hope we can because this matters to the work I’m doing on Zbay.

It sounds like there are two parallel arguments. One argument seems to be whether Zcash can and should have a privacy-friendly fee market. @aiyadt I think it makes sense to concede to @secparam that this can be done and there are good reasons for doing it, and move on.

I say that because as @secparam points out, one can imagine schemes where fees vary but all users choose fees in “the same” way such that they aren’t revealing anything about themselves. (I believe Ethereum’s EIP-1559 achieves this or could be modified to achieve it.) And because once blocks fill up, such a system will be necessary, or people will have to choose between a) choosing the default fee and not having their transaction mined and b) choosing their own fee and not having privacy, which is a failure either way. Sure, that hasn’t happened yet, but it could soon in the case of a deliberate attack and it will happen relatively soon if we’re successful in driving Zcash adoption.

Once we concede to @secparam that a private scheme for flexible fees is both possible and eventually necessary, I think we can have a separate conversation about whether there is any reason to not lower fees 10x in the meantime when the flexible-fee solution hasn’t been implemented yet.

Then we can have a separate conversation about whether we should lower them 100x or more, since whatever reasons there are for not lowering fees 10x are probably also reasons for not lowering fees 100x. Does that make sense?

3 Likes

In this post I’ll focus on the question of whether there are any reasons to not lower the fees 10x in the meantime.

First, I think we can agree that just because a flexible-fee solution would be ideal (and will probably sooner or later be necessary) that is not a reason to not lower fees before such a solution exists. Changing the default fee is a simple matter of altering adding a “0” to one line of code in a still small number of user-facing applications, and in a few places in the Zcash docs. There’s some coordination cost, mostly in discussion, but once we decide there’s a really clear path to doing it. Changing the fee now will not take developer time away from any work to achieve a more permanent solution. If it adds value to users and developers, we can add that value now, and when the other solution arrives, that will add more value.

So the question is whether there are other reasons.

Bad for miners? @_eric points out that shielded transaction fees are negligible to miners and this matches what I’ve heard from a leading miner, Luxor. So it’s not bad for miners.

Bad because it increases DoS vulnerability? @secparam notes the effect on the cost of a DoS attack on Zcash.

First, my understanding is that there are currently other DoS vectors than simply generating large numbers of shielded transactions with the minimum fee, and that some of these are not dependent on the default transaction fee or barely dependent on the fee, such as the attack @Shawn mentioned above where you create very large transactions or transactions with many outputs.

Second, even if the fee is the constraint on DoS attacks, a 10x reduction in fee would result in a max 10x reduction in the cost of a DoS attack, correct? And in practice the increase in the ease of running a DoS attack will be much less than 10x because there are some fixed costs involved (knowing how, bothering to, etc.) On that scale, even if there is an increase in the ease of DoS attacks, it seems worth it for increasing shielded adoption.

Because it should be reduced more than 10x? On the question of whether the decrease in fee should be more than 10x, I’ll leave that to others, but I don’t think this question should become a blocker to reducing the fee 10x. If we agree on reducing it, but can’t agree about reducing it more than 10x, we should reduce it 10x.

Pointless because Zcash already has woefully incomplete network-layer privacy? @secparam you’d agree that this question of whether to decrease the default fee is orthogonal to the existing issues with network layer privacy, right? Work is happening on those other fronts, but we need to take each of these things on discretely, and this is low hanging fruit, especially since the downside of not changing the fee is data leakage on the chain itself for users who need lower fees for a use case, and not just at the network layer, which is extra bad.

Of these questions, DoS vulnerability seems like the least-easily-settled question. I’d recommend that we consider the others settled and focus all our energy on this question, and I hope/expect we’ll conclude that it’s not such a big deal!

3 Likes

Very well put @holmesworcester

@secparam I agree that improvements around fee calculations must be undertaken, only with proper ZIPs around those ideas, in a separate discussion. I do not want to block the low hanging fruit of reducing the default transaction fee by 10x

And I invite protocol/node devs to provide their perspectives on the pros/cons of reducing the transactions fees.

1 Like

@daira you proposed lowering the fee earlier in #2863. How do you think lowering the fee 10x will impact DoS vulnerability? Do you have any objections to lowering the fee?

I agree with most of what you said.

However, Zcash has two paradoxically serious problems 1) we bike shed over perfect and complex solutions 2) we then incidentally pick clearly sub optimal outcomes with no clear path to improvement and that clearly don’t work long term.

The fee thing is one of those. It’s clearly a problem , though some pointedly refuse to acknowledge it and the market realities of block space even with massive scale.

I want to start putting us on paths that we can improve on over time. So in my mind, its preferable to have floating fees with a hacked together universal estimator (which might even just be a suggestion oracle which defaults to @aiyadt’s lower fee unless there’s a dos — which may be unlikely) then go down a dead end.

1 Like

In fact the more I think about this, the more I see it as a good idea.

Tack on to e.g. the DNS seeder/peer discover mechanism a fee suggester/oracle. By social contract it will just be @aiyadt’s fee proposal (actually maybe make it change slightly periodically to test that people are using it). But if there is a DOS problem we can the return it to the original fee.
And then later we can make it pick fees via e.g. EIP-1559 as you suggest. And then after that we can move it into actual consensus at some point

it gets us progress towards an end we need AND mitigates the one downslide of lower fees: cheeper DOS that really is cheap enough it might attract trolls. Remember, the other DoS vectors require some skill and effort to get code to do them. Just scripting sending TXs to yourself really does not.

So in my mind, its preferable to have floating fees with a hacked together universal estimator (which might even just be a suggestion oracle which defaults to @aiyadt’s lower fee unless there’s a dos — which may be unlikely) then go down a dead end.

@secparam I hear what you’re saying overall re: paths to dead ends, but how is the fee change a dead end?

To me, it seems like a very small step towards what you proposed. That is, if we change the fee, and then have to again in the future, it supports the case that we need some more built-in system for changing the fee.

Tack on to e.g. the DNS seeder/peer discover mechanism a fee suggester/oracle. By social contract it will just be @aiyadt’s fee proposal (actually maybe make it change slightly periodically to test that people are using it).

Well, any decision about how adequate this is as a substitute approach needs to be made with full awareness of what the existing development priorities are, where this fits in, and what it displaces.

The advantage of saying “The new fee is hereby 1000 zat” is that it doesn’t displace other development priorities, and that’s a real advantage in practice when it comes to addressing the issue. I think that’s another reason to see the simple fee change as a step on a path, and not a dead end.

3 Likes

Also, re: your idea of an oracle, doesn’t this open up the whole network privacy can of worms?

Like, if the oracle can respond in a special way to a certain IP address, it can fingerprint people, right? Or is that already possible with the peer discover mechanism to the point where it doesn’t matter?

2 Likes

Miners will pick the highest-fee transactions, which will naturally lead to transaction prioritization.

From the miner’s point of view, we’re incentivized to pack as much fee as possible into the block. As blocks start to fill up, it becomes more important to prioritize high-fee transactions.

Currently, fees are so low that we can usually clear the whole mempool every block.

2 Likes

Yep, you already trust the p2p distributor not to give you a unique set of peers that IDs you. And thats harder to verify than the fee estimator is correct.

And the way we end up in dead ends is by removing pain points in the wrong way. And so slowly but surely, we remove anything that will make us turn off of the path we are on. And so inertia caries us along it until the path actually dead ends. This isn’t about technology, its a social thing about how we deal with problems and what gets attention.

Hope we never get to the point where blocks are consistently full. We should look for scaling solutions or increase the block size if we get towards fuller blocks. But that is out of scope for this ZIP.

How large of a mempool can Zcash network handle today?

I think it’ll be important to talk about these issues in terms specific to this question, and then create some space for talking about the broader issue within the teams leading Zcash engineering, because otherwise it will be really hard to make any case for anything based on its merits, in isolation, and it’ll be really hard for people outside the teams leading Zcash engineering to participate in the process.

So, specifically, how does changing the fee right now carry us down a path that eventually dead ends?

Another way of saying that is, if we agree that an oracle or EIP-1559 is a good solution, how does changing the default fee right now prevent us from later taking a step towards an oracle or EIP-1559?

@nitronick600, what you’re saying is that once blocks fill up the highest fee transactions will get included, right? Once that happens, it seems like it will be a very easy decision to take the step towards the oracle idea, or EIP-1559, because we’ll need a privacy-preserving way to dynamically set fees given the scarcity of space in each block. This doesn’t seem like a dead end path?

2 Likes

Is the goal to increase usability? When I use Zcash to buy stuff, I’ve never once been dissuaded by the 0.0001 ZEC fee. It’s negligible. I believe Trezor doesn’t play along and sets an even lower default fee of 0.00000226. That’s even more negligible. Will usability increase from adding a 0 to the tx fee, or from a fee market?

Is there merit to the Bitcoin Cash solution of minimal fees with big blocks?

It’s very low for purchases, since it’s likely to be a very low percentage of what you’re spending.

But for sending memos only, the fee is 100% of the spend. Memos are a popular use case and enable things like public censorship-resistant messages (Zboard, Zbay) username registrations (Zbay) etc.

Even traditional wallets like Zecwallet see a ton of users using memos.

Lower fees work, yes. But the default fee is special, since only the default fee will offer users the full privacy benefits of shielded transactions.

The fee is public, so if you use a nonstandard fee for shielded transactions, your transactions stick out and become much easier to link.

So that’s why several wallet developers are interested in lowering the default fee at this time. There are use cases where the fee is significant, we want to lower it, and we can only do so together if we want to protect privacy.

9 Likes

Hey folks, I wanted to chime in here to try to clarify some points:

ZIP scope

This ZIP, as I understood it, is scoped to establishing a new “conventional” fee, rather than propose changes to the protocol or wallet fee mechanisms. Is that correct @aiyadt?

This ZIP doesn’t change anything about the existing “Zcash conventional fee” approach except the number. I think two good next steps for the bigger scoped question of “how can we improve fee mechanism in Zcash” would be to have a thread with that topic and it could have a running list of proposals (ZIPs, tickets, forum comment links). (I’m too busy currently to set that up myself.)

I think clarifications could help this conversation:

Protocol

First, let’s clarify the existing protocol. As always, I may have mistaken understandings, so anyone else please jump in with corrections.

The Zcash protocol is identical to Bitcoin (pre-segwit) with respect to fees AFAIK. Miners can pick any transactions they want to that follow consensus rules, and they collect the fees. That’s it.

Therefore, there is the same kind of “fee market” as in Bitcoin, and this has the same characteristics in the face of Denial-of-Service attacks and other blocks-are-full conditions. I happen to believe this is a terrible design which exacerbated the block size debate in Bitcoin, but never-the-less that’s what we’ve got for now. (We have one improvement: transaction expiry. This helps avoid “stuck transaction hell” because wallets can tell users “if your txn didn’t get mined by $T o’clock is is guaranteed to never get mined” so they can decide if they want to try again, unlike Bitcoin where all they can do is try to bump up fees with child-pays-for-parent shenanigans.)

When blocks are not full (which has always been the case so far in Zcash, AFAIK), then strictly-rationale-in-financial-terms miners should select every transaction they can find which pays them just enough to cover the cost of validation for that miner. That’s likely something more than 0, but I don’t know exactly what it is. If we had really good historical mempool analytics we might be able to estimate it empirically.

Denial of Service

AFAIK this proposal doesn’t affect DOS at all, because it doesn’t change the protocol, it changes conventional wallet behavior.

History

Before launching Zcash, we decided to deviate from Bitcoin’s design minimally, on the basis that Bitcoin was (at that time) the only blockchain protocol with a lot of in-the-wild testing and verification. So we didn’t change anything about fee mechanism itself.

That mechanism is fairly simple to explain:

  • User select an explicit fee to pay to the miner who includes that txn in a block.
  • Miners choose which (valid) transactions to put into a block, and they can collect those fees.

That’s it. This can be thought of as a first price auction for block space, and mechanism design geeks tell me first price auctions are terrible for UX and economic/social cost. I believe them.

Conventional Shielded Fees

When Zcash launched, we used a flat default fee in the zcashd wallet when generating shielded transactions, based on two beliefs:

  • varying/unique fees are bad for privacy,
  • for the short term before blocks get full, it’s fine for everyone to use a constant fee.

When we launched, at least some miners weren’t selecting shielded transactions, though!

We patched the “priority” system to make an already baroque thing more baroque by basically saying shielded transactions are the highest priority. (Fun fact: we introduced a dangerous bug by doing this! Fun Fact 2: The priority system is an opt-in system miners can use to select some transactions over best-paying transactions, and I consider it a fairly terrible approach to doing that, but c’est la vie)

Conventional Shielded Fees in Practice

This whole approach worked by setting the default behavior of the zcashd node wallet when creating shielded transactions. Users could still override it, and they do.

Where does this leave us?

  • Some users set non-standard fees for their shielded transactions themselves. This is very bad for privacy because it is much more likely that on-chain analysis can determine which transactions came from specific people.
  • Some applications deviate from the conventional fee altogether. This is very bad for privacy because it is immediately identifiable which shielded txns come from which apps, fracturing the anonymity set into smaller “app-specific” sets.

Why do users change the fees?

This is purely anecdotal and speculative, but I’ve only see people mention changing the fee to make it lower. I don’t think I’ve seen anyone want to increase the fee because they want to pay miners more or they’re worried about their transaction not getting mined (remember: blocks are far from full).

How I see this ZIP

I like this ZIP or something similar as a “short term / conventional / wallet only” improvement. (For a better fee improvement, I am a fan of something like EIP-1559, which I believe improves both privacy and UX.)

I like this in the short term for four reasons:

  • It’s a proposal coming from wallet devs who are engaged and motivated that this is an improvement.
  • Users are choosing between private and cheap, but while we’re so far from blocks being full, why not give users both?
  • We can potentially learn if fee price is a hurdle to usage. If this change goes through and we see a huge jump in txn rate, then maybe there was latent demand below the current fee floor. OTOH, if we see no change, it seems likely to me that UX, availability, deployment, and other “last mile” factors are limiting shielded adoption, not fee price.
  • Finally, I hope this re-establishes the conventional fee to improve privacy.

What would really get me on board is if wallet vendors / users somehow commit/pledge to the ZIP before it’s accepted. In fact, I’d like to suggest the scope of the ZIP be modestly extended to say something like:

UX Guidance

Wallets which conform to this ZIP either must prevent users from altering the fee for shielded transactions.

Would that be palatable for both vendors and users alike?

Process Question

Since this is a ZIP about standardizing wallet behavior, I’m not sure how the process may be similar or different than protocol ZIPs. Do ZIP editors have any pointers here? cc @daira @dconnolly

4 Likes

This is great. Thanks @aiyadt for doing the hard work of starting the ZIP and shepherding the conversation forward!

There are a few considerations that I haven’t seen mentioned:

  • checkpoints
  • exchanges
  • hardware wallets

Today, most light clients start from a checkpoint (birthday) and may not have all chain state available. Any solution that is a function of chain state must account for use-cases, such as hardware wallets, where that chain state is not available. Also, shielded withdrawals are now supported on Gemini. We just did a brown bag yesterday with their engineering team and the topic of fees came up. If we change the fee to some kind of complex, dynamic, future-proof solution, then it will have a non-trivial impact on what they’re building. We should definitely factor in their perspective before settling on anything.

This ZIP, as I understood it, is scoped to establishing a new “conventional” fee, rather than propose changes to the protocol or wallet fee mechanisms.

This is a key distinction to make. It sounds like there should be another ZIP, with a longer horizon, that moves us in the direction of something like EIP-1559 with innovative economics. Meanwhile, this ZIP can help us iterate toward a near term solution that could possibly even coincide with the Canopy activation.

3 Likes

I also think it’s a really helpful distinction.

I think Zbay, Zecwallet, and Nighthawk have all informally agreed to adopt the lower fee as the default, and we could get formal commitments here.

Zbay can formally commit right now to using this fee and preventing users from altering the default fee on shielded transactions once the ZIP is approved.

3 Likes

Thanks for the commitment! And thanks to @nathan-at-least & @gmale to add clarity and definitions. I will update the ZIP with your inputs and ask for a review again.

Thanks to @secparam to really dive in to this discussion as the transaction fee is definitely intertwined with the overall network functionality at the larger scale. And since it has been pointed out that this is a conventional fee change, in line with maintaining user privacy, this change will open up novel applications while adding ease of use for z2z transactions.

1 Like

ZIP 313 has been updated with Activation block and grace period. Thanks @daira for the ZIP # assignment!

3 Likes

I’ve added the UX Guidance section with your input Sir!

This is correct, and this new “conventional” fee will require the updating of fee constants used in wallet/transaction building code across Zcash software.

1 Like