ZIP: Reduce default Shielded Transaction fee to 1000 zats

Yep, you already trust the p2p distributor not to give you a unique set of peers that IDs you. And thats harder to verify than the fee estimator is correct.

And the way we end up in dead ends is by removing pain points in the wrong way. And so slowly but surely, we remove anything that will make us turn off of the path we are on. And so inertia caries us along it until the path actually dead ends. This isn’t about technology, its a social thing about how we deal with problems and what gets attention.

Hope we never get to the point where blocks are consistently full. We should look for scaling solutions or increase the block size if we get towards fuller blocks. But that is out of scope for this ZIP.

How large of a mempool can Zcash network handle today?

I think it’ll be important to talk about these issues in terms specific to this question, and then create some space for talking about the broader issue within the teams leading Zcash engineering, because otherwise it will be really hard to make any case for anything based on its merits, in isolation, and it’ll be really hard for people outside the teams leading Zcash engineering to participate in the process.

So, specifically, how does changing the fee right now carry us down a path that eventually dead ends?

Another way of saying that is, if we agree that an oracle or EIP-1559 is a good solution, how does changing the default fee right now prevent us from later taking a step towards an oracle or EIP-1559?

@nitronick600, what you’re saying is that once blocks fill up the highest fee transactions will get included, right? Once that happens, it seems like it will be a very easy decision to take the step towards the oracle idea, or EIP-1559, because we’ll need a privacy-preserving way to dynamically set fees given the scarcity of space in each block. This doesn’t seem like a dead end path?

2 Likes

Is the goal to increase usability? When I use Zcash to buy stuff, I’ve never once been dissuaded by the 0.0001 ZEC fee. It’s negligible.

It’s very low for purchases, since it’s likely to be a very low percentage of what you’re spending.

But for sending memos only, the fee is 100% of the spend. Memos are a popular use case and enable things like public censorship-resistant messages (Zboard, Zbay) username registrations (Zbay) etc.

Even traditional wallets like Zecwallet see a ton of users using memos.

Lower fees work, yes. But the default fee is special, since only the default fee will offer users the full privacy benefits of shielded transactions.

The fee is public, so if you use a nonstandard fee for shielded transactions, your transactions stick out and become much easier to link.

So that’s why several wallet developers are interested in lowering the default fee at this time. There are use cases where the fee is significant, we want to lower it, and we can only do so together if we want to protect privacy.

9 Likes

Hey folks, I wanted to chime in here to try to clarify some points:

ZIP scope

This ZIP, as I understood it, is scoped to establishing a new “conventional” fee, rather than propose changes to the protocol or wallet fee mechanisms. Is that correct @aiyadt?

This ZIP doesn’t change anything about the existing “Zcash conventional fee” approach except the number. I think two good next steps for the bigger scoped question of “how can we improve fee mechanism in Zcash” would be to have a thread with that topic and it could have a running list of proposals (ZIPs, tickets, forum comment links). (I’m too busy currently to set that up myself.)

I think clarifications could help this conversation:

Protocol

First, let’s clarify the existing protocol. As always, I may have mistaken understandings, so anyone else please jump in with corrections.

The Zcash protocol is identical to Bitcoin (pre-segwit) with respect to fees AFAIK. Miners can pick any transactions they want to that follow consensus rules, and they collect the fees. That’s it.

Therefore, there is the same kind of “fee market” as in Bitcoin, and this has the same characteristics in the face of Denial-of-Service attacks and other blocks-are-full conditions. I happen to believe this is a terrible design which exacerbated the block size debate in Bitcoin, but never-the-less that’s what we’ve got for now. (We have one improvement: transaction expiry. This helps avoid “stuck transaction hell” because wallets can tell users “if your txn didn’t get mined by $T o’clock is is guaranteed to never get mined” so they can decide if they want to try again, unlike Bitcoin where all they can do is try to bump up fees with child-pays-for-parent shenanigans.)

When blocks are not full (which has always been the case so far in Zcash, AFAIK), then strictly-rationale-in-financial-terms miners should select every transaction they can find which pays them just enough to cover the cost of validation for that miner. That’s likely something more than 0, but I don’t know exactly what it is. If we had really good historical mempool analytics we might be able to estimate it empirically.

Denial of Service

AFAIK this proposal doesn’t affect DOS at all, because it doesn’t change the protocol, it changes conventional wallet behavior.

History

Before launching Zcash, we decided to deviate from Bitcoin’s design minimally, on the basis that Bitcoin was (at that time) the only blockchain protocol with a lot of in-the-wild testing and verification. So we didn’t change anything about fee mechanism itself.

That mechanism is fairly simple to explain:

  • User select an explicit fee to pay to the miner who includes that txn in a block.
  • Miners choose which (valid) transactions to put into a block, and they can collect those fees.

That’s it. This can be thought of as a first price auction for block space, and mechanism design geeks tell me first price auctions are terrible for UX and economic/social cost. I believe them.

Conventional Shielded Fees

When Zcash launched, we used a flat default fee in the zcashd wallet when generating shielded transactions, based on two beliefs:

  • varying/unique fees are bad for privacy,
  • for the short term before blocks get full, it’s fine for everyone to use a constant fee.

When we launched, at least some miners weren’t selecting shielded transactions, though!

We patched the “priority” system to make an already baroque thing more baroque by basically saying shielded transactions are the highest priority. (Fun fact: we introduced a dangerous bug by doing this! Fun Fact 2: The priority system is an opt-in system miners can use to select some transactions over best-paying transactions, and I consider it a fairly terrible approach to doing that, but c’est la vie)

Conventional Shielded Fees in Practice

This whole approach worked by setting the default behavior of the zcashd node wallet when creating shielded transactions. Users could still override it, and they do.

Where does this leave us?

  • Some users set non-standard fees for their shielded transactions themselves. This is very bad for privacy because it is much more likely that on-chain analysis can determine which transactions came from specific people.
  • Some applications deviate from the conventional fee altogether. This is very bad for privacy because it is immediately identifiable which shielded txns come from which apps, fracturing the anonymity set into smaller “app-specific” sets.

Why do users change the fees?

This is purely anecdotal and speculative, but I’ve only see people mention changing the fee to make it lower. I don’t think I’ve seen anyone want to increase the fee because they want to pay miners more or they’re worried about their transaction not getting mined (remember: blocks are far from full).

How I see this ZIP

I like this ZIP or something similar as a “short term / conventional / wallet only” improvement. (For a better fee improvement, I am a fan of something like EIP-1559, which I believe improves both privacy and UX.)

I like this in the short term for four reasons:

  • It’s a proposal coming from wallet devs who are engaged and motivated that this is an improvement.
  • Users are choosing between private and cheap, but while we’re so far from blocks being full, why not give users both?
  • We can potentially learn if fee price is a hurdle to usage. If this change goes through and we see a huge jump in txn rate, then maybe there was latent demand below the current fee floor. OTOH, if we see no change, it seems likely to me that UX, availability, deployment, and other “last mile” factors are limiting shielded adoption, not fee price.
  • Finally, I hope this re-establishes the conventional fee to improve privacy.

What would really get me on board is if wallet vendors / users somehow commit/pledge to the ZIP before it’s accepted. In fact, I’d like to suggest the scope of the ZIP be modestly extended to say something like:

UX Guidance

Wallets which conform to this ZIP either must prevent users from altering the fee for shielded transactions.

Would that be palatable for both vendors and users alike?

Process Question

Since this is a ZIP about standardizing wallet behavior, I’m not sure how the process may be similar or different than protocol ZIPs. Do ZIP editors have any pointers here? cc @daira @dconnolly

4 Likes

This is great. Thanks @aiyadt for doing the hard work of starting the ZIP and shepherding the conversation forward!

There are a few considerations that I haven’t seen mentioned:

  • checkpoints
  • exchanges
  • hardware wallets

Today, most light clients start from a checkpoint (birthday) and may not have all chain state available. Any solution that is a function of chain state must account for use-cases, such as hardware wallets, where that chain state is not available. Also, shielded withdrawals are now supported on Gemini. We just did a brown bag yesterday with their engineering team and the topic of fees came up. If we change the fee to some kind of complex, dynamic, future-proof solution, then it will have a non-trivial impact on what they’re building. We should definitely factor in their perspective before settling on anything.

This ZIP, as I understood it, is scoped to establishing a new “conventional” fee, rather than propose changes to the protocol or wallet fee mechanisms.

This is a key distinction to make. It sounds like there should be another ZIP, with a longer horizon, that moves us in the direction of something like EIP-1559 with innovative economics. Meanwhile, this ZIP can help us iterate toward a near term solution that could possibly even coincide with the Canopy activation.

3 Likes

I also think it’s a really helpful distinction.

I think Zbay, Zecwallet, and Nighthawk have all informally agreed to adopt the lower fee as the default, and we could get formal commitments here.

Zbay can formally commit right now to using this fee and preventing users from altering the default fee on shielded transactions once the ZIP is approved.

3 Likes

Thanks for the commitment! And thanks to @nathan-at-least & @gmale to add clarity and definitions. I will update the ZIP with your inputs and ask for a review again.

Thanks to @secparam to really dive in to this discussion as the transaction fee is definitely intertwined with the overall network functionality at the larger scale. And since it has been pointed out that this is a conventional fee change, in line with maintaining user privacy, this change will open up novel applications while adding ease of use for z2z transactions.

1 Like

ZIP 313 has been updated with Activation block and grace period. Thanks @daira for the ZIP # assignment!

3 Likes

I’ve added the UX Guidance section with your input Sir!

This is correct, and this new “conventional” fee will require the updating of fee constants used in wallet/transaction building code across Zcash software.

1 Like

We probably need to start getting clarity on this. As @adityapk00 has shown, this change doesn’t necessarily require changes in zcashd. Getting this done and implemented in time for Canopy feels doable.

At the same time, we are kind of charting new territory here and many of us are new to writing ZIPs. So we should make efforts to ensure that everyone is on the same page for how this ZIP process will unfold. Is it possible to be “results oriented” and agile here? Can we make incremental progress and modify the ZIP and code, as needed, whenever we need to make adjustments, later?

Is everyone okay with limiting the scope of this ZIP to just reducing the “Conventional fee”
while saving a larger more comprehensive fee change for another ZIP along the lines of EIP-1559?

If so, can this change be made primarily in one place so that it is less code to manage in fewer locations AND if it needs to be tweaked in the future, everyone gets it “for free?”

Reducing the friction of future changes seems to be an effective way to “favor results over bikesheds” in the present.

2 Likes

I am definitely okay with this. @secparam, since you had some objections to this narrower approach it seems fair to ask if you are okay with this.

1 Like

hey everyone,

I represent horizontal systems and last few years we have been building mobile wallets for various blockchain protocols i.e. SPV client for Bitcoin.

Now, here is our take on network fees.

  1. At this point in time the two blockchains where setting the right fee is problematic are Ethereum and Bitcoin. Each has its own native fee estimate mechanisms and both often fail when network activity changes. In other words, the client developers are not able to fully rely on native fee estimate mechanisms provided by the network nodes.

On a client side we typically show user an estimate (1-2 hours) for a selected fee range (low, normal, high). It’s not uncommon for the transaction to remain pending for a lot more hours then estimates or get processed in a few minutes while user expected it to take hours. So, there is a window for confusion for the sender of the funds. If the transaction remains pending significantly longer than it often results in user being worried as few currently understand what these transaction fees are. So, long story short, existing fee estimate mechanisms for busy blockchains work partially and we are yet to see something that works well for all cases.

  1. I think fixing the fee while blocks are not getting full is a working idea. We already do this for blockchains like DASH, Litecoin and Bitcoin Cash. The 0.0001 comes down to less than a cent which is comparable to those of BCH.

That said, someone mentioned that 0.0001 can be abused as filling the entire block with to-self transactions would be still fairly cheap…

  1. It goes without saying that at some point there should be some kind of fee estimate mechanism which depends on the current state of the mempool as it can be locally accessed by each node and indicates the current activity on the network.

iBek

2 Likes

Haha I love the meme. Tangential topic, but there are too many leading zeroes for me to visually recognize how many it is. That reminds me of this ticket: introduce the ZED unit · Issue #3462 · zcash/zcash · GitHub

ZED is a better unit than ZEC because ZEC is good for large transactions and bad for small transactions, while ZED is good for small transactions and okay for large transactions.

Coffee: 0.0001 ZEC (bad) or 100 ZEDs (good)
Remittance: 1 ZEC (good) or 1 million ZEDs (okay)

Basically, we just need to avoid having prices more than 1 or 2 places to the right of the decimal. People are fine with a thousand or a million. They are not fine with a thousandth or a millionth.

2 Likes

A new Zecwallet release is out (v1.3.3 lite client and v0.9.20 full node) that incorporates this ZIP. Default fees will switch over to 1000 zats after canopy + grace period (Block 1_080_000).

5 Likes

This really helps when pricing in Zcash, ZEDs for 1000th unit instead of worrying about decimals. And ZEC for dealing with base 21 million layer.

This change will need user testing as @daira suggested in the GitHub thread. And light clients might be the ones to help test it out. A community poll might help to check if Zcash users are open to adopt the unit based terminology. Can someone volunteer to create one?

Note: Keeping the terminology simple(& to the brand) should be the goal, as Bitcoin failed the millibit experiment Millibit - Bitcoin Wiki

2 Likes

Wasn’t a π Zec supposed to be like 100 zat or did we ever decide? I cant remember (though it was from a show I never watched, something carbon I think)

It’s help when $15000 per 1zec. Now it’s just $60. Lol

I just posted a github review of the current ZIP draft. Here are high level bits for discussion. Edit: It was super long, so I split two different sections into different replies to improve threading.

What about transparent transactions?

This is one outstanding major gap from my review reading.

The motivation for making privacy cheap to enable new use cases is a great one, but it doesn’t seem to apply to fully t->t transactions. Should this ZIP say that fees for t->t transactions can vary? What about the trickier cases of any transaction with both shielded and transparent components?

Relevant considerations include:

  • Usability - it seems nice to me if all txns have a single flat fee, because it’s easy to understand and plan around UNTIL blocks are full.
  • Ecosystem adoption - there are many many more transparent Zcash wallets and many of them don’t necessarily follow Zcash too closely because they just port a Bitcoin thing and move onto the next coin. So I’m not sure what fraction of transparent wallets would follow the ZIP.
  • Transaction-rate Denial of Service - if all wallets always keep a flat fee, this makes TR-DOS vulnerability worse. (Again, I personally advocate for addressing this in one or more separate follow up ZIPs.) But if some wallets vary fees, especially transparent wallets, then during a TR-DOS event, users may begin unshielding their ZEC in order to make transactions go through, which is worse for privacy.

My personal position: The ZIP should apply to any transaction with any shielded component. This strikes the following balance:

  • Usability - Users are already hopefully aware of the Z / T distinction, so while “all transactions use flat fee F” is very nice, I think “all transactions involving a shielded address use flat fee F” is still pretty good.
  • Ecosystem - Any wallet that can generate shielded components is much more likely to be proactively engaged with Zcash development so the adoption of
    the ZIP is much more likely to be widespread.
  • TR-DOS - Not well addressed. Users who use “legacy Bitcoin-like” t-addr wallets will fare better in a TR-DOS. Since most transactions are t->t currently, there’s a nice systemic defense here because the fees should raise during an attack and limit the attack window, even though the downside is that
    any shielded transaction may not confirm during the window.
2 Likes