Would the impact of sandblasting have been mitigated sooner if ECC had focused on changing the fee structure?
This was discussed during the Sandblasting Retro (minutes, video).
With the benefit of hindsight, I think we were slow to recognise that there was a systemic issue because Zebra wasn’t significantly impacted (although the attack did expose a bug where Zebra was redundantly/repeatedly verifying Orchard proofs). Once the scale of the issue became apparent, ZF did offer to help ECC, and subsequently helped with ZIP 317, and implemented ZIP 317 in Zebra.
For my part, I knew that the risk of denial of service attacks was something the ECC engineers had looked closely at in 2019, and I believed that, in terms of expertise, knowledge of the protocol, etc., the ECC engineers were the best folks to address the issue, and the best thing we could do was to let them take the lead, and offer help and support. That’s what I told the ZF engineers, and (based on Daira’s comments during the Sandblasting Retro) I believe that the collaboration was productive.
Again, with the benefit of hindsight, I wish I had been more proactive (I would say “forceful” but, given all this occurred shortly after a period of strained ECC-ZF relations, that may have been counter-productive!).
I think there’s two factors at play here. The first was identified by @dontpanicburns, who pointed out that (paraphrasing) the Dev Fund has created a “no pay no play mentality” - i.e. anyone who was in a position to make the block-construction changes to zcashd likely did not feel any motivation or responsibility to do so, and instead believed that ECC (as the largest beneficiary of the Dev Fund) should be responsible for fixing such issues. If we had identified suh a person (or team), and asked them why they weren’t working on fixing the issues, I can imagine them responding with “Why do you expect me to do it? Isn’t that what we pay ECC for?”
Certainly, if a similar situation arose with Zebra, I would regard it as our (ZF’s) responsibility to make the fix (at least for as long as we have the resources to do so).
The second factor at play is that fact that ECC possessed – and still possesses – far greater capability to do the things you describe than any other person, team or entity in the Zcash ecosystem. Making changes to zcashd in a competent and timely fashion requires knowledge of the zcashd codebase (which primarily resides with the ECC engineers), the time to do so (which may be difficult to find if one has a day job), and the assistance of at least one other competent developer who can review your code for bugs.
Even if somebody (or a team) had made the changes, they would likely have found it very difficult to persuade miners to adopt their fork because (and this is based on ZF’s experience of trying to persuade miners to adopt Zebra instead of zcashd for block generation) it’s very difficult for anyone who isn’t ECC to gain traction when trying to engage with miners because (a) ECC has traditionally owned all those relationships (so it’s difficult to even know who to contact), and (b) the miners are unlikely to pay any attention to anyone who isn’t ECC.
Imagine some random Zcash community member emailing miners saying “Hey, we’ve forked zcashd to fix the wallet performance issues. Can you please switch to using our fork instead of ECC’s?”. I can’t imagine any miner paying attention to such a request.
Similarly, rallying community support around a fork of zcashd would probably be difficult without the explicit support of ECC.
A prerequisite for that is people being able to do stuff.
At an individual level, when one possesses expertise, experience and knowledge, has the time to devote to a task (because it’s your job, so you can dedicate 8+ hours a day to it), and has the support and assistance of a team of similarly-qualified people, it can be deceptively easy to assume that others possess the same capability. Drawing an owl is easy, right?
I think it’s very easy to underestimate the extent to which ECC has had an effective monopoly on the expertise (e.g. knowledge of the intricacies of the Zcash protocol, expertise relating to ZK proofs, familiarity with the zcashd codebase, and experience of developing zcashd) required to make changes to the Zcash protocol. That, combined that with the fact that ECC has received the lion’s share of the Dev Fund (40% more funding than the next largest beneficiary - i.e. ZF), means that progress on decentralization has been slower than we would all like.
We’ve been developing that some of those capabilities at ZF (albeit with Zebra as the platform, instead of zcashd) but it takes time.
ETA: Expertise relating to ZK proofs is something we have yet to make any meaningful progress on at ZF. I would be remiss in not highlighting the fact that QEDIT’s membership of the Zcash ecosystem is hugely important in terms of reducing our reliance on ECC in that regard.
One way we can speed the process up is by being proactive about sharing knowledge and expertise, which is one of the reasons I’m keen for the ECC and ZF engineers to work together on the zcashd deprecation project (as opposed to in separate silos), so that (a) the ZF engineers can learn more about Zcash wallet functionality, and libraries and crates like librustzcash and zcash_client_blackend, and (b) the ECC engineers can learn more about Zebra.
The more we collaborate, the more we can decentralize.