Cherry-picking Tor onion v3 and other overlay networks from Bitcoin

[nullius]

I regret that looking back, my first reply to Madars may have unintentionally come off as a bit snippish towards Zcash developers. My concern is about the state of Zcash development. Besides the urgent issue of onion support, one of my longtime major pain points is that Zcash is far behind Bitcoin Core in every way except for blockchain privacy. It is even behind in network-layer privacy.

I suffered with that in silence, because I saw Zcash altogether as a small, scrappy upstart with minuscule resources compared to Bitcoin Core’s large, decentralized development effort. I thought I should be grateful that a tiny team with limited funding was achieving one highly difficult, extremely important task to change the world. Their work on blockchain privacy has been superb!

I am not saying that to sugar-coat criticism. I have a hard personality, and a reputation for never sugar-coating anything. I have been clinging to my precious ZECs, for exactly one reason: The Zcash people (inside and outside ECC) created, implemented, deployed, and refined the first and, thus far, only fully-fielded mainnet on-chain privacy technology that I deem adequate. Their key developers are specialized in that, and I want for them to stay focused on it. So, ok, I guess they can’t afford to hire enough node developers to keep the node up to date.

Then, shortly before my last posts on this thread, I perchance happened across this in a massive thread that I have barely even begun to skim—boldface is in the original:

@GGuy

My personal opinion has always been that @ZcashGrants is highly unlikely to find anywhere close to 200,000ZEC worth of high quality grants over the next year so it seems obvious they should fund the full 32,000ZEC ($1.8m) grant. Especially when this grant will contribute to solving the lack of high quality grant proposals.

That, and many similar posts, pertain to funding to make promotional videos. It was even suggested that a fixed percentage of ZEC developer-seigniorage funding should be allocated to the video-makers on an ongoing basis. As a ZEC holder, I protest that that would be outright unethical. As a ZEC holder who has previously defended the Zcash “dev tax” in other venues with my argument that “open-source developers need to be paid somehow”, it suggests to me that perhaps I have erred.

So… @ZcashGrants has so much money that people seriously argue, in substantial effect, that they need to make multimillion-dollar grants just to find a way to spend it? —While the zcashd node is light-years behind Bitcoin Core? While @str4d, a specialist in zero-knowledge proof systems, is struggling to find time for the tedious work needed to merge a few long-overdue improvements from upstream?

Something is wrong with this picture.

The outside view that the Zcash grant system is presenting to others: People are popping up asking free start-up money for unrelated businesses. In this case, a request for $47,000 for a cannabis retail shop:

@ponchobrotherz

I’m seeking $27,000 that will be used for product giveaways and initial payroll for training/staffing.
$20,000 for the neighborhood grand opening expenses. $47,000 total.

Regardless of whether or not that gets funded, money is just being thrown around here:

@nullius

@noamchom

Depending on the proposal amount, i think this should be funded. Lets not fool ourselves about whether or not this qualifies considering that last year we payed a guy $5,000 to go on a parachuting trip that yielded a choppy 45 second video

How about using development funding money to fund development?

---

A constructive solution.

Some people seem completely unaware of any real information about Bitcoin. If one learns about Bitcoin from Bitcoin-bashing Twitter altcoin memes, one may be blissfully ignorant of how far ahead Bitcoin Core is everywhere except privacy. Stunningly far ahead. Upstream is thriving—all funded by voluntary grants, as I have repeatedly noted.

Running both side-by-side tends to make one wish to copy the shielded value pool functionality from Zcash, and paste it onto Bitcoin so as to have everything good in one place. I believe that that was the original intent of Zcash’s design as a Bitcoin codefork—that, and gaining the market advantage of ecosystem compatibility (something that I urge the Zcash developers to keep!)—that, and getting much faster time-to-market, saving millions in development costs, and even making the project feasible. (Many altcoin projects that attempt rolling their own blockchains stall out as vapourware—or worse, produce a buggy, insecure mess.) Zcash has benefitted immensely from Bitcoin Core’s MIT-licensed codebase.

Now, if @ZcashGrants has so much money that it can’t figure out how to spend it (!), perhaps ECC should ask the community to hire a dedicated developer to help get zcashd up to parity with Bitcoin Core 23+, and to maintain it at parity going forward.

No, I am not suggesting myself for that job—and I have not yet asked around about this. I do think that I am competent to help review the qualifications of any candidates, and to give feedback on paid performance. It should be someone with strong qualifications on the Bitcoin side, who has a deep familiarity with the code and who follows Bitcoin development very regularly. Maybe even see if any well-established Bitcoin Core contributors would be interested in a paid position working full-time to get the Zcash node fully up to date, and thereafter part-time on an ongoing basis to keep it up to date.

I am aware of The Mythical Man-Month. But the task here is well-defined: Do for years worth of missing features and fixes the intellectually demanding, but tedious and time-consuming work that @str4d is now doing for Tor v3 onions, and has done before for other fixes and features. Then, submit it for review. I applaud this and other review criteria:

@str4d

For anything that affects consensus code, we require for safety 3 reviews with at least 2 from ECC.

That way, ECC team members only need to perform reviews—instead of doing the preparatory work, and then also performing a sufficient peer review process.

The benefits are many.

In addition to the extensive, ever-improving overlay-network support listed in OP here, catching up with Bitcoin Core would bring:

• Multi-wallet support. This is an especially important UX feature for compartmentalizing identities in a privacy coin. (I used to use some shell scripts, originally developed for #2707, to juggle wallets to avoid any possibility that unexpected transparent coins at addresses used to buy ZEC may inadvertently get merged. I gave up on this long ago; I just couldn’t keep all of the wallets synced.)

• Network-layer/connection manager security protections against eclipse attacks, Erebus attacks, and various other attacks. Bitcoin Core has done much significant work there. (TODO: Gather a list with security research references.)

• Pruning to help poor people run nodes. (zcash/zcash#4080)

• New RPCs.

• Fix and support the REST API (zcash/zcash#2272; see also the bottom of zcash/zcash#2100 (comment). Consumers and retail buyers don’t care about this. For my part, the RPCs are fine. But many business users may want this, for integration with their REST-based software stacks; I believe that is probably why Bitcoin Core has it. I want to expand Zcash use by businesses. The code is already there, and it’s already maintained upstream.

• Support for the SQLite3 wallet based on output descriptors (which would need a ZIP for shielded support). (Primarily bitcoin/bitcoin#19077.) Adding another wallet database format is worth the technical debt of permanently supporting old BDB wallets for safety of user funds. SQLite is reliable and well-maintained, with no licensing problems for new versions.

• Numerous bugfixes, and improvements to security, stability, performance, and in some places even—privacy.

• The major non-consensus-change internal refactoring ongoing for libbitcoinkernel, which is #1 on my Bitcoin Core wish list.

  Lack of any ability to impose hardware-enforced security boundaries between various node functions is a design flaw, of the type that’s not unlikely when one lone-genius inventor was obviously working by himself on a project of large scope. (Transaction malleability was another design flaw, fixed in Bitcoin by Segwit—later otherwise fixed in Zcash.) As a step towards solving this and other problems, Bitcoin Core has tried for years to find a way to refactor a clear boundary between the consensus code and everything else—without risk of breaking everybody’s money. Previous efforts at a libconsensus failed. Carl Dong’s approach with libbitcoinkernel is safe, elegant, and actually happening. Best of all: It will make the core of Bitcoin Core a C library available for other development.

  All of this requires no consensus changes. But the internal code changes are extensive; and Dong designed his plan to facilitate maintainability for much more extensive changes in the future.

  A libzcashkernel delivered as a C library (in terms of API/ABI—thus bindable into most any language) would be awesome for the ecosystem. I can’t want to get my hands on libbitcoinkernel and build things with it; what could be done with an hypothetical libzcashkernel?

• And, and, and… the list goes on, and on, and on…

That is only a sort of off-the-cuff list, for the sake of illustration. The point is that merging support for v3 onions and other overlay networks is only (IMO) the most urgent part of addressing a general problem.