There are two primary reasons this didn’t happen last year before the October deadline for v2 removal:
- The ECC core team was busy with NU5; we’ve been working on it solidly for over a year and a half at this point and are still not finished (there is a bunch of mobile wallet work left to do).
- The backport is not as simple as cherry-picking the ADDRv2 and Tor v3 support. All of the upstream changes assume the existence of a bunch of other upstream PRs that we don’t have (and that I’ve been slowly trying to get backported over the years, but we’re a small team with a responsibility to safely backport changes in a way that does not break Zcash, and Bitcoin Core was not designed to be modular in a way that makes backports easy and safe).
I’m very grateful to ZingoLabs, which put in a bunch of work last year to figure out (most of) the required upstream commits that need to be landed in order to unblock ADDRv2 support. Their current draft PR ostensibly does have Tor v3 support (I believe it was working inasmuch as they had Tor communication functioning in a private testnet). However, it is not at all feasible to just merge the PR as-is: there are far too many changes to review at once, a bunch of tests are currently commented out, and the interactions of the PRs that the changes are sourced from (both with each other and the rest of the codebase) need to be carefully checked (as we have found upstream changes before during review that were innocuous upstream but would have been a subtle consensus-breaking change had we applied them to Zcash).
I’ve just spent several hours mapping the commits back to their corresponding upstream PRs, de-duplicating and sorting them to optimise for path-of-least-conflicts[1]. Currently there are 54 upstream PRs that need backporting, and that is an underestimation as it doesn’t account for the various partial backport commits that ZingoLabs included without upstream attribution (because the corresponding upstream PRs couldn’t be applied due to even more inter-PR dependencies that need to be discovered).
The next steps here are:
- Someone (read: me) has to go through the list of PRs and turn them into a dependency graph, to figure out which PRs absolutely need to be backported before which others, and which can be worked on in parallel.
- Then we start from the front of the graph, and try backporting the PRs in small relevant groups.
- Every time we run into a conflict, we need to investigate it to determine if it is a meaningful conflict that introduces another backport dependency, or if it is a conflict we can work around (either because the upstream PR it conflicts with introduces a feature we definitely won’t be backporting, or we feel that the maintenance burden introduced by the conflict is small enough that we can deal with it later).
- For every backported PR, we also need to scour the affected files upstream to find subsequent PRs that touch the same code, and determine whether we need to backport those as part of the same PR group (because it is quite common for a feature to be merged upstream and then subsequent bugfix PRs to be merged, and it would be irresponsible of us to just merge the PR dependencies of Tor v3 without conducting this kind of review).
- Hopefully the backports proceed smoothly and only touch code that requires only one or two reviewers.
- For anything that affects consensus code, we require for safety 3 reviews with at least 2 from ECC. This usually means 4 ECC core team engineers need to be involved in a single PR. We have 6 ECC core team engineers, so you can see how easily this step can be a bottleneck.
- Given that this feature involves a lot of networking code, we will likely need the ZF engineers to be in the loop for some of the backports as well, given that any network-related PR we backport that has an associated BIP needs “P2P consensus” (in that
zcashdandzebradneed to agree on what the P2P protocol versions mean).
- Keep doing this until we reach the end of the graph (namely, the Tor v3 PR is backported and merged).
Hopefully this gives a bit of insight into why this has been slow. Do not mistake that for a lack of desire to support Tor v3, or I2P (I’m an ex-I2P developer, and I2P Integration · Issue #1111 · zcash/zcash · GitHub has been open since 2016). It’s just a lot, and there’s been a lot going on these past few years (both within Zcash, and in the wider world) that consumes energy and resources.
[1] and then another hour writing this message