`halo2` Community Manager

Community Grants
1

Hi Zcash community,

I’ve applied for a ZCG grant to be a halo2 Community Manager. It would be great to hear your thoughts on the proposal:

Applicant background:

My name is Ying Tong and I’m one of the authors and maintainers of the halo2 library. I was previously an ECC employee. I’ve also contributed to education and outreach efforts around halo2 (e.g. 0xPARC’s “Halo 2 Learning Group” and “Halo 2 Working Group”).

This is my halo2 contribution history.

Description of Problem or Opportunity:

The halo2 library, authored by the ECC team, has been widely adopted by projects such as zkEVM community edition, zKonduit’s ezkl, and Axiom’s halo2-lib. External teams have built many useful features, ranging from low-level proof system changes, to optimisations and API extensions, e.g.:

  • dynamic lookup tables (zcash/halo2#660),

  • multi-phase circuits (zcash/halo2#593), and

  • polynomial commitment scheme abstraction.

Many of these features are not directly used in the zcash project, but are widely useful in a variety of applications: as such, they are often submitted for inclusion upstream. However, reviewing pull requests and maintaining community reviewer/maintainer groups is a significant overhead in addition to the ECC team’s other responsibilities. Maintaining an active ecosystem around halo2 would benefit the zcash project as:

  • more eyes on the code increases the probability of surfacing edge cases, bugs, and security issues;

  • industry-wide standardisation and benchmarking efforts will be more likely to include the halo2 library;

  • a more modular and feature-rich halo2 library could make it easier to support future upgrades and extensions of the zcash protocol.

Proposed Solution: Describe the solution at a high level.

A halo2 community manager would contribute reviews, organise processes for external review and maintenance, and provide support to external teams exploring new features.

Solution Format: What is the exact form of the final deliverable you’re creating?

  1. Document the review policy for the halo2 repository (adapted from zcash/halo2#708):

Currently, the halo2 repository follows ECC’s review policy for code that ends up in zcashd (as zcash/halo2 is a dependency of it), which is designed around “levels of bug impact”. These are centred around reviews from ECC engineers and can require up to three reviews (for PRs that alter consensus rules).

A compatible review policy for halo2 should be formalised and documented in the Halo 2 book. In particular, assuming a “nightly → beta → stable” feature workflow from zcash/halo2#707, the review policy should be equivalent to the ECC review policy for any code landing in (or being moved to) the stable feature set, but could be weaker for experimental features targeting nightly.

  1. Contribute reviews and maintain a workflow for external reviewers (adapted from zcash/halo2#709):

Since many external pull requests are in use, or will be used, mainly by projects outside of the zcash GitHub org, we should enable these stakeholders to contribute reviews. This will also lighten the review load of the ECC team, since external reviews would count toward the repo’s review policy.

This milestone involves setting up and maintaining a workflow for external reviewers to “sign up” for contributing reviews. Some possible workflows include: setting up a bot, having a GitHub team with permissions scoped for review, and having PR delegates along with a Discord channel for coordination.

The halo2 community manager will also contribute reviews.

  1. Administer a workflow for ‘code maintainers’ (adapted from zcash/halo2#710):

There is a clear desire for a wide set of new features to be upstreamed into the halo2 repository or newly developed for it. Some of those features can be developed orthogonally, but many will have some level of interaction with each other. Additionally, while several of the proposed new features are generally useful, others have more targeted use cases that ECC in particular does not have the resources to maintain on behalf of the community.

A workflow for defining “code maintainers” should be established, to delegate responsibility for maintenance of various sub-components within the repository. This would ease the overall maintenance burden, and make feature inclusion more likely overall (though still subject to analysis with regard to the overall UX of the repository’s crates). The halo2 community manager will play an active role in delegating reviews.

Dependencies: What external entities is your project dependent on? What involvement is required from ZF, ECC, and/or other external organizations? Who would have to incorporate your work in order for it to be usable?

This project involves liaising with the ECC core maintainers as well as maintainers of long-lived forks, such as halo2-ce.

Execution risks: What obstacles do you expect? What is most likely to go wrong? Which unknown factors could jeopardize success? Who would have to incorporate your work in order for it to be usable?

Features involving large refactorings may interfere with each other and cause conflicts. In this case, the simple “nightly → beta → stable” feature flagging may be insufficient or lead to large amounts of code duplication. We may eventually have to explore a branching approach to handle such features.

Unintended Consequences: What are the negative ramifications if your project is successful? Consider usability, stability, privacy, integrity, availability, decentralization, interoperability, maintainability, technical debt, requisite education, etc.

Since the external contributor workflow allows for features to be developed in-situ, experimental features will be introduced into the codebase (behind feature flags). Sufficient safeguards and documentation must be put in place to prevent unintentional use of these features in production.

Downstream consumers of unstable features must also be prepared for backward-incompatible changes.

Evaluation plan: What metrics for success will you share with the community once you’re done? In addition to quantitative metrics, what qualitative metrics will you commit to report?

As specified in the previous sections, the deliverables of this proposal include the documentation of review policies, external reviewer workflow, and code maintainer workflow. In addition, the halo2 community manager will:

  • help organise biweekly “Halo 2 Developer Calls” involving the ECC team and the community;

  • provide support in the “Halo 2 Ecosystem” Discord; and

  • participate in external community calls (e.g. halo2-ce’s biweekly call)

Compensation total budget:

$50400.00 USD

Please provide justification for the total compensation budget:

The estimated time for this role is 14 hours/week, at a rate of 75 USD/hour. This works out to 56 hours/month at a cost of 4,200 USD/month, for a total commitment of 50,400 USD over a year.

Milestone 1 - estimated completion date:

01/31/2023

Milestone 1 - USD value of payout upon completion of deliverables:

$0.00

Deliverable 1.1

Document the review policy for the halo2 repository

Milestone 2 - estimated completion date:

01/31/2024

Milestone 2 - USD value of payout upon completion of deliverables:

$25200.00

Deliverable 2.1

Contribute reviews and maintain a workflow for external reviewers

Milestone 3 - USD value of payout upon completion of deliverables:

$25200.00

Milestone 3 - estimated completion date:

01/31/2024

Deliverable 3.1

Administer a workflow for defining ‘code maintainers’

Total proposed USD value of grant:

$50400.00 USD

How was the project timeline determined?

This proposal is scoped to a 12-month period, and should be re-reviewed for renewal or possible restructuring at the end of this period.

Application submission date:

01/13/2023

22 Likes
2

Be sure to check out Ying Tong’s Halo 2 Community Showcase at Zcon3! I vote, yes! :grin:

12 Likes
3

I’ve said it before, I’ll say it again – I’m a fan of funding education. @therealyingtong is brilliant and I think they are perfectly placed to build the halo2 community. :+1: :zebra: :owl:

8 Likes
4

Developers, Developers, DEVELOPERS!

2 Likes
5

Let’s make sure we fund this proposal. Halo2 is popular and will increasingly be more popular. I would like to see the Zcash community becomes one of its stewards.

3 Likes
6

I’m generally in favour of funding this, but I wonder if there could be an opportunity to collaborate with some of these other communities to set up multi-year funding which will benefit all Halo2 builders.

Maybe ZCG could do one year, Ethereum another, Filecoin another etc. Or maybe we could explore some type of joint funding mechanism or even leverage the joint ECC/Filecoin grant pool…

7

I think this is definitely a good idea for the long-term. However, I also think the ide should be pursued independent of this specific proposal.

8

@therealyingtong & Zcash Community, I am happy to announce that the @ZcashGrants Committee unanimously voted to approve this Grant Proposal at the most recent meeting.

12 Likes
9

Over the past year, I have been engaging with the halo2 community through:

  • providing support in the Halo 2 Ecosystem Discord and Office Hours;
  • understanding and documenting the forks and projects building on halo2; and
  • contributing to new features in halo2.

Having developed a better understanding of the ecosystem, I believe that the popular forks have diverged significantly enough that we can expect them to be long-lived. However, community contributions that can benefit zcash/halo2 include:

  • low-level optimisations (e.g. curve operations, lookup arguments);
  • general-purpose features (e.g. Challenge API); and
  • security analysis and formal specifications.

I will finish the year by identifying a set of features that can be realistically upstreamed, and aim to fully upstream one of these to demonstrate the community workflow.

The scope of work I have completed this year diverges from the initial milestones set out in this grant: I have focused more on community engagement, and less on upstreaming features. I have not yet claimed any part of the grant, and would be happy to adjust / reduce the grant accordingly as the ZCG decides. (Iit’s been lots of fun serving as halo2 community manager, and I plan to continue after this year regardless of the grant.)

Community resources

I help to moderate and maintain the following community resources:

New features

  • mvlookup: an implementation of a more efficient lookup argument based on the logarithmic derivative. This has been tested in Scroll’s fork and ezkl, and could be a candidate for upstreaming
  • cq: an implementation of a more efficient fixed lookup argument using cached quotients. Since this uses a trusted setup under-the-hood, it is likely not suitable for upstreaming.

Contributor workflow

I will finish the year by identifying a set of features that can be realistically upstreamed, and aim to fully upstream one of these to demonstrate the community workflow.

Further, I would like to highlight another effort to standardise contributor workflows for zcash-maintained cryptography projects: the Algebraic Improvement Proposal. I plan to contribute to discussions around this process.

12 Likes
10

Hi @therealyingtong. Thank you for the update! Please reach out if you want to discuss changing or adjusting your milestones.

11

After further discussion with the ZCG, I’ve decided to retroactively withdraw this grant application (i.e. I will not be claiming the amount granted). I’ll still be continuing this role in an informal capacity though – it’s been lots of fun!

10 Likes
12

I think we should retroactively pay this out as intended. :+1: :student: