Sarah Jamie Lewis announces her candidacy for the Major Grants Review Committee

If you go look at my comment on @mistfpga application you’ll see that I was quite adamant about him using his auditing knowledge and reverse engineering experience actively in the decision-making process. He brought up an interesting point though that the word audit holds a specific legal responsibility and he didn’t want to include that responsibility in his application because it would make it more complicated. So I see your concern but I think if we have someone on the team who acknowledges the importance of auditing and there’s a responsibility from the foundation to commit to those audits then it won’t be a problem.

Perhaps we could arrange a bilateral agreement between grant applicants and the community to perform a 3rd party audit but add a clause of sorts that guarantees the auditors from frivolous legal repercussions. We could define very precisely what we consider to be a pass and a fail for the audit, a fail would just require a revised application imo.

Auditing is a very overloaded term, so I’m going to break it down slightly to clarify meaning.

  • Financial Auditing: A full review of the books and operations of an entity - generally very expensive when it comes to small teams, and a major hurdle in the technical grant space as many grant committees require it. I do not think it wise to require such audits of grantees as it drastically slows the process and limits the potential pool of applicants. If there are special circumstances (i.e. very large grants) where such an audit would be prudent, I’d like to see it built into the budget of the grant.

  • Security Review / Auditing: Also a very overloaded term, sometimes part of a legal requirement (e.g. handling credit cards directly), sometimes not. It is very clear to me that several Zcash ecosystem grants have not had sufficient funding dedicated to security review and if elected I would push to have a standardized security review process be a mandatory requirement of a technical grants. This may involve funding a team dedicated to this work, or working with existing teams to conduct the reviews.

I’ll note at this point that I used to be a “Security Certifier” (and a Security Engineer) for Amazon and have extensive experience conducting security audits across a range of systems handling extremely sensitive data and managing external teams to run large scale, multi-month security engagements (both assessments and red teams) - I’ve also at this point found, reported and occasionally several vulnerabilities in existing Zcash ecosystem projects.

As such I understand the amount of work that goes into such reviews (from all sides), and even with a template and standardized process there is always going to be need for customized reviews for each grant (either when defining scope, or directly reviewing such consequences).

We could define very precisely what we consider to be a pass and a fail for the audit, a fail would just require a revised application imo.

Audits are rarely pass/fail - ideally an audit (whether financial or technical) provides an essential insight into how a system operates, what risks are prevalent, how those risks are being managed and how the system will evolve in the future.

An audit might occasionally raise a risk so severe that the programme needs to stop and immediately address it, but more often than not you end up with a long list of potential risks that need to be managed. - (see the updated Zbay Grant which acted on my suggestions to include more milestones for better tracking and managing risks as they develop.)

Ultimately, this is why I am skeptical of proposals that require the committee to work less than full time, someone needs to be keeping tracking of how the MGRC is distributing funds, what risks are at play (no grant is ever 0-risk), and how teams are dealing with them.

6 Likes

Hi,

There seems to be a lot of security engineers applying for this. That is always a good thing. I completely agree with how you are defining audit and the amount of work they involve.

For me, I have done FIPS ( Federal Information Processing Standards) testing and actual audits for Hardware Security Modules and Datacryptors. I have farmed this testing out to companies like NGS to get our products tested too.

I have also professionally done the much more ambiguous “security audits” these normally were as part of pen testing assignments. where the whole thing was the audit and I was responsible for a specific subsection (generally runtime analysis stuff and memory data management).

Real, legal audits, cost a lot of money and take a lot of time. You cant get FIPS certifications over night and without spending a lot of money (FIPS seems the most relevant one that I have had experience with.) For nearly all financial and government security audits part of the process is using a requirements based testing model (think of software like IBM DOORS to track progress) - This could well be a step too far. You would be forcing teams to adhere to a development process. This is generally very limiting for attracting developers, it certainly was an issue where I worked. This needs to be avoided if possible.

What I would like to see (and what I think people mean when they say audit) would be a certification scheme more inline with how console games are certified.

For developers to use the zcash trademark if they need to pass a series of tests. These would be requirements and unit tests. For wallet software this might include:

  • zcash logo must not be altered.
  • change must be sent to a zaddr
  • many other requirements.

These would be general, like the case of logo usage and they would be project specific unit tests set by the MGRC, like the case of change. @PhusionPhil is this more inline with what you were thinking?

I really think certification is the way to go not audits. I mean what would be audited against? what standards? We could come up with some eventually, but certification requirements is the first step in that direction.

Defining the standards for certification and doing the testing, including milestone testing really is not that big of a deal. It is certainly not a part time job in the first year, but after the standards are defined, the testcase database made and a bug tracker put in it is not full time work. I did this back in the day for the launch of the xbox, so I have relevant experience.

Sure but micromanagement is not part of this either. 6 week to 3 month milestones with community reports should not be an issue for the MGRC. Just tie funding to passing milestones. Then define passing the milestones based of unit tests and predefine requirements outline at grant approval time.

I think the work will be oddly weighted with it occurring in fits and spurts. so for 4 weeks there might be nothing to do, then you might be full time for 2 weeks. then part time for 2 weeks then nothing for 6 weeks. We just dont know. So there needs to be flexibility in the candidates.

I would love to hear what you think on this.

cheers,

steve.

1 Like

While I agree with you, to some extent, that certification standardization is a front-loaded process, I also think that tracking grants along any such standard is a very minor portion of what I would consider the work of the committee. These are the ideal cases and the major work of the committee needs to be focused on where these tests fail.

Referring back to a reply I gave in the main thread on the MGRC I think it likely that the committee will be called upon to spent a lot of time responding to deviations from the milestones and ensuring that grant money is invested effectively - I feel the need to emphasize that that often means the most work happens when things don’t go to plan:

I also think it is worth pointing out that even if we were to accept the idea that the work of the committee might taper off once all the processes and procedures have been put in place, time still needs to be spent building and field testing those processes and procedures in the first place. We can debate the potential steady states of the review committee, but we must acknowledge that the initial term will very much be not a steady-state, it will require dedicated effort from committee members (and with planning that effort should eventually become sublinear to the number of grants). As you put it:

It is certainly not a part time job in the first year

Some more specific thoughts:

This seems explicitly outside of the scope of the MGRC given that it has no authority over the zcash trademark (while I could see such a scheme in the future of the MGRC, I don’t think it is a wise use of the MGRC time or resources in the initial few years).

As I mentioned, I don’t think audits are the most effective mechanism for evaluating grant recipients or outcomes but there are several domains in which audits can and might be considered (although these would primarily be financial / organizational audits and I will stress again that I think these should be the exception to applicants, not the rule)

That being said, I will clarify:

The MGRC is in a position to help set standards of excellence and practice - and back that up with funding to help teams reach those levels.

Privacy and Anonymity are very peculiar risk models that aren’t served well by most commercially focused security review processes. As Zcash starts gathering a larger number of apps promising the mitigations of such risks I think it is vital that we don’t repeat the mistakes we have seen in other ecosystems and in the early days of the zcash app ecosystem where anonymity is heralded and then apps fall over and expose users at the first hurdle or inappropriately promise anonymity in the name of zcash while doing little to mitigate risks outside of the ones that zcash can protect against.

I think this includes an expectation (and funding of):

  • Ethics Review for studies and software (see the Zbay link in my previous post for an example) to discover and capture risks. Grantees (and the committee) have a responsibility to ensure that they aren’t harming the communities they are purporting to support - this means setting up and engaging in ethics reviews and responding to the outcomes. Sometimes this will mean stopping a grant, other times it will be reworking it. I have lost count of the number of community groups, charities and marginalized people I know who will not touch cryptocurrency because earlier groups showed up promising the world, parachuted in a bunch of half working technology and then left those groups to clean up the mess. We need to be better than that.

  • Design Review milestones than include explicit risk models and mitigations. One of my biggest fears around Zcash is we end up with a large ecosystem in 5 years that does nothing to advance financial privacy or censorship, and Zcash suffers the same fate as many other privacy ecosystem projects - offering great privacy at one layer while ignoring all others.

  • Peer Review of new technologies and software outside of traditional academia and industry committees (e.g. community review where grants are targeting specific groups) - the MGRC should act as a bridge between various parts of the ecosystem, connecting and encouraging work between groups to avoid rework and encourage reuse and integration.

All of that is why I think it is important to push for an active and full time inaugural committee; some of these processes will shape out, others won’t. But developing them in concert with grant applicants, recipients and the community at large is going to take dedicated time and effort. Anything beyond is speculative.

6 Likes

The certification scheme would very much appeal to me. Thank you for clarifying the audit process for the community as well as myself @sarahjamielewis

2 Likes

There is a lot of talk right now in the main MGRC thread regarding compensation and related issues.

These are issues that are most likely to be decided at the ballot rather than prior, so in the interests for transparency I will lay out my position on this topic explicitly:

If elected to the MGRC I will vote for internal processes that:

  • Require 2-3 MGRC members to work full time (as outlined in this thread) - sourcing and meeting with applicants, reviewing applications, developing metrics and all the other work I previously talked about in the main thread:

  • Compensate MRGC members for that work. As wonderful as the mission is, missions don’t feed people and we need to establish that this committee is not just for those that are already wealthy or who already have secured positions within the ecosystem. If the community values diversity of thought, opinions and backgrounds then it is essential that MGRC members are paid for the work that they do, and is seen as a viable option to as many people as possible.

  • I will vote for processes that denote this compensation as a mixture of ZEC and/or ZEC futures to ensure that committee members have skin in the game.

  • Minimize the influence of ECC and ZF employees, associates and/or affiliates in the running of the MGRC. One of the primary goals of the MGRC is the decentralization the ecosystem, there are rightfully concerns within the community that capture of MGRC seats by such affiliates risks turning the MGRC into a proxy arena for the two incumbents in this space. It is vital that the MGRC establishes independence and primary accountability to the community.

I believe that all of the above are essential for the MGRC to be successful.

3 Likes

So MGRC employees should be compensated for hours worked.
But your proposal, as with a number of people’s, seems to want to implicitly make compensation an incentive by making it in ZEC and thus subject to appreciation.

What do you make of the fact that we had such incentivization for the past four years for ECC leadership and it is, unarguably, a failure. For the past 4 years zcash has trended down relative to most cryptocurrencies of merit. From that we can either conclude Zcash is doing well and therefore market performance isn’t what we want to incentivize or we can conclude Zcash is doing poorly and it wasn’t an effective incentive. What will be different going forward?

1 Like

Skin in the game is necessary but not sufficient for success.

Zcash launched with a lot of hype and was unable to deliver on much of the hype for (what I see as) a few key reasons:

  • Both ECC and ZF spent a lot of time (months to cumulative years of time) arguing with each other instead of (directly) investing in the ecosystem. What ecosystem investment there were came across (at least from an external perspective) flustered, in-considered, and led to the creation of countless security vulnerabilities across the zcash ecosystem.
  • During that 4 year period ECC also played a year long shell game with a critical vulnerability which, despite maybe being the best way to handle it, also deeply damaged the zcash brand.
  • Finally, how Zcash fairs in absolute terms is pretty meaningless - Zcash, and the rest of cryptospace, are inextricably tied to the price of Bitcoin (at least for the foreseeable future), which is tied to all the other integrations and short-lived experiments going on in that space. Zcash price relative to Bitcoin on any significant scale is a flat line. The decline of the last 4 years can be attributed almost 100% to the decline in the price of Bitcoin and related offerings.

The MGRC doesn’t control the price of Bitcoin. The only levers it has to incentivize people are ZEC, ZEC Futures, derivatives thereof and some amount of power within the Zcash ecosystem. There is inherent risk in someone aligning themselves to those incentives - and we must not forget that those who serve on the MGRC are taking on that risk.

Are other systems needed to maximize the output of the MGRC? Absolutely, and I’ve talked in this thread and elsewhere about the processes and policies that I would bring to vote on the committee to bring ethical and security and risk review criteria into the grant process, track the progress of grants beyond one-and-done checkpoints and bind committee members to strict disclosure policies for conflicts of interests.

Should you also vote for people who are in it for the mission? Absolutely, I think my bio speaks for itself in that regard. But as I outlined above, missions don’t feed people, and some incentives structure is needed - and the MGRC only has a few levers to play with.

4 Likes

I agree your bio speaks for yourself and I think i’m going to ask this question of everyone. I don’t mean to pick on you . In the main MGRC thread the incentives issue came up and at least there, it seemed to be giving people a false hope. “If incentives are aligned, things will be good.” So, implicitly, we need not create other structures/incentives to make sure the right things happen.

You are right we care about relative performance, not absolute. But it’s gone down relative to everything. The below graph is log scale and shows a steady decline relative to BTC. It’s a 10x slip. The same would be true of Eth. And we’ve dropped in relative market cap by a large amount. We can ignore the market, but everyone suggesting incentives is saying we should listen to it. So , let’s listen to its past performance.

I’m skeptical we can attribute this to ZFND and ECC fighting. That’s a development over the last year when the trend is much longer than that. And the market largely ignored the security issue. But I could be wrong there and maybe those are the mistakes that lead us here. The question is, what do we do going forward to avoid those kind of mistakes. Because incentives clearly didn’t cut it. We need other mechanisms.

5 Likes

Well it hasn’t helped that there is little marketing and not enough effort to get Z addresses on exchanges so far. I assume that is because the wallets are not ready yet. Whenever ZEC holders want the ECC/ZF to do things that would help the price of ZEC they always say they don’t want to influence the price. Yet they expect people to invest in Zcash to give them money to continue developing and the holders have gotten little to nothing in return. This has also directly affected the incentive alignment. So thus far it has been a combination of still needing development and promotion. The ECC is happy to give their work away and not get the value back for their holders. That is one of many reasons why Ethereum is doing well and will continue to do so. How does sharing or giving away the tech benefit ZEC holders? They should be required to use Zcash in the other projects that use Zcash technology.

If Zcash doesn’t move to Z addresses and monetize its’ technology it will fail.Also, Zcash is GPDR compliant and has/will have? viewing keys so why is it not more accepted? It should be an accepted privacy coin for those reasons. I am still optimistic the value will go up, but these issues need to be addressed. I think most of this is moot, however until the wallets are mature to fully enable z address functionality.

1 Like

I agree we need wallets but I want to draw a line between individual technical choices and the way choices are made. When people talk about incentives, they are saying we need to make sure choices are made by a correct process. It seems to be obvious to everyone we need something to cause that to happen. Multiple people are talking about incentives and it’s not primarily because they themselves want money, it’s because they think we need to fill the decision making/governance mechanism void. And I agree. But there’s no way, given zec’s market performance to argue incentives are it. Either it failed to work because it didn’t cause us to do well in the market for four years, or it failed to work because we don’t care about market performance. So we need something else to fill the void. Something else that will lead people to develop wallets, or custom assets, or scaling, or whatever other technical fix it is.

2 Likes

Hi !
Can you be more clear about what you intend to fund ?
Your resume says you worked with “marginalized” and “targeted” communities (sounds like politically charged words), is it a hint that you want to focus spending money that won’t directly benefit the common folks ?

If yes, I think this is a big strategic mistake and looks like an easy money grab from the Zcash community to the benefits of only few people.

They are politically charged words. In many cases they are risk models where failure means death or imprisonment. As Emma Goldman once remarked “As an anarchist my place has always been on the side of the persecuted.”

I work with sex workers, queer communities, activists and journalists because they are the communities that have the most to lose from a lack of privacy. They are communities that understand what is at stake. They live their risk models.

If you can’t protect the privacy of those that society deems deviant or criminal then you can’t protect the privacy of anyone.

is it a hint that you want to focus spending money that won’t directly benefit the common folks ?

This is an incredibly naive take. Do you want “privacy” or do you want privacy? To me the core robustness, dare I say the anti-fragility, of the privacy and security (and thus the future) of Zcash must be on the margins, on the extreme ends of the risks that undermine freedom - that means building systems that understand those extremes, and can work with them to ensure that everyone shares in the power that the system controls.

My research has always focused on those extremes. It’s why I first started delving into the extent of deanonymzation vulnerabilities on dark web drug markets - because if people who have skin the game when it comes to not getting caught screw up the technology then guess what? "common folks " don’t stand a chance in hell.

Can you be more clear about what you intend to fund ?

Here is a short and incomplete wish list:

  • Wallets that have a utility beyond the simple sending and receiving of transactions - I’d like to see more thought given to:
    • Organizational Accounting - Beyond CSV export and into integrating with various other software, providing safe & automated tracking of worth against various other assets (BTC / USD etc.)
    • Trading Integration - One day soon I’d like to see sheilded zcash become an option on decentralized exchanges like Bisq, but even basic integration would go a long way
    • Donations
  • Research and Development into Memo based applications (more experiments like Zbay and Token Based Services) and the future of memos within the Zcash ecosystem.
  • Projects within targeted communities to understand the limits of Zcash adoption within those communities. Projects within those communities to experiment with possible on-the-ground applications. Along with the development of an ethical review process (as I helped Zbay design: Request for feedback on Zbay user-interview design) to ensure that such work is held to the highest standards and prevents the parachuting that haunts the rest of this space.
  • Libraries to make efficient / secure in-app light client integration a (secure & private) possibility.
  • Security and Privacy reviews of the software and hardware stacks around Zcash (Tor, new Mixnets, User Interface Libraries, Kernel Extensions etc.) to ensure that holes in our environment don’t compromise the privacy of Zcash and Zcash Ecosystem projects.
9 Likes

Thanks for the answer, I understand better. :+1:
I’ve always seen Zcash privacy as an all-inclusive one so it raised me an eye brow, but I’m happy to see the kind of project you would fund

OK, just read this. I will remove a redundant question (apologies).
Right after reading this, I feel jazzed about your candidacy.
I feel like you articulate and represent many of my concerns and interests.

Of course… there are other candidate threads to read…

But I would vote for you, if I could vote right now!

Given your expertise, it seems reasonable to ask how you would handle things in the face of a bug like this?

It seems like there’re a couple of ways to respond:

  1. Generally what’s the right approach to such a situation?
  2. Given the actual counterfeiting vuln mitigation that occurred what did you learn? How would you apply that Zcash-specific learning to the next vuln?

If the way the vuln was handled was maybe the best way, what are the other hypothetical ways in contention for best?

Specific to the MGRC role, if the MGRC is involved in technical audits (which sounds good to me), how would disclosure of vulnerabilities be handled? Would it differ significantly from how the ECC disclosed the linked vuln?

6 Likes

Probably worth starting with a note that I have a history of tweeting out critical vulnerabilities in the zcash ecosystem - that is where we are today - critical vulnerabilities being discovered hours after a release to users. It is not a good place to be for a coin whose main differentiation is privacy and security.

I want to get beyond simply discovering and disclosing vulnerabilities and into the realm of of a high standard of security engineering across the ecosystem. In an ideal world I want grant rewards to be tied to design and implementations that are accompanied by
thoroguh security assessments prior to shipping and advertising features to users.

Where projects lack the expertise to that I want to be able to provide them with options to access or acquire them.

Of course we are not going to get there overnight, and so I think it is likely worth exploring the establishment of a far-reaching disclosure reward program to flush out the rest of the low-medium hanging fruit in the ecosystem. The cryptocurrency ecosystem is a hostile one and mistakes have financial incentives tied to their exploitation and so we need to tie financial incentives to disclosing bugs to those who are best place to fix them, and maintain that standard across the ecosystem. And projects who decide to take on the responsibility of building infrastructure for the Zcash ecosystem need to understand that responsibility and accept that responsibility.

Specific to the MGRC role, if the MGRC is involved in technical audits (which sounds good to me), how would disclosure of vulnerabilities be handled?

In an ideal setup the MGRC would act only as the sponsor of these kind of programs and wouldn’t be directly involved in disclosing these vulnerabilities e.g. a bug found in a zcash node would be reported directly to ECC or ZF teams, a bug found in Zecwallet would be reported there.

Generally what’s the right approach to such a situation?
Given the actual counterfeiting vuln mitigation that occurred what did you learn? How would you apply that Zcash-specific learning to the next vuln?

The counterfeiting bug was a catastrophic bug. There is no right way to respond to them. As I mentioned at the time I think the zcash team handled it well, but I think the precedent they
set for masking an ongoing recovery with operational incompetence was a pretty far-reaching one. I’d have liked to have seen a deeper postmortem regarding the operational aspects of recovery and the impact of those kinds of decisions on community perception and future bugs.

That being said, the decisions that have to be made once critical or catastrophic bugs are found are never going to be ideal or free from consequence or compromise. The priority should be in robust up-front analysis and design to minimize the impact of such discoveries in the first place - energy invested there has far greater returns than in a disclosure process.

Personally, I believe that the biggest security-related risk to zcash no longer lies in catastrophic full-node vulnerabilities but in the quickly expanding ecosystem. Regardless, it is clear to me that the MGRC is well placed to provide an avenue for addressing the concern more generally through the kinds of initiatives I’ve described above.

6 Likes

Are you aware of Zecwallet Lite?

Is there a sense in which it is not ready for exchange use?

I’ve been using it on mobile, and it’s been pretty slick! It’s the kind of system that I worry about the security of, and would love to have more assurances of security ala the audits @sarahjamielewis proposes.

In a slight tangent, I’d like to get @sarahjamielewis 's response to the diasporization of Rust from Mozilla (yup, I did just make that word up).

Is there something the MGRC could/should do with respect to the the nascent Rust Foundation and/or the potential hireable engineers from Mozilla? Any ideas?

I’m excited. I’ve come to love Rust as a language and as a community. I also understand some amount of the vast amount of work that setting up an independent organization entails. There is a great team working on it and, as I said, I’m excited about what they come up with.

ZIP1014 mostly precludes the MGRC from actions outside of the Zcash Ecosystem directly:

Major Grants SHOULD be restricted to furthering the Zcash cryptocurrency and its ecosystem

While there is some flexibility there, I don’t think it is within the intent of the MGRC to take action. Given the reliance of Zcash on the Rust ecosystem, with both ECC and ZF making heavy use of it, it likely makes more sense for those organizations to consider sponsoring the nascent foundation.

That being said, I do think it is within the remit of the MGRC to put forward “Requests for Proposals (RfPs)” that would appeal to the influx of talent coming from Mozilla (although I imagine most have already found new work already so we would have to be quick about it) e.g. RfPs for reviews of the Zcash Rust ecosystems (I note that Mozilla let go a significant portion of their security & system engineers so there is a lot of talent out there) or RfPs for integration projects or far-out experiments (what if we funded a privacy focused web browser with zcash payments built in?)

2 Likes

I don’t believe any wallet is at least version 1.0. They are still being worked on plus the Ledger Z address wallet won’t be ready for roughly another few months.

1 Like