Request for Input: Securing the Zcash Ecosystem

Community Grants
4

I am in full support of this idea. A concern struck me after I read this:

Finding people who are cross-disciplinary enough to be able to audit both protocol and application level projects seems like a huge ask. It may be good to have someone or some people in the community who are a part of the team in a very limited role, but are responsible for helping source the auditors on a per-project basis. The idea of having 1-2 people who do shifts over the course of a quarter-year time is good if they are incredibly cross-disciplinary, but that will be a difficult find.

5 Likes
5

I would recommend collaborating with OSTIF to help identify and negotiate with security auditors.

1 Like
6

When you say team you are mixing lots of different skillsets.

  • I suggested this in my application. It would be better to have a general purpose team with crypto ability, like NCC and use them to farm out the rest.
  • Webapp pentesting is not cryptographic maths assurance which is not network segregation or HSM/container management, which is not the same as app development.
  • all carry unique issues that need specialists (this is why OWASP was made, hardened PHP, etc.)
  • Yes, I know a number of different companies that I could reach out to over this.
  • I would like to see it organised, as in “this software needs to pass these requirements” - as outlined by the mgrc, then tested for by people with insurance and reputation in that area.

btw - your link. kinda 404’s but shawn explained removed image.

A great example of of this would be the bug you found. That should never have made it to security review. it should have been part of release test procedures. There is a lot of room for automation and manual conformation. but these test harnesses need to be built on an technology or device undertest. (that might translate bad. I mean. You build a test harness to test zcashd, or zecwallet, or if you are particularly brave and really good, libraries.)

2 Likes
8

The link is good, Sarah will generate a CCR, there are none currently open @mistfpga , only proposals.

9

Hi Sarah

– Just me, you can skip this part.
I’m Benjamin Winston, I work for the ECC as their Director of Security. I’ve been lucky enough to work on the test team for several great consultancies, and I’ve worked directly for a few vendors in the past, most recently at Fitbit looking at their full stack and firmware. The transition from external assessment to internal vendors really taught me a lot about being a security practitioner, including making my fair share of mistakes along the way.

I had been hired while CVE-2019-7167 was being worked on, but I wasn’t briefed on it until just prior to Sapling launch, at which point I handled the disclosure process. Many sleepless nights and stress on that one lol :slight_smile:

I handle the engagement of ECC with external security assessors, and I run our internal security function doing stuff like this Fuzzing Zcash with Kubernetes - Electric Coin Company
– end of me, please stop skipping here

I’m glad that we’ve got representation of security interests on the ZOMG and I agree that forcing spend into silos will reduce the effectiveness of the effort to check the security of the Zcash ecosystem by doing external assessments.

It also makes sense to me that there be a resource for security input into designs ahead of the “oops we didn’t mean to do that” and reengineering that inevitably follows someone engineering without first doing some security planning.

Certainly I don’t want to see basic TLS security or any other table-stakes control omitted from any product, which is why we started an outreach program with Aditya, unstoppable, nighthawk around their security. Our (I keep saying our - I mean mine and Taylor’s) intentions around that was to provide maximum reward for the ecosystem with a minimum invetment of effort on our part.

After you dropped that bug in Aditya’s code on Twitter with no warning or forward disclosure, we reached out to him and tried to provide a service. We created a key result around our performance in this area in response to that event to the effect of “no simple-to-discover issue gets by this program”. We defined “simple to discover” that a competent attacker would find it in the first few moments of analyzing the software.

I must confess I do wonder about your commitment to Ethics Reviews around projects harming the communities they’re intended to support. I wonder whether you consider dropping that bug to not be harming the community, or whether you think it only doesn’t detract from your ability to make that determination in other people’s efforts.

Thus far this program, as ah-hoc as it has been has seemed to provide at least some assurance to projects that might otherwise have experienced none. We had problems regulating our time on it though - it’s hard to dedicate exactly half a day per week to something while we’re operating in what is basically a high throughput interrupt-driven environment like ECC’s security.

One thing we’ve been able to organize while the Zcash project was just ECC (well, ZcashCo at the time) and the foundation is to adapt the norms of responsible disclosure to the wilds of the cyptocurrency space. I was lucky enough to be invited to a working group that ultimately produced this standard: GitHub - RD-Crypto-Spec/Responsible-Disclosure: A standard point from which crypto projects may derive their responsible disclosure policy. and we committed to it here: https://github.com/zcash/zcash/blob/master/responsible_disclosure.md

Briefly, the standard is intended to provide a commitment scheme to the destinations and conditions under which we will disclose security issues, so as to be able to provide continuous user safety in a distributed consensus-critical environment. The intent is to separate, basically, the transport negotiations from the communication itself so as to speed up our response times and reduce overhead when dealing with critical bugs. This complements ECC’s internal processes on handling security issues.

We just completed negotiations (yesterday) with Deirdre and Teor at the foundation to add them to our public commitment (expect a PR for that in zcash/zcash later today, I’m delighted to have them on board for this) and with Nighthawk and Unstoppable who have adopted our wallet code (expect a PR for that later today or early next week in those repos).

My hope is that if you come into contact with any bugs, critical or otherwise, in any of the code I’ve been employed to coordinate the security of that you’ll work with us on our polite request to please report them via our security disclosure comms point, which is documented on our website here: Security Information - Zcash and we will deal with them seriously.

I stand ready to provide whatever other input might be helpful to ZOMG in the effort to secure the ecosystem, although of course ultimately we don’t have too much spare capacity in terms of individual contributions, I’m here to help you in any other way I can.

Thanks

bambam

6 Likes
10

This point was rehashed numerous times in my candidacy. I’m not sure what there is to add, and I can only speculate as to why you feel you need to add your voice to this now instead of then, but my answer remains the same.

On that latter point, I am glad that my actions were the prompt for ECC to take ecosystem software risks somewhat seriously.

If you had actually read my post you would understand that my goal here is to establish funding for a team that can proactively address these challenges. Instead you have chosen to impugn my ethics and make this about some nebulous concept of “responsible” disclosure in an inherently hostile environment.

Thank you for your comments. There are clearly deep divisions that I thought had been somewhat resolved through the candidacy period and election results, but which I now see through recent conversations are deeper and systemic to core parts of this ecosystem. Given such constraints I’ll have to consider if my participation can be ultimately useful or effective.

3 Likes
11

<rant>

I’m officially annoyed that we (I mean everyone) have failed to move on from that event.

Bruised egos, sore ethics…yeah, I get it, but I really don’t care & it was a long time ago.

Please can we now move towards what we want to happen in future ?

(not aimed at anyone in particular)

</rant>

6 Likes
12

I think @zebambam owes @sarahjamielewis a public apology. Her involvement with ZOMG was one of the things that made me most optimistic about the future of Zcash attracting more open source developers that make meaningful contributions.

Losing @sarahjamielewis is a huge blow. @zooko please think carefully when hiring and retaining people like @zebambam that are part of the reason for a toxic culture at ECC and create toxicity in the community.

1 Like
13

Sarah wants to take on the surveillance state but can’t handle a little criticism? This has to be a joke.

14

I was extremely surprised to hear this weekend of Sarah’s resignation, and sad to see that my reply was a contributing factor. I’m not active on these Forums at all and clearly there was a wider situation that I stepped into that I was unaware of. I am taking the time to listen, hear and reflect on all the comments around what I posted so that I can do better in future.

12 Likes
15

Good to hear that you are taking some time to reflect on how your reply may have impacted her.

Being more active on the forum would be a positive. Public comments from someone in your position acknowledging the importance of open security issues will help rebuild confidence from the community in your stewardship. Absent that, some many will falsely assume that you either don’t take some issues seriously or do not find it important to keep the community updated. Zcash is a decentralized blockchain with many stakeholders. Please interact with the wider ecosystem as much or more than you interact with ECC.

5 Likes
16

I lik what this guy said.

17

I think it would be a tragedy on top of a tragedy if Sarah’s resignation distracted us from this initiative that she’s kicked off with this request for input.

I’m really interested in pushing this forward and would still appreciate feedback!

The positive feedback from @aiyadt here—i.e. that this would be helpful for Nighthawk—is a very good sign. I feel exactly the same way for Zbay.

It seems like once we start reaching out to these firms, they’ll have ideas about the best ways to structure it. Ideally (and I’m cribbing this from Sarah’s suggestions) they’d be involved in the grants process, in the initial design of new applications or features, and in review and testing—both of projects ZOMG has funded and really any widely used projects in the ecosystem.

My vision for this also includes the security team publishing regular reports on the state of ecosystem privacy and security (like our own version of Google’s Threat Analysis Group) and on progress towards goals.

I think this can help focus our values and brand around privacy, as secparam suggests here, and communicate to both users and investors that—while it will be a long road and a ton of work to get there—Zcash is on a certain trajectory to becoming the leading private cryptocurrency.

“Zcash Ecosystem Security Team” has a zesty ring to it :slight_smile:

4 Likes
18

It doesnt work like that. @zebambam @earthrise @alchemydc have all explained in previous posts why not. im too tired to look them up sorry.

My NCC option was the best imho. but you will spend all your funds on “security testing” when normal testing and release procedures would fix them problem. now how do you get skilled people to test complex hardware across different platforms? you make a test department! im nt sure if the ecc or zfnd have dedicated test outside security.

19

Sarah is not professional on the bug report how is that even a loss for community Infact it’s good she left with such unprofessional behaviour she doesn’t deserve Associated with zcash anyway…losing her is good that we found such vicious behavior early better than late.

20

(My personal opinions, not necessarily those of ECC, are:)

What are your opinions on the creation of such a team?

Yes please! There are so many great projects in the Zcash ecosystem from hardware wallets to message boards to Slack replacements. It’s really hard to try to keep up with them all, and they all have different levels of security expertise, so the community could really benefit from a dedicated and well-funded effort here.

Doing after-the-fact audits alone will likely lead to lots of “oops, this design is totally broken, we need to re-do a lot of work.” IMO, it’s important for there to be security people working closely with the teams to help build a “safety culture”—a sense of pride around delivering reliable code and an eagerness to notice when things are security- or privacy-critical and call out for extra scrutiny.

Early and continuous input is especially important for privacy, because it’s nearly impossible to “add” privacy later to a design that’s already leaking a bunch of information in a bunch of different ways.

Do you have thoughts on the structure of such a team?
Do you have ideas on who you would like to see involved in the team?
Is this something you would prefer to see handled by e.g. a dedicated/contracted industry partner or constructed from scratch?

I would like to see it staffed by full time security and privacy people. Because:

  • There are enough different projects now that it’s a full-time job just to keep up with them.
  • Audit firms (NCC et al.) operate on a timeboxed contract basis, so I don’t think they don’t really provide the kind of from-the-ground-up continual service that fledgling projects need to grow into high-assurance software shops.
  • Full-time allows them to develop personal relationships with the people working on the projects, hopefully earning them some easy buy-in when they want to make recommendations around development practices and feature selection.
  • They can become Zcash security/privacy specialists over time, building up a deep understanding of the protocol and all of the subtle Zcash-specific things that can go wrong, as opposed to using third-party firms who won’t be as well-versed in the tech and privacy goals. (I love Holmes’ idea of something like Google’s TAG, I’m thinking take it even to the next level and create a Project Zero for the Zcash ecosystem.)

So perhaps a couple people on the team who are really great security engineers and communicators, combined with a budget for standard post-hoc auditing according to the team’s prioritization?

(By the way I’ve heard of https://latacora.com/ which does something like I’m imagining, but I have no experience of actually using them. It might be worth reaching out to them for an opinion on how best to implement this.)

7 Likes
21

This makes a lot of sense to me.

Is this similar to what @earthrise is saying about creating process and culture inside the team itself, and guiding design decisions?

Like, you’re saying to leverage the work of skilled people within the teams as much as possible and not wastefully consume the security consultants’ time, is that it?

I’ll reach out to them. Who else should we reach out to, besides NCC and Krebs as mentioned above? (Assuming they’d be interested in a more long-term retainer-type engagement than something time-boxed.)

1 Like
22

We need to separate privacy from security in this conversation.

Security is not all about testing. In fact, security testing is pretty useless to make software secure (odd thing to say about my own job, but it is true)

This post is about security specifically.

You can only get secure software from secure design and defensive/secure coding. Security testing is meant to verify this has been done.

I think @zebambam put it best with this entire post. I dont want to quote it all, but please read it again. - Application for Major Grants Review Committee - #41 by zebambam

Security has to start with Security Engineering and Security Research, the whole time being supported by Security test. - I have a few ideas, zcash is in a slightly different and much better position than most.

Ive been trying for ages. Hopefully you will have more luck. (in fact I think i had a massive whine about it - not like me, huh.

Also this:

@holmesworcester may I suggest that whoever is in on those calls/comms has at least one person who knows about these types of companies, testing and reporting and can speak the same language.

This is in very early stages, but I would like to see a zcash test dept, that covers this sort of thing as well as milestones, signoff on features, test plans, test cases, etc. I am not sure of the logistics of this yet.

Other companies - Yes, contact me via PM. I dont want to advertise companies I dont work with or dox the ones I do.

Thinking more on this makes me think we need to do this ourselves (the finding the right companies) - bambams post I linked reminded me of a few conversations.

If we give XYZ company who does do crypto and other pentesting the jb of finding pentest teams and specialists, we are just going to pay 3 times for contractors. we want to be establishing long term knowledge if possible.

We have at least two “internal” development teams and no “internal” test team. - who does the testing out of interest?

1 Like
23

kinda, its saying you cant retrofit security and if it wasnt developed with security in mind and constant minor security tests (Alongside the normal unit test ) you will spend a lot of money and have made nothing secure

It feels (not specifically you) that people think security is a testing job - whilst testers imho make the best researchers they dont make the best security engineers.

When I keep referencing “Requirements based testing” thats because it is the only way to have accountability - but its a nightmare for dev teams. and we want to be agile, and work with as many teams as possible, so it would, imho be a full time job working and testing (remember at this point all testing is security testing) and then using that to develop a software design plan that can be scrutinised by 3rd parties. (note, document not program). and worked into test cases, unit tests, test plans and milestone signoffs.

oh and you can automate a lot of this. so maybe before approaching someone like NCC, sit and have a private voice chat with myself, bambam and earthrise (if they are willing).

24

What does the new @ZcashGrants think about this idea?

Is it something you are still interested in following up on? I am, and I am pretty sure @earthrise is too.

Id be happy to help get the ball rolling.

5 Likes