Application for Major Grants Review Committee

I strongly urge it, it was a revolution for privacy freedoms. It solved a massive accessibility issue with tor

I take it you are familiar with tor? When you used to use it you had to set up a proxy on your own machine to route your traffic thorough and a myriad of other things that if you got wrong you would leak DNS queries over the tor network, cookies, etc. it was pretty easy to deanon people because they had it incorrectly configured.

The EFF released the Browser bundle which is a couple of clicks and you have firefox running a connecting over tor and the only thing that is using the tor connection is firefox. it solves a lot of the data leakage issues.

you can get it for your phone too.

1 Like

Oh sweet… I’ve just been using tor-browser.

I think thats it. all tor packages now days are the browser bundle - i just remember when there was tor and tor-browser so i call the new one the browser bundle. do you remember when you had to configure privoxy? I kind of see zcash is now at that stage, it works if you know how to configure it and use it right. but it needs to be packaged up in a nice ux and idiot proof config.

Imagine where the no script button is, next to that is a z2z button, click on it and you can z2z people.

1 Like

Theres an active Zcasher or two on the Rust discord, Im on there but really only in the beginner thread sometimes
(And for the purpose of Rust, not to shill Zec, I was merely stating the fact)

3 Likes

Shilling Rust could be a mutually-beneficial thing, the more that I think about it

2 Likes

@daira asked in the live stream chat, “what would you do if a really good project is asking for too much/little funding”

Negotiation is the short answer, to be in a position to negotiate effectively you need to know what too much or too little funding is.

We already have a project, arguably the first MGRC. - Zebra (parity client) - This is a major milestone for the community and zcash. This client is owned and maintained by the foundation. The costings from the zfnd will be really handy, also any feedback parity can give as to the operational expenses, ramp up time, etc, would be invaluable.

In addition all grants awarded by the zfnd could help add clarity, I would like to see more information on these two. (the zfnd has awarded outreach grants too, so we are not completely blind on everything)

The transparency reports for this are not available yet, and I assume the post mortems for the other grants are only available under NDA to the MGRC. I have reached out to the ZFND and will see where that leads.

(I really feel like I have posted this before… I cant seem to find it in my history tho.)

2 Likes

The current Zebra client is different from the original Parity Zebra client. It’s a rewrite.
Further, I wouldn’t consider Parity’s work on forking their bitcoin client to implement Zcash as the first MGRC, simply because I think other grants or service agreements that the ZFND did in the past have been given before that particular service agreement and they are equally likely to be counted at MGs. For example, the developer that worked on a Windows wallet and port for Zcash who currently no longer works on Zcash.

3 Likes

heh. I was trying to be poetic :slight_smile:

good point on zebra v parity. I kind of blurred them in my mind.

I have updated my original application with more information regarding how I see remuneration. tl;dr I am applying out of love, and whilst money helps, it is in no way a deal breaker for me.

I know I have posted these answers elsewhere or answered on the livestream, but I typed this up anyway.

The main ambigiouty is around how the MGRC will be structred and will operate. I dont have strong feelings on this in the first few weeks. I think it will become evident pretty quickly what the areas of ambiguity are.

I would see expert council in regards to activities not covered by the zip. This will primarily come from fellow MGRC members, the community, the zfnd and hopefully the ecc. as the MGRC grows I expect them to hone their communication skills to ensure efficient use of the resources available to them. The MGRC should at all times look to the forums as a primary means for asynchronous feedback and communication. I am happy to submit any zips for further resolving ambiguities, but feel this should be a last resort, we don’t want weekly CAP votes.

The MGRC should in the first instance look to solidify its oversight position. They have a lot of information available to them in the short term to get this part sorted asap (4 - 8 weeks max). Once this is done I would like to see them expand, if required, to a more proactive role - what this role should be and how they should execute it is very dependant on community feedback.

Because the zfnd is doing the remuneration there is no reason why this cannot be up and running before the halving ready for the first grants. (like thesis)

Please see my in depth answers here: - MGRC candidates teamwork questions - #6 by mistfpga

I would state that the MGRC’s officially supported and preferred means of asynchronous communication is the forum. Not everyone will create a forum account or want to submit ideas here so the MGRC should at least adopt all methods of communication that the zfnd does and try to reflect communications from other mediums onto the forums.

For within the MGRC I would like their communication to be both asynchronous and synchronous. Because of time zones it is pretty hard to hard define what the best mediums for these would be, but I expect google hangouts or similar, email and forum communication to be covered at a minimum. Work flow will be needed but that software depends on what the work is.

Apart from that it would be standard office stuff, professionalism, etc.

Being primarily associated with an alternative cryptocurrency is not inherently a conflict of interest. Depending on their role within that currency. The benefits someone like that could bring in general will far outweigh the risks. I expect standard Conflict Of Interest polices and community vetting via the voting process to be enough to identify any potential issues.

Hello @mistfpga For my vote, please answer my questions frankly:

  1. Are you pro BTC? If yes, Why? If not, Why?
  2. What is the largest account size you’ve handled in USD? How many end users did it impact?
  3. MGRC will control 8640 ZEC per month or 25920 per quarter, how will this be roughly spent? (provide napkin calculation).
  4. MGRC announcement attracts 100s of applicants from all over the world with all random ideas, all matching your goals, how would you evaluate them?
  5. KPIs aren’t entirely possible on a privacy preserving payments protocol project’s level, it’s all z2z, how will you evaluate funded team’s impact?
  6. DeFi fever made ETH run 2x compared to every cryptocurrency this year, thoughts?
  7. What locals, regions, languages, ethnicities, educational backgrounds of people have you worked with? What are your preferences of assembling teams that deliver?
  8. We live in a remote world now, how do you evaluate applicants for grants?
  9. Projects in Zcash are going to go through a huge change beyond the handful, driven teams funded via Zcash Foundation, thoughts?
  10. Zcash is a protocol at its core, ZEC price is volatile. How will you handle a single digit ZEC? ($9 x 8640/month = $77,760) How will you handle a 5 digit ZEC? ($21,000 x 8640/month = $181.44MM) Thoughts…
1 Like

i will, it ia 6am hre Ill do itn in 12hrs of that is okay.

Thanks for really food questions. I hope to get your support

1 Like

Hi,

Sorry for the late response. I am not sure I have addressed everything either. I need some sleep then will check it over again.

Im on about 3 hrs sleep in 2 days so I will probably edit this when i wake

I like BTC, but nowhere near as much as I used to. I got involved back in late 2010 i think amd 1100XT’s cpus. and a lot more so when GPU’s were a thing. BTC is not a panacea. It is just a very rough guide. (hashcash is more elegant in my minority opinion)

Im very pro privacy, and not too keen on coins without it.

For example I remember posting back on bitcointalk 2011ish) saying how BTC is an accountants wet dream. That being said I am strongly against the idea of quantitively easing. but that is probably because my economics is not that that strong. I am much more in this more for the tech. It is also really fun to be part of something new.

I have been responsible for +300,000 of GBP in salaries and about the same as that for R&D tech, in yearly budgets (although this feels a but of a push, HR and purchasing did all this, i just told them what to buy and signed stuff)

Short answer “forwarding the zcash ecosystem”

Due to napkin restraint i see it something like this ( I have pie charts…but thought it unfair)
weeks 0 - 7 = 17,280
I see this as monies unlikely to be spent in any significance. maybe 50 zec on preliminary DD reports

weeks 8 - 16, This depends on the Thesis funding. I would like to repeat actions from month 1

weeks 24+, Hopefully results from Thesis - glowing press everywhere and people now are coming to us.
If not, I believe that some of that warchest can be used for prizes, or to encourage people to tender for contracts.

Now we revaluate the current proposals and warchest. Apart from thesis I expect many others to come in within the first few months. I just know thesis want to work with the MGRC and will be forgiving of any mistakes (that goes both ways)

I do not believe it is constructive to say we will give z to this project and y to that project. The dev fund is a series of pie charts 50%roi and 50% non roi, then sub divided again.

With the same due diligence I would if there was 1 application. If language was to be come a barrier, then I would see professional advice for those.
To help with the volume of submissions I would create a rigid template and set up an official submission procedure. If an application is rejected for not following process then that needs to be communicated to the application so they can submit a more formal one next time (as I said with the dev fund proposal, no idea is a bad idea)

% based Milestones, tied to code review and sync. daily build unit test results, 2 hrs weekly manually testing reports. (this is done by the vendor) but put in a database the MGRC can access.

I have the skills to quickly tell if something has been done or not. This is standard in all software industries. I personally favour requirements based testing for the KPI’s, so while the KPI might not be provable, the work is. This is an issue that the media (specifically games) solved a long time ago. (this is something I can do)

The final Alpha and Beta need to be done by a proper 3rd party company NCC or Portcullis (or do they just do web stuff now) anyway these people exist and can audit crypto. They are not cheap. 10k+ a week. but we need it.

[quote]
DeFi fever made ETH run 2x compared to every cryptocurrency this year, thoughts?
[\quote]

Glad we are not eth.

Various. As a European (until jan 2021) he UK really is pretty diverse. However I have worked in America, Australian, Germany and France. I was responsible for the all language transition of PhotoShop CS2 - have you seen my spelling?

For recgonising languages, i am fine with European, Chinese/Korea/Japanese/Vietnamese, etc. and no clue with Cyrillic or similar.

Merit.

The dev fund proves its worth. is what I take away from that.

There is also talented teams that might want to learn zcash.

In my personal opinion ZEC is a mechanism to allow zcash to become something. If zec is worth pennies and zcash is still sound then there is hope and I will be here.

At the other end of the scale, I would hope the foundation would be able to cope with the large influx of funds and and not go crazy spending it. providing sustainability for privacy for years to come.

I want zec to moon, but it would devastate me if that mooning was the cause of the downfall of zcash.

Im on about 3 hrs sleep in 2 days so I will probably edit this when i wake

2 Likes

Good Luck Steve.

2 Likes

I was contacted via another method than the forum asking why I updated the security aspect of my application and what specifically does it have to do with how I see the MGRC remit.

I purposely left this out earlier, but the conversation moved in that direction so I thought I would let people know my experience in this area and would like to do this as part of my MGRC role (as well as the community stuff, I really like doing that).

What I would like, and will advocate for is relevant software/services to go through proper audits. These are not cheap.

I do not know how much the Sapling report cost, however I do know that from experience 14 days of FIPS related cryptographic work by NCC (the people that did the sapling report) cost in excess of 30k GBP. This is for three people, 1 senior expert, one “trainee” (they had 5+ years experience) and one crypto expert + myself and two other members from my team assisting. (sapling was 40 days and 6 NCC consultants)

I would like the MGRC to fund this work as part of MG applications. So say for example Thesis came to the MGRC with an application for a GO based client, then I would ask for at least one audit, at the expense of the MGRC. - If this audit fails badly there will need to be something in the grant payments (fit for purpose is a legal UK term).

depending on the amount of work for different applicants it could well be worth he MGRC open negotiation with NCC. - Yes other auditors need to be used too, but I believe the MGRC is in a strong position to negotiate an “on going” contract work/retainer with the NCCGroup if required.

This is the sapling report - https://www.nccgroup.com/us/our-research/zcash-overwinter-consensus-and-sapling-cryptography-review/?research=Public+Reports

Understanding these reports and making sure you get the right report/testing done in the first place is very much a skill. You need to know how to question and work with these companies. It is something I can bring to the MGRC - especially since a lot of people are being very vocal about stuff being audited to ISO and RFC security standards.

It is very easy to handwave and say “we need to include this stuff” - but the practicalities of this requires people with solid professional experience in getting this work done otherwise it will be a very big waste of time, money and will probably miss security flaws.

I know of at least two other people as from myself with professional experience with this sort of thing and a vested interest in making sure the work is appropriate, fair cost and understand the limitations of the work. @alchemydc and @zebambam both of whom currently work for the ECC. Im 99% confident bambam was the main point of contact between the ECC and NCC. I strongly urge the MGRC to seek advice from these people, and trust their input.

There are plenty of other companies that specialise in other forms of security testing that would be relevant to MG applicants, be it web app, network access, etc, etc. being able to identify, verify the reputation and competitive pricing of these companies that will be most applicable to a MGRC is going to be a key skill.

4 Likes

Going a bit further,

This could tie in with my “seal of approval” idea too - one @jmsjsph elaborated on in his application, and added the security seal concept (that would have to be an audit). - if a “thing” with NCC would be beneficial then why not use them for a zcash security seal?

I would not use them for a quality seal though.

quality seals, like the nintendo seal rather than a verisign seal would probably be better done with requirements based unit tests - way beyond the scope of this post, but i wrote a lot of those for the original xbox…

here are the xbox 360 ones so you can get an idea of what they are like. I have a feeling you will like them zcash vr.

yes games fail these and still get released - I found several games that could break the xbox sandbox through savegame buffer overflows because they didnt hard fail some triple A games during certification for not doing save game content protection. (it was this and using a dev xbox with a kernel debugger that got me into fuzzing and RTA back in 2003)

The UK testing was a lot better than the USA testing. but USA games got released first, so when we found security issues in already released USA games there was little we could do to block them. Not many people know that the orginal xbox would recognise a microsoft usb keyboard and a few games you could press the tilde key and get the unreal console - (had to be in controller port 4)

Sony charge games publishers for submitting games to their certification process. Microsoft didnt for the original xbox because they wanted lots of games and they wanted them to be customised for the xbox rather than just an obvious PS port. I dont know if they charge for this testing now.

The whole seal of approval idea came from nintendo and they used it as a method to stop piracy rather than having to rely on hardware solutions (which were easily cracked) - in fact, there is a nintendo gameboy requirement that the game must scroll the nintendo logo on first boot, and the drm checks to see if its there. you can never stop piracy but I thought this was a novel idea. Just sue people. heh.

1 Like

This is all very fascinating and valuable information to me

Piracy seals, quality seals (including UX?), security seals (including UI?)…

So, I guess we start by agreeing upon a base set of “Technical Certification Requirements” for whatever seal(s) we seek to create? If the seals can be related to one another, maybe they can be offered in sequence… easy, medium, hard, from least to more safe and approved… by MGRCx(ECCxZF), eventually all 3.

Copy/pasted from the link you shared… this is what a base unit test looks like? Cool!

Base Requirements (BAS)

The following requirements apply to general coding quality standards, security, restricted-access system components, game behavior, and acceptable library usage.

TCR # 001 BAS Game Stability
Requirement On a functional console, the game must not enter an extended unresponsive state, cause unintentional loss of player data, crash, or cause an unintended reboot of the machine.
Intent Console game players expect that console games just work. Games that crash or hang or lose player progress reflect poorly on the Xbox 360 experience.

Privacy and Security are the same seal. Everything else would be Quality or Zeal of approval.

yes and no. this is a mammoth task in the first instance. you cant just come up with stuff. it really needs a couple of people who know they idea behind requirements based testing/development and then the areas of detail, like ux or web interface, change addresses, etc. (I did list a bit of this in another post.)

Edit: my post gave the wrong impression, I was posting requirements but it does look like I am calling them unit tests.

the document contains the base requirements. depending on what you are doing would define how the requirement applies to you. Microsoft don’t publish their unit tests or their testcases. only the requirements. (even then they are not public, i have no idea how those 360 ones are public…)

its the unit tests that are time consuming, testcases are more generally applicable and once written are easy to modify or update.

Once you have the unit tests passed you put it to certification testing (which is manual but for a game is about 20 man hours) - then it moves on to functional testing (which is ~1000 man hours, but this is not the responsibility of the mgrc.)

So if this was something people wanted to follow up on, I would start with basic general requirements, then manual testcases for those, then come up with what unit tests need to exist, then ask the MG applicants to write them, if they want the seal. The tests will need to be reviewed buy an independent party tho.

1 Like

Privacy & Security are the biggest words we have - so yes, we need people with proven skills and experience to defend those standards.

3 Likes

Audits are definitely important and necessary, but I feel that there needs to be some kind of ongoing/continuous security support provided to major grant recipients, or that grants should only be awarded to recipients with plans to provide their own security support.

This is especially true for any project developing private protocols (e.g. light wallets) – small information leaks in a protocol can combine together to create big information leaks, and it’s possible to end up with a design that’s irreparably broken if privacy isn’t considered in early designs.

Audits are great for getting a point-in-time snapshot of a project’s security level, but they don’t help with other important pieces of a secure software development lifecycle:

  • Incentivizing and responding to incoming vulnerability reports.
  • Secure coding practices, including code peer review, prioritizing test coverage, and running retrospectives after vulnerabilities are discovered.
  • Threat modeling and making sure the threat model matches users’ expectations.

To have those, the team building the software either needs to already be familiar with those practices, or they should have embedded security staff helping to educate them. Not all software needs to meet these high standards, but anything handling funds or impacting users’ privacy arguably should.

8 Likes

To elaborate, expand and reinforce somewhat what @earthrise just said, and sorry for the delay since the mention, I wanted to get accurate figures for you:

Hey @mistfpga

Thanks for @ing me. Indeed I’m the primary point of contact at ECC for all our external security assessments with commercial vendors. I’ve been planning, coordinating and responsible for our spend in this area since I joined ECC as director of product security in July of 2018.

I’m writing this for a general audience, I know that I’m repeating myself to you and that you know a lot of this stuff already from your experience. I’m just very quiet on this forum and I don’t want anyone to mistake what I’m saying and why I’m saying it. To that end, it’s worth me giving a bit of background on my experience because it’s relevant to the insight that has gone into the way we’ve been buying external security assessments, where they fit into our security program and why I’ve chosen to do things the way I have. If you’re not interested in this part or you already know or you just don’t want to know any more about me, feel free to skip the next paragraph entirely and instead move on to the next one about strategy.

Inserting myself

All I knew until my job prior to this one was security assessment work. Breaking into systems. Pentest, audit, people have different names for it, but basically finding security weaknesses and writing up reports. I started at the bottom and over several years worked up in rank to the higher technical levels through field promotions. Along the way I worked with lots of vendors on lots of contracts, formats, some ad-hoc, some on retainer, some systems (tragically?) were vulnerable in the same ways each time, others required significant amounts of new effort to break into. I broke into middleware applications, messaging, databases, database applications/stored procedures, web applications, network infrastructure, embedded software, door access systems, voip systems, basically anything customers were interested in, so long as it was theirs to attack. I worked with lead developers and system commissioners around the world in their offices and data centers for about ten years as part of several consultancies. Once I worked at a mobile phone vendor on their internal team for a while, and once I worked as a technical risk assessor, translating pentest reports into business risks. In that time the number of systems billed as unbreakable or that a pentest wasn’t necessary, that ended up being flawed and broken was staggering. It wasn’t until March of 2015 that I swapped over to defence full time though. I was the first technical security hire at Fitbit, so I started looking at general security, seeing somewhat more intimately the processes of a company that create security issues and for the first time trying to change those directly rather than just filing them in a report and moving on. I worked with hardware manufacturer vendors on new designs, external security tech vendors, their firmware team, etc. to drive in security into their design at every layer. Great bosses that I’ve had over the years all went on to better things - Google, linkedin, facebook etc. I try to stay in touch somewhat. I am not now nor have I ever been a cryptographer. I have analyzed products for cryptographic security, but only using universal reverse engineering, accepted practices for AES and cipher modes and basic key derivation functions. In short, I’m not a cryptographer, rockstar or otherwise, by any means.

If you got through my life story, hopefully it will be clear where I’m coming from when I say some of this stuff, and the perspective it’s coming from. I don’t believe it to be controversial, but I do feel the need to CYA when posting publicly:

Failure modes of security programs and thinking

There are some strategies that I have seen fail in commercial and open source endeavors with respect to security:

  1. All you need are pentests.

Having played both sides of this game, I can tell you that I am 100% confident that only having external security assessments in your security program is a quick path to repeated failure. I’ve seen time and again software vendors use only pentests to find the same class of bugs again and again. I know you’ve seen this too because we’ve spoken about it before.

  1. All you need are bug bounty programs.

This is an extension of 1) and equally wrong. Projects with only bug bounty programs when they get a real pentest get destroyed. Ergo, often necessary but never sufficient.

  1. All that’s needed is better education.

Education is important but not sufficient. If there’s no reason for someone to implement an appropriately secure system, having the means to do so means nothing.

So really each project, device, offering, system, needs to have a whole security program applied. There have been some attempts over the years to describe what a successful security program looks like, none of which fit any situation perfectly. They don’t fit, much in the same way that the various attempts to quantify security vulnerabilities through some scheme have basically failed over the years. Ref: heartbleed had a CVSS score of 5.0/10.0

I see my job as trying to create and run an entire program for ECC. My superiors at ECC see that job as including describing that program externally - something that was on our Q3 KRs but looks like it will get slipped to Q4 at this point.

Given all the above caveats about the place of external and independent security assessments in ECC’s security program and their usefulness in general, of course they are an important and necessary check on how well you’re doing. One way to think of them is as wellness visits. A bit like going to see the doctor - necessary for good health, but not sufficient. Can provide some useful insight into how well you’re doing.

Finally I’ll talk a bit about the unusually high quality of work that the ECC core engineering team kicks out. This isn’t just me promoting and congratulating their work, it’s of such high quality that it makes a material difference to our purchasing strategy.

Assessments of new core ECC software features typically turn up no materially impactful findings. This is important when we purchase assessments.

ECC’s purchasing strategy

[ Note: If you’re one of our vendors reading this post then know that we appreciate your efforts and nothing here was meant to be duplicitous in any way, but hopefully you see it rather as a solid application of market forces and respect for motivations around pwnage. ]

  1. We can’t be telling our vendors what to put in their reports.
  2. We can’t be telling our vendors how long they should be doing security assessments for.

Why? Well we’re forcing them to publish their work, for a start. This is unusual for them, most of their engagements are conducted in secret, used in secret, kept classified by whatever company they are commissioned to work for, and never see the light of day. The same is true for me - my life’s work, basically, is lost to time and kept in darkness. [ side note - computers and security were an escape for me from judgement of the people around me, but now that same mechanism has prevented them from seeing my best work, a cruel twist of irony ].

So working for us is a chance for these vendors to shine publicly, because the reports are published. Which means they have to stand by their work. Which means they’re open to criticism. Which means they’re going to be very sensitive to criticism about how much time we gave them to assess within a given scope.

[ another side note - most people see pentests as “secure” vs “not secure” but for any real system they’re really just a pressure test for how long it could withstand the pressure from these assessors at this time. It’s a limited result and most people don’t understand that. The system under test could have given out completely five minutes after the assessor ran out of time, but we’ll never know. For this reason the assessor’s job is typically to find the biggest problem the fastest possible way. ]

I’ve seen one of our vendors publicly decry the investment strategy of one open source project similar to ours - for dividing their investment between them and another vendor. I think they were probably right in what they were saying, but I didn’t envy them being put in that situation, and I find it difficult to see how that relationship could continue to be productive after that problem arose. We thus far have avoided that kind of fracas. I intend to keep our vendor relationships mutually beneficial for as long as I’m responsible for doing that.

For a while we were doubling up our investment on security assessments on a particular network upgrade. This is in stark contrast to dividing it in half. We did this by contacting one vendor and asking them to use their understanding of our threat model to scope an appropriate amount of time and then quite independently and without telling either party engaging with another vendor to do the same thing. It seems duplicitous, but when you think about it we’re treating each vendor with respect and giving them every chance to succeed.

After we did that the first time, and as part of their general market awareness, they’re then free to select their own scope based on the various competing interests - the desire to defend their work publicly, their desire to promote good work publicly, and their desire to keep us as a client by providing great work at a reasonable price.

We’ve had to cut this down because of financial constraints relatively recently, and now we only have one vendor per network upgrade to save costs. We reserve the right to switch vendors at any time, and we do that from time to time. That risk is still there and is in the mix when they’re scoping work.

Finally, in the case of security assessments that typically turn up no substantive issues, you really don’t want to overstress your assessors. Complacency, learned helplessness and a routine of not finding issues are enemies of great security assessment.

Having the same firm keep hammering away at our stuff and finding nothing that brings them ‘the joy of root’ again and again on retainer seems like a good way to lower the effectiveness of the assessments over time. This is just human nature, not a criticism of our world-class vendors.

tl;dr

All’s to say, we have some particular (maybe even peculiar?) requirements around our security assessment purchasing because of the exceptionally high quality of ECC core team’s work, and the fact that we insist on working publicly. The intersection of those issues is how I’ve concluded that we shouldn’t try to put firms on retainer for this purpose.

Importantly, as I said above, security assessments are really just a check that your security program is working.

In terms of cost, we typically spend anywhere from a thousand dollars to fifteen hundred dollars per person-day for general application security assessment and fifteen hundred to two thousand dollars per person-day for cryptographic security assessment reflecting both the increased specialism involved in that work and the scarcity of services in the marketplace that can perform competent assessments of cryptographic systems.

ECC spent 250k last year on external security assessments, the vast majority of which was for product assessment. We have spent 100k so far this year on external security assessments, all of which have been for product assessment. These numbers aren’t secret, and are published as part of our transparency reports. I believe that we get value for money from these services, despite their rarely finding very significant security issues.

Security proofs, which are a super important key element for our success, and I see as the most specialized cryptographic work we engage externals for, are perhaps somewhat ironically done at significantly lower cost by engaging academic institutions on an ad-hoc basis.

13 Likes