The infosec field is evolving in the era of AI cybersecurity and slop sophistication.
Every open source project, especially blockchains face the same problems: AI is getting really good at finding very critical obscure bugs, and AI makes it really cheap to report slop bugs.
Bounty programs are overwhelmed with false positive submissions, yet responsible disclosure of true existential threats needs to be rewarded.
In the Zcash ecosystem, we saw our teams get overwhelmed by both patching of true vulnerabilities while also getting swamped by spam reports, leading to the shutdown of the bug bounty program. Other open source ecosystems have done the same.
Now, while looking through the CHG (Coinholder Grants) submissions, I see bounty requests for obviously critical bugs (as evidenced by the entire ecosystem coordinating emergency upgrades) and requests for bugs I’ve never heard of. It seems the spam of the previous bounty programs is overflowing to CHG, which is also ill equipped to respond to anything besides the very obvious Orchard soundness bug.
Let’s figure out how to fix this reliably. We need to reward the Taylor Hornsby’s of the world while deterring sloppers.
My best idea so far, is to create a bonding system for responsible disclosures, so that the cost of submitting a bug is higher than the cost of the AI generation + the EV of payout. This should make submitting bugs that are rejected a substantial financial loss, while submitters that are highly confident in the bug submission outcome will feel comfortable taking the risk for profit. The bonds forfeited can go towards funding humans and AI reviewers if the slop remains high volume and low signal, or the cost deterrent may reduce the volume to manageable levels that can be funded by a grants program.
No, I believe this is honestly the worst strategy. Bug bounties is not something that should be gatekept because the impact is on the TVL, not the owned assets, which affects everyone.
Take AAVe. They run one of the most prolific bug bounty programs upto $10,000,000 on Sherlock, but the entry fee is $100 USDC and you cannot escalate later. That means bugs are obviously missed, as evidenced by the Kelp DAO hack draining $100+ million in liquidity and causing $9 billion in outflows.
I believe incidents like this, whether real or not, are an unacceptable risk. Instead, a much better alternative is for the Foundation or ZCG to vote to appoint a full-time reviewer for ~$100,000 per year to review submissions full-time, whether AI or not. ZCG can obviously afford it, and it is much better than relying on the goodwill of the finder, especially when AI can actually yield real good results too. Taylor Hornby reported this because he’s a board member.
The average person, when discovering a vulnerability that allows them to mint ZEC undetected, would not have reported it in a million years, because the pros outnumber the cons so bad its insane. At least with no barrier to entry, they would have a little incentive. But if they have to actively pay to do it, the hassle is not worth it.
The goal of a bug bounty program is to make the pros of reporting so attractive and hassle-less that the cons lose.
I’m also thinking some sort of guide rails for Retroactive Grants could be put in place like:
Projects that have already been submitted to ZCG or FPF and declined aren’t eligible for a RG.
Anything to do with bug bounties unless it’s a highly unusual and network breaking circumstance, such as the Orchard soundness bug.
Zcash has never had a formal Bounty program until ZCG decided to put up funds for one, then later due to the circumstances you mentioned, it was discontinued.
If the community wants a real bug bounty program then it should go through a separate system with set aside funding and has personnel equipped to review and decide what qualifies for that program.
Further, not to mention that “Taylor Hornsbys of the world” is a bad example
The ethics of him receiving a “bug bounty” for something he was paid to do as an audit… You’re paid for a service (the audit). Finding bugs is the explicit goal of that service. Requesting a separate bounty on top of your fee for doing the job you were already paid to do feels… greasy. It’s like a chef charging you for a steak and then sending a separate bill for “the service of not poisoning you.”
It is only acceptable here because it is a VOLUNTARY grant from the coinholders, based on whether they liked it or not. It is in no way equivalent to a bug bounty, which is tiered, often absolves all legal responsibility, is a formal contract, and obliges payment.
Taylor Hornby is in no way, shape or form, in this specific case, representative of actual bug bounty finders.
Instead, I suggest (I’m pinging them) getting the opinions of actual bug bounty hunters, such as @sangsoo, who were not doing paid audits or any other work. Their opinion is what matters here.
ZF and ZCG already had multiple people reviewing bugs and they were effectively DDOSed. The net effect of an unbound bounty program is that reported bugs will not be reviewed. Our default is no bounty at all.
Putting up a bond is not gatekeeping it is opening the doors to real reviewers. AI has changed the economics of bounties such that the cost of reporting becomes effectively zero while the cost of reviewing is high.
Onlookers should also note that @strahncryptography posted a “bug report” publicly on another thread (irresponsibly disclosed) and the report itself was not actually an exploitable bug, which an actual review by the reporter would have revealed. So this response is coming from exactly the type of “auditor” we would want to disincentivize.
Putting restrictions on Coinholder votes is not something we should ever do, nor would it be possible. Forming norms around it is fine but the entire reason to have Coinholder votes is to error correct and override exceptions.
I hope that we find a solution to the problem so that either ZCG or CHG can fund it and CHG can collectively form a consensus framework for thinking about how to handle these requested bounties.
I use the Taylor Hornsby report as a clear example of something deserving of a bounty because there was clear 100% consensus that it needed to be urgently fixed, however Coinholders have not yet decided that it is something they should fund, so that precedent is yet to be established.
On the other hand, “ZF and ZCG already had multiple people reviewing bugs” were they paid exclusively to review bugs 5 hours a day and nothing else? Or, wait for it, they were normal engineers that had to pick bugs ASIDE from all their existing duties, and probably didn’t apply for the “Full Time Triager” job role.
It’s like complaining the scientists at CIMS are bad at cleaning up spills. Sure, they clean up spills when it’s a big spill and they’d drown without it, but it’s not their job, and they would suck at it.
Hi, thanks for opening this topic and sharing your thoughts. I’ve been thinking about the same issues and have been following the discussion closely.
I don’t think requiring a large bond to submit a vulnerability is the right approach. It creates a real barrier that could deter legitimate researchers, especially independent researchers or people who aren’t already well-funded. The goal of a bounty program should be to make responsible disclosure as frictionless as possible for credible findings.
I think the idea of having dedicated reviewers could be a better approach. I’d take it a step further and suggest putting several reviewers on retainer with solid compensation.
Reviewing a high volume of AI-generated reports is genuinely difficult and time-consuming work. Someone still has to reproduce the issue, inspect the relevant code paths, test assumptions, determine exploitability and impact, identify duplicates, and separate legitimate findings from low-quality AI-generated reports.
Rather than making vulnerability review an additional responsibility for engineers or other ecosystem contributors who already have full workloads, these could be part-time or full-time independent contractor roles, with reviewers covering different areas of expertise: protocol and consensus, cryptography, applications, networking, wallets, and so on.
Recruiting them could also be fairly simple and quick. An online application form could ask:
Who you are and your relevant background
Time availability (part-time / full-time)
What specific strengths you bring (cryptography, protocol-level analysis, applications, AI tooling familiarity, etc.)
This keeps the submission door open while creating dedicated capacity for the exact problem that overwhelmed the previous bounty program.
I’m starting to be convinced by the replies that a bounty program may not be good idea and funds would just be better spent on hiring full time bug hunters instead of on reviewers.
For the rare case where a true existential bug is found then token holders can vote on a bonus payout to the reporter as a good will gesture but no formal guarantee of a bounty would be made.
This enables coinholders to use their judgement on the reward without committing funds to people endlessly trying to game the system.
Under this strategy, the Coinholder vote should reject most of the bug bounty requests and only vote on those it finds truly existential.
That is a great approach as I see a project whose grant was stopped by the zcg who told them to stop working and they went on ahead to continue working and are now requesting for funds from coin holder after their requests to zcg wasn’t responded
Coin holder votes are very important and we need to find a way to shun bad actors
I fail to see why someone would submit a bug bounty to ZCG. If its an audit instead (plausible for ZCG), then I fail to see why a found bug not under a ZCG-sanctioned grant would fail to qualify on its own.
First, it would be helpful to look at how other projects run their bug bounty programs.
Most major projects use platforms like Immunefi, which act as an intermediary before reports reach the engineers. They use things like spam filtering, PoC requirements, AI triage, and a 1 USDC submission fee.
In our case, bug reports submitted through GitHub Security Advisories probably had to be reviewed directly by the engineers. Without a dedicated professional triager, this likely ended up overwhelming the engineering team and disrupting their regular work.
For a sustainable bug bounty program, we basically have two options:
I strongly discourage against bug bounty platforms because their terms of service are often inherently incompatible with the philosophy of Zcash (which pays out in ZEC, which is shielded), and they are often prohibitive with exclusionary terms. In short, they may lead to legal battles Zcash does not want to fight, and compel law enforcement requests that Zcash may find in bad taste.
Yes I don’t disagree, but Taylor is a great example. We basically have a sorting problem, where on one end of the spectrum we have the bug Taylor found (obviously existential due to entire ecosystem effort to mitigate) and on the other end we have total spam false reports.
The challenge is how to discern which of the bugs in between the two extremes of the spectrum are worthy of a bounty?
On top of that, which bounties should Coinholder Grants cover, as it is not explicitly a bug bounty program nor is it equipped with an engineer review process.
A professional platform like Immunefi sounds appealing, but the 1 USDC submission fee seems low, there is a game theoretical value that enables quality submissions and deters bad actors.
Am really interested in your ideas here. The default case is no bounty program at all.
Someone like you who seems to submit useful reports is someone we want to hear from, but your submissions are drowned out by a sea of spammers which make bounty programs non-viable due to the asymmetric economic attack on the funding source. On top of that, the majority of bounty submitters are adversarial to the bounty program, so 99+% of the time we have to assume that bounty hunters are bad actors.
We shun bad actors by exposing them and rallying a vote against them. If you see bug report from a bad actor, please post the known facts on their thread so others can form their own opinion.
Hi @thowar2! Small world, we’ve actually met in person before!
I don’t disagree with your framing, but let me offer a perspective from the middle of your spectrum.
GHSA-2p4c-3q4q-p463 (rated High) in Zebra was my first security disclosure ever, fixed in 6.2.1 and credited in the 6.3.0 release. This would’ve been a $75k award under the now-closed ZCG bounty program.
Finding the bug took about a day, but the report took quite a bit longer, especially creating a high-quality Proof of Concept (PoC) against a live Regtest Zebra node.
The challenge as I see it is that filtering by people needs precedent, and my first report had none, so basically every “assume bad actor” filter would have caught aspiring talent, although my report might have been beginner’s luck.
One idea is to have an AI agent that runs the reporter’s PoC before a human looks at it. Every submission runs in a sandbox against the affected release, if it reproduces the team gets called in for manual review with the evidence attached, if it doesn’t run it’s closed as spam and no engineer time is spent.
Working PoCs that prove the exploit is confirmed are valuable, so even if “spam” reproduces it wasn’t a useless finding. We’ll probably see fully autonomous submissions soon, so it doesn’t matter whether the submitter is human or an agent, only whether the exploit is confirmed.
I pray the core teams will eventually be able to solve all these bug classes themselves with AI-assisted discovery and patching - the Zakura team seems ahead on this!
@ValarDragon, we’d love your thoughts, you’ve seen this from many sides.
I also thought about something along these lines after reading the discussion yesterday (: I like the idea of having an automated pre-triage step that checks whether a PoC actually reproduces before sending the report to a human reviewer.
My only concern would be automatically closing anything that fails to reproduce as spam. Not every legitimate vulnerability will fit neatly into a standardized reproduction environment, and the tooling itself could fail or miss something depending on the vulnerability class.
Maybe reports with a successfully reproduced PoC could go directly into a priority review queue, while reports that fail automated validation could go into a lower-priority/manual queue or be sent back to the researcher for additional evidence rather than being immediately rejected.
That would still reduce the amount of low-quality AI-generated reports reaching reviewers without making the automated system the final authority on whether a vulnerability is legitimate.