Hi David!
I think it’s necessary. Others think so too.
I understand and sense that this is a very widespread feeling throughout the community and must be taken seriously.
Yes, I can add that the anecdotes were sometimes adding noise and overhead to the development teams and also that using odd terminology as well from “fair use” to bloat, “sandblasting” or spam didn’t contribute to create a clear vision of the problem itself throughout the community
Yes and it is a problem for developers too. Because it’s really hard to prioritize problems from anecdotes, impressions, sentiments and other forms unstructured feedback
Also is not possible to optimize when you have no baseline to compare to. In the case of the syncing issues, the problem was so huge that one could say “oh well, if it takes 4 days now, I’ll notice If it’s faster really easily”.
This is incorrect, because you could increase performance a 100% and your wallet will still take 48 hours to sync. You always need reproducible and systematic baselines to optimize.
I think this is related but also a separate problem. I think it’s more related to Josh’s governance issue post than anything else.
ECC already said they are willing to relinquish control of z.cash. So it would be good that candidates to take over it propose an app listing process to the community.
I agree that an objective and serious process would involve benchmark and other scoring.
Yes. Carter and I advocated for it and our team worked on adding metric APIs to the wallet SDKs for this very reason.
Zingo Labs has shown great interest in this kind of testing. They inherited a lot of code when they forked ZecWallet code and they are working very hard to update and optimize, so they are also investing on tools to have a clear vision of their progress. I started to work on some regtest darksidewalletd datasets that will allow wallets to run integration tests that are applicable to all wallets.
Testing tools are expensive and paid upfront in terms of development resources. That’s why they often get punted and deprioritized. This is one of the main reasons that spawned Test-driven development (TDD). methodology in software engineering. The core principle is that you must not develop something that can’t be tested and for it you first need to create the tests that will verify that your development fulfills the requirements you’ve set.
If we have a problem that’s affecting a lot of users and actors of the ecosystem, we can’t know if we are out of it until we don’t have a clear definition of:
- what the problem is
- how to reproduce it consistently
- tests that show the current state of things and allow us to determine if we’ve improved, worsen or solved the problem entirely.
all of the above exceeds what a single person can do. But having someone that can be our of the daily grind of maintaining products and raising this matters will definitely contribute positively and it’s a good start.
Thank you for your ideas.