May 5, 2017 - Dev update

Before diving into the working groups categories, it’s worth noting that a good amount of engineering resources went into investigating some heavy load on the network which started last weekend. It turns out a large exchange had started a process consolidating funds from many unspent outputs, transmitting them in batches which put strain on nodes to verify the many inputs on those transactions and filled up blocks (due to size + higher priority from txn fees paid) creating rather large sets of transactions queueing in the mempool. Discussion and more details about this situation are in this forum thread, the community chat and ticket #2333.

Our current status is that we will address the underlying engineering issues in our normal (non-incident-response-mode) engineering process.


Now, onwards with your regularly scheduled dev update categorized by our in-progress working groups strategy to manage Zcash development, operations, collaborations, and related events.

Meta:

While individual working groups have started taking shape, not much focus has been put on the overall direction of working groups this week due to network performance investigation. We anticipate to re-visit this process this month.

Protocol:

Formal discussions on Sapling and HF0 have recently been delayed to focus on more pressing improvements such as infrastructure and stability/security. We intend to get back into this full swing in the coming weeks. An interesting discussion has been happening, however, on improving the MPC ceremony for Sapling for improved security and efficiency which you can read about in #2247.

PR 2297 was submitted to implement the auto-senescense feature for deprecated versions in HF0.

You can keep track of relevant tickets in Github projects for the Sapling protocol upgrade and HF0.

Release:

As stated previously, upcoming 1.0.9 release will focus on essential improvements to stability/security and continuous improvement. Next week will focus on final triaging, review and testing for the May 16th 1.0.9 release date.

We released a blog post on Monday covering our updated release lifecycle and policy.

Development infrastructure:

We’re working on improving our development infrastructure by creating a new development CI environment that is separate from production CI #2302.

You can track progress on this working group in the Development infrastructure project.

Network infrastructure:

Due to the network performance issues this week, we created a patch (PR 2342) for miners which allows them to exclude transactions with greater than 100 inputs. We’re also looking into solutions to reduce strain on nodes when the mempool grows beyond a threshold in #2344.

On the testnet side of things, we finished making the testnet faucet ready for the public. Grab some TAZ to play around or develop with. If you have unused TAZ, feel free to donate it back into the faucet for others to take advantage of!

Tickets related to this working group can be tracked in the Security and Stability project.

Berkeley DB replacement:

This working group is making progress on discussions to replace Berkeley DB for use in wallets, specifically with sqlite (#2221).

Related tickets can be tracked in the Wallet DB to SQLite project.

Miscellaneous:

Documentation on wallet backups is in a draft stage but you can see it’s current text in PR 2340. We’re also looking into a central location for documentation, currently exploring https://readthedocs.org as a solution.

We’re starting a preliminary UX project to research some of the existing user experiences with third-party wallets. One goal from this is to have better insight for creating a UX guideline document for third-parties. While we don’t have any plans to build a UI for zcashd, it would be really helpful to have some standardization across third-party implementations to provide users with safer and more familiar experience across platforms.

We just hosted our first “Understanding zk-SNARKs” livestream driven by @arielg. Aside from technical difficulties at the beginning and end (I blame Google and Chrome respectively :upside_down:), it went very well! Check out the recording if you didn’t catch it live and keep an eye out for a continuation on this topic as we didn’t have enough time to cover the full spectrum!

Next week is another quarterly technical AMA, so join us if you are so inclined. We’ve really enjoyed hosting the past two AMAs and hope y’all enjoy participating!

6 Likes

Also new this week ICYMI, to help get visibility into the network issues that were happening, I launched a public block explorer running Insight: https://insight.mercerweiss.com

It doesn't have all the bells and whistles that https://zcha.in has, but it does have full insight/bitcore api access enabled for broadcasting transactions onto the network by hand, for wallets to use, etc.

It is going to take a bit of cashflow to run it, especially as it gets more traffic, so donations to support it are much appreciated! https://zcash.mercerweiss.com/index.html#Donations

4 Likes