Memory Requirements Estimation

ZCash originally required 8GB to compute a transaction, but this was trimmed down to 4GB by launch. I am aware of the Github tickets on this issue, but I was hoping someone would be willing to provide a guesstimate on what the memory usage could be brought down to.

Edit: Just wanted emphasize that the estimates below are just that :slight_smile:

The memory usage for creating a shielded transaction has been 2.9 GB since launch - the oft-quoted 4 GB required is for system memory, not for zcashd itself. What is interesting however is that the memory required to load the network parameters (specifically the proving key) is 2.25 GB, which means that the actual shielded transaction creation only requires 666 MB :ghost:

The current work being done to reduce memory usage is to load required parts of the key into memory as needed (which can be done because the key has several independent parts). We have not run comparative benchmarks yet, but anecdotal measurements indicate it significantly reduces the memory usage at a cost of a few seconds extra per JoinSplit (because the key is loaded every time a JoinSplit is created, instead of once at start).

Other ideas that still need to be investigated (that may have other trade-offs, and would likely be much more work than the current PR) include:

  • Using mmap to load the key parts into memory, so they don’t use system RAM.
  • Streaming the key parts from disk, using them on-the-fly in a single pass.
9 Likes

In addition, it’s very likely that the Sapling circuit upgrade will substantially reduce memory usage. There are some very rough and preliminary estimates of how much Sapling could reduce the circuit size here. The reduction is given in terms of its effect on proving time, but the memory also scales roughly proportionally to the circuit size. So, a switch from SHA256Compress to Pedersen commitments for all of the hashes, combined with the as-needed key loading @str4d mentioned, might reduce the memory usage to roughly 1.5 * 4/42 ~= 0.14 GB, for example. This is before we’ve added any circuit support for new Sapling features, though. In any case I estimate the memory usage for creating a Sapling shielded transaction, spending from new addresses, is likely to be less than 0.3 GB (perhaps substantially less; it depends how much time we have to spend on memory optimization). This is not any kind of guarantee or commitment, just my best estimate at the moment.

Edit: improve estimate based on a recent measurement.

5 Likes

That’s amazing! Well within the budget of any computer with a HD large enough to store the blockchain.

I’ve updated the OP to emphasize this point.

This question comes up very frequently. I often refer to these two charts. I hope that helps! (I also posted a new topic just to link to those charts.)

I’ve seen those, any guesstimates on how much of a speedup we might see in the future?