Memory Requirements Estimation

The memory usage for creating a shielded transaction has been 2.9 GB since launch - the oft-quoted 4 GB required is for system memory, not for zcashd itself. What is interesting however is that the memory required to load the network parameters (specifically the proving key) is 2.25 GB, which means that the actual shielded transaction creation only requires 666 MB :ghost:

The current work being done to reduce memory usage is to load required parts of the key into memory as needed (which can be done because the key has several independent parts). We have not run comparative benchmarks yet, but anecdotal measurements indicate it significantly reduces the memory usage at a cost of a few seconds extra per JoinSplit (because the key is loaded every time a JoinSplit is created, instead of once at start).

Other ideas that still need to be investigated (that may have other trade-offs, and would likely be much more work than the current PR) include:

  • Using mmap to load the key parts into memory, so they don’t use system RAM.
  • Streaming the key parts from disk, using them on-the-fly in a single pass.
9 Likes