Shielded Wallet Performance on Entry-Level Android
Benchmarking Methodology · Version 1.0
Zcash India · Zcash World, Mumbai Published before any data is collected
1. Pre-registration
This document is published before any measurement is taken. It fixes, in public and in advance, what we measure, on what hardware, under what conditions, what we exclude, and how we publish.
We commit to the following:
- All results are published, including results unfavourable to software Zcash India contributes to.
- Runs excluded from headline figures are still published, in a separate file, with the reason.
- Data is released under CC0.
2. The question
There is no public data on how shielded Zcash wallets perform on the hardware most of the Indian market owns. We will prioritise the phone models most widely used in the Indian market, taking the most popular brands and the most commonly owned models and benchmarking on those. If a shielded transaction takes minutes to prove on a ₹8,000 phone, or a first sync consumes a week of someone’s mobile data, that is a first-order adoption barrier that nobody has measured.
3. What we measure
Everything below comes from two runs per device. This is the central design decision: we do not run a separate test per metric. We run one sync and one send, and every other figure is recorded automatically while those two runs happen.
| # | Metric | Unit | Captured during |
|---|---|---|---|
| M1 | Cold sync duration | seconds | Run A |
| M2 | Send time, end to end | seconds | Run B |
| M3 | Proving time | seconds | Run B, where the wallet exposes it |
| M4 | Peak memory | MB | Both runs, sampled every 5 s |
| M5 | Out-of-memory kills | count | Both runs, from system logs |
| M6 | Thermal state | status 0 to 6, °C | Both runs, sampled every 5 s |
| M7 | Battery draw | % | Run A only, start minus end |
| M8 | Mobile data consumed | MB | Both runs, per-app counter delta |
| M9 | Failure mode | category | Both runs: crash, stall, OOM, timeout, none |
| M10 | Sync progress at 10 minutes | % | Field runs |
On M3. Not every wallet exposes a distinct proving phase in its logs. Where it does, we report proving time separately. Where it does not, we report total send time only and mark M3 as not available rather than estimating it. We will not guess.
On M7, and why it is not reported per transaction. Android reports battery level in whole percentage points, and a shielded send completes in seconds. The energy drawn by a single transaction is below the resolution of the platform battery API, and any per-transaction figure we published would be measurement noise presented as a result. We therefore report battery draw across the cold sync, where the effect is large enough to measure honestly, and report no per-transaction battery figure at all.
On M10. Field runs are conducted on devices belonging to people who are attending an event. Nobody will stand at a table for forty minutes while their phone syncs. Rather than collect a handful of completed syncs, we record the percentage synced after exactly ten minutes on each consented device. Ten minutes per person is a reasonable ask, and across thirty devices it produces a far more informative distribution than three or four completed runs would.
On M8. Most of this market is on metered mobile data. A first sync that costs hundreds of megabytes is a barrier regardless of how fast it is. This is the metric least often reported elsewhere and one of the most relevant here.
4. Two runs, and nothing else
| Run | What happens |
|---|---|
| A. Cold sync | Clear app data. Restore benchmark wallet B1 from seed. Measure until the wallet reports fully synced. |
| B. Send | From a synced wallet, send one shielded payment to a fixed address. |
There is no warm-resume test, no sustained-session test and no scripted usage test. They were considered and dropped: they add hours of collection time and answer questions the proposal does not ask.
One run of each per device, per wallet. Every figure published is a single measured run, reported exactly as measured. Nothing is averaged and nothing is smoothed. Where a run fails it is recorded as a failure under M9 rather than repeated until it succeeds.
5. Coverage
| Tier | Devices | Wallets | Purpose |
|---|---|---|---|
| Primary | All 20 | Zodl | The headline dataset: 20 devices benchmarked |
| Comparison | 6, spanning the RAM range | 1 further wallet | Cross-wallet comparison under an identical protocol |
Running the comparison tier on a six-device subset rather than all twenty is a deliberate reduction that keeps collection to a size we can complete properly.
Two wallets is a floor, not a target. Zcash India contributes to Zodl, so a single-wallet study published by us would not be a benchmark. If a third wallet can be added without putting the rest of the collection at risk, it will be, and Appendix B will say so before collection starts.
On light-client endpoints. All runs use each wallet’s default light-client server as shipped. We considered also measuring against the Zcash India server in Mumbai, which would isolate the latency cost of serving Indian users from distant infrastructure. It is not part of this study: it requires reconfiguring the server inside each wallet on every device, some wallets do not expose that setting at all, and a comparison we could only run on some of the matrix would not be worth publishing. If we run it opportunistically on a subset we will publish it as a clearly separate observation, not as part of this dataset.
Device matrix. Twenty handsets in the ₹6,000 to ₹12,000 band across Redmi, Realme, Poco, Lava and Samsung M-series, spanning 2 to 6 GB RAM and Android 11 to 15. Each is recorded with model, chipset, RAM, storage, Android version and build. The exact models are fixed in Appendix A before collection begins. No device is dropped afterwards for producing poor results.
Wallets. Fixed in Appendix B before collection, recorded by version and build. Zcash India has an open localization pull request against Zodl. That conflict is the reason more than one wallet is tested under the same protocol.
6. Controls
Three controls, chosen because each one costs almost nothing and without them the data is not reproducible.
6.1 One benchmark wallet. Every sync run restores the same wallet, B1, with a fixed birthday height and a fixed set of shielded transactions, so every device performs identical work. B1 is disposable and holds no meaningful value. Its seed phrase is published with the dataset so anyone can reproduce the exact workload. No funds should be sent to it.
6.2 Chain tip recorded per run. Start and end chain height are written into every row, and sync duration is reported alongside the number of blocks actually scanned. This is what keeps runs taken on different days comparable.
6.3 Network quality recorded per run. Throughput and latency are measured immediately before each run and written into the row. Network conditions cannot be held perfectly constant, so instead every measurement carries the conditions it was taken under and readers can filter accordingly.
All collection runs on a single dedicated network. We are not testing multiple network types.
Run hygiene, checked before every run and recorded in the row:
- Unplugged, battery above 50%. The actual level is recorded rather than forced into a narrow band, because battery level has no material effect on sync speed and holding twenty devices inside a ten point window costs hours for no analytical gain.
- Thermal status 0 at start
- At least 10 minutes since the previous run on that device
- Screen on, brightness 50%, sleep disabled
7. Instrumentation
All measurement is over ADB via USB. No device is rooted, because rooting would change the thing being measured.
| Metric | Source |
|---|---|
| Timing | Operator marks start and finish against the wallet’s own on-screen sync state |
| Memory | dumpsys meminfo every 5 s |
| OOM kills | logcat filtered for lowmemorykiller and ActivityManager: Killing |
| Thermal | dumpsys thermalservice every 5 s |
| Battery | dumpsys battery, reset via batterystats --reset |
| Data | dumpsys netstats detail, per-app delta |
| Device facts | getprop, captured once per device |
Timing is taken by the operator, deliberately. Automated detection of a wallet’s synced state depends on parsing on-screen text or log strings that differ between wallets and between versions, which would make the protocol wallet-specific and introduce a failure mode of its own. A person watching the screen applies identically to every wallet.
The accuracy cost is negligible at the magnitudes involved. A two second human reaction on a twenty minute sync is an error of roughly 0.2%, far below the run-to-run variation on the same handset. We are not measuring something where sub-second precision carries meaning.
Everything that genuinely cannot be done by hand, which is memory, thermal state, battery and network counters, is sampled automatically every five seconds for the duration of each run.
8. Two kinds of run
Reported separately. Never combined in any figure.
Controlled runs
The protocol above, on the 20-device matrix, under the run hygiene set out in section 6. This is the core dataset.
Field runs
At the venue on the day of the event, capturing what controlled runs cannot:
- Consented attendee devices, which are the real distribution of hardware in an Indian room rather than a selection we made in a shop. Sync progress at the ten minute checkpoint (M10), plus one send where the participant is willing.
- A congested shared network, which is what a real user in a crowded place actually has.
- Live results on a screen through the day, updated as rows are recorded.
Every field row is labelled field under run_class in the dataset.
9. Consent and privacy for attendee devices
- Written consent before any measurement, in the participant’s own language.
- The participant’s own wallet is never used. Measurement uses B1 or a fresh disposable wallet. We do not touch a wallet holding someone’s funds.
- No addresses, seed phrases, viewing keys, balances, transaction contents, phone numbers, IMEI or account identifiers are recorded.
- Recorded fields are limited to device model, chipset, RAM, Android version and the metrics in section 3.
- Participants may withdraw at any time, before or after publication.
- Any app installed for measurement is removed and its data cleared before the device is returned.
10. Data published
One row per run, as CSV.
run_id, run_class, timestamp_utc, device_model, chipset, ram_gb, android_version,
build_id, wallet_name, wallet_version, endpoint, net_throughput_mbps,
net_latency_ms, run_type, battery_start_pct, chain_tip_start, chain_tip_end,
blocks_scanned, m1_cold_sync_s, m2_send_s, m3_proving_s, m4_peak_mem_mb,
m5_oom_kills, m6_max_thermal, m6_max_temp_c, m7_battery_delta_pct,
m8_data_mb, m9_failure_mode, m10_pct_synced_at_10min, excluded,
exclusion_reason, notes
Three files: results.csv, excluded.csv, devices.csv.
11. Exclusion criteria
Fixed in advance. A run is excluded only for one of these, and is still published in excluded.csv:
- USB or ADB disconnection mid-run
- Device not at ambient thermal status at start
- Battery below 50% at start, or the device plugged in during the run
- Measured throughput below 1 Mbps or latency above 500 ms at start
- Operator error, described in the notes field
- A crash caused by something other than the run itself
A slow result is not an exclusion. A failure is not an exclusion. Crashes, OOM kills and stalls are recorded under M9 and are among the most useful findings this study can produce.
Timeout rule. Any sync exceeding 120 minutes is stopped, recorded as sync_timeout with the progress reached, and included.
12. Analysis
- Every figure is a single measured run, published as measured. Nothing is averaged.
- Results reported per device and per wallet. No single aggregate figure, because one number across a 2 GB Unisoc and a 6 GB Snapdragon would be meaningless.
- Failure rates reported as a proportion of attempted runs per device class.
- No wallet rankings, no scores, no league table. We publish measurements. A composite score would embed our judgement about which metric matters most, and that is not a judgement we are entitled to make for every wallet team.
13. Publication
| Item | When |
|---|---|
| This methodology | Before collection |
| Raw dataset, three CSVs, CC0 | By 30 November 2026 |
| Findings report on the forum | By 30 November 2026 |
| Benchmark wallet B1 seed, for reproduction | With the dataset |
Reproducible defects found during collection are filed as issues in the relevant wallet’s own repository, with device, build and steps to reproduce.
14. Conflicts of interest
- Zcash India has an open Hindi localization pull request against Zodl.
- Zcash India operates an independent light-client server in Mumbai. It is not used in this study, and all runs use each wallet’s default server as shipped.
- Field data is collected at an event funded by Zcash Community Grants.
Mitigations: an identical protocol for every wallet, pre-registered exclusion criteria, publication of all runs including excluded ones, no composite scoring, and publication of unfavourable results including any affecting Zodl or our own server.
15. Limitations
- Twenty devices is a matrix, not a representative sample of the Indian handset market.
- A single run per device per wallet means device-level run-to-run variance is not characterised. This is a deliberate trade against collection time, and it means small differences between wallets or devices should not be read as meaningful.
- Mobile network conditions cannot be fully controlled, which is why per-run network figures are recorded.
- Wallet versions are a snapshot. These numbers describe specific builds on specific dates and will age.
- Field data is uncontrolled by design and describes what happened in one room on one day.
- Per-transaction battery draw is not reported, because it falls below the resolution of the platform battery API. See section 3.
- Field results are measured at a fixed ten minute checkpoint rather than to completion, so they describe early sync behaviour and cannot be compared directly with the controlled cold sync figures.
- All runs use each wallet’s default light-client server, so any difference between wallets may reflect their server infrastructure as well as their client software. We report the default because that is what a real user gets.
- We measure performance only, not correctness, privacy properties or security.
16. How this maps to the funded deliverables
| Commitment in the grant proposal | Where it is met |
|---|---|
| Twenty entry-level Android devices, ₹6,000 to ₹12,000 | Section 5, device matrix, fixed in Appendix A |
| Plus consented attendee devices | Section 8, field runs; consent terms in section 9 |
| Cold sync duration | M1 |
| Proving time per transaction | M3, where the wallet exposes a distinct proving phase |
| Peak memory and OOM kills | M4 and M5 |
| Thermal throttling | M6 |
| Battery drain | M7, across the cold sync. Not reported per transaction, see section 3 |
| Failure modes | M9 |
| Results displayed live | Section 8, field runs, updated through the day |
| Published as an open dataset | Section 13, three CSVs under CC0 |
| Reproducible methodology | This document, plus the published B1 seed in section 6.1 |
| Methodology published before any data is collected | This document, published ahead of collection |
| Unflattering results published, including for software we contribute to | Section 1, commitment 1, and section 14 |
Appendix A: device matrix
Fixed before collection begins.
Appendix B: wallets and versions
Fixed before collection begins.
Zcash India