Recovering Funds From Wallet.dat

Heres a google translated version of that thread

Foreword. I am not a programmer, not a system administrator, my knowledge of databases is minimal, but I was able to restore my broken wallet.dat. And therefore I will describe the procedure here in simple words that an ordinary user can understand.

I was looking for a solution for about 3 days, re-read many articles on the forums, numerous threads on github about how zcashd works. Learned what Berkeley DB is and more. But nevertheless, I did not find any instructions on how to restore the wallet, and the correspondence on the forums in search of solutions that I came across ended in nothing.

The ZecWallet developer told me in a correspondence that with the errors that I received, this was basically impossible to do. But logic pushed me to search.

I slept very little, but I’m very glad that everything worked out for me. Hope my experience will help someone in a similar unpleasant situation.

Attention! First of all, make a few backups of your wallet.dat (or renamed wallet.000000.bak) - how to find it is written below. Copy it to backup directories or to a USB stick. Without this file, nothing will work. Read the entire text to the end and only then perform any manipulations with the wallet. 

  1. First, let’s define what error we are talking about, how I got it and how to understand that you have it.

How I got it. Those who use the ZecWallet FullNode wallet may have come across a situation where the update of blocks for some reason stops and whatever you do the wallet will give a connection error until you reindex.

I use ZecWallet FullNode on MAC OS and after version 9.1 I noticed that it became impossible to re-index from the program itself, since the corresponding tab with this function was removed after version 8.3.

I kept version 8.3 in order to re-index from it, if necessary, in an understandable graphical interface (a week ago I did not even know how to start zcashd manually with attributes from the MAC OS terminal).

After another problematic situation (perhaps my macbook was discharged and went out with the wallet turned on), I should have re-indexed, but this process is so long (it takes me exactly more than 14 hours) that I decided to just replace the blocks and chainstate folders in the directory: ~ / Library / Application Support / Zcash /

I always have a full node open on a Windows PC and I previously did such a trick that solved the connection problem - but this time something went wrong …

FullNode version 8.3 (old graphical interface) - gave an error:
“Zcashd said: Error: wallet.dat corrupt, salvage failed”

FullNode version 9.1 and higher (new graphical interface) - gave an error:
"Failed to start zcashd. Giving up! Please file an issue with Zecwallet "

The wallet was restarted as if nothing had happened, but with completely new empty addresses.

I described the primary visual symptoms, now about what happens inside the / Zcash / folder, which is located in:
• Windows:% HOMEPATH% \ AppData \ Roaming \ Zcash \ debug.log
• macOS: ~ / Library / Application Support / Zcash / debug.log
• Linux: ~ / .zcash / debug.log

It will contain your damaged wallet.0000000.bak file (any numbers), which is saved with the .bak extension and a new wallet.dat is created

The log files (located here) will contain the following entries:

debug.log

020-04-26 05:25:14 Renamed wallet.dat to wallet.1587878714.bak
2020-04-26 05:25:14 CDBEnv :: Salvage: Database salvage found errors, all data may not be recoverable.
2020-04-26 05:25:14 Salvage (aggressive) found no records in wallet.1587878714.bak.

db.log

BDB2506 file unknown has LSN 448/450072, past end of log at 1/204362
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment
BDB0522 Page 0: metadata page corrupted
BDB0523 Page 0: could not check metadata page
wallet.dat: BDB0090 DB_VERIFY_BAD: Database verification failed
BDB2506 file unknown has LSN 448/450072, past end of log at 1/205410
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment
wallet.1587877729.bak: BDB0090 DB_VERIFY_BAD: Database verification failed

  1. Awareness of the problem

In general, the situation did not bother me until this moment. I assumed that walled.dat might have undergone some kind of unsuccessful operation that damaged it, but for this case I had a wallet.dat backup 1 month old, which contained all my active addresses. I created it via a standard function in the FullNode 8.3 GUI and burned it to disk.

I copied my backup to the \ Zcash folder with a simple replacement, saving the broken version in advance. Looking ahead, I will write that I was able to restore the broken version too - we must save it, because you may not have a backup.

After restarting ZecWallet, the error did not go away. My backup was also not accepted by the wallet and the internal zcashd handler logged all the same errors.

The logical action was to rename the Zcash folder to a temporary one, uninstall all wallets, erase all folders with the internal data of the wallet, reinstall everything again, download a new backup and re-index the entire database.

I did just that - no result, both my wallet.dat and “corrupted” and backup were not accepted by zcashd and again and again gave an error: Salvage: Database salvage found errors, all data may not be recoverable. 

  1. Possible causes

I noticed that both my wallet.dat files were unusually large for such files, not that I had many transactions on my addresses, but my Z-address was listed on the zecpages.com service and therefore I regularly received mailing list with a large amount of encrypted information in memo-fields.

Since wallet.dat contains all the information about all transactions made on addresses, as well as information received in memo-fields, its volume has grown significantly every day. The zecpages.com service was launched in February. So, my backup for March 22 wallet.dat weighed 15.6 MB, and the damaged version from April 23 was already 36 MB. And this, it seems to me, is a reason to think. For the sake of fairness, I note that the flow of information in mailing lists was large: 5-8 messages loaded with information daily in the form of z2z transactions.

I cannot name this as a reason, at least my current recovered wallet.dat also has a weight of more than 36 MB and this does not prevent it from being loaded onto the newly installed wallet. However, this led me to believe that wallet.dat backups are not a reliable backup method. I will write about this in PS.

  1. Standard recovery methods.

So, we figured out the symptoms, now I will describe the standard recommended recovery methods that did not help me in any way.

There is documentation that describes various attributes and commands for executable files - clients that allow a node to interact with the network (blockchain), receive and send commands, and perform operations with databases.

Information with conditionally up-to-date documentation is here -
Troubleshooting Issues - ZecWallet Docs 3
Wallet Backup Instructions — Zcash Documentation 5.2.0 documentation 1

Why “conditionally relevant”? Because some of the current ZecWallet FullNode versions may not contain separate executable files:
Zcashd.exe
zcash-cli.exe

In addition, you may not always be able to find documentation for your operating system, since it is written primarily for people who understand the interaction architecture. And as a rule, the development environment is Linux - not the most popular operating system among ordinary users, you must agree. This is also due to the fact that the official wallet for a long time existed only for Linux.

I will not duplicate the documentation. I’ll just write that recovery attempts should start with simple commands in the command line with the executing client zcashd, the documentation is here - Troubleshooting Issues - ZecWallet Docs 3

Commands:

zcashd -rescan
zcashd -reindex
zcashd -salvage

How to start:

Windows

cmd / C ““ C: \ Program Files (x86) \ zecwallet \ zcashd.exe ”-rescan”

MacOS

In Finder, open the programs, find ZecWallet FullNode, right-click to bring up the context menu, “Show Package Contents”, inside go to “Contnets”, and there in “Mac OS”. Open a terminal window, drag zcashd there, add a command attribute, for example, -reindex

By the way, if you do not find it in the current version of ZecWallet FullNode, then download the special version, which definitely has it, two options:

Zecwallet Special Build (master branch zcashd) - https://www.zecwallet.co/masterzcashd.html 1
ZecWallet FullNode v.0.8.3 - Release zcashd update · ZcashFoundation/zecwallet · GitHub 2

It is also recommended to delete the folder with parameters, so that in case of failure of these databases, the wallet can be reinstalled:
• Windows:% HOMEPATH% \ AppData \ Roaming \ ZcashParams
• macOS: ~ / Library / Application Support / ZcashParams /
• Linux: ~ / .zcash-params /

Delete the entire folder, do not hesitate, the wallet will restore it automatically the next time you start ZecWallet.

Actually, this is where the entire standardized arsenal of tools ends.

But be sure, if your wallet.dat is corrupted, the logs will contain the same wording “CDBEnv :: Salvage: Database salvage found errors, all data may not be recoverable.” and all manipulations will be performed with the newly created wallet.dat, and your wallet.dat will be renamed with the * .bak extension (and in some cases it is simply deleted in case of an unsuccessful backup, so always make a backup copy).

  1. Non-standard recovery method

I got to the most interesting place for which I wrote this long post.
Even after reading the hopeless predictions, my logic told me: Ok, even if my current file was broken, then a backup? Why is the backup not loaded? 

I created a backup in my sane mind and sober memory;

I clearly remember how I did it;

I made it standard functionality ZecWallet FullNode v.0.8.3;

the node was fully synchronized;

the binary copy of this file continued to work successfully after the backup - simply because the backup operation is a banal wallet.dat copy-paste with a new name.

Why is zcashd spitting it out now?

I started by simply viewing my backup wallet.dat with a text editor
As I scrolled back and forth through this meaningless set of characters, I found that the file had a clear structure. That is, most likely it is a database.
And since FullNode does not support encryption, it is also an unencrypted database.

Having no idea how everything works there, I assumed that wallet.dat is not always an independent database where keys and transactions are stored, but interacts precisely with the block structure that was at the time of its relevance (that is, as of the state and date backup) and if any process is incomplete, it is saved in some “open” position, which further prevents it from acting as a backup copy.

Thus, I thought that I needed to know exactly the moment of the backup and see in the logs what block it was in order to somehow rewind to it - however, I quickly realized that this was anrial and abandoned this attempt, but thanks to this I was more attentive studied all the available logs.

Among other things, in the logs I saw the line
Using BerkeleyDB version Berkeley DB 6.2.23: (March 28, 2016).
And that was the lead. 

So, the first thing you need to do is download the source code for working in the environment for Berkeley databases, they are here - Oracle Berkeley DB Downloads 1

I chose the closest release to Berkeley DB version 6.2.23, it is 12cR1 Releases
Berkeley DB 12cR1 (12.1.6.2.38) - https://download.oracle.com/otn/berkeley-db/db-6.2.38_64.msi 2 link for Windows 64. By the way, to download you have to register on the site Oracle.

I will not bore you with a story about all unsuccessful attempts, I will write specifically the commands that helped me, and I hope they will help you too.

There are only three of them:

db_verify (description - db_verify man page - libdb-utils - General Commands) - allows you to check the wallet.bak structure for integrity

db_dump (description - db_dump man page - libdb-utils - General Commands | ManKier) - allows you to decompile wallet.bak

db_load (description - db_load man page - libdb-utils - General Commands | ManKier) - allows you to compile wallet.dat back

After installing the Berkeley DB package, this joy is located in the C: \ Program Files \ Oracle \ Berkeley DB 12cR1 6.2.38 \ bin folder

Open PowerShell in this folder (in the window of the Windows folder in the context menu, right-click with the pressed Shift).

Let’s say your file is called wallet.bak - move it to this \ bin folder where all these exe-files are located, then the commands with attributes will be like this (in the PowerShell terminal, be careful with extensions):

db_verify wallet.bak

And if the answer sounds: “BDB5105 Verification of wallet.bak succeeded” - you can already send someone for wine 

db_dump -f 1.txt wallet.bak
which means to decompile your wallet.bak to 1.txt file
By the way, by opening it in the viewer, you can enjoy the clear structure of the records.

db_load -f 1.txt wallet.dat
which means reverse compilation of wallet.dat from 1.txt

Copy the resulting file back to the Zcash folder (where we store the block structure and logs).

Start the wallet, if you see familiar addresses, but of course there will be a zero balance (since you now have a new file structure), then you are at the finish line - do the re-index from the command line (or MAC OS terminal):
zcashd -reindex

After re-indexing, you will see your balances 

Such a really not tricky operation somehow corrects the structure of wallet.dat, which, as a result, is remarkably readable by zcashd and even loses a little weight.

 If db_verify does not return a successful result, it means that your wallet is really seriously damaged. In this case, try the command db_recover man page - libdb-utils - General Commands
On the forums of 1C programmers (1C, Karl - I was even there), I read that this command helps to correct the tables.

  1. Conclusion.

After long ordeals and the restoration of my wallet, I was able to experience very pleasant feelings of relief and joy.

And if you feel them too, you can let me know about it:

zs1f2hlu6vzl65256mr0k2nh3n49wtrtj2vvalathwvzxt9asn6v0xtda2nfwgukvaaeuvcq3gpzqv
Thanks!

PS in order not to face all this smut,

I recommend using ZecWallet Lite for transactions - Zecwallet Lite - Fully featured Zcash wallet 2
because unlike FullNode:
• It has a recovery seed-phrase, by which you will restore all your created addresses, and not only those that you had at the time of the wallet backup;
• will not force you to carry out many hours of reindexing in case of improper shutdown;
• allows you to encrypt your wallet.dat

Always pull private keys from FullNode if you still want to have one.


This is a good write-up, I’m sorry I didnt translate sooner

6 Likes