Fss
July 2, 2018, 7:54am
1
Hello everyone!
About 3 weeks ago, in I and a few friends stopped working wallets zcash4win and winzec compiled “developer” - David Mercer.
None of the standard procedures that are on the forum did not help.
opened 10:18AM - 20 Nov 17 UTC
closed 06:32PM - 14 May 18 UTC
A-wallet
M-potential-loss-of-funds
A-wallet-database
### Describe the issue
I can not load wallet.dat after computer crash. I had sa… ved file wallet.dat but can`t import it.
### Can you reliably reproduce the issue?
Yes, I tried Win10 and Linux (Ubuntu).
#### If so, please list the steps to reproduce below:
1. Copy paste wallet.dat to zcash dir
2. fire zcashd deamon (after clear all blocks data)
3. Deamon is shutting down. I had in debug.log
### Expected behaviour
Client should start normally and get blocks.
### Actual behaviour + errors
Deamon shutting down with log:
`2017-11-20 10:02:27 Renamed wallet.dat to wallet.1511172147.bak
2017-11-20 10:02:27 CDBEnv::Salvage: Database salvage found errors, all data may not be recoverable.
2017-11-20 10:02:27 Salvage(aggressive) found no records in wallet.1511172147.bak.`
### The version of Zcash you were using:
Zcash Daemon version v1.0.12
### Machine specs:
- Ubuntu 14.04
- Intel i5 4c
- 16 GB DDR3
- 128 GB SSD
- Linux kuba 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
- Compiler version (gcc -version):
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.5) 5.4.0 20160609
### Do you have a back up of `~/.zcash` directory and/or take a VM snapshot?
Have only backup wallet.dat (created about 3 months ago when I installed zcash)
Issues was just closed. Jakub Socha problem has not been solved(((
opened 09:24AM - 31 May 17 UTC
I-SECURITY
A-wallet
A-dependencies
M-potential-loss-of-funds
database corruption
C-tech-debt
spring_cleaning
A-wallet-database
**Dependencies:** #2505, #2580, #2587, #2518, #2156
We currently depend on BD… B 6.2.23. The differences between that version and the latest 6.2.32 are here: http://download.oracle.com/otndocs/products/berkeleydb/html/changelog_6_2.html
Note for example:
> 28. Fixed a bug that could cause a client undergoing internal initialization to fail to request all the necessary logs for a correct recovery. The major symptom of this bug was that some client database files were left with empty pages that would later cause log sequence errors or other failures. [\#25624]
Normally BDB only makes incompatible file format changes on minor versions, not on micro versions. In this case the release notes say:
> Database or Log File On-Disk Format Changes
>
> 1. The log file format changed in release 12.1.6.2.
>
> 2. Existing SQL databases will have to be reindexed after upgrading to this release. No actions are required for non-SQL databases. Check the upgrade documentation for more details. [\#23469]
(Ignore the "12.1" prefix on the version number; it means 6.2 for the log file format change.)
It's unclear to me whether the second point refers to upgrading to this micro version, or only upgrading to 6.2. Also I do not know whether we're using "SQL database" mode.
opened 12:18AM - 12 Aug 17 UTC
C-bug
A-wallet
A-testing
special to Zooko
I-heisenbug
M-potential-loss-of-funds
database corruption
usi
A-wallet-database
Based on user supplied info, when the `~/.zcash/database` folder is empty on sta… rtup, the wallet gets corrupted `debug.log` like this:
```Zcash version v1.0.10-1 (2017-06-23 20:13:47 -0700)
Using OpenSSL version OpenSSL 1.1.0d 26 Jan 2017
Using BerkeleyDB version Berkeley DB 6.2.23: (March 28, 2016)
Default data directory /root/.zcash
Using data directory /root/.zcash
Using config file /root/.zcash/zcash.conf
Using at most 125 connections (1048576 file descriptors available)
Using 16 threads for script verification
scheduler thread start
Loading verifying key from /root/.zcash-params/sprout-verifying.key
Loaded verifying key in 0.003090s seconds.
HTTP: creating work queue of depth 16
HTTP: starting 16 worker threads
Using wallet wallet.dat
init message: Verifying wallet...
CDBEnv::Open: LogDir=/root/.zcash/database ErrorFile=/root/.zcash/db.log
Renamed wallet.dat to wallet.1502470197.bak
CDBEnv::Salvage: Database salvage found errors, all data may not be recoverable.
Salvage(aggressive) found no records in wallet.1502470197.bak.
Error:
Error:
Shutdown: In progress...
StopRPC: waiting for async rpc workers to stop
StopNode()
zcashd: /zcash/depends/x86_64-unknown-linux-gnu/share/../include/boost/thread/pthread/mutex.hpp:127: void boost::mutex::unlock(): Assertion `res == 0' failed.
scheduler thread interrupt
Shutdown: done
```
Note: this case is running in Docker with an ext4 for the datadir and a tmpfs for the `database` dir.
opened 09:25PM - 28 Oct 16 UTC
A-wallet
I-error-handling
M-potential-loss-of-funds
database corruption
usi
A-wallet-database
Hi,
I send ZEC from T to Z address and once received by Z address, daemon start… ed to crashing and it was restarted by monitoring scripts. I took a backup then of wallet.dat but now I cannot recover in in any way. I tried -reindex and -salvagewallet but it does not help - I have only logs like this:
`2016-10-28 20:44:42 CDBEnv::Salvage: Database salvage found errors, all data may not be recoverable.
2016-10-28 20:44:42 Salvage(aggressive) found no records in wallet.1477687482.bak.
2016-10-28 20:44:42 Shutdown: In progress...`
I did an investigation with daira and we found following:
db.log:
```
BDB2506 file unknown has LSN 1/189046, past end of log at 1/1076
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment
wallet.1477681161.bak: BDB0090 DB_VERIFY_BAD: Database verification failed
BDB1538 Program version 6.2 doesn't match environment version 5.3
BDB2506 file unknown has LSN 1/189046, past end of log at 1/1168
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment
BDB0522 Page 0: metadata page corrupted
BDB0523 Page 0: could not check metadata page
wallet.dat: BDB0090 DB_VERIFY_BAD: Database verification failed
BDB2506 file unknown has LSN 1/189046, past end of log at 1/2216
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment
wallet.1477687482.bak: BDB0090 DB_VERIFY_BAD: Database verification failed
BDB2506 file unknown has LSN 1/189046, past end of log at 1/2216
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment
BDB2506 file unknown has LSN 1/189046, past end of log at 1/2216
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment
BDB2506 file unknown has LSN 1/189046, past end of log at 1/2216
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment
BDB2506 file unknown has LSN 1/189046, past end of log at 1/2308
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment
BDB0522 Page 0: metadata page corrupted
BDB0523 Page 0: could not check metadata page
wallet.dat: BDB0090 DB_VERIFY_BAD: Database verification failed
BDB2506 file unknown has LSN 1/189046, past end of log at 1/3356
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment
wallet.1477688722.bak: BDB0090 DB_VERIFY_BAD: Database verification failed
```
dpkg -l 'libdb*':
```
un libdb4.6++-dev <none> <none> (no description available)
un libdb4.6-dev <none> <none> (no description available)
un libdb4.7++-dev <none> <none> (no description available)
un libdb4.7-dev <none> <none> (no description available)
un libdb4.8++-dev <none> <none> (no description available)
un libdb4.8-dev <none> <none> (no description available)
ii libdb5.1:amd64 5.1.29-7ubuntu1 amd64 Berkeley v5.1 Database Libraries [runtime]
ii libdb5.1++:amd64 5.1.29-7ubuntu1 amd64 Berkeley v5.1 Database Libraries for C++ [runtime]
ii libdb5.1++-dev 5.1.29-7ubuntu1 amd64 Berkeley v5.1 Database Libraries for C++ [development]
ii libdb5.1-dev 5.1.29-7ubuntu1 amd64 Berkeley v5.1 Database Libraries [development]
ii libdb5.3:amd64 5.3.28-3ubuntu3 amd64 Berkeley v5.3 Database Libraries [runtime]
ii libdbus-1-3:amd64 1.6.18-0ubuntu4.3 amd64 simple interprocess messaging system (library)
ii libdbus-glib-1-2:amd64 0.100.2-1 amd64 simple interprocess messaging system (GLib-based shared lib
```
opened 04:56AM - 28 Mar 17 UTC
A-wallet
A-dependencies
S-needs-clearer-scope
licensing
A-wallet-database
On #2194, @zooko wrote:
> I am a huge fan of sqlite and would happily give my :… +1: to using that. It has a long track record of reliability in many deployments, [their test policy](https://www.sqlite.org/testing.html) is top notch and we could learn from it! I have had extensive — and entirely positive — experience working with sqlite. The developer docs are great.
>
> If anyone wants to argue against sqlite on performance grounds, that's fine, but I'm going to listen to this argument only if you bring a realistic performance test that shows (at least approximately) how many z-addresses/t-addresses/transactions/etc. incur how much of a performance penalty in a realistic scenario. My guess is that sqlite is more than performant enough for most or all practical scenarios.
@veikkoeeva wrote:
> A vote for SQLite here too, an excellent cross-platform and development friendly choice.
>
> An added note, as this may not be evident, but if ZCash were to adopt the convention that all queries will be explicitly identified and used so that the ZCash "engine" would care only that the variable and column names and types remain the same, it would be possible to switch the DB engine or the implementation on the fly. This gives the added benefit one can adapt the structures taking account the hardware and amount of data, for instance, as in sending "computation and data" between the engine and the program. Prior art is availabe at https://github.com/dotnet/orleans/tree/master/src/OrleansSQLUtils, see different scripts (the queries are keyed [like this](https://github.com/dotnet/orleans/blob/master/src/OrleansSQLUtils/CreateOrleansTables_SqlServer.sql#L252) and loaded and used by these keys). This way one can make use of the various tooling suites and backends depending on any platform variables. This is, I suppose, mostly like Stackoverflow operates too. As to why this might be important for a running system, maybe [this post mortem](https://blogs.msdn.microsoft.com/bharry/2016/02/06/a-bit-more-on-the-feb-3-and-4-incidents/) indicates as to why. As the amount of data grows or there are problems, one could adjust queries on the fly.
>
> On tangential, this would also allow one to transparently to make use features such as at https://www.pipelinedb.com/ (assuming one would switch from SQLite to PostgreSQL) or pipe data through the DB interface to any other storage (e.g. https://msdn.microsoft.com/en-us/library/mt143171.aspx) by simply modifying the queries appropriate – as it would be transparent to ZCash. I.e. making ZCash perhaps more readily integrable to various other systems.
@radix42 wrote:
> One thing I'd like to make note of is that currently all calls to access either database (LevelDB or Berkeley DB) are very much peculiar to the database in use. I'd recommend first factoring out all calls that result in database access into some nice abstraction layer, starting of course with the current databases that are in use. Then safely swapping out the backend(s) will be a whole lot easier. I bloodied my head against this some months ago when I made the start of a switch to lmdb. It was not pretty because of this.
@veikkoeeva wrote:
> @radix42 I second that too. Which is, in fact, how the aforementioned _Storage Providers_ in the just linked Orleans work. This is repeating the obvious, but for completeness sake, there's an interface and an implementation class and then a loader the loads the implementation. One backend is ADO(.NET), i.e. "relational databases" and that specific backend works as just mentioned.
@ebfull wrote:
> @nathan-at-least We tentatively believe that sqlite would be a nice replacement for Berkeley DB, and would like you feedback. If we don't hear from you soon, we will assume you agree.
@zooko wrote:
> [...] there are two different uses of DBs in Zcash — 1. the chain state DB that contains UTXOs, tree states, and stuff like, and 2. the wallet data file. The latter is where we currently use BerkeleyDB, and we definitely want to get rid of it (unreliable, problematic licensing, frequent backward-incompatible releases), and the latter doesn't ([I guess](https://github.com/zcash/zcash/issues/2194#issuecomment-288827202)) need better-than-sqlite performance.
>
> I also [agree] with what @veikkoeeva and @radix42 are saying above — we should use an abstraction layer: https://github.com/zcash/zcash/issues/2194#issuecomment-289211140, https://github.com/zcash/zcash/issues/2194#issuecomment-289028249
See also #1469, which focuses on licensing issues with BDB; and #1694, an example of a user losing funds apparently due to BDB database corruption.
After carefully reading everything and realizing that the problem in Oracle Berkeley DB I wrote a script on a Python that can recover wallets, but alas not all(((
If your zcash4win/winzec wallet shows an error in the log file - " CDBEnv:: Salvage: Database salvage Fund errors, all data may not be recoverable."
paige
July 2, 2018, 5:27pm
2
Note that the user support team closes tickets after a timeout period when the user doesn’t respond and we can’t move forward with further investigation. Tickets can be reopened if the user eventually gets around to responding.
Regarding your tool, that sounds like it might be a saving grace for some users but is it open source for users to run themselves? I would suggest users be careful before sending any wallet files (corrupted or not) to an untrusted party. And also be cautious of the medium you choose to send the wallet, should you decide to trust a third-party with wallet recovery.
Fss
July 2, 2018, 5:39pm
3
Why I do not spread opened the source code?
David didn’t shows the source code zcash4win and winzec. Then why should I post an open source bug fix on Berkeley DB?
