Why does Zcash installer download additional binaries/packages from z.cash website?

Technical Support
1

Recently i’ve tried to compile new version 1.0.14 from source. I was able to compile 1.0.13 and previous versions just fine according to 1.0 User Guide, but this time there was an error saying I don’t have curl installed. Ok no problem, i quickly fixed situation by install curl with apt-get.

It turned out that curl was used to download more packages/dependencies from z.cash website such as Zeromq, libevent or ccache… This is one example

cd /home/<user>/zcash/depends/sources/download-stamps; (test -f /home/<user>/zcash/depends/sources/ccache-3.3.1.tar.bz2 || ( mkdir -p /home/<user>/zcash/depends/work/download/native_ccache-3.3.1 && echo Fetching native_ccache... && ( curl --location --fail --connect-timeout 10 --retry 3 -o "/home/<user>/zcash/depends/work/download/native_ccache-3.3.1/ccache-3.3.1.tar.bz2.temp" "https://z.cash/depends-sources/ccache-3.3.1.tar.bz2" || curl --location --fail --connect-timeout 10 --retry 3 -o "/home/<user>/zcash/depends/work/download/native_ccache-3.3.1/ccache-3.3.1.tar.bz2.temp" "https://www.samba.org/ftp/ccache/ccache-3.3.1.tar.bz2" ) && echo "cb6e4bafbb19ba0a2ec43386b123a5f92a20e1e3384c071d5d13e0cb3c84bf73  /home/<user>/zcash/depends/work/download/native_ccache-3.3.1/ccache-3.3.1.tar.bz2.temp" > /home/<user>/zcash/depends/work/download/native_ccache-3.3.1/.ccache-3.3.1.tar.bz2.hash && sha256sum -c /home/<user>/zcash/depends/work/download/native_ccache-3.3.1/.ccache-3.3.1.tar.bz2.hash && mv /home/<user>/zcash/depends/work/download/native_ccache-3.3.1/ccache-3.3.1.tar.bz2.temp /home/<user>/zcash/depends/sources/ccache-3.3.1.tar.bz2 && rm -rf /home/<user>/zcash/depends/work/download/native_ccache-3.3.1 ))

My question is: why don’t we install those packages from ubuntu software repository just like we used to do? I’m sure that my Ubuntu already had those packages installed. Even if we need a specific version of those packages, why don’t we fetch them directly from their official repositories/websites instead of z.cash website? Users can’t verify what’s inside the package.

And also, why don’t we include those dependencies within Zcash repository? This installer makes it impossible to compile zcash on offline computers.

I’m not a developer and that’s why i find it a bit suspicious and difficult to understand. I’m not saying that i don’t trust Zcash team/website with their additional packages, but we’re using trustless currency and I think I should ask the question, it might be a stupid question.

2

Hi. Your concern is very valid, and it’s a smart move to follow up on anything unusual or suspicious. You may not be a developer, but your instinct already puts you ahead of many developers.

This response has three parts: History & Rationale, Under the Hood, Build It Yourself.

History & Rationale

We inherited the build system from Bitcoin, and it achieves important goals which we’ve maintained. The part of the build system that fetches third party dependencies is called the depends system because it lives in the depends folder of the main repository checkout.

This system is designed to retrieve third party source code (not binaries) and build them directly into the Zcash (or Bitcoin) binaries. This is used in Zcash and Bitcoin for doing deterministic builds, to ensure everyone who builds uses the exact same versions of all dependencies across all platforms. This let’s us or upstream Bitcoin use extremely specific versions of all dependencies, even with custom patches, if necessary.

So for example the zcashd binary built from this process will use the exact same version of libsodium regardless of whether or not Debian, Ubuntu, and MacPorts ship different versions. This is especially important to help ensure the network maintains “operational consensus” where subtle variations in libraries might lead to different behavior that the developers don’t notice by inspecting Zcash (or Bitcoin) code.

In Bitcoin (last I checked) the default build process does not use this depends system, and instead it is only used when creating deterministic builds. In Zcash, we decided to invert that default so that by default everyone who builds from source will use these dependencies. If you want, you don’t have to use this system and you can build your own dependencies or use your own platform’s system packages. See Build it Yourself.

Under the Hood

The depends system is built directly from make (and it might be GNU Make specific). Unfortunately it’s fairly hard to read and maintain, but at least it accomplishes its goal given certain restrictions.

Each dependency has it’s own .mk file inside depends/packages/<PACKAGE NAME>.mk. If you look at one of those files it has:

a. the URL where the original source code came from.
b. a cryptographic hash of that source code.

This ensures that if you already trust the zcashd codebase which contains the depends system, that no one can sneak in a different version of any of these dependencies from what is specified in the zcashd repo (because of b.). You can use a. to fetch the original source and examine it if you wish, although there is a way to tell the build system to save all fetched source code locally so you can examine it later. (I recommend this for anyone who wants to be able to do deep investigation of any behavior after the fact. Even if you aren’t a coder, having that full codebase of all dependencies could be a crucial piece of evidence you can share with others.)

One quirk is that we try to fetch all of the source code from our own website where we mirror the original source code. The main reason for this is that when trying to fetch N packages from N different websites, there’s a fairly high chance some of them will fail due to normal outages, so this is just a speed/availability improvement. Again, because of b. we don’t have to worry about an attacker controlling our website to insert back doors here if the users got the “correct” version of our zcashd source code.

Build it Yourself

You can still build zcashd and the other tools without using the depends system. ZcashCo doesn’t support this “officially” because there are too many variations between all the user platforms for each of the dependencies, so it’s very hard to ensure we’re testing all possible combinations carefully. Still, it’s your machine and you are responsible for the code you run on it. :wink:

It’s been ages since I have done that, but to do so you can follow the “standard unix” source building process: autogen.sh, then ./configure [your options], then make. The instructions for that process are in the bitcoin docs for building on unix. (We’ve removed them because we didn’t want confused users building in a non-standard way.)

Another option to get all of our other build changes is to replicate all of the ./zcutil/build.sh script, just skip the make -C depends step (right before autogen.sh is run), and drop the the --prefix=… argument from the final make command.

Best of luck!