Jan 20, 2017 - Dev update

Ecosystem Updates Weekly Forum Updates
1

Greetings!

This week, the engineering team put most of its focus into getting the next release ready and set for a Monday announcement. This consists of reviewing pull requests (PRs) slated to go into 1.0.5, any necessary debugging and testing of PRs, fixing PRs that require changes and final triaging for what will actually make it into the release. We’ve set a regular schedule for non-urgent releases with the goal to have one out every 3 weeks. This allows us to take full engineering time at the beginning of the 3 weeks to do longer-term planning of protocol upgrades, with the final two weeks in the cycle having more engineering focus on the next release and more immediate fixes.

In addition to the needs for the upcoming release, we took some time for a topical engineering meeting regarding selective disclosure. While this feature is not slated for immediate release and requires more discussion, we made progress in understanding the implementation changes needed to support more immediate use cases. We’ll be continuing these discussions in another topical meeting in the near future. Next week’s topical meeting, however, will be about formalizing the Zcash Improvement Proposal (ZIP) process.

Some engineers who were not focused on the release process did research surrounding cross-chain atomic swap support for Zcash - which we’re referring to XCAT. More progress will be made on this front next week with continued discussion and research. Further, more research was done regarding zk-SNARK on Ethereum (ZoE) as a collaborative effort between our team and Ethereum developers and documented in a blog post.

Next week, further work will be done on faster verification for transactions involving shielded addresses and finishing touches on the protocol spec which we intend to submit for formal publishing soon after. The web development team will also be putting lots of time to finish up a revamp of https://z.cash.

In addition, a couple of Zcash engineers gave presentations on zk-SNARKs at meetups in Kiev and Milan this week.

Next week, expect two concrete events:

  1. The 1.0.5 release announcement will be made on Monday.

  2. Our first official Show & Tell livestream will take place next Friday at 3pm PT/6pm ET. Our idea for these Show & Tell sessions is to highlight various projects related to Zcash from the community (including team members from time to time) with 15-20 minute presentations and a Q+A afterwards. They will be hosted on the Zcash Youtube channel and I’m very pleased to announce that our first Show & Tell presenter will be David Mercer (@anon47418038) who has been taking on a huge amount of work porting the Zcash client to MacOS and Windows in addition to GUI implementations. We hope you can join us for this special event! If you’re interested in presenting in a future Show & Tell, feel free to reach out to me (Paige) either on the forum, community chat or email paige@z.cash.

Lots of movement happening with projects that @anon47418038 (see above) and @lustro (https://explorer.zcha.in/) work on so shout out to them and any quick dev updates would be most welcome!

Until next week!

9 Likes
2

Thanks! here’s the skinny on the Windows port, this is gonna get technical :slight_smile: …working with @veikkoeeva I finally got a build to complete but not sync past block 395…block 396 is the first shielded transaction and the libsnark stuff was failing, but we finally had a working build on Windows of the test suite and could dig into. Veikko is working on a branch that uses Cmake instead of autotools to build on Windows (and Mac’s native Xcode once that branch get’s there!), and once we had the test suite working, I discovered that it was because on linux (and most everywhere else!) a “long” and “unsigned long” C++ data type is 64 bits, but on Windows (even 64 bit!) it is 32 bits. I changed all the longs and related types in libsnark to be C++11 standard fixed width data types, and @movrcx (on twitter and elsewhere) from Zclassic then jumped in to lend a hand. We got a cross-compiled mingw build (from linux->windows) working with syncing blocks with private transactions in them, sending and receiving from a taddr on Windows, and receiving at a zaddr in a wallet on a Windows machine with the native build. Sending from a zaddr still crashed it though.

He dug deeper and found that in a section of the libsnark code that calls the libgmp library for its fixed width integer implementation that is involved in generating JoinSplits when you send from a zaddr, libgmp returns, you guessed it, 32 bit longs!!

We’re working on a couple of different ways to work around this, hopefully with as few changes to upstream code in zcash, libsnark and/or libgmp as possible. I’ve been backporting his changes to Zclassic to Zcash as we go, and we’re tossing patches back and forth between the source trees. We’re trying to keep as much of it as we can in #ifdef’s for Windows, to make it easier to send back upstream to Zcashco’s main Zcash repo.

On mac, in the process of integrating my changes to the Zcash port of Mac to make it compatible with Zclassic (they paid a nice bounty for the 2 days of work it took me to convert it to Zclassic and get the Mac gui wallet working and rebranded), he went behind me and stuck all my changes for Mac in proper platform specific #ifdefs, which I’ve partially backported to Zcash and will send upstream from my Zcash fork once I’m done merging Zcash 1.0.5.

More updates soon as we have news!

-David Mercer
Tucson, AZ
radix42@gmail.com

Again, I’m not a ZcashCo employee, don’t have a day job, and live primarily off of donations!
https://zcash.mercerweiss.com/index.html#Donations

6 Likes
3

Oh and here’s my twitter: https://twitter.com/radix42
github for my zcash stuff: radix42 (Jane Mercer) · GitHub
and my Zclassic things (can’t have 2 forks of the same thing on the same github account!):
MercerWeiss Consulting · GitHub

1 Like
4

Thanks a lot for the enthusiastic work. Great that you are working on making Zcash (and Zclassic) more accessible. I remember having similar issues, about inconsistency of type length, when working with C++ 11 on windows with code written on Linux.

2 Likes
5

Cheers - here’s the latest on the ZChain development front:

Network status page is up and running at network.zcha.in. IPs are automatically imported from nodes which connect to mainnet.z.cash, so if you’re running a ZCash client you should show up somewhere on the map (WebGL required).

Transparent and shielded value definitions on explorer.zcha.in have been updated with much assistance from @tromer. The “transparent value” of a transaction now refers to the higher of the sum of the transaction’s transparent inputs and the sum of the transaction’s transparent outputs, while the “shielded value” of a transaction is the lower bound on the known shielded ZEC involved - calculated as the absolute value of the difference between the transaction’s transparent outputs and transparent inputs (note that for certain complex transactions involving multiple joinsplits, it may be known that more shielded ZEC were involved depending on the joinsplit ordering, but the currently displayed value should be accurate as a lower bound). The statistics page - explorer.zcha.in/statistics/network - now displays information about what percentage of transactions included shielded ZEC, and what fraction of total transaction volume is shielded, using the shielded/transparent value definitions above. (if you aren’t seeing anything new, force-reload the page with Ctrl-Shift-R)

A v2 API and frontend are under construction at the moment, expected in beta sometime late next week. At the moment v2 includes the following features:

  • Fork (chain tip) tracking - in two elements: (1) a page tracking chain tips, with metadata - height, chain tip hash, branch length, miner if known - and (2) a selector in the footer for which fork you want the explorer to use for display; the rest of the UI will change based on that (so one could browse blocks/transactions on a fork, for example)
  • Better network evolution-over-time graphs for miner distribution and shielded ZEC usage
  • Transaction/block propagation latency measurements using geo-distributed ZCash nodes
  • More advanced block/transaction/account search capabilities

Let me know what you think of those; I’m also quite open to ideas for any statistics / tracing / graphs that would be useful. ZChain can be reached on Twitter or here on the forums.

5 Likes
6

@lustro: Nice job!
A small comment,
when looking at this transaction
https://explorer.zcha.in/transactions/834385b914276bea5d539eaf73575de9bf321150e1a45ada0d3da47d9b4fddad
I was confused for a while how the transparent output could be 0 - cause it seemed to mean there is no miner’s fee.
Only after a while and after hovering over the values on bottom right corner I understood you are subtracting mining fee from both input and ouput values…
maybe that can be made clearer somehow

1 Like
7

@arielgabizon Are you referring to the transparent output of the JoinSplit?

8

Hi! First of all, I feel @anon47418038 is giving too much credit for me as he’s the one working hard. And indeed, I started a CMake port in order to see how it would like. It would like CMake would make it easier to reach other platforms and maybe have development process benefits, but naturally it’s all plain thought and talk until (I’m) walking the walk. Unfortunately I got stalled by influenza (which naturally wife and kids catched too) for the last 1-2 weeks and though I briefly mentioned in the chat I’m gearing back to online and could make progress towards the end of this week, it came evident I had other accumulated cruft I needed to take care of first. Now hopefully all clear and a bit luck not too busy at work, let’s see what this week brings with it!

1 Like
9

I’m glad you’re feeling better!

When you get to the point of adding libgmp and libsnark to Cmake, let me know, the versions of them for Windows in general and VS particularly are very fussy (stock libgmp won’t work, need to use mpir fork of it and my C++11 branch of libsnark).

10

Let’s chat when I get there! I noticed, there’s also LevelDb that might give trouble. I took a preliminary look at something such as at NuGet Gallery | libgmp_vc120 5.1.3.2 and if we could use something like that. This won’t come easy, but the bright side is that likely the platform reach isn’t Windows only, but easier everywhere.

11

hm, well mpir is also specifically designed to be 64-bit clean on Windows, and to build under Visual Studio, and has higher performance than libgmp. Let’s cross that bridge when we get there!

12

Yes that’s what I meant

1 Like
13

Update on this: we’re postponing 1 week! The first Show & Tell with @anon47418038 is set for Friday, Feb 3rd at 3pm PT/6pm ET/11pm UTC. Expect a new Show & Tell on the first Friday of each month!

3 Likes