Measuring Shielded Adoption

Concur! I was totally biting my tongue and not saying this because I didn’t want to complain, nor to give extra work to Gareth, but it is really easy for people to misunderstand representations which show an incomplete bucket in the same visual way that it shows complete buckets (Figure A below).

Here’s are a couple of ideas for better representation — no idea how much work it would be to implement in this case:

  1. Whenever you’re visually representing a series of buckets as stacked bar charts, and you have an incomplete bucket (like a month that is not yet over) as one of them, make that one thinner proportionally to how incomplete it is. Like, if we’re halfway through the month, make that bar be half as wide as the bars representing complete months. Maybe show a dotted line showing how wide it will be at the end of the month. (Figure C)

  2. Another related concept is show a dotted-line linear extrapolation, i.e. if you’re halfway through the month, and you have 1000 items in the bucket, then show the 1000 items in the same color as the items from completed buckets (previous months), plus show a dotted line outline to indicate that, if the same rate holds for the rest of the month, that there will probably be 2000 items in this bucket by the end of the month. (Figure D)

Might be possible to combine both ideas.

Just some thoughts. Would be curious what Gareth thinks. Obviously it would be easier to just do what Daira suggested and omit incomplete buckets from your visualization. That would definitely be better, IMHO, than showing incomplete buckets side by side and visually identical to complete buckets. You could perhaps show the latest, incomplete bucket separately somehow, so that it is visually distinct from the complete buckets. (Figure B)

P.S. Ooh, you could even conceivably have that last, incomplete bucket be a dotted-line outline which is partially filled with mini-buckets. For example, if the last bucket is a month that is halfway over, it could be a dotted line that is one-month-wide, and it could have 15 little mini-bar-charts in it each of which is 1/30th of a month wide, showing the 15 days that have already passed and already have data. i think that should result in a visual indicator to the reader that both (a) shows them that this isn’t the same kind of data as the complete-buckets-data, and (b) gives them an indication or a prediction of where it is likely to end up. (Figure E)

P.P.S. This is probably a job for whoever makes the graphing libraries that Gareth is using rather than a job for Gareth. :laughing:

4 Likes

I actually considered recommending the extrapolation approach, and decided against because it can easily be misleading or cause scaling errors. For example, if you’re extrapolating a single day’s data to a month, then it won’t be at all accurate, and it might interfere with the scaling of the whole graph if the mis-estimate is too large.

7 Likes

Here are sketches of all the possibilities I mentioned above.
Figure A:


Figure B:
Figure C:
Figure D:
Figure E:

Like Daira says above, the linear extrapolation ones (Figure D and Figure E) can produce really inaccurate predictions.

If I were doing this today I guess I would go with Figure B, but it would be cool if some Edward-Tufte-loving dataviz wizard could come up with a good solution to this common problem.

4 Likes

Ok last idea and then I’m going to go back to my actual job.

Figure F:

This one is potentially actually implementable, and it avoids both the “partial buckets misrepresented as complete buckets” problem from Figure A as well as the “sketchy linear extrapolation” problems from Figures D and E. Not sure if it would actually be a good and useful representation in practice, compared to Figure B.

7 Likes

I personally would much rather see actual complete data from the actual chain, and nothing else. It’s far too easy to misread or misinterpret anything else, and I would hate to see any decisions or opinions be made based on extrapolations on data that has frequently changed over time. Hopefully no decisions are being made that require up-to-the-day information on this anyway!

5 Likes

Great feedback, thanks. This is why I always published complete monthly data as per the original post :laughing:

I agree with this and I’ve taken the dashboard down. It was intended as a POC, and as a result is very limited by the software used so it simply isn’t feasible to implement any of the suggestions as is. The plan was to develop further using a fully developed charting library (as that is where the bulk of the work is) using the same backend but I’ll save that for (much) later or perhaps someone can use this example and the above discussion to implement it in an existing solution.

5 Likes

@zooko: This data seems very important. While it’s been very kind of @garethtdavies to volunteer their time to maintain it, might it fall within ECC’s funding to keep it going? It is not currently possible for anyone to request ecosystem funding through the existing ZF Grants platform to do so.

2 Likes

Hey, do you have the CSVs for the data somewhere? I’m happy to mess around with different graphs.
I keep coming back this thread to track shielded adoption, so least I can do.

3 Likes

Sounds great! Here’s the raw data used which is at the tx level so it’s ~7 million rows and ~850MB. Script used to determine types is in the original post.

https://keybase.pub/garethtdavies/zcash-all-tx.csv

I’ll also continue to publish the monthly data as I have been doing, just dropping the real-time dashboard until it can be better implemented as per the suggestions above.

7 Likes

Updated stats for August. While monthly transactions were the highest since August last year, 90% of those were fully transparent :confused: A mere 1.5% of tx in August were z2z tx.

I’ve separated out shielded coinbase stats and will later show as a % of all coinbase tx when this ratio becomes more meaningful. In August there were 24 of them.

There was 1 fully shielded Sprout tx and 0 migration transactions in August. The latter is interesting given there is still ~115k in the Sprout pool.

12 Likes

How many total coinbase outputs were generated in August, to compare to the 24 shielded ones?

1 Like

There were 35,513 transparent coinbase tx in August.

3 Likes

Thanks! Much appreciated.

1 Like

So shielded adoption clearly needs massive improvement. But I don’t think it makes sense to focus on shielded coinbase separately.

Shielded coinbase was hyped a bit too much. In and of it self, it has nothing to do with privacy: the source, amount, and recipient of a “shielded” coinbase are, i’m told, public. Which is the same level of privacy you get from a transparent coinbase. So why bother? Well, now the money is in the shielded pool and as a mining pool operator you can distribute your payouts privately without extra work.

Mining pools clearly aren’t doing this but this shouldn’t be surprisingly :

  1. Their pool management software needs to be updated. That takes time and there needs to be demand to do it.
  2. Well, there needs to be demand for shielded.

So, we’re back to shielded usage needing to go up. Which it does. But shielded coinbase is a sideshow.

5 Likes

Shielded coinbase is a necessary prerequisite to moving to a fully shielded protocol. Might as well have done that part early.

9 Likes

BTW, I see this currently for the dashboard link:

Whoops, we couldn’t load the dashboard

The dashboard does not exist, or you don’t have permission to view it

See my earlier message. I removed based on feedback and just publishing monthly reports.

1 Like

Updated for September. 18,716 fully shielded transactions - that’s a new high and 11% of all transactions :smile:

For those measuring shielded coinbase adoption, there were 128 shielded out of 34,399 total coinbase transactions. There were also 7 migration transactions.

17 Likes

this is awesome

1 Like

tested ACK. :nerd_face:

1 Like