Zcash Node and lightwalletd Running PoC Proposal

@chris-remus — we discussed the proposal at our last meeting, and I’ll summarize our decision here:

Lightwallet infrastructure is definitely on our list of funding priorities, so we want to encourage proposals like this, and we’re grateful for your being one of the first people to submit a proposal to ZOMG!

To proceed with considering the proposal, we would need to see need more transparency around costs. For example, for the pieces of this that are developer time, how many hours do you estimate, and what rates are you billing at? And for the pieces that are server costs, what are your cost estimates?

We’re also open to you increasing the scope of the project, if that makes sense for you.

At least a few of us agreed that it’s helpful and healthy for there to be highly reliable lightwallet infrastructure that is managed independently from existing wallet app teams, and the things we’d look for would be proposals that provide high levels of privacy, reliability, and security for users.

If you’re open to discussing this more and refining the proposal, we’d welcome that, and thanks again.

2 Likes

FYI, ZecWallet Lite allows the user to choose between two preselected servers or a custom one (see: Wallet » Server info » Switch LightWalletD Server)

2 Likes

Thank for the feedback @aiyadt. To answer your question, the idea here was to develop -

1 - User documentation to help users understand how to connect to the service

2 - An FAQ documenting common questions and error messages with their resolution that may arise when trying to connect

Please see inline -

Sure, I hope the dialogue the submission sparked is beneficial to the community.

To proceed with considering the proposal, we would need to see need more transparency around costs. For example, for the pieces of this that are developer time, how many hours do you estimate, and what rates are you billing at? And for the pieces that are server costs, what are your cost estimates?

Did you see the breakdown in this spreadsheet? It was included in the original proposal. Is there anything else you need?

We’re also open to you increasing the scope of the project, if that makes sense for you.

Yes, it does, since if approved, we intend to be in this for the long haul. To clarify, would we be talking about increasing how long we run the services and adding a testnet implementation?

At least a few of us agreed that it’s helpful and healthy for there to be highly reliable lightwallet infrastructure that is managed independently from existing wallet app teams, and the things we’d look for would be proposals that provide high levels of privacy, reliability, and security for users.

If you’re open to discussing this more and refining the proposal, we’d welcome that, and thanks again.

Please let me know if you need anything else regarding the cost breakdown and confirm the potential scope increase intention and we’ll be glad to continue the discussion!

1 Like

@holmesworcester once I receive your feedback on the above response, I can update our proposal.

Please let me know if you need anything else regarding the cost breakdown and confirm the potential scope increase intention and we’ll be glad to continue the discussion!

Thank you!

@chris-remus thanks!

I have two remaining questions, and I can post questions from others if they come up today:

  1. If it turns out that support or maintenance take much less time than expected, how would you deal with that? For example, will you bill ZOMG for less?

  2. Do you have a sense of who will use this in the period it’s available for? This seems a bit tricky because even if in general we want this to be available, it’s hard to fund if it seems unlikely that people will use it. Do any existing wallets need a “backup” solution in case their main server goes down? Are there any teams that need a testnet solution?

1 Like

As far as I understand, the second offer includes the same work, but for 2 years, what comes out as a result is 3 times less if you count by months, will you consider an offer that is more expensive if there is a more profitable one or is there a price reduction option taking into account the market on this?

1 Like

I think it’s great for the community that there are multiple competing proposals to provide lightwallet infrastructure.

Engaging like this and giving applicants a clear idea of our doubts and questions can give them (and other potential applicants!) ideas about how to navigate potential pitfalls and make their proposals more compelling.

It can also help shape future proposals or encourage competing proposals.

Digging into the proposals, the big difference in cost comes from the fact that this proposal bills at $150/hr for engineer time, and the other at $50/hr, and the fact that the other proposal doesn’t break out support costs, either because they’re handling that with volunteers or because they’re baking that in to the 6 hours / month of devops time, which means they’re really confident that they won’t be doing much support or putting out too many fires.

I think in practice this proposal could end up consuming fewer hours, and the other could end up consuming more hours, over the next few years. I’d say the biggest difference is the hourly rate.

In my opinion, this still does not make it necessary to spend more than you can, in the right business work it is important to get more for less money, the money is not endless to overpay for a reason why not. If there is a personal interest in certain service providers, then these services should not differ several times, otherwise there will be question “why”.
Self-interest is trust, not something bad.

True! And we haven’t spent anything yet!

I would say that this is bad and not good, a month and a half of downtime. Well, I have my own point of view, not popular here, so do not take it as a criticism of the work.

I agree with you that it is important to fund projects soon, and that it’s important to fund projects as cost effectively as we can.

Right now we have a shortage of applicants, and I think this will continue to be our problem, so I’m keen on doing everything we can to encourage strong proposals.

One way to do this is to spread the word.

Another way to do this (and what I’m trying to do here) is to engage as constructively as possible with the proposals we have. That is, what can I (or we) say or suggest to resolve any issues and bring proposals from qualified teams up to the level where we would consider funding them. I think if we all take a proposal when we see it here and try to make it stronger, that’s perhaps the easiest way to get more proposals and more projects funded—much easier than trying to increase the pool of applicants (which I think we should also do, but which is harder, since that’s “top of the funnel” work.)

Also, I’m really not trying to make this oppositional! I just really want it to be clear why I’m asking the questions I asked above, since if it wasn’t clear I can see where it would seem strange or suspicious given that there’s another proposal billing at a lower rate.

Solution for shortage of candidates: announcement of a grant for finding candidates with suitable ideas, you have a section on what you want to fund, give 500 (example) dollars for work on finding proposals for selection in well-known organizations, I think the forum is a good place to create a request and then twitter? Many (almost all are not at all aware of the movements in the zcash project)
The idea of ​​job rewards is a very old way to attract attention, it always worked great, and waiting for contractors to knock on the door themselves did not work, so discussing the methods that work is the first decision that should be made.
Roughly speaking, hire a temporary worker from the circles in which you can find what you need, and continue to apply this method until the expectation method works, competition (economy) will cover these expenses many times
Again, the “talent hunt” helps a lot, specialists are always at work, you need to be able to find not just offers, but some of the best, so you need to hire a person who will create hype in the media, that there is a search for specialists and companies to improve zcash, this will go to benefit to all, I think, I know that this is not your job, but otherwise there will be crumbs from the table (no offense to the available grants! I’m talking about the prospects of the crypto market)

I’m back from a delayed holiday vacation. Please see inline.

  1. If it turns out that support or maintenance take much less time than expected, how would you deal with that? For example, will you bill ZOMG for less?

We’ve been consulting in one way or another for 15+ years. In that time, we’ve come to learn that billing for time and materials is a poor way of aligning the interests and motivations of the value creator and beneficiary. It also introduces time and attention consuming overhead that often negates any value to the beneficiary in disputing charges, etc.

What we’ve done instead and with success over this time is to try and bill for value provided. This means agreeing to the value a deliverable provides prior to the project starting, then tracking whether that value was delivered.

That said, the approach we’ve taken in the pricing is to combine a T&M approach with a value based approach. This results in the flat fee. The flat fee is basically the mid-point of a range of time estimates.

So rather than track time, we’d much prefer to agree on a value up-front, with measurable parameters, then track those parameters. If we come-up short against the parameters, we don’t get paid the full fee. (In a symmetric example of this, a service provider gets paid the full fee if they meet expectations, lesser fee if they don’t and greater fee, i.e. bonus if they exceed them.) We feel this maximizes the alignment between the Zcash community and us, the service provider.

  1. Do you have a sense of who will use this in the period it’s available for? This seems a bit tricky because even if in general we want this to be available, it’s hard to fund if it seems unlikely that people will use it. Do any existing wallets need a “backup” solution in case their main server goes down? Are there any teams that need a testnet solution?

The way we approached minimizing this risk (and others) was by limiting the project’s duration. The idea was that after the initial period of performance, the community would look at the value created, including usage, and determine whether the project was worth continuing. (Hence PoC :slightly_smiling_face:in the title.)

We could also then adjust our original quote to expand it over a longer-term, amortizing any start-up costs in this original post, while applying learnings to possibly reduce ongoing costs, if possible.

Of course, one risk with this approach is that the services aren’t around long enough to gain traction. Yet as I’ve been saying all along, we’re in this for the long run and are glad to provide services as long as it makes sense for both sides to do so.

Our sense was that the Zcash community has recognized the need for additional infrastructure. It’s to that
stated need we submitted this grant application. We have’t done direct user research to validate demand. In these early days, we feel an empirical approach is the prudent one, rather than trying to estimate user demand. Yet we can imagine that any developer who wants to get started building on zcash quickly but doesn’t want to deal with the hassle of running infrastructure would find this service valuable.

I hope this response answers your questions. Please let me know if it doesn’t and I’m glad to fill in the gaps!

Thanks for your reply!

We ended up moving the proposal from the nighthawk folks to the next stage in the process, for reasons including: the costs were more in line with what we expected, there was a guaranteed user in nighthawk wallet, and the longer period of time period gives time for an ecosystem to build up around it (though we haven’t settled on 2 years yet and there are some questions).

You can follow the kinds of questions we’re asking over on that thread: https://forum.zcashcommunity.com/t/2-years-of-lightwalletd-infra-hosting-maintenance/38126

There’s also a proposal for the Zecwallet folks for the same thing, as part of a larger maintenance proposal, which ZOMG hasn’t discussed yet: https://forum.zcashcommunity.com/t/zecwallet-code-maintenance-and-infra/38170

I have no idea, and I’m not really pushing this one way or another, but Zecwallet’s lightwalletd setup is different than Nighthawk’s and has some performance optimizations, and I wonder if Zecwallet might see lightwalletd maintenance as a distraction and be interested in another party taking this on.

@holmesworcester I appreciate the feedback. I wanted to be sure you saw this response I sent to you a while back.

I don’t recall receiving a response from you -

If there’s still an interest in us expanding the quote to extend for a longer period of time, we’d be glad to do so. In that case, we’d be able to reduce our costs, as the commitment would be for a longer term.

Otherwise, we’ll pause the conversation here and hope that our initial proposal was helpful to both the ZOMG and Zcash Community. If there’s any interest in returning to it in the future, e.g. to diversify the network infrastructure, or if Zecwallet might be interested in us running infrastructure for them, please let me know.

Apologies for missing that.

As I remember the conversation, there were concerns raised about bang-for-buck, just as there were concerns raised here in the thread.

Others could chime in here, but I think the reasons why we moved to diligence with the Nighthawk proposal were:

  1. At least one project will use it, i.e. Nighthawk.
  2. Costs are apparently lower for the amount of services offered, for a few reasons (to be fair, some are not apples-to-apples comparable.)
  3. A longer time period makes more sense, provided 1 and 2.

For this reason, i think if there were projects lined up to use it (which means there are projects that need it), and the time period was long enough to make switching to it worth it for other projects (e.g. 1 year or more), and it was clear to everyone that the costs made sense for the scope of the project, it would be in a good position to be funded.

I get the idea of applying for proof of concept which would then apply for further funding, but I think it will feel like an irresponsible use of money, to us and others here in the thread, if it literally isn’t used at all.

That’s where my suggestion about reaching out to Zecwallet came from.

I can just ask here, to simplify. @adityapk00 — does Zecwallet have any interest in moving its devops work on lightwalletd outside the organization, and having another grantee take it on? Would that be helpful at all or at any point this year? (There’s no pressure from me or ZOMG to do this, and I’m just curious if it would be helpful!)

Also, just to note for others in the thread, the reason why I’m asking about this is that there are currently two parallel and separate lightwalletd setups (ECC’s, and Zecwallets) that aren’t currently compatible and that—even when they’re made to be compatible—will still have different performance characteristics, with Zecwallet’s being able to sync faster. (I don’t have metrics on what the difference is.)

At the next meeting, we’ll be discussing Zecwallet’s proposal for funding for their version of the lightwalletd infrastructure, and it’s great to have a competing proposals when possible!

2 Likes

@chris-remus Thank you for submitting the POC proposal for running lightwalletd and Zcash nodes for the community.

@ZcashGrants members met today and reached the consensus that the costs associated with this proposal did not meet expectations for the scope of work to be performed. Therefore ZOMG has decided to decline funding your proposal at this time.

Thank you for the prompt responses to ZOMG and the communities questions.

Also, please note: A rejected proposal does not prevent you from submitting proposals in the future should you wish to revise or submit a different project.

hi Brad,

Looks like I am somewhat late to reply here :slight_smile: I will be keeping an eye on notifications moving forward!

From developers point of view, having the ability to spin up a node and maintain it ourselves would be beneficial given that it’s easy (i.e. docker container) and not expensive (I believe it’s fairly cheap to run Zcash light node). It costs us about 40$ a month to keep it. With some further effort it can be brought down some more). Overall we found Zcash light client fairly easy to maintain.

That said, as a developer we would rather have users running their own nodes. So, if there is a way for a user to do it themselves we feel it’s worth exploring. Using something like https://www.ankr.com can work. With a click of a button from a wallet app the user can spin up his own node at Ankr and communicate only with that node.

This obviously would not be a feature for all but mostly for power users with even higher requirements for privacy.

iBek

2 Likes