Arti Year II Funding Plan: Onion Services and more!

Hi!

I’m Nick Mathewson, one of the co-founders of Tor. We’ve been working on Tor for a long time now (since 2002!), and over the years it has become a widely used and widely analyzed tool for internet privacy.

Last year, I made a post on the Zcash Community forum about a project called Arti—“a Rust Tor implementation”—and its network-level privacy benefits for the Zcash privacy portfolio, not only as a communications privacy layer for Zcash, but also for all the other communications tools that Zcash users employ. We discussed the project with stakeholders, and used their suggestions about Zcash’s needs to improve our project plan. After more revisions, we applied for a grant to fund the first year of Arti work from what was known at the time as ZOMG, and is now ZCF.

Thanks to ZCF’s generous funding of this project, we’ve taken enormous steps forward in developing Arti from an experimental project to a usable privacy backend. And today, I’m returning to the forum to discuss our progress, talk about what comes next, and propose our idea for a second-year grant to the Tor Project.

Brief introduction to Tor and Arti + Zcash

In my post last year, I wrote about Tor and its network-level privacy benefits, and defined Arti in depth. I encourage you to read this post if you are just now learning about this project and want to dive in!

Here’s what I said last year, and I think it still applies:

Tor provides robust network-level privacy for a huge variety of protocols, hiding their traffic among millions of active Tor users. Tor is available to use, well established, and well analyzed today, and works with most any protocol that works over TCP. Our network’s performance and scalability have been improving over time, and we layer with an ecosystem of tools to circumvent different censorship regimes around the world.

But the current Tor implementation bears the signs of its age. Tor is implemented in C, as a stand-alone network proxy. Because Tor is a stand-alone proxy, it’s cumbersome for third parties to embed. And because Tor is written in C, it’s difficult to have the same confidence in its memory-safety as would be possible with a higher-level memory-safe language. Moreover, because C is unsafe and error-prone, our development requires more time and energy than it should, which slows us down when delivering improvements to our users.

Here’s where Arti comes into play. Arti is a pure-Rust implementation of the Tor protocols designed from the ground up for flexibility in embedding, straightforward maintenance, flexible deployment, and reasonably good performance. We believe Arti is the future of Tor: by using Rust, we get memory safety “for free,” and will eventually get compilation to WASM. Moreover, we get a codebase that others can maintain and adapt to fit with their projects, taking only the parts that they need to keep their binaries small for mobile or other restricted environments.

Arti has the potential to benefit Zcash in many ways, both in the short term and the long term, as it resolves some existing painpoints for using Tor for network privacy: (1) Arti makes Tor easier to embed, (2) Arti allows for more fine-grained control over Tor’s behavior, and (3) Arti ensures Tor improvements can be developed faster.

By adding a layer of communications privacy between wallet users and the Zcash network, we expect Arti to improve the overall privacy of Zcash by hiding which users are communicating to which servers when, and thereby protecting users against a wider variety of traffic analysis and correlation attacks.

Moreover, Arti’s communications privacy benefits other applications in the privacy space that Zcash users depend on. After all, it’s hard to use Zcash’s private transactions in a meaningful way if you cannot have private communications with your transaction partners. By improving the ecosystem around online private communications, Arti provides more opportunities for Zcash users to privately coordinate and communicate.

Progress of our current grant with ZCF

We’re a little past the half-way mark in our year-one plans. Since we started, we’ve put out a number of releases, making Arti more secure, more stable, and more embeddable. You can read the details in our announcement posts for each of the versions we’ve released:

For people who like statistics: since we started this grant, Arti has grown by about 35,000 lines of code, and our test coverage remains well above 80%.

Now we’re focusing on the missing features needed to use Arti in production. When we release version 1.0.0 in a few months, we intend for Arti to be ready for at least some production use. This will require implementing important security features, tolerating more kinds of network failures, and converging on even more stable APIs.

In August and September, we’ll focus on censorship resistance in Arti 1.1.0, making sure that Arti can take advantage of Tor’s current group of bridge relays and pluggable transports to avoid censorship on the internet. When we finish 1.1.0, we will have finished the work laid out in our original grant with ZCF.

Lessons learned so far

Our attempts at consructing APIs before finalizing related functionality proved to be somewhat ill-conceived. In at least two places (our event-reporting API and our configuration API), initial work has proven to need API and implementation revision based on deployment experience and needs. Up-front design was not an adequate substitute for iterative refinement. One possible lesson to take from this is that in the future, we should avoid stabilizing a definite API for a piece of functionality before a couple of versions’ worth of experience trying to use it in practice.

One surprising piece of good news for us is that as this grant gave us the chance to hire more Rust developers, we had a much easier time than we typically have had when hiring C programmers in the past. The applicant pool was large, and of very high quality. We attribute this to the fact that writing new security-critical high-performance network programs in Rust is simply a more attractive job than maintaining 20-year-old security-critical legacy code in C.

Arti in the wild

At least three projects have already tried using Arti experimentally. In one, a group of researchers developing a COVID-19 contact-tracing app adapted Arti to anonymize their client-server communications for added privacy. In another, an internal team of developers at Tor is using Arti to construct a new cross-platform mobile-friendly Tor-based VPN application. Both groups have reported that using Arti was, for them, far easier than trying to embed the C Tor implementation. The third project is Zebra: @teor added experimental Tor support using Arti 0.0.2, and reported the results.

What’s next

When we end our year 1 grant, we intend that Arti should be production ready for at least some users and use-cases. But there is more work to do, which we believe will also be of benefit to the Zcash community. In year 2, we have two major milestones planned. In 1.2.0, we hope to be able to implement onion services for responder anonymity in Arti, and in 2.0.0 we plan to make Arti a viable replacement for most current users of the C tor client. I’ll talk about each of these phases below.

Year 2: Phases of Work

Implementing onion services (Arti 1.2.0)

The Tor client allows users to run onion services: a location-hidden server on the network that you can only connect to if you know its public key. For many users, this is their main use case for tor. Some applications (such as Onionshare, Ricochet Refresh, and cwtch) rely on onion services as their primary transport for private communications. Some developers in the broader Zcash community have also told us that onion services are a critical feature for meaningful integration with their applications.

To start the second year of Arti’s work, we plan to produce a complete implementation of the current (v3) onion service protocol, with support for both client-side and server-side implementations.

We also plan to include a full range of Tor’s tools for security and denial-of-service improvements to onion services, to make them safer to run on the internet. As we go, we’ll incorporate ideas from existing research to try to produce the most secure onion service implementation possible.

This is no small task: as you can see from the main ticket, onion services require a complex set of subprotocols in addition to the main Tor implementation. We’ll be drawing on the experience we’ve gained in the past from building these protocols in C, and from designing Arti to support protocol extensions.

Replacing the C tor client. (Arti 2.0.0)

In time, we want Arti to become the primary Tor implementation, so that we can focus our efforts on a high-productivity, high-security program environment without all of the risks and drawbacks of our old C implementation. But before we can do that, we need Arti to work well enough as a replacement for existing Tor users that they can migrate to it comfortably.

In the second half of our second year, we will be “filling in the gaps” to make Arti 2.0.0 a better alternative to the C client implementation for nearly all users. This will include work on circuit padding, a stable C FFI, support for more proxy protocols, and whatever missing features users and developers judge to be most important.

Once that’s done, we’ll be able to deprecate client usage of the C Tor implementation (though we’ll have to keep supporting it in maintenance mode for a couple of years at least).

After year two

Although this is beyond the scope of our proposed grant, I thought it would make sense to share what we’re planning for future work on Arti.

Our long-term goal with Arti is to improve the pace of Tor development by breaking free from our legacy code and architecture, and from the dangerous of the memory-unsafe world. So once we have parity with the C client implementation, we’re going to switch our attention to implementing the relay and directory-authority parts of the Tor protocol. Once that is done, we’ll be able to deprecate our C implementation completely, and deploy new features and improvements to the protocol as “Rust only”. We’re at the early stages of estimating this work, but we think it should be something we can accomplish in one or two further years.

In closing

Thanks for reading this post! Based on our work so far on Arti, we believe that this second year of work is of interest to the Zcash community. We’d like to hear from the community about this idea before we propose a grant to ZCF, so that we can integrate your feedback. Thanks again: we’re looking forward to your ideas!

(This seems like a good time to say that the Tor Project is a 501(c)3 not-for-profit charitable corporation, incorporated since 2006. We don’t have shares, or answer to shareholders. We are committed to free software, and we don’t do patents. We also believe in transparency, and we post our annual financial filings online once they’ve been audited. Our mission is to develop and promote privacy software, and we believe in it strongly.)

21 Likes

Great update and really cool work. Having a rust implementation of tor will make it much more portable as it seems many projects are moving to rust these days

1 Like

ZOMG is now Zcash Community Grants (ZCG) :slightly_smiling_face:

1 Like

Sorry I’m a bit late commenting on this proposal, NU5 activation was a bit busy for all of us!

These seem like features that would benefit wallet users.

If you’re using a private cryptocurrency, you likely also want network privacy for:

  • checking your wallet balance (Arti’s Onion Services)
  • the websites you visit to make Zcash purchases (Tor Browser does this already)
  • submitting your transactions (Arti’s Onion Services)

So you probably want to chat with:

  • light wallet developers
  • full wallet developers
  • node developers (if the node accepts transactions directly from users)
5 Likes

Join the Zcash Foundation + @ZcashGrants this Thursday at 17:00 UTC for a call w/ the Tor Project to discuss their ZCG funded grant, Arti: a Rust Tor implementation.

(https://zfnd-org.zoom.us/j/84070218472)

5 Likes

:crab: :onion:

4 Likes

Hi all, here’s our first update on our Arti Year 2 grant (Gallery View: Zcash Community Grants Program)

6 Likes

Arti 1.1.1 is released, and announced at Arti 1.1.1 is released: Groundwork for onion services | The Tor Project

7 Likes

Hello Zcash!

We are thrilled to let you know that the Arti team here at The Tor Project has begun building Onion Services support into the Arti codebase as part of the grant provided by you. The team managed to finish 2022 with the deliverable that opens up Tor’s anti-censorship technology to be used by Arti users and developers. The work here in 2023 will become a huge milestone that we have been looking forward to reaching, and we are very happy for your continued support of the Arti project.

The Arti team has worked with the rest of the Network Team and our project management on planning how the first two quarters of 2023 will look. We will make some changes to how we do Arti development this year, which will streamline the process and make it easier for external parties to understand what is happening and the name of The Tor Project’s goals of being transparent with our development work. In addition, we also have some deliverables on the Arti codebase that do not originate from the ZCash-provided grant. These will land in the codebase over the year and become available with our Arti releases.

To give a summary of some of the things we are going to change:

Releases

We want to move to a release model for Arti, where the team makes a new monthly release. This release model will differ from the year one release model, where each release was tied to a specific feature deliverable. The new release model should make it easier for others to follow when releases are readily available, and they can read the ChangeLog and our announcement blog posts on what has changed since the last release.

There may be releases over the year that comes out where you will be wondering if this isn’t a “small” release of the content from the ZCash grant. That may be because the specific release will add one or more significant features from one of the other grants we also have. One feature we will be working on this year in Arti from outside the ZCash grant is upgrading the Tor network with support for UDP packets on top of the already existing TCP streams. This work will be part of our experiments with building a VPN mode for Tor using the Arti ecosystem.

FFI and RPC

Another reason we want to split out the releases to monthly is that it allows us to work on building a solid and usable FFI and RPC developer experience over the entire duration of the grant while the team is also working on other deliverables. We have already opened up for the first meeting where we can have discussions on what needs developers have to integrate Arti into their existing codebases.

We will run these meetings over the next year and open them up for more participants. If you are interested, please reach out to ahf at torproject dot org, and he will ensure you get an invitation to the next meeting. These meetings are very technical by nature :slight_smile:

Features

The team has already dived head-first into the big Arti project this year: Onion Service support. The team has been working on improved estimations with our project management and has split the targets into three components: onion service client connectivity, onion service hosting, and additional onion service security features. Once all three components have landed, we will have feature parity with C Tor’s Onion Service support, but splitting up the work allows us to release some usable code to the world for early testing. This means developers interested in experimenting with Onion Services in Arti can begin their experiments after the first component has landed, and host their own Onion Services using Arti once the second component has landed. After the security component is in, the safety of using Arti for hosting Onion Services will be on par with C Tor today.

According to our current estimates (and remember, we could improve estimates once we start doing the work on each area), we believe that we will have client support done towards the end of July, service support done towards the end of September, and the security work done around the end of November.

These dates are later than those we had estimated when we applied for this grant for two primary reasons. First, we are starting later than originally anticipated (January 2023 rather than November 2022). Second, as discussed above, we have decided to move the start phase of our FFI/RPC work earlier into our timeline, since we believe it will need a great deal of refinement through experience.

This means that milestone #1 (with the deliverable “Onion Services: Arti has client and service support for Onion Services.”) of this grant will be completed on November 30, 2023. We will make a better estimation in December 2023 for the following milestones #2, #3 and #4.

Conclusions

We are all very excited about getting started with this work. 2022 was an exciting year for Arti’s development, and it would not have been possible without your help. We genuinely look forward to delivering these new experiences and taking Arti to the next level here in 2023 with Onion Services’ support. We believe these features will open up a much more rich ecosystem where privacy-related projects can integrate Tor’s state of the onion anti-censorship technology with our Onion Services for building exciting new applications :slight_smile:

Please reach out to us if you have anything you want to talk with us about!


Editing this post with a new announcement! Arti 1.1.2 is released: HsDir groundwork and cell handling improvements | The Tor Project


Announcement! Arti 1.1.3 is released: More onion service development | The Tor Project


Update: Arti 1.1.4 is released: Development on Onion Services and RPC | The Tor Project


Update: Arti 1.1.5 is released: Onion Services, RPC, and a security patch | The Tor Project


Update: Arti 1.1.6 is released: Now you can connect* to Onion Services! | The Tor Project


Update: Arti 1.1.7 is released: incremental improvements, numerous bugfixes | The Tor Project


Update: Arti 1.1.8 is released: Onion service infrastructure | The Tor Project


Update: Arti 1.1.9 is released: Assembling the onions | The Tor Project


Update: Arti 1.1.10 is released: Assembling the onions | The Tor Project


9 Likes

@bdavila Can you post an update for your grant, please?

2 Likes

Sure!

Hello,

Earlier this year, the team wrapped up support for Arti Onion Services connectivity such that an Arti client can connect to Onion Services within the Tor network.

Last month, an exciting new milestone was reached by the team: we now can use Arti to connect to an Onion Service that was also made available via Arti’s API. Since then, the team has worked on stabilization and eliminating bugs that will make the Onion Services feature work as you would expect from well-tested code.

The team is working hard now to wrap up the service side, such that Onion Services can be hosted internally via Arti’s API or using a reverse proxy similar to that seen in the C implementation of Tor.

However, we will not be able to meet the deadline we originally had for Onion Services support (Deliverable 1.1). We will detail here where we are at with the project so you will better understand the causes of the delay and when we expect to be completed.

Beyond the client-side and service side Onion Service work that was done, the team also did work that was not originally in the grant but that we identified as being essential to make the availability of the Onion Services feature better for end-users and third-party integrators:

  • We added an internal key management subsystem in Arti for handling cryptographic material that Arti needs to store (Onion Services identity keys, Client Auth keys, Certificates, etc.). Since this is a better format and design than C Tor’s, the team will be working on a command-line interface tool to generate and work with these objects.

  • As we have mentioned before, the Arti team at Tor also wanted to work on the RPC/FFI milestone from an early point of view so that we could gather some feedback, both internally and externally, on designs. This should make the engineering work more approachable once the team continues this engineering deliverable after the Onion Services work is wrapped up. This work continued from where we previously reported last time.

We additionally had some other unexpected delays come up when we had to move some staff around internally earlier in the year, resulting in a ramp-up phase. Fortunately, that was significantly shorter than we thought it would be back in March.

We believe the team will be able to wrap up Onion Services work around February 2024 with inclusion in the Arti release that will happen at the end of February/early March 2024.

This, of course, impacts the timeline for the entire duration of the grant, but we will return in January 2024 with some better estimates for when we think the grant as a whole can be finalized. As some of you may be aware, we recently got external funding to deliver a Relay implementation of Arti, which means we can finally move the majority of the Network Team at Tor to Arti engineering in 2024, this will likely slow us down a little as the C engineers transition into the Arti project, but once they do we anticipate an increased pace.


Update: Arti 1.1.11 is released: More onion progress | The Tor Project

6 Likes

Hi pkr, Tor has had excellent communication with us throughout their grants. They proactively reached out to us as soon as they knew their originally submitted timeline would no longer work and informed us that they would post this update on the forum. I checked in with them last week about this and it turns our they edited their original post with the update instead of making a new post. I asked them to make a new post so the community would be notified of the update and the correct date of the update would be reflected on the forum. I had to reply to their post to allow them to post this update because the forum does not allow someone to reply 3 times in a row to their post. My question asking to provide an update was simply a technical workaround to allow them to write a new post.

7 Likes

@pkr Thanks for this great question!
@Danika Thank you for responding :slight_smile:

4 Likes

Thanks for the update, super excited about this project :grin:

2 Likes

Could you suggest some ways Arti could be used in Zcash projects?

Thanks,
–h

1 Like

Thanks for the context. Not sure why (wait yes I do), the “community” flagged my post :roll_eyes:

Arti could be used to make it possible for wallets or nodes to transmit new transactions to the Zcash network over Tor, to prevent potential leakage of the sender’s IP address. This would require integrating Arti with Zebra (to enable Zebra to receive new transactions over Tor), as well as changes to wallets (to enable them to transmit new transactions over Tor), and some mechanism to enable discovery of Tor-enabled nodes.

A hypothetical future light client protocol that isn’t as bandwidth-intensive as it is today could also leverage Arti to allow wallets to connect to the Zcash network over Tor.

3 Likes

Also important when light wallets fetch txs to decrypt memos, so that nodes don’t learn which txs are yours.

1 Like

@dodger @conradoplg I think there are issues regarding connecting to GRPC servers over TOR. Are they resolved?

1 Like

Zcashd doesnt support tor v3. It wasnt upgraded after NU5 because of the (workload imposed by the) sandblasting.

You can read through the arti v1 thread and maybe get a better idea of the progression