Arti: A pure-Rust Tor Implementation for Zcash and beyond

[tokidoki]

Congrats @nickm_tor and all people at Tor

Happy to be a part of community who cares about the future of privacy for all

https://twitter.com/ZcashID/status/1402235896649293825

[Shawn]

Tor Project Receives $670K From Zcash Open Major Grants for Coding Language...

[extrapo]

Congrats for the Tor Team!

Heavy Tor user here, 5 years ago on the CaosComputerCamp I gave a workshop on making a crawler for Hidden Services, also generating some DataVizualizations… Was very lucky to meet some of you with Gabriella Coleman in the train back to Berlin… lots of intellectual generosity…

Long Live to Tor!

https://events.ccc.de/camp/2015/wiki/Session:Deep_Graphics

[bdavila]

On October 29th we launched the first release of Arti, v0.0.1 [0]. This release is ready for general testing, but not yet ready for production use. You can read about the features and changes in the changelog for this release [1][2…8].

0. release: Arti 0.0.1 is released: An embeddable Tor client in Rust - News - Tor Project Forum

1. changelog: CHANGELOG.md · main · The Tor Project / Core / Arti · GitLab

2. [tor-dev] Arti development report: 23 June through July 7

3. [tor-dev] Arti Report 3: July 21 through August 4

4. [tor-dev] Arti report 4: August 4 through August 18

5. [tor-dev] Arti report 5: August 18 through September 15

6. [tor-dev] (Short) Arti Report 6: September 15 through September 30

7. [tor-dev] Arti report 7: September 30 through October 13

8. [tor-dev] Arti report 7: September 30 through October 13

[bdavila]

Arti 0.0.2 is released: API groundwork, refactoring, config, and optimism

Arti 0.0.2 is released: API groundwork, refactoring, config, and optimism

[teor]

@nickm_tor

we at Tor aren’t necessarily the right team to develop patches for apps in the Zcash ecosystem. (That’s not where our expertise lies, and our performance at this task would probably be less efficient than more experienced cryptocurrency developers.) But we would be more than happy to put forth the work and effort to support wallet developers at every stage of their integration plans. We especially want to make sure that their needs are met by our APIs, and have included the cost of that that needs-discovery within the Zcash ecosystem as part of our project estimates.

I recently spent about a day adding Tor support to Zebra’s isolated connection API using arti-client.

When Zebra adds support for sending user-generated transactions, we can use this API to make sure they aren’t linked to the sender’s IP address. (Or to any other transactions we send.)

Overall the process went really smoothly. The API was easy to understand, the code is short and easy to read, and basic tests seem to work. (I wasn’t able to do comprehensive end-to-end tests, because Zebra doesn’t support creating transactions yet.)

Here’s some detailed feedback about some tricky parts of the integration.

Dependency Versions

The dependency versions that arti requires seem quite strict for a library crate. We needed to do a few dependency upgrades for Zebra as part of integration. These upgrades were simple using cargo upgrade. But sometimes dependency upgrades might need code changes.

Can arti depend on the earliest supported minor release of each dependency in Cargo.toml?
You can use Cargo.lock to specify exact dependencies for binary crates.

Dependency Troubleshooting

It would be helpful to mention:

• installing pkg-config, and
• setting OPENSSL_DIR and SQLITE3_LIB_DIR for troubleshooting header or linker errors.

I also needed to set LD_LIBRARY_PATH, but that’s probably a Nix derivation bug.

The troubleshooting doc was a bit hard to find, maybe it could be linked from the readme.

Useful APIs

There are a few missing APIs that would help make our code more efficient, and might reduce our load on the Tor network:

An API that bootstraps Tor if needed, then returns a cloned Tor client

Making a shared TorClient instance was a bit tricky, because it’s hard to lazily call async code in Rust, and then share the result. So I ended up wrapping the shared value in an Arc<Mutex<_>>. But this seems like a bit of a waste, because there’s already a lot of locking inside TorClient.

Here are some API changes that might have made that easier:

• an API or example code that initializes a shared TorClient when it is first used, then clones the shared instance for future requests
• a non-async API that configures and allocates a shared TorClient, then bootstraps it when it actually gets its first request (or when an async bootstrap method gets called)
• if possible, making arti_client::Error cloneable, so arti futures be used with FutureExt::shared

A TorClient config that makes sure all connections are isolated

In Zebra, we don’t plan on sending blockchain blocks over Tor, because it’s all public information. So we’ll just be occasionally sending new user-generated transactions over Tor.

So it would be helpful to have a TorClient config that guarantees all connections are isolated. It might also save some preemptive circuit load on the network. (Unless all preemptive circuits are guaranteed to be isolated already - I can’t remember.)

But I can understand if you don’t want to expose this as a configurable client mode. Because it will place more load on the network.

Minor API nitpicks

arti_client::Result is a useful type alias, it would be nice for it to be available outside the arti crates. That way, our tor-specific APIs don’t have to use BoxError everywhere.

It seems like the stream timeout config builder isn’t exposed by the client config builder. But we didn’t need it anyway.

In Zebra we’ll probably end up creating a tower::Service that takes hostnames, and returns arti_client::DataStreams. So we might not actually need any of these API changes - we can do it all in the service. Cloneable errors would be nice though!

[nickm_tor]

Thanks for the detailed feedback, @teor! So you know, we’re on vacation till 6 January, but once we’re back we’ll be opening a bunch of tickets for these issues, and probably asking some questions too. I want to give this feedback a high priority in January. (And a happy new year to you!)

[teor]

Thanks @nickm_tor!

Just letting you know that I’m taking some leave in January. I can answer questions once I’m back.

I’m happy to answer questions on individual tickets on your bug tracker. It would be hard to track all those details on a single forum thread. (But we should post the ticket links here, so interested people can follow along.)

[nickm_tor]

@teor

Overall the process went really smoothly. The API was easy to understand, the code is short and easy to read, and basic tests seem to work. (I wasn’t able to do comprehensive end-to-end tests, because Zebra doesn’t support creating transactions yet.)

Thanks, @teor! This is really helpful. Below I’ll link to the tickets we’ve opened on our tracking system, and include any questions or comments we have for you.

Here’s some detailed feedback about some tricky parts of the integration.

Dependency Versions

The dependency versions that arti requires seem quite strict for a library crate. We needed to do a few dependency upgrades for Zebra as part of integration. These upgrades were simple using cargo upgrade. But sometimes dependency upgrades might need code changes.

Can arti depend on the earliest supported minor release of each dependency in Cargo.toml?
You can use Cargo.lock to specify exact dependencies for binary crates.

This is arti#275 and arti!228, now merged. One question we had here, though, is why cargo upgrade was necessary on your end. (Were you using earlier versions that weren’t forward-compatible with the ones that Arti required? Do you happen to remember which crates were trouble?)

Dependency Troubleshooting

It would be helpful to mention:

• installing pkg-config, and
• setting OPENSSL_DIR and SQLITE3_LIB_DIR for troubleshooting header or linker errors.

I also needed to set LD_LIBRARY_PATH, but that’s probably a Nix derivation bug.

The troubleshooting doc was a bit hard to find, maybe it could be linked from the readme.

This all sounds like a good idea; it’s now arti#277.

Useful APIs

There are a few missing APIs that would help make our code more efficient, and might reduce our load on the Tor network:

An API that bootstraps Tor if needed, then returns a cloned Tor client

Making a shared TorClient instance was a bit tricky, because it’s hard to lazily call async code in Rust, and then share the result. So I ended up wrapping the shared value in an Arc<Mutex<_>>. But this seems like a bit of a waste, because there’s already a lot of locking inside TorClient.

[…]

Agreed that this is a good idea. The cloneable-Error issue falls under arti#247; I’ve opened arti#278 for the rest.

A TorClient config that makes sure all connections are isolated

In Zebra, we don’t plan on sending blockchain blocks over Tor, because it’s all public information. So we’ll just be occasionally sending new user-generated transactions over Tor.

So it would be helpful to have a TorClient config that guarantees all connections are isolated. It might also save some preemptive circuit load on the network. (Unless all preemptive circuits are guaranteed to be isolated already - I can’t remember.)

But I can understand if you don’t want to expose this as a configurable client mode. Because it will place more load on the network.

I think we should try to find a way to provide this feature without too much risk of people turning it unnecessarily. I’ve opened arti#279 for this.

(As an aside: preemptive circuits are treated normally with respect to circuit isolation. That is, two isolated streams can be placed on the same preemptive circuit iff those streams can share a circuit.)

Minor API nitpicks

arti_client::Result is a useful type alias, it would be nice for it to be available outside the arti crates. That way, our tor-specific APIs don’t have to use BoxError everywhere.

Agreed; this is now arti#280.

[qoute]
It seems like the stream timeout config builder isn’t exposed by the client config builder. But we didn’t need it anyway.
[/quote]

Thanks; that omission sounds like an oversight. It’s now arti#281

In Zebra we’ll probably end up creating a tower::Service that takes hostnames, and returns arti_client::DataStreams. So we might not actually need any of these API changes - we can do it all in the service. Cloneable errors would be nice though!

Thanks for the feedback: It’s exactly the kind of thing we were looking for at this stage, and it’s been extremely helpful Even if Zebra does wind up doing all of this with a [tower::Service], making these changes will probably help other projects as well.

[teor]

@nickm_tor

@teor

Dependency Versions

The dependency versions that arti requires seem quite strict for a library crate. We needed to do a few dependency upgrades for Zebra as part of integration. These upgrades were simple using cargo upgrade . But sometimes dependency upgrades might need code changes.

Can arti depend on the earliest supported minor release of each dependency in Cargo.toml ?
You can use Cargo.lock to specify exact dependencies for binary crates.

…

One question we had here, though, is why cargo upgrade was necessary on your end. (Were you using earlier versions that weren’t forward-compatible with the ones that Arti required? Do you happen to remember which crates were trouble?)

In 2021 we paused our dependency upgrades for a few months while we focused on implementing the Zcash consensus rules. We also delayed some upgrades until the rest of the ecosystem caught up, because we wanted all our dependencies to use the same version of common crates.

So sometimes we run behind the latest crate versions by 3-12 months.

We also try to minimise the number of duplicate crate versions we have. I wrote a detailed comment on the arti issue explaining our reasons.

Here are the exact lists of dependencies we upgraded at each stage:

Dependencies that Zebra uses even when arti is turned off:

• upgraded bytes 0.5.6 via http 0.2.1, this got rid of some duplicate dependencies
• upgraded a bunch of proc macro crates, more duplicate reduction

I’m not sure if there were any specific incompatibilities here. But bytes has had some in the past.

The rest of the version changes probably aren’t that interesting. But I’m listing them just in case.

Arti-only dependencies, after unification with Zebra’s existing dependencies.

Version changes when we moved from a specific commit to the tagged 0.0.2 release.

Version changes since I opened the PR at the end of December.

[Shawn]

Arti 0.0.4 is released: Refactoring, rustls, and more! | Tor Project

[anon35140610]

I love that they always post blog updates and that at the end of the updates, they always give a shoutout to ZOMG / Zcash Community Grants. It’s a good way to bring exposure to the program.

[bdavila]

Thanks for posting @Shawn!

Tor is here if anyone has questions/comments/concerns regarding this project update

[teor]

Thanks to the Tor team for this update!

We haven’t had a chance to try it with Zebra yet, but it looks like it resolves all our feedback. (And does some other useful things as well.)

[decentralistdan]

TwitterSpace: Grant Discussion ~ Arti: a Tor Rust Implementation for Zcash & Beyond

We will be hosting our 2nd Twitter Space alongside the @ZcashGrants and will be joined by members of the Tor(Arti) team on March 2nd at 1pm ET. Set a reminder and come join us in this Community Grant Discussion! We would also like to crowdsource some questions from the community beforehand so please post in this thread with Qs for the Tor team, and also please include your Twitter handle if you we will be joining the call and would prefer to come up as a speaker to ask your question live.

[bdavila]

Arti 0.1.0 is released: Your somewhat-stable API is here! | Tor Project

[alsmith]

Hi! When the Tor Project received this grant last year, ZCF was ZOMG. We like to publicly recognize sponsors (who opt-in) on our sponsor’s page: Tor Project | Sponsors, to say a bit about their work and what about Tor they are supporting.

I’m wondering if there’s new ZCF branding and if we should update this blurb in any way? What do you think @ZcashGrants?

Blurb below screenshot below:

Blurb text:

Zcash Open Major Grants (ZOMG)

ZOMG (Zcash Open Major Grants) funds projects that advance the usability, security, privacy, and adoption of Zcash, a privacy-focused cryptocurrency which pioneered the use of zk-SNARKs. The Zcash ecosystem is driven to further individual privacy and freedom. ZOMG is funding a project to create an implementation of the Tor protocols in Rust (called “Arti”).

[tokidoki]

The current Zcash Community Grants (ZCG) website is at zcashcommunitygrants.org. It currently uses the Zcash logo that can be found on z.cash/press/.

The website states:

The Zcash Community Grants program (formerly known as ZOMG) funds independent teams entering the Zcash ecosystem, to perform major ongoing development (or other work) for the public good of the Zcash ecosystem.

Note: I am not on ZCG board. Just sharing info published on ZCG website. Cheers.

[anon35140610]

ZCG has a horizontal logo which I think is better to use when talking about the grants program on pages that are not zcash-related. So that people skimming* through the text can instantaneously see that this is funded by Zcash Community Grants. The previous ZOMG logo also had it spelled out which made grants awareness easier when not on their site.

I added it in here but maybe a member of @ZcashGrants can clarify how best to communicate about the program on other websites!

[tokidoki]

@alsmith

I’m wondering if there’s new ZCF branding and if we should update this blurb in any way? What do you think @ZcashGrants?

Has @ZcashGrants responded to this?