Request for feedback on Zbay user-interview design

Hi everyone!

As part of our ZF-funded project to research Zbay’s potential as a communication tool for journalists, activists and whistleblowers, we’re about to embark on a series of user interviews to better understand these use cases and inform our threat model.

And as part of that, we’re soliciting public feedback on our plan for those interviews.

We’d love feedback from anyone, but we’re especially interested in feedback from anyone with experience doing work like this, as well anyone who’s familiar with current debates about the best ways to do this work from a political and ethical perspective.

(And of course, if you are a journalist, an activist who supports journalists, an activist or public policy professional who is often a source for journalists, or a whistleblower yourself and you would like to be involved, feel free to be in touch—with the caveat that the process we’ll go through is still being fleshed out: h@zbay.llc or @holmes on Zbay.)

Thanks to @sarahjamielewis for reading an earlier draft and providing very helpful feedback.

And thanks in advance to anyone who can weigh in!

–Holmes @ Zbay

P.S. due to the fairly quick pace of the project, feedback arriving by early next week would have the biggest impact on the project, but if you see any issues to be addressed but need more time to distill your thoughts, just DM me and I can hold off on moving forward until we talk.


Proposal overview

Project summary

Purpose: To learn what anonymity, privacy, and security features would make Zbay uniquely useful for journalists and their sources, what specific threats they face, and what factors would make them choose Zbay over alternatives.

Aim: Yo build a product that can one day meet the security, privacy, anonymity, and usability needs of journalists and their sources.

Justification: Communicating privately and anonymously online remains extremely difficult, and this is a problem for journalists and sources. Zcash is a privacy-focused cryptocurrency with plans for scalability and network-layer privacy. It may be possible to leverage the Zcash network to provide a solution for private, secure, and anonymous communication.

Investigators

Holmes Wilson, Zbay LLC

Funding status

Funded by a grant from the Zcash Foundation, July 2020 - October 2020.

Dates

User interviews run July 2020 - October 2020

Conflict of interest

Holmes Wilson is a principal of Zbay LLC and holds Zcash.

Results will not be published and will be used to inform work on Zbay, free software available under the GNU GPL.

Methodology

We will use a typical user-interview approach approximating this set of recommendations. We will start with prepared questions, following up with conversation-specific questions to gain more detailed information. Each user will be invited to two interviews, the second of which is optional but requested. Each interview will be scheduled for one hour but could go longer if the interviewee chooses to. Interviews will be conducted over secure video calls.

Participants

Criteria

  • Journalists are:
    • Practicing professional journalists
  • Sources are one of:
    • Activists and public policy experts who encounter sensitive information that the public may have an interest in knowing.
    • Known whistleblowers
  • Activists are:
    • Activists or security trainers whose work includes recommending security tools for journalists and sources, supporting their security posture in some way.

Geographic focus

Due to the limited scope of the project, the need to begin with a specific niche and collect meaningful data about that niche, we choose to focus on journalists, sources and activists in the U.S. This is not ideal, but collecting data globally will likely yield more variance in data and user requirements, which would require 10x the number of interviews or more to ensure reasonable coverage.

Diversity

In order to maximize the range of viewpoints and risk models captured by the research we will:

  • Commit to ensuring that at least 60% of participants are women, non-binary and/or intersecting with communities facing gender-based oppression.
  • Commit to ensuring a minimum 40% rate of BIPOC (black, indigenous and people of color) participants.

Recruitment

Channels

  • Email outreach to qualifying colleagues, requesting share
  • Facebook/signal outreach to qualifying colleagues, requesting share
  • Email outreach to some relevant listservs, requesting share
  • Advertisement on sites frequented by U.S. journalists

Benefits to participants

There may be no benefits to participants beyond having an interesting conversation, sharing their views, and possibly gaining a useful tool at some point in the near future.

Social benefits

Because the results of the work will not be published, there are no known benefits to the scientific community beyond whatever lesson or patterns are revealed in Zbay or its source code. However, establishing a new way to communicate online that values usability, security, anonymity, and user freedom could provide a broader social benefit.

Risks to participants

  • It could be a waste of participants’ time.
    • Mitigation: keep interview to one hour, schedule flexibly, keep second interview optional.
  • Questions may prompt participants to discuss experiences that were stressful or traumatic
    • Mitigation: make it clear to participants that they are welcome to avoid any subject they would rather not discuss, or to cut short a line of questioning.
  • Participants may disclose sensitive information
    • Mitigation: in addition to making it clear that participants need not discuss sensitive information, we will use clear data protection safeguards and make these clear to participants. (See section on confidentiality of data)

Informed consent process

The invitation to participate will include a link to the study details and an overview of how the study will be used, including the steps we are taking to protect the security and anonymity of interviewees.

We will receive written consent by email or message (e.g. Signal) before each interview begins. This study does not involve persons who cannot give their own consent (e.g. minors.)

This study will not use partial disclosure or deception.

Anonymity of participants and confidentiality of data

  • Call will use end-to-end encryption (e.g. Signal)
  • Interview audio will be recorded on local device of a single interviewer, not in the cloud, and deleted after processing for aggregate notes and insights
  • Only aggregate notes and insights will be stored, with identifying details (like name, employer, and names of sources, organizations or scenarios) removed.
  • Specific requests made in the call about privacy or security will be honored if possible
  • The data will not be linked with any other dataset, or sent outside the location it was collected (the home office of the interviewer.)
  • Aggregate data in the form of a written summary will be stored locally, never on cloud services other than encrypted cloud backup solutions (e.g. Backblaze, Acronis) secured by a strong passphrase.
  • Interviewee identity will be known to the interviewer, but not to others.
  • No interview data will be published.
  • Conclusions from the interview data (e.g. “several journalists indicated they needed x”) may be published in our threat model document or our issue tracker.
  • One known situation in which confidentiality of data cannot be guaranteed is if the interviewer receives a subpoena before data has been destroyed, though this seems highly unlikely.

Remuneration

Participants will typically not receive remuneration, but exceptions will be considered on a case-by-case basis.

Feedback

Participants will receive a feedback letter, including a statement of appreciation, details about the purpose and predictions of the study, restatement of the provisions for confidentiality and security of data, contact information for the researchers, and a link to the forum post where our research proposal received community feedback.

Appendix B: Interview questions (1st round)

Journalists

Learnings sought

  • How they communicate with sources, normally
  • How they communicate with sensitive sources
  • How they communicate with colleagues
  • How they communicate with colleagues about sensitive matters
  • The most serious threats they face
  • The most common threats they face
  • Acceptable solutions to these threats
  • Any threats for which solutions do not yet exist
  • If lack of security, privacy, anonymity as ever harmed or limited their work
  • How they evaluate the suitableness of a tool for security and privacy
  • How they learn about security and privacy tools
  • What people and organizations they trust for recommendations
  • How often they try new tools
  • What convinces them to try new tools
  • If they face any privacy/security/anonymity problems right now that they need to solve
  • How urgent these are
  • Examples of steps they have taken to find a solution

Questions:

  • What does a typical day look like for you? What are the hard parts?
  • How do you communicate securely with sources and colleagues?
    • What software do you use?
    • What devices do you use?
    • What’s your workflow?
    • What do you typically use for less secure communication?
  • What are the most serious threats around privacy, source identity, and anonymity that you worry about?
    • Has security ever been an issue communicating with sources?
    • How do you protect your sources?
    • Has a source ever needed to be anonymous to you? How did you achieve this? What works well in these situations?
    • Are any of these threats not addressed by the tools you use? Any problems you need to solve? How urgently?
    • Has a lack of anonymity, privacy, or security ever harmed your work?
    • Do steps you take for security and privacy negatively impact your work in some way, e.g. do they create work, roadblocks, or slow you down? What are the worst things that happen?
  • What tools do your sources seem the most comfortable with? What do they tend to use? How much control do you feel you have over what tools your sources use to communicate with you?
  • Do you try new tools often? When was the last time?
  • What would make you try a new tool, or depend on it for something sensitive? How strong would the recommendation have to be? From who? What problem would it have to solve?
  • Do you use Signal? What works well? What doesn’t?
  • Do you use SecureDrop? What works well? What doesn’t?

Sources

Learnings sought:

  • How they typically communicate with journalists
  • How they communicate sensitive information with journalists
  • How they communicate the most sensitive information with journalists
  • What the most serious risks / threats are
  • What measures they consider acceptable to protect against these threats.
  • What kind of additional protection would be valuable
  • What factors inhibit or prevent them from communicating sensitive information with journalists when there is a good reason to
  • If they might sometimes need to make information public directly, without going through a journalist.
  • How they would do this
  • What factors inhibit or prevent them publishing information when there is a good reason to
  • Are they satisfied with existing tools for communicating with journalists? What are the gaps?
  • Have they used Signal and SecureDrop? What are the gaps?
  • Do they feel like they can trust these tools
  • What most builds trust in communication tools or methods?
  • How do they learn what tools are best?
  • What sources do they trust for advice on security tools?
  • What gaps do they see in existing tools for privacy, security, anonymity?
  • What would make them more confident when sharing sensitive information?

Questions:

  • What does a typical day look like for you? What are the hard parts?
  • When you need to communicate something sensitive to a journalist, how do you do this?
  • Do you ever worry that the information, or your identity as a source, will not be protected?
  • What methods or assurances did you use to ensure that the information would be protected? What additional protection would be valuable to you?
  • Have you ever considered sending information to a journalist anonymously? (If not, did you ever need to communicate anonymously for any other reason?)
  • Did you imagine using any specific software for this?
  • Did that software meet your needs? Were there any big gaps?
  • Have you used Signal? Did it meet your needs? What were the gaps?
  • Did you feel like you could trust it? Why or why not?
  • What would make you more confident when sharing sensitive information?
  • Have you used SecureDrop? Did it meet your needs? What were the gaps?
  • Did you feel like you could trust it? Why or why not?
  • What factors inhibit you from publishing information or providing it to a journalist when there are good reasons to?
  • What sources do you trust for information about tools or methods for protecting your security or privacy?

Activists that provide security support and training to journalists

Learnings sought:

  • What gaps they see in tools available to journalists for privacy, security, anonymity
  • What shortcomings they see in existing tools
  • What barriers they see to adoption of existing tools
  • What requirements do we need to meet to make them enthusiastically recommend a tool to fill those gaps?
  • What do they see as barriers to adoption?
  • How do they evaluate tools for suitability for a given purpose?
  • What sources or metrics do they trust, if any?

Questions:

  • What does a typical day look like for you?
  • What are the hard parts in supporting the security needs of journalists?
  • What tools and methods do you recommend/train/support for secure communication?
  • What’s missing? What are the gaps?
  • What about for anonymous communication?
  • What do you see as the barriers for journalists adopting Signal?
  • What do you see as the barriers for journalists adopting SecureDrop?
  • What are the shortcomings of Signal and SecureDrop, if any?
  • How do you decide whether or not to recommend a tool. What are the key factors?
    • Security audit?
    • Recommendations?
    • Your own checklist?

Appendix C: Interview questions (follow up)

Learnings sought:

  • Does our brand want to make them try the app when they hear about it?
  • If they see a link to our app on Facebook or Signal does the image, title, and description metadata make them curious or want to try it?
  • Does our website make them want to try and/or recommend the app?
  • Does the first time user experience make them excited about using the app?
  • Do they understand how to use it as their inbox, or contact another user?

Questions:

  • I’m sending you a link to the app. If a friend shared this with you, would you be interested in it?
  • What does the name remind you of or evoke for you?
  • Here’s our website. What does the logo remind you of or evoke for you?
  • Does this page make you want to download the app and try it?
  • What would make it better?
  • Would you be willing to try the app right now?
    • Is the experience of running it for the first time positive?
    • Is everything clear?
    • Does it make you happy you’re trying the app out, or excited to have it?
    • Pretend you had to contact a journalist and you knew their username was holmes. Would you know how to do that?
    • Would you know how to use this as an inbox? Does everything make sense?

Appendix D: Outreach content

  • Personal email
  • Personal Facebook/Signal message
  • Listserv email
  • Relevant listservs
  • Ad content
  • Sites for ads
  • Feedback letter
5 Likes

Found this on Twitter about using Signal (not as secure as people think):

Yeah, Signal has the TOFU issue and the number porting issue, though they’re beginning to address the latter with this notion of PINs I think.