Learning from Authoritative Security Experiment Results

The 2017 LASER Workshop

Dead on Arrival: Recovering from Fatal Flaws in Email Encryption Tools

Juan Ramon Ponce Mauries, Kat Krol, Simon Parkin, Ruba Abu-Salma, and M. Angela Sasse

University College London and University of Cambridge

Abstract

Background. Since Whitten and Tygar’s seminal study of PGP 5.0 in 1999, there have been continuing efforts to produce email encryption tools for adoption by a wider user base, where these efforts vary in how well they consider the usability and utility needs of prospective users.

Aim. We conducted a study aiming to assess the user experience of two open-source encryption software tools – Enigmail and Mailvelope. The study combined labbased sessions with remote tasks.

Method. We conducted a three-part user study (installation, home use, and debrief) with two groups of users using either Enigmail or Mailvelope. Users had access to help during installation (installation guide and experimenter with domain-specific knowledge), and were set a primary task of organising a mock flash mob through encrypted emails in the course of a week.

Results. Participants struggled to install the tools – they would not have been able to complete installation without help. Even with help, setup time was around 40 minutes. Participants using Mailvelope failed to encrypt their initial emails, due to usability problems. Participants said they were unlikely to continue using the tools after the study, indicating that their creators must also consider utility.

Conclusions. Through our mixed study approach, we conclude that Mailvelope and Enigmail had too many software quality and usability issues to be adopted by mainstream users. Methodologically, the study made us rethink the role of the experimenter as that of a helper assisting novice users with setting up a demanding technology.

Important Dates

04/18 Call for Papers
07/15 Submissions Due
09/01 Authors Notified
09/11 Registration Open
          Accepting Student Grant Apps
09/15 Program Announced
09/29 Student Grant Application Deadline
09/22
Hotel reservation deadline
09/29 Pre-workshop papers due
*** EXTENDED ONE WEEK
10/07 Early Bird Registration Closes
*** EXTENDED ONE WEEK
10/18-10/19 Workshop
11/22 Final Papers Due

Apply for student travel grant LASER Venue

Important Links

LASER Workshop Home

Past Workshops

LASER Mailing List

Further Information

If you have questions or comments about LASER, or if you would like additional information about the workshop, contact us at: info@laser-workshop.org.

Join the LASER mailing list to stay informed of LASER news.