The 2018 Biometric Technology Rally was an evaluation, sponsored by the U.S. Department of Homeland Security, Science and Technology Directorate (DHS S&T), that challenged industry to provide face or face/iris systems capable of unmanned, traveler identification in a high-throughput security environment. Eleven selected systems were installed at the Maryland Test Facility (MdTF), a DHS affiliated biometrics testing laboratory, and evaluated using a population of 363 naive human subjects recruited from the general public. The performance of each system was examined based on measured throughput (efficiency), capture capability and matching capability (effectiveness), and user satisfaction metrics.

This research documents the operational tradeoffs between transaction times (efficiency), identification rates (effectiveness), and user satisfaction. Here, we perform two tradeoff analyses: efficiency vs. effectiveness and satisfaction vs. efficiency and effectiveness. The results indicate that no clear relationship was evident between how quickly a system can process a user and how well it can identify the user. Further, a strong relationship was evident between how well a system identifies a user and how satisfied the user is with the system. However, no significant relationship exists between how quickly a system processes a user and how satisfied the user is with the system. These outcomes suggest that some systems are not benefiting, in terms of system effectiveness, from spending additional time collecting with the intent of collecting higher quality images. Further, these outcomes suggest that, in regards to public acceptance, systems designers should place more focus on identifying the system user properly rather than how quickly that user can be processed.