COMPARISON OF BIOMETRIC TECHNIQUES

Prepared by Thomas RUGGLES

 

Initial Release:

April 17, 1996

Revised:

March 15, 1998

Revised:

February 11, 2001

Revised:

May 31, 2002

Revised:

July 10, 2002

Copyright 1996

For a comprehensive follow-on document, please refer to

Biometric Technical Assessment


This drop-down table of contents requires a Java-enabled browser.

REPORT OUTLINE

The original objective of this report was to examine various techniques for biometric identification and identify which biometric would be most suitable for the California Welfare Fraud Prevention System.  Since its original publication in 1996, many changes have occurred in the field of biometrics; in addition, this document has been referenced in more than forty web sites dealing with the topic of biometrics.  Therefore, beginning with the May 2002 revision, Comparison of Biometric Techniques has been re-written to, hopefully, reflect the latest level of technology in the field of biometrics.  

The biometric methods compared in this report are: Fingerprints, Facial Recognition, Hand Geometry, Retinal Scan, Iris Scan, Vascular Pattern, Signature Dynamics, and Voice Dynamics. Each biometric will be considered in the light of the California Welfare Fraud Prevention System requirements.

Please refer to the biometric overview at the end of this report for a general outline of biometric techniques.

 

EXECUTIVE SUMMARY

Fingerprints, Facial Recognition, Hand Geometry, Retinal Scan, Iris Scan, Vascular Pattern, Signature Dynamics, and Voice Dynamics biometric were evaluated with respect to accuracy, CLOSED and OPEN search requirements, data collection error rates, and client acceptance.

The Vascular Pattern technique has not had enough operational data produced to make a proper evaluation.

Signature Dynamics and Voice Dynamics have the lowest accuracy rates of any biometric considered in this report. In addition, these two techniques rely on behavioral measurements as opposed to physical measurements and, as such, are difficult to reproduce. Signature Dynamics and Voice Dynamics are not considered to be viable candidates for an OPEN SEARCH biometric technique.

The Hand Geometry biometric was eliminated from consideration because of its small per-record data size (9 bytes per record). Given the expected large database size of an average biometric-based system (i.e., >1,000,000 records), the database must be divided into partitions with only a (relatively) small number of records per partition. Accuracy decreases with an increase in the number of records compared per search; therefore, partitioning must be done to increase the accuracy of the system. A database consisting of a biometric with a record size of only 9 bytes per record cannot be divided into a large number of partitions with a small number of records per partition.

Retinal Scan has high accuracy but also has high data collection error rate and low user acceptability. Retinal Scan has been used as a CLOSED search biometric; few, if any, applications have used Retinal Scan as an OPEN search biometric.

Iris Scan has the potential to have very high OPEN search accuracy.  Iris Scan is unobtrusive and, as such, is generally accepted by clients.  The Iris Scan image capture may be impaired by dark glasses, eye disease, and the percentage of the iris area that is exposed with the eyelid open in a "natural," static environment.  There is some question as to whether a low light condition (that increases the size of the pupil thus decreasing the total area of the iris) may affect the proper imaging of the iris.  As with Retinal Scan, above, few, if any, applications have used Iris Scan as an OPEN search biometric.

Facial Recognition does not seem to be a dependable technique to establish identity because the error rates for this biometric appear to increase with time, angle of the image captured, lighting, and facial expression.

The Fingerprint biometric has a low data collection error rate and high user acceptability. Further, Fingerprint technology has had the most research and development money applied to both the CLOSED and OPEN biometric search problem. Finally, the Fingerprint biometric has the highest acceptance in the identification community and virtually every large biometric system in operation today uses the Fingerprint biometric. Notwithstanding its association with "criminal" applications, the Fingerprint biometric is generally accepted by clients.

After consideration of the performance levels of Fingerprints, Hand Geometry, Retinal Scan, Signature Dynamics, and Voice Dynamics biometrics, I recommend that the Fingerprint biometric be used in the California Welfare Fraud Prevention System.

 

REFERENCES

The following sources were used in the compilation of this report:

A Performance Evaluation of Biometric Identification Devices, J. Holmes, L. Wright, R. Maxwell (Sandia National Laboratories, SAND91-0278/UC-906, June 1997).

Biometrics: Who Goes There?, J. Fenn (Gartner Group, Inc., Spring 1995).

Personal Identifier Project Executive Summary (State of California Department of Motor Vehicles (CA DMV), 16 May 1990).

Electronics Benefits Transfer - Use of Biometrics To Deter Fraud In The Nationwide EBT Program, GAO/OSI-95-20 (September 1995).

 

 

BIOMETRIC ACCURACY

Biometric accuracy is measured in two ways; the rate of false acceptance (an impostor is accepted as a match - Type 1 error) and the rate of false rejects (a legitimate match is denied - Type 2 error).

Every biometric technique has a different method of assigning a "score" to the biometric match; a "threshold value" is defined which determines when a match is declared. Scores above the threshold value are designated as a "Hit" and scores below the threshold are designated as "No-Hit."  A Type 2 error occurs if a true match does not generate a score above the threshold.  A Type 1 error is made when an impostor generates a match score above the threshold. If the Type 1 and Type 2 error rates are plotted as a function of threshold value, they will form curves which intersect at a given threshold value.  The point of intersection (where Type 1 error equals Type 2 error) is called the crossover accuracy of the system. In general, as the value of the crossover accuracy increases the inherent accuracy of the biometric increases.

 

Biometric

Crossover Accuracy

Retinal Scan

1:10,000,000+

Iris Scan

1:131,000

Fingerprints

1:500

Hand Geometry

1:500 (against a very small background database)

Signature Dynamics

1:50

Voice Dynamics

1:50

Facial Recognition

no data

Vascular Patterns

no data

Table 1: Biometric Crossover Accuracy

Facial Recognition is still in the research stage and will not be considered any further in this report.

Vascular patterns, also a new biometric technique, is very similar to Retinal Scan and, within this report, will be considered equal to Retinal Scan in terms of accuracy.

Of all biometric techniques listed in Table 1, only Signature Dynamics and Voice Dynamics rely on behavioral characteristics rather than physical characteristics. A person’s voice and signature are prone to variability in execution and, in the case of Voice, variability in the instruments used to take the measurement.  Because of these reasons and because of the fact that the crossover accuracy for the biometrics are the lowest of all the biometrics examined, Signature Dynamics and Voice Dynamics will be dropped from further consideration in this report.

Biometrics that are considered to be accurate enough to fulfill the requirements of a large-scale (one million plus individuals) Biometric-Based Identification System are: Retinal Scan/Vascular Pattern, Iris Scan, and Fingerprints.

As a final note, it should be pointed out that the crossover accuracy values in the table above were generated by use of a single instance of the biometric; i.e., the results are based on single fingerprints matches, etc.  If multiple instances of biometrics are obtained from an individual and used in the search (for instance, if the images of two fingers from each client are matched against a corresponding set of two finger images in the database) the identification system accuracy is expected to increase. 

If biometric-based identification systems are designed to use multiple instances of biometric data from each individual and the system analyzes all search results synergistically, the expected accuracy rate of the identification system is expected to be significantly higher than the rates shown in Table 1. 

To achieve higher accuracy rates by using multiple biometrics, there is no reason to limit the "identification engine" to just one type of biometric; while accuracy may be increased using multiple instances of just one type of biometric (using two fingerprint images instead of just one, for example) even higher accuracy levels should be attainable by using more than one type of biometric in the system.  A system that used Iris and Fingerprint biometrics would be inherently more accurate than a system that used just one type of biometric.  In addition, a system that used multiple examples of multiple biometric types (for example, a system that used two fingerprint images and two iris images) would be much harder to "spoof" that a system that used only one type of biometric.

 

TYPES OF BIOMETRIC SEARCHES

There are two broad types of biometric searches; CLOSED searches and OPEN searches. A CLOSED search occurs when the client claims to be already enrolled in the system; in this case, the client’s biometric will be read and the biometric will be compared to the biometric data already on file for the client.  An OPEN search will occur when the client is (or claims to be) unknown to the system (i.e., not in the biometric database); in this case, the client’s biometric will be searched against all biometric records in the database.

Of the two types of searches, the OPEN search is technically the most challenging and costly.  OPEN search accuracy generally decreases as the size of the database increases; for this reason, biometric records must be categorized according to some broad characteristic within the biometric data.  Records that fall into a like category are put into a fixed "partition" (or "bin") of the database. Subsequent searches for a particular record, or a record belonging to the given category, are searched within the small subset or partition of the total database. This technique lowers the relative size of the database per search and thereby increases the accuracy of the system.  The risk in this technique is in the accuracy of the "binning" process; a mistake at this point usually, if not always, results in a "Miss" if the person is already in the database.

In order to divide the biometric data into partitions, the per-record size of the biometric data must be considered. In general, the larger the per-record data size of the biometric, the easier it is to assign the biometric record to a particular partition within the database.  Also, the database can be divided into a greater number of partitions as the per-record data size of the biometric grows larger (i.e., there are more possibilities for generating distinct partitions).

 

Biometric

Data Size Per Record

(bytes)

Retinal Scan

35

Iris Scan

256

Fingerprints

512 - 1000

Facial Imaging

 

Hand Geometry

9

Table 2: Biometric Data Size per Record

 

[Note:  With respect to Retinal Scan and Iris Scan, the Biometric Crossover Accuracy figures from Table 1, above, for these two biometric techniques do not correlate with the Biometric Data Size per Record given in Table 2.  It is my opinion that the crossover accuracy should be a very nearly linear function of data size per record.  Given the high crossover accuracy for Retinal Scan in Table 1, I would expect that the data size of a Retina Scan would be much higher that either Iris Scan or Fingerprints.  This data size vs. crossover accuracy inequity indicates that accuracy claims for Retina Scan may not be well-founded.]  

From Table 2, we see that the per-record data size for Hand Geometry is 9 bytes vs. 1000 to 35 bytes for the other biometrics listed. With a record size of only 9 bytes, it will be very difficult to divide the welfare system’s 6,000,000-record database into a sufficient number of partitions so that a single OPEN search is matched against a relatively small number of records.

Given that the Hand Geometry database cannot be partitioned into a large number of data "bins" so that a single search is matched against a relatively small number of records, Hand Geometry is not a viable candidate for a large-scale biometric-based identification system. Further, given the small per-record data size and the fact that there are usually a large number of records in the database, there is a high probability that a particular record will have a large number of identical - or nearly identical - matching records in the database.  In this case, discrimination of one record from among many identical or nearly identical records will be difficult to achieve.

In the recent edition of The Biometric Journal, Mr. Bill Willson (V.P. of Recognition Systems, a supplier of Hand Geometry biometric systems) concedes that Hand Geometry is "just not practical" for large systems.

At this point, the biometrics still under consideration are; Retinal Scan, Iris Scan, and Fingerprints. 

 

DATA COLLECTION ERROR RATES

Data collection procedures differ according to the biometric used. Some biometric measurements are prone to error during the measurement while others produce consistent results from read to read.

Biometric collection procedures are listed as follows:

Retinal Scan - "The (Retinal Scan) reader contains an aperture where the user looks to align his eye with an optical target, which appears as a series of circles. As the user moves his eye around, the circles become more or less concentric. Proper alignment is achieved when the circles appear concentric and the user is looking at the contour of the circles." With the eye properly aligned with the reader device, an infrared light illuminates the retina (heating the blood vessels of the retina) and a camera captures an image of the infrared-enhanced blood vessel pattern.

Fingerprints - "The user places his or her finger on a small (flat) glass plate. The system captures a high-resolution optical image of the fingerprint, typically using a charge-coupled device (CCD) camera."

Of the four reports referenced in this study, two (the Gartner report and the CA DMV report) cited data capture problems associated with Retinal Scan.  In the CA DMV report, three test results are listed for Retinal Scan and Fingerprints; the following table lists these results:

Data Collection Error Rate For: 

 

 

Retinal Scan

 

Fingerprint

Test 1

3.41%

0.42%

Test 2

18.28%

6.01%

Test 3

9.16%

1.88%

Table 3: Data Collection Error Rates

From Table 3, we can conclude that the data collection error rate for Retinal Scan is considerably higher than that for Fingerprints.

In Table 3, the fingerprint "Live Scan" device that was used in the testing was an [Identix] product that only produced binary (i.e., black & white) fingerprint images. This reader is now at least two generations "old"; the current versions of the Live Scan readers from any vendor produce 256-level gray scale images of the fingerprints. As a result, the current versions of Live Scan devices can reasonably expect to produce a significantly lower data capture error rate than that shown in Table 3.

The data collection error rate in Retinal Scan is due to "errors resulting from operator’s instructions, dependency on each participant’s ability to focus correctly, and operator lapses in quality control." The capture of a correct Retinal Scan biometric requires the operator to verbally "walk the clients through" the mechanics of the scanning procedure. The operator does not have a real-time display or feedback of the client’s actions and the operator must rely on the client’s ability (and willingness) to follow instructions.

On the other hand, Fingerprint data collection provides a real-time display of the fingerprint image during the capture procedure; the operator can instantly see if the image is correctly placed on the reader and the operator can select the moment when the image placement has been optimized for capture.

Further, while it is hard for the client to "cheat" during fingerprint image capture (i.e., purposely sabotage the fingerprint image capture process) because of the real-time monitoring capabilities of the operator, there is much more opportunity to cheat during Retinal Scan data capture.

Of the published studies that compare various biometric techniques for identification, most (if not all) deal with access control-type problems. In an access control-type problem, the system is faced with verification of a client’s stated identity; this is a CLOSED search problem. In an access control-type problem, the client tries his best to get a positive verification by the system - even if the client is an impostor.

I have not reviewed a study that addresses the biometric matching problems associated with searching a large database for possible matches (an OPEN search problem). In an OPEN search problem, the welfare client who is trying to circumvent the system will state that he is not in the system whereas his record is in the system (under, perhaps, a different name). This welfare client must try to cheat the system during data collection in order to be successful. Regardless of the accuracy of the system, the successful read of a biometric that is in the database will result in an increased chance of detection; therefore, the impostor must try to degrade the collection of the biometric data. Any biometric that can be purposely corrupted during data collection, especially if the fact that the corruption has occurred is undetectable to the operator, cannot be used in any system that relies on OPEN searching.

Considering data collection error rate, it would seem that the Retinal Scan biometric is not suitable for the California Welfare Fraud Prevention System. On the other hand, the Fingerprint biometric, given its relatively low data collection error rate and the fact that the system can provide the operator with real-time monitoring and quality control capabilities during data capture, seems to be suited for the California Fraud Prevention System.

 

CLIENT ACCEPTANCE

According to the Sandia report, Retinal Scan had the most negative reaction compared to all other biometric techniques. The "users have... concerns about retina identification, which involves shining an infrared beam through the pupil of the eye..." Also, Retinal Scan requires "a precise alignment and a pause while the scan is done, while (other biometric techniques) such as voice and fingerprint can be done in a more natural and casual manner."

Some biometric techniques must capture a large sample of the biometric (signature and voice) in order to determine if the biometric is typical of (i.e., is matched to) the data on record.

The CA DMV study cites the summary of responses to a survey of 9,709 participants rating public acceptance of Retinal Scan and Fingerprint technologies. The survey results showed that there was a 96.48% favorable public response to Fingerprints vs. a 93.44% favorable response for Retinal Scan. Significantly, the CA DMV study notes that, of all the people approached to participate in the DMV project, 2,515 refused to participate in Retinal Scan while only 619 refused to participate in Fingerprint.

The ease of data collection impact client acceptance of the biometric:  One of the factors affecting ease of data collection is the percentage of unacceptable reads; if the percentage of unacceptable reads is high, the client’s confidence in the system is likely to decrease. The CA DMV study reports that Retinal Scan had an unacceptable data collection rate of 5.9% vs. 1.8% for Fingerprint.

 

_________________________________________________________

_________________________________________________________

 

BIOMETRIC OVERVIEW

 

FINGERPRINT IDENTIFICATION

[Please see “Fingerprint Identification Update” that was added in the 10 July 2002 revision of this document.]

Fingerprint identification is, perhaps, the oldest of the biometric sciences. Fingerprints were used in China as a means of positively identifying one person as the author of a document. During the British occupation of India in the 1800’s, a British policeman named Henry established the first systematic classification of fingerprints. The ten finger classification scheme allowed fingerprint image records to be divided into relatively small groups. The fingerprint set of a person who was arrested and classified according to the Henry system could be compared to a small group of similar records - this narrowed the search process and led to better identification rates.

In recent years, fingerprint comparisons have been based on "minutiae", i.e., individual unique characteristics within the fingerprint pattern. Within a typical fingerprint image obtained by a live scan device, there is an average of 30-40 minutiae. The Federal Bureau of Investigation (FBI) has shown that no two individuals can have more than 8 common minutiae. The U.S. Court system has consistently allowed testimony based on 12 matching minutiae; in some courts, a lower number of matching minutiae have been allowed.

In the early 1970’s, the U.S. government commissioned a study by Scandia Labs to compare various biometric identifiers; the conclusion of this report was that fingerprint technology had the greatest potential to produce the highest accuracy rate for identification purposes. The Scandia study is dated but the impact of the study was to shift focus onto fingerprint technology. Because of this early emphasis on fingerprint technology, the years since 1970 have produced a large body of research and development in fingerprint identification algorithms and integrated systems.

Fingerprint images contain a large amount of data. Because of the high level of data present in the image, it is possible to eliminate false matches and quickly reduce the number of possible matches to a small number, even with large database sizes. Because of the fact that Fingerprint Imaging Systems use more than one finger image in the match process, the match discrimination process is geometrically increased.

Fingerprint identification technology has undergone an extensive research and development effort over the past twenty years. The initial reason for the effort was in response to the FBI requirement for an Automated Fingerprint identification System (AFIS). Today, in the criminal justice AFIS application, the fingerprint identification process has a 98%+ identification rate and the false positive identification rate is less than 1%.

 

FACIAL RECOGNITION

[Please see “Facial Recognition Update” that was added in the 10 July 2002 revision of this document.]

Facial recognition is the most natural means of biometric identification; this method of distinguishing one individual from another is an inherent ability of virtually every human. However, until recently, facial recognition has never been treated as a "science" and has been largely subjective in nature. Police artists have tried to categorize different parts of the face (chin line, hair line, nose features, mouth features, etc.) into sets of templates that could be assembled into a composite face which would, hopefully, resemble the face of the person in question.

Facial recognition technology has recently developed into two areas of study; facial metrics and eigenfaces.

Facial metrics technology relies on the measurement of specific facial features (e.g., the distance between the inside corners of the eyes, the distance between the outside corners of the eyes and the outside corners of the mouth, etc.) and the relationship between these measurements.

Within the past two years, an investigation has been made into categorizing faces according to the degree of fit with a set of "eigenfaces". It has been postulated that every face can be assigned a "degree of fit" to each of 150 eigenfaces; further, only the template eigenfaces with the 40 highest "degree of fit" scores are necessary to reconstruct a face with over 99% accuracy. The difference between the eigenface method of facial categorization and the police artist method of building a face from template parts is that the eigenface method is based upon an actual photo of the individual and the "eigenface" information is derived from a computer-based analysis of the digital image of the photo. Eigenfaces are (reportedly) highly repeatable and are not affected by human subjectivity.

Eigenface technology has some promise, but it is a technique that is just in the infancy stage of development. Very little data regarding eigenface error rates (false negative, false positive) exists at this point.

 

HAND GEOMETRY

Hand geometry is based on the fact that virtually every person’s hand is shaped differently than another person’s hand and that the shape of a person’s hand (after a certain age) does not significantly change its shape. Various methods are used to measure the hand; these methods generally fall into one of two categories - mechanical or image-edge detection. Either method produces estimates of certain key measurements of the hand (length of fingers and thumb, widths, etc.); this data are used to "categorize" a person.

Hand geometry, as compared to some other means of biometric identification (notably fingerprints), does not produce a large data set. Therefore, given a large number of records, hand geometry may not be able to distinguish one individual from another who has similar hand characteristics. Simply, as the size of the database grow, there must be an increase in the number of distinguishing characteristics of the biometric used in order to place the individual into an increasingly narrow "band" of individuals who share similar biometric characteristics. With hand geometry, there is not enough data available; the individual is placed in a "band" within the database structure that contains many individuals.

To effectively use hand geometry in a system with 5,000,000 plus individuals, it would be necessary to actively review large candidate lists. Since it would be very difficult to verify that two sets of hand data are identical, a second biometric - probably a photo image - would have to be used to make the verification. Considering the cost of the manpower necessary to accomplish this task and the number of potential searches per day (11,000), hand geometry may be prohibitively expensive.

  

RETINAL SCAN

Retinal Scan technology is based on the blood vessel pattern in the retina of the eye. An infrared light source is used to illuminate the retina of the eye; the infrared energy is absorbed faster by blood vessels in the retina than by surrounding tissue. The image of the enhanced blood vessel pattern of the retina is analyzed for characteristic points within the pattern.

A retinal scan can produce almost the same volume of data as a fingerprint image analysis. Based on the fact that a high data volume equates to a high discrimination rate (identification rate), it would seem that retinal scan may be an alternative to fingerprint identification.

Retinal scan technology has several drawbacks that are not common to fingerprint imaging technology; 1) the retinal scan is more susceptible to disease (notably cataracts, etc.) that change the characteristics of the eye; 2) the method of obtaining a retinal scan is personally invasive - a laser light (or other coherent light source) must be directed through the cornea of the eye; and 3) the method of obtaining a correct retinal scan depends heavily on the skill of the operator and the ability of the person being scanned to follow directions.

Importantly, retinal scan technology has not had the level of research and development funding (both from private and government sources) that fingerprint imaging technology has had within the past twenty years.

 

IRIS SCAN

[Please see “Iris Scan Update” that was added in the 10 July 2002 revision of this document.]

Iris Scan technology is based on characteristics in the iris of the eye. A person must stand approximately 12-14 inches from a camera which frame-grabs an image of the iris for analysis. An iris scan produces a high data volume which equates to a high discrimination rate (identification rate).

Iris scan technology may be more acceptable to user than retinal scans and, as opposed to retinal scan, it does not use an infrared light source to highlight the biometric pattern in the iris.

Iris scan technology is not yet in production and is currently in prototype testing.

 

VASCULAR PATTERNS

Vascular pattern technology is very similar to Retinal Scan technology in that it uses infrared light to produce an image of the vein pattern in the face, in the back of a hand, or on the wrist.

Vascular pattern technology is generally acceptable to users except that some users still object to any biometric method that uses infrared.

 

SIGNATURE RECOGNITION

[Please see “Signature Recognition Update” that was added in the 10 July 2002 revision of this document.]

Signature recognition is based on the dynamics of making the signature, i.e., acceleration rates, directions, pressure, stroke length, etc., rather than a direct comparison of the signature after it has been written.

The problems with signature recognition lie in the means of obtaining the measurements used in the recognition process and the repeatability of the signature. The instrumentation cannot consistently measure the dynamics of the signature. Also, a person does not make a signature in a fixed manner; therefore, the data obtained from any one signature from an individual has to allow for a range of possibilities.

Signature recognition has the same problem with match discrimination (i.e., finding a match in a large database) as does hand geometry.

 

VOICE DYNAMICS

Voice dynamics relies on the production of a "voice template" that is subsequently used to compare with a spoken phrase. A speaker must repeat a set phrase several times as the system builds the template.

This biometrics technique relies on the behavior of the subject rather than the physical characteristics of the voice and is, therefore, prone to inaccuracy.