Monthly Archives: November 2011


On Biometric Suppliers Publishing Accuracy Figures 2

Of late there have been repeated calls on Twitter for biometric suppliers to publicly release statistics pertaining to the performance of their biometric algorithms, specifically False Accept Rates (FAR) and False Reject Rates (FRR).

Whilst not a response to those calls, this post is in part motivated by them.

Those repeatedly calling for the release of these figures know in advance that their calls will not be heeded. As they are already well-versed in the technology, they already understand the reasons why. Yet I believe they persist so they can cite the non-responsiveness of suppliers as “evidence” that the technology does not work.

Let’s examine why suppliers keep this information secret.

1. THEY DON’T

I have been involved in negotiating multiple contracts for deployment of biometric technology, ranging from large government infrastructure programmes, through to enterprise access control solutions. I can emphatically state that in every one of these instances, the customer has been absolutely fully aware of the performance metrics of the technology they are deploying, from accuracy through to HW requirements. In fact, before securing any contract, it is very common for the supplier to have to benchmark their technology on customer supplied data, and often the adherence to pre-defined accuracy SLAs is written into contract, with penalties for non-performance.

2. There is no single correct answer

Anybody versed in biometrics knows that the answer is almost always “It depends”. The accuracy is dependent upon multiple factors, many of which will be under the control of the customer, not just the supplier, such as:

      – Quality of the data being matched against
      – Representative population
        – Environmental conditions

     

          – Performance required
          – Budget

      Again, required levels of accuracy will often be pre-agreed with the client, and it is often down to a matter of how much budget the client has available. Faster and / or more accurate will require more computing power, and the determination is often down to a cost benefit analysis.

      3. It is Competitive Confidential Information

      Accuracy of biometric technology can pose a strong competitive advantage, and suppliers often don’t want this information to be in the public domain (or more specifically, available to their competitors). Though the release of this information is often required, for example to prospective clients, it will almost always be under a non-disclosure agreement.

      4. There is no Commercial Reason to do so

      Suppliers, like anybody, don’t like having their time wasted. They’ll apply their resources to those who wish to engage with them seriously, and as mentioned above, they will have no problem in releasing the information as required. A car salesman will spend his or her resources on the individual who wants to buy a car, and ignore the tire kickers.

      My Point

      To only ever argue the facts on one side of a debate to follow a predefined agenda generally results in a loss of credibility. The irony is that people who do so often have valid concerns or issues that quite rightly should be aired and considered, but end up falling by the wayside.

      These are my own personal opinions, and not necessarily the opinions of any suppliers I may happen to work with.


      Is the Head of UK Border Force being made a Scapegoat?

      Pressure mounts on Theresa May, and Brodie Clark steps down, citing constructive dismissal.

      Clark states “Despite pressure to reduce queues, including from ministers, I can never be accused of compromising security for convenience.”

      Did Brodie Clark exceed his authority?, or is he being made as a scapegoat?, or is this simply a breakdown of internal communication, with both sides believing they are right?

      The story continues…


      UK BA Suspensions

      air travelThe news last week that Brodie Clarke and Graeme Kyle were suspended from the UK Borders Agency following claims that identity checks were relaxed during busy periods at Heathrow raises some interesting questions.

      Without passing any judgement, I understand in part both why there may have been pressure to do so,  and the government’s decision to undertake suspensions.  The latter is easier to address. Whatever concerns may have existed, freedom to exercise authority cannot fly in the face of direct ministerial guidance.

       

      Having said that, I’m sure the reasons for doing so were well intentioned, and may have resulted from trying to meet conflicting requirements, mainly ensuring:

      • High security and appropriate passenger screening.
      • Passenger throughput and avoidance of queues / delays.

      While I’m not close to the environment in question, at initial glance it appears that the former requirement may have been sacrificed to an extent to ensure the latter during busy periods.

      Delays and queues, in a very real and commercial sense, cost money, and it is easy to quantify exactly how much. So it appears the dilemma faced was the age old one, being : “What is an acceptable cost for increased security?”

      It appears that some at least felt the benefit delivered did not warrant the disruption to existing processes. Unfortunately it also appears that the decision was taken without due process and consultation.

      This situation  highlights the importance of understanding the overall cost of any new security system (which invariably is significantly higher than the cost of procuring it), and the benefits it delivers. Invariably, any system will have an impact on existing workflows, and if carefully designed, should deliver an improvement in workflow in addition to an increase in security.