Governments are implementing the most severe budget cuts of recent times. Against this backdrop, threats from terrorism, organised crime and public disorder continue to rise. Yet recent statistics in the UK demonstrate that authorities can remain resilient and still ensure law-and-order. The targeted application of technology can further increase resilience and the readiness to respond to major events. The relentless advance in the accuracy of face recognition technology, increase in the availability of digital media and mass availability of cheap computing power now provide unique opportunities to meet challenging budgets by drastically enhancing the operational efficiency of forensic investigators while even further enhancing public safety. Digital media can be bulk-ingested in an automated fashion to be processed in a cloud computing environment to identify and extract potential actionable intelligence. Processing is continuous, consistent and predictable. Multiple identification technologies can be deployed and the most suitable algorithms integrated to meeting evolving requirements. Analysts can now focus on investigating and confirming suggested results rather than having to manually watch countless hours of media in the hope of stumbling across the required information. Expanding beyond traditional sources of media is increasingly being accomplished by engaging the public and crowd-sourcing intelligence in response to incidents.
Having previously written on the subject of the application of face recognition in airports[i] and privacy concerns of face recognition when used by retail[ii], this article focusses on the application of face recognition to support bulk processing of media by what has traditionally been the first and thus far most proliferate user of biometric technologies: law enforcement. The convergence of multiple advancements now provides a whole new set of opportunities to use identification technologies in manners that provide benefits that are only now being realised.
1.A Need for Enhanced Safety and Operational Efficiency
Austerity is the order of the day and public budgets are being slashed. Against this backdrop, security risks are continuously increasing. The threat from terrorism, organised crime and public disorder is not abating. Indeed, as reported by the BBC News on the 17th July 2013[iii], the threat landscape is “substantial” and becoming ever more fragmented, consisting of a greater number of smaller and less sophisticated plots.
However, the UK’s police forces have demonstrated that it is possible to maintain and even improve upon public safety despite the relentless pressure of austerity. Recent reports indicate that crime in the UK is at an historic low, being at its lowest level since 1981 [iv]. As always, it appears that necessity is the mother of invention and it is likely that technology is playing an innovative role in improving police efficiency.
What is not apparent from these recent reports, however, is the current level of readiness to respond to a major event. Indeed, the UK’s Police Federation, the body representing rank and file police officers, warns that the police “could not handle more riots”[v] after the budget cuts and Her Majesty’s Inspectorate of Constabulary (HMIC) warns that neighbourhood policing risks being “eroded”. [vi]
2. A Relentless Increase in Digital Media
The increase in the creation of digital media is relentless. Law enforcement and intelligence agencies have amassed large collections of biographical, video and photographic information from multiple sources such as:
- Computer hard drives.
- Mobile phones and portable cameras.
- Flash memory devices.
- Online sources on the Internet such as Facebook and YouTube.
Additionally, when tragic events or social disorder occur, investigators have a long and arduous task of reviewing countless hours of CCTV footage, generally with a varying degree of concentration and scrutiny.
A solution that minimises manual effort in the extraction of actionable intelligence from amassed media by automating this process with a consistent and repeatable level of scrutiny will deliver concise and consistent information in a fraction of the time taken by operators undertaking the task completely manually.
3. An Automated Media Processing and Exploitation Solution
Police, intelligence and other public order agencies would benefit from the application of a powerful media processing solution designed to process, ingest, analyse and index in an automated fashion very large quantities of photographs and videos to transform them into usable assets.
Such an automated solution ingests and processes media from multiple sources. Once processed, law enforcement agencies can analyse and make use of the extracted assets and manage them in a centralised repository of information. Data links, associations and metadata inferences can be managed across the whole dataset by multiple users from a single common user interface. Backend processing services are run in a cloud-computing environment, the capacity of which can be configured and incrementally scaled up and down to meet an organisation’s changing demands; peaks arising from specific events can be easily accommodated.
- Automatically find, extract and index faces to enable biometric and biographic searching of media.
- Create and manage watchlists of people of interest via a web-based interface.
- Find and cross-reference all media instances in which a person of interest has been seen.
- Identify, locate, and track persons of interest, their associates and their activities across all media.
- Discover, document and view links between people of interest, their activities and networks.
- Use of metadata (including geo data) in the media to enhance investigations and association of data.
- Integration into existing system environments, databases and components via a flexible API.
3.1 Incorporating Other Detection Capabilities
In addition to face detection and recognition, other detection engines can be incorporated, such as:
- Automatic Number Plate Recognition. (ANPR)
- Voice Biometrics.
- Object / Logo Recognition.
(Other identifying features can be used to track individuals through other processed media.)
- Scene Recognition
(Identify similarities in the entire frame, often used in child-exploitation investigations)
3.2 Biographic Filtering and Fuzzy Match Capability
Forensic investigations are complex and require a holistic view of all available data. This involves not only analysing media, but making full use of all textual and biographic data available as well. This can include text from files recovered from hard drives and other storage devices, online sources, metadata associated with photo or video files and data entered by investigators during the investigation.
Traditional Boolean search techniques only work within a black and white, true and false paradigm. More applicable within a complicated forensic analysis are techniques that use advanced “fuzzy” algorithms that to calculate similarities and aggregate match scores using multiple criteria to enable a “shades of grey” analysis.
Such an approach can fuse match scores across multiple disparate search criteria and even allows for fusion and aggregation of search results across multiple biometric and biographic criteria.
3.3 Working with Geo-Location Data
An ever-increasing amount of media available to investigators is captured on mobile devices and cameras affixed with location determining technology. This includes media obtained from CCTV, confiscated hardware and devices, online sources and voluntarily made available by members of the public. The majority of the time, this geo-location data is incorporated into the media metadata, thereby providing significant potential to further enhance the analysis of the media. For example, geo-location can be used to:
- Compartmentalise and refine analysis by location of where the media was created.
- Overlay location of proposed matches onto maps.
- Chart movements of individuals of interest by location and time of sightings.
- Link individuals at the same location and time even if they do not appear together in media
3.4 Architecture and Integration with Existing Systems
There are significant similarities in organisation and methods of operation in many western law enforcement agencies facilitating increased levels of co-operation. Operational systems should support full control of information and data as well as have sufficient in-built flexibility to enable authorised data exchanges.
In addition to utilising COTS components, adhering to common standards and being cloud-architected to enable massive scalability, a well delineated scope of functionality and open API enables:
- Flexibility in customisation and integration with existing systems and workflows.
- Well-defined mechanisms of loading data and automating ingestion of media for processing.
- Dynamic alteration and sharing of watchlists, media, system-generated results and operator analysis.
3.5 Hosting, Cloud and Virtualisation Options
Full architectural flexibility enables flexibility of hosting options. Organisations can elect to:
- Take advantage of IaaS and SaaS options on cloud offerings.
(UK accreditation of IL0 to IL3 is available via hosting partners)
- Fully host the solution on their own private and secure premises and datacentres.
- Deploy in a hybrid manner.
(Thereby taking advantage of external processing power whilst retaining the most secret data)
3.6 Working Hand-in-Glove with Trained Forensic Investigators
The human operator will always remain the critical and essential part of intelligence analysis; media analysis solutions are not designed to replace the intricate skills and knowledge of trained investigators. Rather, the operator is enabled to intelligently direct and apply their extensive training at suggested results, eliminating the necessity of rote viewing of countless hours of media either in a sequential our random fashion.
4. Potential Use Cases
There are myriad different applications of a solution architecture as described herein within military, law enforcement, intelligence and public site security agencies. These are summarised into three broad categories:
4.1 Time Critical Investigations, Media of Critical Importance
In certain major incidents, timeliness of response is of the essence. Authorities need to quickly process evidence to identify and apprehend individuals. The scale of the investigation is often huge and the amount of media that needs to be processed massive. Examples include terrorist events such as the recent Boston bombing and the Woolwich attack in South London.
Often, the media acquired in these instances is of such critical importance that the authorities may choose to review it all in its entirety, frame-by-frame. However, in the early stages after the incident, decisive and immediate action is critical. Rather than having to sift through the media in a random or sequential fashion, a media analysis solution can quickly direct the investigators to the portions of the media that are most likely to deliver immediate results. Full review of the media can be conducted during subsequent stages of the investigation.
4.2 Bulk Ingestion of Media Arising from Criminal Investigations
During routine operations or specific criminal investigations, authorities may recover significant quantities of media on confiscated hard drives, mobile phones, flash / thumb drives and other sources that need to be processed to either further the investigation or to assist in building an evidence base for criminal prosecution.
- Military or counter-terror officers raiding terrorist training facilities.
- Specialist organised crime investigators raiding the offices of organised crime syndicates.
- Child protection officers raiding premises of individuals or organisations involved in child exploitation.
This media can be bulk ingested in an automated fashion to provide the investigating officers an overall summary of the contents including focus areas for further investigation.
4.3 Continuous Background Processing of Media Sources
Authorities may as a matter of routine have access to masses of media which may contain actionable intelligence, but typically would never be viewed or processed due to a lack of resource and the time consuming nature doing so. Examples include:
- Media from specific cameras installed at high-profile or sensitive locations.
- Media from known or suspect online sources or accounts from social media sites.
- Media made available to the authorities by the general public.
Intelligence in these sources may be missed entirely and never acted upon.
This media can now be bulk ingested and processed in an entirely automated fashion to flag any relevant intelligence, using operator controlled criteria, to the authorities as required for follow-up processing.
5. A Compelling Business Case
The solution and optional IaaS / SaaS components can be made available on a monthly service-charge basis, thereby requiring a minimal capital outlay and enabling a compelling operating expenditure business model.
Whilst the human operator is an essential part of intelligence analysis, an entry level solution empowers the analyst to process up to an order of magnitude more media on a daily basis. This enables trained operators to apply their expertise and training by focussing on the analysis of results generated by the solution in a more focused effort than manually watching hour upon hour of media.
6. Engaging the Public to Crowd-Source Media to aid Investigations
Increasingly, especially from crowded public events, authorities are making greater use of media captured intelligence in the form of photographs and videos that have been recorded by members of the public.
Most members of society would welcome the opportunity to assist the authorities with their investigations, but often do not know how or are fearful of being involved.
Allevate’s proposed PublicEye service is aimed at empowering the public to take a greater collective social responsibility and assist law enforcement in much the same manner as the phenomenally successful CrimeWatch. It enables members of the public to (at their discretion) upload media directly from their mobile phone or other internet device to a public portal for processing and dissemination to the relevant authorities.
A PublicEye portal could be used:
- In response to appeals by the police to the public who were present at an event or disturbance.
- When individuals witness a crime being committed.
- Upon suspected sightings of missing persons or individuals wanted by the authorities.
Security concerns are ever increasing. However, public budgets are being slashed. Law enforcement agencies are rising to the challenge of implementing budget cuts partly through the focussed application of technology. The accuracy of face recognition has increased dramatically over the past 10 years. This, coupled with the massive increase in the creation of digital media and the availability of cheap computing, now provides authorities with the ability to bulk ingest and process media in an automated fashion. Results are continuous and predictable. Trained analysts can now focus their skills on investigating suggested results and on intelligence extracted by automated systems. Not only does this provide the ability to process critical media even faster than ever before to respond time critical investigations, but it also enables authorities to extract intelligence from media sources that in the past may never even have been looked at because of the significant resource this previously would have entailed.
Additionally, the availability of smartphones means almost everybody is carrying in their pocket a high quality camera. The ability to process media rapidly and cheaply means the authorities will be able to, on a continuously increasing basis, engage with members of the public to crowd-source media in response to major investigations.
8. About the Author
Carl is the founder of Allevate Limited (http://allevate.com), an organisation that works with law-enforcement, intelligence and government agencies to enhance public safety by ensuring positive identification through the application of biometric and identification technology. With over 20 years’ experience working in the hi-technology and software industry globally, he has significant experience with identification and public safety technologies including databases, PKI and smartcards, and has spent the past 10 years enabling the deployment of biometric technologies to infrastructure projects. Carl started working with biometrics whilst employed by NEC in the UK and Allevate subsequently supported NEC’s global and public safety business internationally.
Residing in the UK since 1993, Carl was born and raised in Canada and holds a Bachelor of Science Degree on Computer Science and Mathematics from the University of Toronto.
[i] Allevate, July 2012:
[iv] The Telegraph, 18th July 2013
[v] The Telegraph, 18th July 2013
[vii] In cooperation with Tygart Technology
[viii] The UK’s G-Cloud Programme is a cross government initiative led by Andy Nelson (Ministry of Justice) supported by Denise McDonagh (Home Office) under the direction of the Chief Information Officer Delivery Board as part of the Government ICT Strategy.