Modernizing the Security Clearance Process through Machine Learning and AI

Late last month, President Donald Trump signed an Executive Order transferring responsibility for security clearance screening from the Office of Management and Budget to the Defense Department. 

The Administration has called the clearance process a target for government reform, noting in 2018 that “background investigations are critical to enabling national security missions and ensuring public trust in the workforce across the Government.”

The Administration’s efforts are part of an ongoing focus on reforming the clearance process, and reducing the existing backlog. 

That backlog peaked at 725,000 open investigations in 2018, with some Americans waiting more than 500 days just to start their first day at work.  As part of these efforts in 2018, the Federal Government hired 2,500 additional investigators to address the backlog.

In addition to the Executive Order, in February, Senator Mark Warner (D-VA), reintroduced The Modernizing the Trusted Workforce for the 21st Century Act (S.314), which calls for a major overhaul of the system and sets targets to reduce the backlog to 200,000 by the end of 2020, and shorten the time required to issue a secret level clearance to 30 days or fewer and top secret level clearance to 90 days. 

The legislation establishes the “clearance in person” or “one-clearance” concept, which enables clearances to follow employees who change agencies, within two weeks or fewer.

Continuous evaluation is another key component, moving from the existing periodic reviews, to reviewing on a dynamic and ongoing basis.

Achieving these goals requires a complete re-thinking of the security clearance process.  In his legislation, Senator Warner rightly notes the critical role that technologies will play in preventing, detecting and monitoring threats; as well as the role data integration and analytics can play in expediting or focusing reinvestigations through delta reporting and continuous evaluation. 

As many security experts have pointed out, the current system is not only time consuming and slow, it is also out of sync with how people live in the 21st century.  As it currently works, a field investigator is assigned to confirm information from the applicant’s form, and to make sure that individual does not represent a threat to national security.

These determinations are based on the 13 adjudicative guideline criteria, which include, among others financial considerations, foreign preference and influence, alcohol consumption, drug involvement.

To be sure, 50 years ago, interviews with neighbors, colleagues and other associates could help provide meaningful insights into our lives and habits.  But today, we share these very same insights publicly, willingly and knowingly across a variety of online platforms, making the Internet a useful, but largely untapped resource.

In fact, according to Gary Reid, Director of Defense Intelligence patterns of life, including scans of public-facing social media could one day be considered.

A significant challenge is the volume of data on the web. 

With more than 2.5 quintillion bytes of data created on the Internet every day, searching for relevant content finding relevant content can be like looking for the proverbial needle in a haystack.

One way to solve for this is through machine learning and AI capabilities – a super-charged web search, allowing for all that publicly available, open-source data to be searched for risk behaviors – in this case, associated with those 13 established adjudicative guidelines.

But rather than having to weed through thousands of pages of search results, these technologies can quickly synthesize the data and cull out high priority risks associated with guideline selectors. 

This provides analysts with the most critical data first, helping streamline their search process and gather the most relevant information.

Call it the Radiance solution. 

Lumina’s AI-powered Radiance technology is specifically designed to overcome the challenges of massive unstructured data ingestion, evaluation, and prioritization. This provides a rapidly deployable, scalable and user-friendly solution for the security clearance process. 

The technology is comprised of three modules, for edge-to-edge risk detection.

Radiance Open Source Intelligence (OS-INT)

OS-INT is a deep-web listening tool that uses machine learning and artificial intelligence to assess and prioritize risk.  OS-INT scours publicly available data across the entire Internet, correlating names entered into the system with content related to its exclusive BRPs, and cross-referenced with more than one million queries into Lumina’s proprietary databases of risk.  Unlike social media monitoring, OS-INT is not reliant on a single platform or social media API, allowing for continuous ingestion of all open source data.

OS-INT’s security clearance bundle includes more than 16,220 terms related to the SEADs. OS-INT performs nearly 325,000 searches across the entire web, correlating names with associated risk behaviors. Similar results would take an individual running a manual web query more than 18 years to read and analyze.

OS-INT completes searches in an average of 4-5 minutes, providing prioritized, high resolution, and actionable results. The system allows for continuous monitoring and evaluation, mapping previous results against results from more recent queries.

The configuration of BRPs ensures that collection of such information adheres to SEAD 5 guidelines by only collecting publicly available information, within the scope of the investigation and does not use account creation or digital interaction with POIs.

Radiance Internet Intelligence’s (NET-INT) 

NET-INT’s proprietary algorithms continuously identify, monitor, capture, and prioritize IP addresses exhibiting anomalous behavior across multiple risk dimensions.  Its massive system of data ingestion has the capability to catalogue, index and redeploy Internet content related to risk dimensions associated with SEADs.

The system captures an IP addresses’ pattern of life data, prioritizing anomalous behavior. NET-INT also screens IP addresses associated with an entity or person of interest against all IP addresses displaying anomalous behavior collected over the system’s lifespan. 

NET-INT’s continuous monitoring of a POI’s Internet research behavior helps predict emergent behavior indicative of a violation of the guidelines.

Radiance Human Intelligence (HUM-INT)

HUM-INT is powered by the S4 app, a crowd-sourced, mobile application that allows users to confidentially report concerns in real time. S4 app can be configured as a workplace tool, allowing employees to submit information related to potential risk behaviors exhibited by co-workers. A centralized management portal allows clients to access real-time threats to geo-fenced facility locations.

As Washington continues its efforts to reduce the security backlog, and modernize the existing process, machine learning and artificial intelligence will play an important role.

As Senator Warner recently said, “There is much more we can do to reform decades-old policies and processes to reflect today’s threat environment, adapt to the dynamic of a modern mobile workforce, and capitalize on opportunities offered by modern information technology.”