Understanding the power of OSINT

Sir Mark Rowley QPM details the advantages that OSINT (open source intelligence) can bring and calls for its increased use across the security sector.

OSINT (open source intelligence) can be a powerful intelligence and investigative tool but is too often overlooked. In many organisations there are significant barriers to the adoption of effective OSINT, as well as a failure to adapt fast enough to emerging technologies. A cultural shift is needed in order to elevate the status of OSINT and ensure that it is used to its full potential.

The case for OSINT

OSINT is a critical component of modern intelligence and investigative tools. The volume of data available online is constantly growing, providing investigators with a rich information source to draw from. The insights that OSINT can offer are unlikely to be found in internal datasets, curated databases, or sanctions lists. There are many powerful examples where OSINT was instrumental in the solving of a case: Bellingcat’s insights on the downed flight MH17 in 2014 relied exclusively on OSINT.

OSINT should also be considered an essential element of counter-terrorism and counter-misinformation programmes. The mapping of terrorist networks on social media – especially the more grassroots right-wing extremist groups that are now popping up on platforms like Parler – is a highly effective means of identifying those behind these crimes. In 2015, our teams convicted Imran Khawaja, who received 12 years for preparing for acts of terrorism, attending a training camp and possessing firearms. OSINT provided much of the evidence.

It is hard not to conclude that open source investigations are of growing strategic significance. Furthermore, they can save money as a rapid and economic way to understand an offender early in an investigation before deploying more expensive and intrusive tactics. Why then are so many organisations still failing to take advantage of OSINT?

What are the barriers to adoption?

Misconceptions

The reasons for lack of investment in OSINT are often based on a misunderstanding of what exactly it entails. ‘Open source intelligence’ can conjure an image of invasions of privacy. However, the type of OSINT whose adoption I am arguing for can be better described as online open source investigation: making use of freely available online information in a targeted and non-invasive way.

Cultural and technological barriers

Culture and technology deficit are also factors in this attitude towards OSINT. Many organisations wrestle with outdated technology architecture and spend most of their efforts focusing on how better to curate internal data. However, increasingly, the insights OSINT provides into individuals and companies will be greater than those found internally.

Where organisations are realising the importance of open source data, they are often only using it in the form of curated datasets which don’t capture all of the rich, valuable information available on the internet. For example, a well-known curated dataset offers 6 petabytes of data. The entire internet is thought to have over 1200 petabytes (as of 2020). By relying solely on this database, investigators could be missing out on 99% of available data.

Lengthy and bureaucratic processes

Whilst there is clearly a need for thorough and fair procurement processes in every organisation, their complexity and length can also stifle innovation. This was evident in my own experiences: I was most able to deliver cutting-edge technological change at speed when there was an especially urgent requirement. In early 2012 I joined the Metropolitan Police, tasked with dealing with the aftermath of the 2011 riots, where rioters had organised themselves on social media. The forthcoming Olympics meant that there was an urgent requirement for capability to counter this sort of risk. In this case, the bookends of the 2011 riots and 2012 Olympics created a unique forcing function that facilitated operational clarity and the circumvention of normal procurement rules. I was able to set up the UK police’s first serious OSINT team in just a few months.

Yet after this success, the lack of obvious urgency around OSINT capability meant that progress continued to be slow. Years later, I found myself outside New Scotland Yard announcing to the world that Sergei and Julia Skripal had been subjected to a nerve agent attack in Salisbury. Subsequently, Bellingcat identified the two Russian agents responsible using advanced open source investigative techniques.

Increasing flexibility and the role of technology

To facilitate increased investment in OSINT, systemic, strategic and technological change is needed.

Firstly, organisations need to shift towards more flexible procurement methods that reflect the reality that many high-quality solutions are found in early-stage companies. These companies often find that they are accidentally designed out of the complex processes in large institutions.

Secondly, there is a need for a new strategic approach to investigations. Organisations need to allocate a proportion of technology investment and training budgets towards equipping investigators with cutting-edge OSINT tools.

Thirdly, sophisticated technologies that are designed to professionalise the OSINT investigation should be supported. Technology plays a vital part in reducing operational difficulties in using OSINT by increasing:

• Security: gathering online data risks revealing the investigator’s identity, undermining operations
• Speed: data can overwhelm without technology that helps you get to the relevant information quickly
• Insight: finding connections and presenting data from disparate sources.
• Connectivity to other data: OSINT will always be one part of a wider strategy. The ability to combine data from different sources, both structured and unstructured, is essential.

Today there is an exciting portfolio of companies I work with in this field. Blackdot Solutions provides some of the best software to assist open-source investigators; Deloitte are helping big organisations transform their investigations through use of social media; and Quest is a specialist security and investigations company which has set up a ‘threat matrix’ with Signify to tackle the racist abuse of leading sports men and women – especially in football.

Conclusion

There are numerous advantages to including OSINT in an investigation strategy. Tools such as Blackdot’s Videris platform are available to help investigators use open source information quickly, securely and effectively. However, without a strategic drive to ensure the open source tools are part of a deliberate mix of capabilities in the investigator’s toolbox, many organisations will find that cultural, technical and commercial barriers leave this part of their armoury underpowered.

About Sir Mark Rowley QPM

Sir Mark Rowley was one of the most senior police figures in the UK with 31 years of service. He led UK Counter Terrorism Policing, 2014-2018. Previously, he held positions as Assistant Commissioner at the London Metropolitan Police and Chief Constable of Surrey Police.