International Defense Security & Technology Your trusted Source for News, Research and Analysis
Intelligence, in military science, information concerning an enemy or an area. The term is also used for an agency that gathers such information. Military intelligence is as old as warfare itself. Even in biblical times, Moses sent spies to live with the Canaanites in order to learn about their ways and about their strengths and weaknesses. In the American Revolution George Washington relied heavily on information that was provided by an intelligence net based in New York City, and in World War II the results of a lack of good intelligence were realized in the destruction of the U.S. Pacific fleet at Pearl Harbor.
Indeed, technological innovations have always been central to the intelligence profession. The technological breakthroughs of recent years have led intelligence organizations to challenge the accepted truths that have historically shaped their endeavors. The hierarchical, compartmentalized, industrial structure of these organizations is now changing, revolving primarily around the integration of new technologies with traditional intelligence work and the redefinition of the role of humans in the intelligence process.
During the 1980s, the military and intelligence services began to shift some of their information-gathering activities away from covert activities like trying to read an adversary’s mail or tapping their phones to discover hidden secrets. Instead, effort was put into looking for useful intelligence that was freely available or even officially published.
Open-Source Intelligence (OSINT) – a concept created by the intelligence community to describe information that is unclassified and accessible to the general public. Traditionally, this kind of information was inferior compared to classified information; and as a result, the investments in OSINT technologies were substantially lower compared to other types of technologies and sources. This is changing now; agencies are now realizing that OSINT is easy to acquire and more beneficial, compared to other – more challenging – types of information.
With the advent of the internet, vastly more information became publicly available and OSINT became increasingly useful not just to sophisticated government agencies and law enforcement, but to financial crime analysts, fraud and brand misuse investigations and particularly cybersecurity.
OSINT sources can include: Newspaper and magazine articles, as well as media reports; Academic papers and published research; Books and other reference materials; Social media activity; Census data; Telephone directories; Court filings; Arrest records; Public trading data; Public surveys; Location context data; Breach or compromise disclosure information; Publicly shared cyberattack indicators like IP addresses, domain or file hashes; Certificate or Domain registration data; Application or system vulnerability data.
While most open source data is accessed via the open internet and may be indexed with the help of a search engine like Google, it can also be accessed via more closed forums that are not indexed by search engines.
But OSINT can also be conducted on the deep or dark web. The deep web is a layer below the surface web that requires login or subscription services. These sites can include academic journals, court record databases or even services like Netflix. OSINT can still be applied even to sites requiring login or subscription — as long as analysts can access the information legally, without hacking.
And, that extends to the dark web. While the surface and deep web can be accessed by any common browser, the dark web requires specific software, like Tor (The Onion Router). Once inside, there’s lots of information that can be beneficial to threat intelligence gathering and other investigations.
Traditionally, OSINT was a technique used by the national security and law enforcement communities. However, in recent years it has also become a foundational capability within cybersecurity.
OSINT is a valuable technique for OPSEC, but it can also be used to gather threat intelligence to proactively reduce cyber risks. Cybersecurity teams frequently use OSINT for OPSEC (operational security) by understanding what of their company’s information is publicly available. This information may be on assets they control that are designed to be public-facing or become so through error, or on assets outside the company perimeter, like social media or third-party websites that may accidentally leak information. OSINT is used to analyze, monitor and track cyberthreats from targeted or indiscriminate attacks against an organization by malware and bad actors.
It is also important to note that there is often a tremendous amount of secondary data that can be leveraged from each open source of information. For example, social media accounts can be mined for personal information, such as a user’s name, birthdate, family members and place of residence. However, the file metadata from specific posts can also reveal additional information such as where the post was made, the device used to create the file and the author of the file.
One of the most common reasons cybercriminals leverage OSINT is for social engineering purposes. They will often gather personal information of potential victims via social media profiles or other online activity to create a profile of the individual that can then be used to customize phishing attacks.
Additionally, the analyst can provide more detailed information regarding the breached information including who may be impacted at their organization along with how the breach occurred for more amplifying information.
OSINT operations, whether practiced by IT security pros, malicious hackers, or state-sanctioned intelligence operatives, use advanced techniques to search through the vast haystack of visible data to find the needles they’re looking for to achieve their goals—and learn information that many don’t realize is public.
Today, nations have at their disposal information collection and processing systems that permit gathering and producing intelligence more rapidly and more accurately than ever before. Satellites, ultramodern aircraft, electronic systems, human sources, cameras, imaging and electronic devices, and a host of other systems permit the amassing of information on a scale that was unheard of in the past.
Traditionally when it came to processing, analyzing, interpreting, and acting on intelligence, however, human ability – with all its limitations – has always been considered unquestionably superior. From 1995 to 2016, the amount of reading required of an average US intelligence researcher, covering a low-priority country, grew from 20,000 to 200,000 words per day. And that is just the beginning. According to forecasts, the volume of digital data that humanity will produce in 2025 will be ten times greater than is produced today. Some argue this volume can only be processed – and even analyzed – by computers.
The national security establishment already recognizes that the private sector and academia are the main drivers of technological innovation. In the United States there is dynamic cooperation between these bodies and the security community, including venture capital funds jointly owned by the government and private companies.
Take In-Q-Tel – a venture capital fund established 20 years ago to identify and invest in companies that develop innovative technology which serves the national security of the United States, thus positioning the American intelligence community at the forefront of technological development. The fund is an independent corporation, which is not subordinate to any government agency, but it maintains constant coordination with the CIA, and the US government is the main investor.
It’s most successful endeavor, which has grown to become a multi-billion company though somewhat controversial, is Palantir, a data-integration and knowledge management provider. But there are copious other startups and more established companies, ranging from sophisticated chemical detection (e.g. 908devices), automated language translations (e.g. Lilt), and digital imagery (e.g. Immersive Wisdom) to sensor technology (e.g. Echodyne), predictive analytics (e.g. Tamr) and cyber security (e.g. Interset).
Actually, a significant part of intelligence work is already being done by such companies, small and big. Companies like Hexagon, Nice, Splunk, Cisco and NEC offer intelligence and law enforcement agencies a full suite of platforms and services, including various analytical solutions such as video analytics, identity analytics, and social media analytics . These platforms help agencies to obtain insights and make predictions from the collected and historic data, by using real-time data stream analytics and machine learning. A one-stop-intelligence-shop if you will.
Another example of government and non-government collaboration is the Intelligence Advanced Research Projects Activity (IARPA) – a nonprofit organization which reports to the Director of National Intelligence (DNI). Established in 2006, IARPA finances advanced research relevant to the American intelligence community, with a focus on cooperation between academic institutions and the private sector, in a broad range of technological and social sciences fields. With a relatively small annual operational budget of around $3bn, the fund gives priority to multi-year development projects that meet the concrete needs of the intelligence community. The majority of the studies supported by the fund are unclassified and open to public scrutiny, at least until the stage of implementation by intelligence agencies.
Open-source intelligence in the 2022 Russian invasion of Ukraine
A war has perhaps never been covered in such details as seen in Ukraine, with the creation of a new class of open-source intelligence (OSINT) analysts, harvesting and examining content posted on social media for intelligence into orders of battle, equipment and personnel losses, and more.
In the early hours of 24 February, just before the start of the invasion, OSINT researchers at the Middlebury Institute of International Studies at Monterey used Google Maps to track a significantly large traffic jam on a road in Russia leading to the Ukrainian border. Jeffrey Lewis subsequently tweeted “someone’s on the move.” An hour later, Russian troops began the invasion.
Netherlands-based investigative journalism group Bellingcat has published interactive maps of destroyed civilian targets and has worked on authenticating potential documentation of war crimes. In July 2022, Bellingcat was banned as an undesirable organisation by the Russian government, with the Prosecutor-General of Russia saying that it posed “a threat to the security of the Russian Federation.”
The Free Buryatia Foundation, which was founded in opposition to the invasion, has used open-source intelligence to try and track the number of Buryats killed in action in Ukraine. As of April 2022, the Foundation has estimated that around 2,8% of Russian casualties were Buryat, one of the highest death tolls among the Russian federal republics. OSINT groups have also used tools such as facial recognition apps to try and identify perpetrators of war crimes, such as the Bucha massacre
Many of these OSINT specialists are civilians and gaining huge followings on social media, providing effective analysis to an audience keen to get behind the mainstream headlines and soundbites.
The sharing of open-source intelligence on social media has raised ethical concerns, including over the sharing of graphic images of bodies and of potentially military-sensitive data. Matthew Ford of the University of Sussex has noted that “Ukrainians fear such images will reveal their tactics, techniques, and procedures,” and that Ukrainians have therefore undertaken a degree of self-censorship. Concerns have also been raised about the potential dissemination of misinformation, such as through fake accounts posing as insider sources
References and Resources also include:
https://en.wikipedia.org/wiki/Open-source_intelligence_in_the_2022_Russian_invasion_of_Ukraine
