DARPA Media forensics to detect fake photos and videos on social media to support National Security

In recent years consumer imaging technology (digital cameras, mobile phones, etc.) has become ubiquitous, allowing people the world over to take and share images and video instantaneously. Mirroring this rise in digital imagery is the associated ability for even relatively unskilled users to manipulate and distort the message of the visual media. While many manipulations are benign, performed for fun or for artistic value, others are for adversarial purposes, such as propaganda or misinformation campaigns.

 

The most infamous form of this kind of content is the category called “deepfakes” — usually pornographic video that superimposes a celebrity or public figure’s likeness into a compromising scene. Though software that makes that makes deepfakes possible is inexpensive and easy to use, existing video analysis tools aren’t yet up to the task of identifying what’s real and what’s been cooked up.

 

The problem isn’t limited to the fashion and cosmetics industries where photos are “touched-up” and “augmented” to make models look better and the results of skin-care products look (instantly) appealing — it’s spread to politics and now even business. Manipulated videos and images that may be manually indistinguishable from the real thing present a series of real-world problems, including election and evidence tampering, blackmail, general propaganda and targeted social media misinformation efforts.

 

This manipulation of visual media is enabled by the wide scale availability of sophisticated image and video editing applications that permit editing in ways that are very difficult to detect either visually or with current image analysis and visual media forensics tools. DARPA’s four-year, $4.4 million Media Forensics, or MediFor, initiative will underwrite research to, for example, develop better algorithms that can be used to spot fake images. The tools would then allow analysts to conduct forensic investigations to determine precisely how and why images were manipulated.

 

Tools already exist to scan Internet images, but not on the scale required by U.S. intelligence agencies. The forensic tools used today lack robustness and scalability, and address only some aspects of media authentication; an end-to-end platform to perform a complete and automated forensic analysis does not exist.

 

“A key aspect of this project is its focus on gleaning useful information from massive troves of data by means of data-driven techniques instead of just developing small laboratory solutions for a handful of cases,” Walter Scheirer, a principal investigator at Notre Dame, noted in a statement.

 

Researchers noted that such a capability would require specialized machine-learning platforms designed to automatically perform processes needed to verify the authenticity of millions of videos and images.

 

“You would like to be able to have a system that will take the images, perform a series of tests to see whether they are authentic and then produce a result,” explained Edward Delp, director of Purdue’s Video and Image Processing Laboratory. “Right now you have little pieces that perform different aspects of this task, but plugging them all together and integrating them into a single system is a real problem.”

 

In rolling out the program , program officials said MediFor would attempt to integrate machine learning and image analysis technologies into a forensic-based platform to “detect manipulations, provide analysts and decision makers with detailed information about the types of manipulations performed, how they were performed… in order to facilitate decisions regarding the intelligence value of the image [and] video.”

 

DARPA’s MediFor program brings together world-class researchers to attempt to level the digital imagery playing field, which currently favors the manipulator, by developing technologies for the automated assessment of the integrity of an image or video and integrating these in an end-to-end media forensics platform.

 

If successful, the MediFor platform will automatically detect manipulations, provide detailed information about how these manipulations were performed, and reason about the overall integrity of visual media to facilitate decisions regarding the use of any questionable image or video.

 

DARPA hopes the fruits of the MediFor program will be distributed far and wide, picked up by tech companies such as Facebook and YouTube, which handle a big fraction of the world’s user-generated videos.