DARPA Ocean Of Things(OoT) to provide automatic detection of Russian and Chinese submarines and ships

Rajesh Uppal July 14, 2022 AI & IT, Navy Comments Off on DARPA Ocean Of Things(OoT) to provide automatic detection of Russian and Chinese submarines and ships 1,934 Views

The Internet of Things (IoT) is a system of interrelated computing devices, mechanical and digital machines, objects, animals or people that are provided with unique identifiers (UIDs) and the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction. By 2025, it is predicted that there can be as many as 100 billion connected IoT devices or network of everyday objects as well as sensors that will be infused with intelligence and computing capability.

The growing Internet of Things is mostly a land-based phenomenon, frequently in large cities with loads of sensors. It is heavily influenced by the future prospects of warfare in an urban environment and involves the use of sensors, munitions, vehicles, robots, human-wearable biometrics, and other smart technology that is relevant on the battlefield

But researchers at the Defense Advanced Research Projects Agency (DARPA) foresee a wide range of military and civil benefits from extending the Internet of Things out to sea. While satellites can provide some information, DARPA project manager John Waterson points out that there are gaps in their coverage – optical satellites cannot see through clouds, radar satellites only have limited coverage, and none of them can say much about what is going on underwater. DARPA launched OoT program which can be considered as one of the aplications of the Internet of Military Things (IoMT) in the maritime and underwater domain for the purposes of reconnaissance, surveillance, and other combat-related objectives.“Oceans cover more than 70 percent of the earth’s surface, but we know very little about them,” said Ersin Uzun, vice president and general manager of the Internet of Things team at Xerox.

Existing naval and commercial platforms are constrained to localized situational awareness from their organic sensors, and support by remote sensors is often restricted by the physical environment (e.g., fog, rain, cloud cover). Ocean of Things will address these gaps in ocean understanding to benefit all users of maritime data.The Ocean of Things program will provide persistent, wide-area sensor coverage across the maritime environment through the employment of thousands of intelligent floating platforms, the agency said. Along with machine learning and multi-sensor fusion techniques, the Ocean of Things would detect object and sensor motion used to track vessels, aircraft and even marine mammals.

The similar concept proved highly successful during the Cold War, when the U.S. Navy laid fixed networks of underwater hydrophones on the ocean floor called the “Sound Surveillance System” (SOSUS) to detect Soviet submarines transiting from their bases to patrol areas in the Atlantic and Pacific Oceans. Listening arrays placed in strategic chokepoints that those submarines would necessarily have to transit, like the waters between Greenland, Iceland, and Scotland — the so-called “GIUK Gap” — notionally let the United States know every time a Soviet submarine entered the North Atlantic, allowing the U.S. Navy to direct its own ships or submarines to track them. But since then submarines have become much stealthier and require advanced analytics and machine learning to detect them.

Floating sensors, known as floats, can gather far more detailed information, and can remain at sea for months at a time. There is a network of almost 4,000 Argo science floats around the world, gathering data on ocean temperature and salinity, that uses several thousand battery-powered, robotic floating devices to measure temperature, salinity and current for climate and oceanographic research. The floats mostly drift 10 days at a time below the ocean surface. After rising and transmitting their data to satellites, they return to depth to drift for another 10 days. The floats go as deep as 2,000 meters, according to the Argo website.

However, the Internet of Underwater Things (IoUT), is much larger world-wide network of smart interconnected underwater objects that would transmit data from existing and planned roaming, autonomous vehicles and underwater sensor networks to networks above surface in real time. Waterson wants to see much larger arrays of low-cost floats with more sensors, floats which would carry out missions lasting up to a year before scuttling themselves and degrading. The floats are environmentally friendly, avoiding the use of toxic materials. The Defense Advanced Research Agency’s “Ocean of Things” initiative would then transmit data collected by commercial sensors and crunched by new analytics tools via satellite to a cloud network. There it would be stored and prepped for real-time analysis, program officials said.

“The persistent coverage provided by dispersed sensors provides round-the-clock coverage that other sources of data like an MPA [maritime patrol aircraft] or a SAR [synthetic aperture radar] satellite, which cover a given area for a certain amount of time before moving on, cannot,” says Dr. Sidharth Kaushal, an expert on sea power at the U.K. defense think tank RUSI. Kaushal notes that as well as directly observing vessels and aircraft, the OoT will be able to measure variables like temperature, ocean salinity and ambient underwater noise, which are important for calibrating sonar during anti-submarine operations. The OoT will not be limited to any specific role; the variety of sensors, coupled with powerful data-processing techniques, mean it might be reconfigured to deal with emerging threats. For example, OoT might form a defensive picket to detect, track and locate incoming Russian Poseidon nuclear torpedoes so they could be intercepted. “This fits into a wider concept of Mosaic Warfare creating a system the components of which can reform and interact in multiple ways rather than relying on a hierarchical system,” says Kaushal.

DARPA is partnering with the National Oceanic and Atmospheric Administration, the National Weather Service, the University of Southern Mississippi, the University of Miami, multiple private companies and other entities. Waterston notes that he has brochures from Chinese and Spanish companies working on similar systems.

According to Waterston: “It’s so immense, covering 70% of the Earth’s surface, yet even with all of the ships, all of the aircraft, all of the satellites, and all of the existing sensors, we are severely undersampling this environment.”

China is also planning to build a massive underwater observation system across the disputed East and South China seas, that experts say could be used to detect the movement of foreign ships and diminish the stealth capabilities of US submarines. However China’s Blue Ocean sensors are tethered in place and appear to be mainly for radar and optical observation; there are believed to be hydrophone arrays on the sea bed. By contrast the OoT is much smaller and expendable, and could be deployed anywhere that the U.S. requires detailed, persistent observation of maritime activity, thousands of floating eyes to see over, on and under the sea.

DARPA’s Ocean of Things program

The agency announced its Ocean of Things program in 2017 , which seeks to enable persistent maritime situational awareness over large ocean areas by deploying thousands of small, low-cost floats that could form a distributed sensor network. While several programs over the past decade have measured the ocean environment on a large scale (>10 km), the Ocean of Things (OoT) program will provide measurements that are orders of magnitude finer (<10m). Potential applications include more accurate fish population estimation, oil spill monitoring, improved weather forecasting, and better tsunami prediction models.

The floats will carry sensors that will autonomously generate a large amount of heterogeneous dataset for real-time analysis and generation of high-resolution mission products. Each smart float would contain a suite of commercially available sensors to collect environmental data—such as ocean temperature, sea state, and location—as well as activity data about commercial vessels, aircraft, and even maritime mammals moving through the area. The floating platforms would include a “mission sensor” packed with a AIS receivers, hydrophone, camera, magnetometer and radio communications. Other components include a microphone, GPS, accelerometer and temperature gauges.

The data includes dynamic display of float locations, health, and mission performance; processing of environmental data for oceanographic and meteorological models; algorithms to automatically detect, track, and identify nearby vessels; and identification of new indicators of maritime activity. Along with high-resolution environmental data, the floating network could track passing vessels and predict their likely destination. The network could also provide early warning of unusual maritime activities.

Waterston says one of the most interesting missions for the sensor might be to simply determine whether GPS signals are available in an area of interest for military operations. “It doesn’t risk a plane or risk whatever platform you were going to use to go over there and say, ‘Oh, GPS is good. Go ahead and do whatever mission you were going to do.’ Other requirements include algorithm development for detecting, identifying and tracking ships. “We plan to create floating sensor networks that significantly expand maritime awareness at a fraction of the cost of current approaches,” said John Waterston, program manager in DARPA’s Strategic Technology Office.

Data would be transmitted via satellite to a cloud network for real-time analysis. The floats would transmit data periodically via satellite to a cloud network for storage and real-time analysis. The Ocean of Things would be linked to command datacenters via the 66-satellite Iridium constellation that includes dedicated Defense Department gateway. According to preliminary plans, sensor data would then be downlinked via DoD’s Enhanced Mobile Satellite Services gateway.

“We will see acoustic communications transmitting information to AUVs over long distances, while optical modems enable data transfer between sensors and vehicles over shorter distances,” says Sonardyne’s Tena. “The entire network will enable the provision of near-real-time updates to surface-based operators.”

Floats will be designed using commercial hardware components as a low-cost design approach will allow for the manufacture of large numbers of floats to cover large operating areas and provide robust data from areas where limited visibility exists today. According to reports, DARPA wants to build about 50,000 floating platforms with a unit cost of around $500. John Waterston, a program manager within DARPA’s Strategic Technology Office, says the sensors will float along the surface for at least one year, transmitting short messages via the Iridium satellite constellation back to a central location for analysis. “It’s a 280-byte in and 340-byte out message, so it’s a little bit more than a tweet. I like to say these things tweet about their environment,” he says.

“The goal of the program is to increase maritime awareness in a cost-effective way,” said John Waterston, program manager in DARPA’s Strategic Technology Office (STO). “It would be cost-prohibitive to use existing platforms to continuously monitor vast regions of the ocean. By coupling powerful analytical tools with commercial sensor technology, we plan to create floating sensor networks that significantly expand maritime awareness at a fraction of the cost of current approaches.”

“In phase one of this program, we’re building 4,500 sensors and we’re deploying them in 15,000 square kilometers right off of Southern California, inside the Catalina Islands. We’re going to try to do about one sensor per 3 square kilometers. The next phase, we’re going to expand the area to 150,000 kilometers and put in 15,000. That gives us 1 per 10 [square kilometers]. I think the final system is going to be about 50,000 floats for 1 million, so that’s 1 per 20 square kilometers,” Waterston reports. “Initially, we’re going to deploy overly dense to figure out what the right density is going to be.”

DARPA said it plans to conduct proof-of-concept demonstrations in a first phase. A second phase will test data analytics capabilities during actual sea trials.

DARPA plans to carry out tests with thousand-float arrays in the Southern California Bight and Gulf of Mexico in later 2020. Initially they will be arranged at about one float per three square kilometers. Waterson believes separation can be increased to one float per twenty square kilometers while maintaining coverage. He is also talking about much bigger arrays in future, of tens of thousands of floats.

In addition to obvious military and border protection use – no vessel could slip through the dense field of OoT sensors, on or under the water – the OoT will produce a mass of data of interest to oceanographers, meteorologists and biologists, with plans to share raw data online with researchers. The OoT may be able to monitor marine mammal like whales, watch hurricanes form from the inside, and track changes in ocean temperature.

In this context, Waterson says that people often mention Flight MH370, the Malaysia Airlines Boeing 777 that disappeared in Southern Indian Ocean in 2014. If there had been an OoT in the area, a plane crash could have been detected and the crash site located. However, DARPA’s main interest is likely to be in military applications.

Technical challenges

The technical challenge for Ocean of Things lies in two key areas: float development and data analytics. Under float development, proposers must design an intelligent float to house a passive sensor suite that can survive in harsh maritime environments. Each float would report information from its surroundings for at least one year before safely scuttling itself in the deep ocean. The floats will be required to be made of environmentally safe materials, pose no danger to vessels, and comply with all federal laws, regulations, and executive orders related to protection of marine life.

A key element of the OoT is its hydrophone, a sensitive underwater microphone or passive sonar which can pick up engines, screws and other sounds from ships and submarines. Floating sonobuoys dropped from aircraft have been used to locate submarines since WWII, but these only operate for a few hours. The OoT hydrophone has to operate for a whole year – and it has to be affordable. In 2019, researchers from Scripps Institution of Oceanography made a prototype high-fidelity hydrophone for the OoT which they estimated could be mass produced for $100-$150.

To effectively use constrained floats, each onboard sensing modality will require research into efficient signal processing methods that can maximizing the information content while utilizing limited communications bandwidth available underwater and also energy efficient to minimize the use of stored energy on a float.

DARPA is also planning the availability of cloud computing resources on vessels and on shores connecting these ocean of things through satellite network , to perform data processing and analytics algorithms on large volumes of data produced by these floats. These Ocean Internet-of-Things can benefit from the scalability, and performance of cloud computing infrastructures and could provide them with opportunities for cost-effective on-demand scaling.

DARPA’s OoT project will process the information that has been received in the cloud-based data center which particularly focuses on converting raw field measurements into oceanographic/environmental/meteorological products in these fields. (For example: convert recordings from temperature sensors into an interpolated, time series grid.) It is also planned for importing/formatting products from sources external to the DARPA project to use as a ground truth comparison to the field-collected data. Data types can include: Sea Surface Temperature (SST), salinity, solar irradiance, wave/swell height/periodicity (derived from IMU/accelerometer), barometric pressure, precipitation, wind speed/direction, surface current measurements.

Additional research will focus on the implementation of advanced analytic techniques in a cloud-based architecture, on approaches to visualize the dynamic capabilities of the system, and on methods to assist operators interacting with large numbers of floats, says DARPA.

The data analytics portion of the Ocean of Things program will require proposers to develop cloud-based software and analytic techniques to process the floats’ reported data. This effort includes dynamic display of float locations, health, and mission performance; processing of environmental data for oceanographic and meteorological models; developing algorithms to automatically detect, track, and identify nearby vessels; and identification of new indicators of maritime activity.

It is possible an ocean-based Internet could provide data on demand to a variety of customers inside and outside the Defense Department. If, for example, a government agency needs the water temperature in a given area reported every six hours, or a combatant command needs to know what’s happening in the Mediterranean, or NATO officials want information between Gibraltar and Sicily, or commercial fishermen need data on where the shrimp or tuna are, they could simply request it. “It’s about serving the end users. If you can use that data, we can generate it for you,” he offers. “It’s a little bit like floats-as-a-service or data-as-a-service.”

Ocean of Things will deliver distributed awareness of both the physical and operational ocean environments to provide improved environmental and activity characterization. In addition to the primary sensing mission, a successful Ocean of Things program will be able to support testing of specialized payloads/behaviors to interact with its surroundings and improve system performance.

Ocean of Things program partners

DARPA initially was working with three teams led by the Palo Alto Research Center, better known as PARC, Areté Associates and Numurus LLC to develop the floats. DARPA planned to select two suppliers to go head-to-head on the last phase of the competition. The new contract was awarded in July 2020 to technology company PARC, whose 18-kilo, solar-powered glass float design won out ahead of two others in the first phase. The floats are sensor nodes which will pass data via satellite to a cloud network for real-time analysis. The OoT will combine data from multiple floats, seeing the whole picture rather than the single pixel gathered by one sensor, as Waterson puts in.

Leidos, Draper Laboratory, SoarTech and Geometric Data Analytics are providing software for data visualization, performance prediction, float command and control and detection. The program will begin phase two in the 2020 fiscal year and phase three the next year. It will end in 2022.

DARPA awards Xerox PARC a contract for the next phase of the Ocean of Things project in Oct 2020

PARC built 1,500 drifters for the first phase of the project and will deliver up to 10,000 that are more compact and cost-effective for the next phase. Each solar-powered drifter has approximately 20 onboard sensors, including a camera, GPS, microphone, hydrophone, and accelerometer. The different sensors can provide data for a broad array of areas including ocean pollution, aquafarming and transportation routes.

PARC leveraged its more than fifty years of experience developing industry-leading technologies to design a drifter that best fit the DARPA requirements for the program. Among other things, the float needed to be made of environmentally safe materials, be able to survive in harsh maritime conditions for a year or more before safely sinking itself, and use advanced analytic techniques to process and share the data gathered.

Data gained in this round will help further optimize the final design, at which point DARPA expects to deploy large volumes of these drifters to provide continuous information and a better understanding of oceans that is missing today.

DARPA Selects Numurus’ Smart IoT Ecosystem for Ocean Monitoring Program

Numurus has won a $2.3M contract from DARPA’s Strategic Technology Office to integrate its Smart IoT Ecosystem into thousands of satellite connected remote monitoring devices.

To achieve the OoT program goals, DARPA will leverage Numurus’ SB-SDK™ Edge software and NEPI™ Smart IoT management platforms. These tools together produce a full-spectrum sensor-to-cloud solution capable of detecting minor changes in the environment, reducing megabytes of sensor data into kilobytes of information at the collection source, and transmitting that information through Iridium satellites to a secure firewalled cloud infrastructure.

Through NEPI’s fleet level monitoring and software management solutions, DARPA can access data products, tweak mission configurations, and apply software updates to deployed smart devices. Outside of the marine environment, the technology advances developed under this program are immediately applicable to many challenging problems such as measuring ice sheet thickness and permafrost across remote areas of the world or search and rescue operations in mountainous and dense forested environments.

Forecasting Floats in Turbulence Challenge

DARPA’s recent Forecasting Floats in Turbulence (FFT) challenge took an exploratory first step toward trying to understand the turbulent convergence of wind, waves, and currents on the surface of the ocean – and its effect on objects floating at sea. The competition was part of DARPA’s Ocean of Things program, which uses low-cost distributed drifters for maritime situational awareness.

The goal of the challenge was to spur development of algorithms to better predict where free-drifting floats will travel over time. Competitors developed algorithms or other methods to predict where 90 Sofar Spotter drifters – about the size of a basketball – floating freely in the Atlantic would travel over a 10-day period. Each team received the previous 20 days of location data from the drifters’ GPS coordinates. Using that data, teams attempted to predict where the spotters would travel a week and a half into the future.

Three teams from a field of 32 competitors took top spots, winning $25K, $15K, and $10K, respectively: First place went to Second Sight Predictions with 498 points; Deltares took second with 425 points; and the Center for Ocean-Atmospheric Prediction Studies (COAPS), came in third place with 395 points. Second Sight Predictions was a single-person team, Chris Wasson, a guidance, navigation, and control engineer working on satellite development in southern California.

“I was very happy to discover this challenge where the focus was split equally on algorithms and their application to an engineering problem,” Wasson said. “There are still many examples in the challenge float data where the float trajectory disagrees strongly with all available ocean current and wind data at that latitude/longitude. I think this is a strong indication that one of the biggest remaining hurdles in this problem is not in modeling surface effects but in identifying errors in these models, particularly ocean surface current models.”

Second-place team Deltares leveraged their work in hydrodynamic modeling near the coasts and employed a physically-based modeling approach integrated into a forecasting platform. “We are passionate in improving our understanding of particle movements in ocean and coastal seas,” said Kun Yan from Deltares. “As well as its wide application, e.g. mitigating oil spills, search and rescue operations, determining the trajectory of fish larvae, etc.”

Points were awarded based on prediction accuracy of drifters’ location in 2-day, 4-day, 6-day, 8-day, and 10-day forecasts. The further out the forecast, the higher the point value for accuracy. For example, a two-day forecast within 16km of the actual float position was worth one point, but a 10-day forecast within 16km was worth 20 points for added difficulty. Lastly, to ensure competitors had strong trajectory models, they would no longer receive points for any drifter prediction that experienced greater than 32km of error for any forecast. So, if a forecast for a particular float was off by 50km on Day 6, but within 1km on Days 8 and 10, they would receive no points for that float because the trajectory was too far off.

Results varied significantly by float and by team: No points were awarded for 8 floats, while Second Sight Predictions and Deltares each predicted a float within 4km on Day 10.

“Using inexpensive, large scale in-situ drifters is a new way of looking at ocean sensing and addressing the lack of fidelity in current models of the ocean’s surface. We wanted to provide access to this float data and get people excited about approaching the problem in novel ways. The challenge has shown that we’ve just scratched the surface in understanding the complex turbulence at the convergence of air and sea, and we hope to spur further research into making models more accurate,” said John Waterston, program manager for the Ocean of Things program in DARPA’s Strategic Technology Office.