International Defense Security & Technology Your trusted Source for News, Research and Analysis
We see things because our eyes are sophisticated light detectors: they constantly capture the light rays bouncing off nearby objects so our brain can construct an ever-changing impression of the world around us. Similarly in a camera, All the light traveling from the object enters a single lens before it hits the light-sensitive image sensor chip (the CCD or CMOS chip in a digital camera), enabling camera to record two-dimensional pattern of light, dark, and color. But for the most part, imaging applications are limited to light propagating in a straight line. A direct line of sight between an object and a camera or detector is typically needed for every imaging application.
However, that is starting to change as some cutting edge research is opening up possibilities to image around corners and around obstacles. A combination of lasers, sensitive cameras, and computational reconstruction methods can be used to detect objects hidden by obstacles by scattering light off of surrounding objects.
NLOS imaging has the potential to be transformative in important and diverse applications such as medicine, robotics, manufacturing, and scientific imaging. NLOS imaging is also very useful to security forces during counterterrorism operations to complete mission and reduce casualties. For cases when the line of sight is blocked by corners of walls or other building obstructions within the environment of city streets and buildings , traditional imaging techniques and equipment are unable to achieve imaging beyond the direct field of view. Non-line-of-sight imaging can bypass the corners of walls or other obstructions to achieve imaging beyond the direct field of view, which can be used to reduce casualties in modern urban street fighting, and to increase battlefield awareness and operational efficiency.
However, taking this emerging technology and creating a practical solution for real-world use that is portable and not dangerous to observer’s eyes is extremely challenging. More development is still needed before non-line-of-sight imaging technology becomes available in practical commercial systems, but it is a promising solution for the next generation of imaging applications.
NLOS technology
The process for non-line-of-sight imaging is similar to that of LiDAR (light detection and ranging), where a laser pulse is sent towards an object and the time-of-flight of the light scattering back off of the object is used to measure the distance between the object and a detector. However, non-line-of-sight imaging images objects obscured by obstacles by adding another scattering event to this process.
Non-line-of-sight (NLOS) imaging allows to observe objects partially or fully occluded from direct view, by analyzing indirect diffuse reflections off a secondary relay surface. Whereas conventional imaging involves direct line-of-sight light transport to recover the visible objects, NLOS imaging aims to reconstruct the hidden objects from the indirect light paths that scatter multiple times, typically using the information encoded in the time-of-flight of scattered photons.
Reconstructing a Model of the Hidden Target
Since non-line-of-sight imaging based on laser range gated imaging is affected not only by the parameters of the laser range-gated imaging system but also by the reflection characteristics of the intermediary reflective surface and the scene, construction of an optical imaging model is important to the development and application of non-line-of-sight imaging.
Highly sensitive cameras such as single-photon avalanche photodiode array cameras are needed to measure the propagation of picosecond and femtosecond pulses of light in real time. The detector receives two different return signals: an initial signal of light scattered directly off of the wall and a secondary signal of light scattered off of the target, which is the signal used for non-line-of-sight imaging. This time-of-flight information is then used to reconstruct a series of ellipsoids that all overlap at a given point on the hidden target, allowing computational software to calculate the distance between the camera and the hidden target and recreate a 3D model of the target.
A 3D object can be broken down into a collection of many individual points that scatter light. The summation of all of these points can reconstruct a model of the original object. If the detector can distinguish return pulses with a temporal resolution of 100ps, this corresponds to a spatial resolution of points on the hidden target of approximately 1.5cm.
Challenges
Taking this emerging technology and creating a practical solution for real-world use that is portable and not dangerous to observer’s eyes is extremely challenging. One of the main issues with non-line-of-sight imaging is the limited amount of light that makes its way back to the detector, which must be able to pick up this very small amount of light and differentiate it for any ambient light sources. The return signal to the detector is the result of two consecutive scattering events, leading to an extremely high loss. Return signals can be as low as one photon per laser pulse.
In contrast to conventional LOS imaging , the diffuse nature of light reflected from typical surfaces in NLOS imaging leads to mixing of spatial information in the collected light, seemingly precluding useful scene reconstruction. To address these issues, optical techniques for NLOS imaging that have been demonstrated include transient imaging, speckle correlations , acoustic echoes , intensity imaging , confocal imaging , occlusion-based imaging , wave-propagation transformation , Fermat paths, and so forth . With few exceptions , most of the techniques that reconstruct NLOS scenes rely on high-resolution time-resolved detectors and use the information encoded in the time-of-flight (TOF) of photons that scatter multiple times. Using such techniques, NLOS tracking of the positions of moving objects has been also demonstrated.
Despite its many potential applications, existing methods lack practical usability due to several shared limitations, including the assumption of single scattering only, lack of occlusions, and Lambertian reflectance. Line-of-sight (LOS) imaging systems, on the other hand, can address these and other imaging challenges despite relying on the mathematically simple processes of linear diffractive wave propagation.
The major obstacles to extending NLOS imaging to long ranges are signal strength, background noise, and optical divergence. Due to the three-bounce reflections and the long standoffs, the attenuation in long-range NLOS imaging is huge. Also, the weak back-reflected signal is mixed with ambient light, leading to poor signal-to-noise ratio (SNR). The sunlight contributes ambient noise, and the backscattering from the near-field atmospheric will also introduce high noise.
Moreover, unlike long-range LOS imaging, the signal detected at each raster-scanning point in NLOS imaging contains light reflected by all of the parts of the hidden scene. Consequently, it is more difficult to tolerate low SNR in NLOS imaging than in LOS imaging. On the other hand, the optical divergence over long range introduces a strong temporal broadening of the received optical pulses. Such broadening renders the idealizations of virtual sources and virtual detectors in previous short-standoff NLOS imaging experiments inapplicable.
Furthermore, previous NLOS experiments typically required high-precision timing measurements at picosecond scale—as obtained from a streak camera or a single-photon avalanche diode (SPAD) detector —to recover hidden scenes. However, the temporal broadening in long-range situations can contribute time jitters on the order of nanoseconds. Lastly, long-range NLOS imaging needs a higher scanning accuracy, which will effect the resolution of the reconstruction results. All of these issues prevent the useful reconstructions of NLOS imaging over long standoffs.
Technology advances
Since 2007, the Media Lab at the Massachusetts Institute of Technology (MIT) has been studying transient imaging based on femtosecond laser range-gated imaging technology and proposed an algorithm for extracting object information from multireflection, realizing a means of seeing around the corner. In 2012, researchers at MIT realized a three-dimensional reconstruction of hidden targets utilizing diffuse reflections of ultrashort laser pulses using an intermediary reflective surface. Since 2012, researchers at the Beijing Institute of Technology (BIT) have been studying non-line-of-sight imaging. The BIT group uses a nanosecond laser range-gated imaging system to realize non-line-of-sight imaging of an USAF 1951 standard target at different distances with glass and ceramic tile as the respective intermediary reflective surfaces. These existing researches show the potential application prospects of non-line-of-sight imaging in areas such as urban warfare, counter-terrorism, and disaster relief.
Stanford Computational Imaging Lab has developed a non-line-of-sight imaging system that works outdoors under indirect sunlight. They successfully imaged an object made out of retroreflective tape that was obscured by a wall, which bodes well for the future of this technology.
The lab of Aristide Dogariu of University of Central Florida is investigating non-line-of-sight imaging utilizing the spatial coherence of light hitting a wall instead of laser light scattered off of that wall and the target behind it.3 This could lead to modeling of the hidden target without requiring ultrafast laser illumination, making real-world applications of the technology more portable and easy to use.
China Has Built a Laser That Can Spot Hidden People, Objects From Over 1 Km Away
But scientists in China have developed a new technology that can help with non-line-of-sight (NLOS) imaging, i.e. identifying objects which aren’t in the line-of-sight of the device or even hidden behind a screen. In a video shared by South China Morning Post, the laser-device was able to track a person 1.43 kilometres away, at a diagonal, and hidden behind a screen.
“This range is about three orders of magnitude longer than previous experiments. The results will open avenues for the development of NLOS imaging techniques and relevant applications to real-world conditions,” the paper said. The laser emitter was placed within the university campus in urban Shanghai whereas a mannequin was kept behind a screen in the balcony in an apartment building 1.43km away. The laser pulsed and bounced off of the apartment’s wall thrice in multiple directions. As the light scattered, some photons re Then, a computer algorithm did the rest.
We develop both hardware and software solutions to realize long-range NLOS imaging. First, we construct an NLOS imaging system operating at the near-infrared wavelength, which has the advantages (as compared to visible light) of low atmospheric loss, low solar background, eye-safety, and invisibility. Second, operating at near-infrared requires previously undescribed detection techniques, since the conventional Si SPAD does not work. To do this, we develop a fully integrated InGaAs/InP negative-feedback SPAD, which is specially designed for accurate light-detection and ranging applications.
Third, we develop a high-efficiency optical receiver by employing a telescope with high coating efficiency and a single-photon detector with large photosensitive surface. The collection efficiency is >4.5 times higher than previous experiments . Fourth, we adopt a dual-telescope optical design for the confocal system to reduce the backscattering noise, thus enhancing the SNR. The dual-telescope design can separate the illumination from detection and remove the use of beam splitter, thus allowing high illumination power and the optimization of receiver optics for high collection efficiency. Fifth, we optimize the system design to balance the collecting efficiency and the temporal resolution to realize high-resolution imaging . We achieve a fine precision scanning with 46× and 28× magnification, where the scanning accuracy is as low as 9 microrads.
Finally, we derive a forward model and a tailored deconvolution algorithm that includes the effects of temporal and spatial broadening in long-range conditions. With these efforts, we demonstrate NLOS imaging and tracking over a range up to 1.43 km at centimeter resolution. The achieved range is about three orders of magnitude longer than previous experiments
References and Resources also include:
https://www.edmundoptics.com/knowledge-center/trending-in-optics/non-line-of-sight-imaging/
https://www.pnas.org/content/118/10/e2024468118
