ALEXANDRIA, Va.- Recognizing that accurate and timely terrain models are critical for Soldiers in many areas of the world, scientists at the U.S. Army Engineer Research and Development Center’s (ERDC) Geospatial Research Laboratory (GRL) invented a cost-efficient method to generate high-resolution 3-D terrain data using existing unmanned platforms.
Led by physical scientist Dr. Richard “Ricky” Massaro, a research project team earned a U.S. patent in February 2020 for its “Method of Processing Full Motion Video Data for Photogrammetric Reconstruction.” Team members in GRL’s Information Generation and Management and Data Signatures and Analysis Branches included Jeff Ruby, Jonathan Cotugno, Dr. John Anderson, Dr. Rob Fischer and Jarrod Edwards.
Massaro used his expertise in photogrammetry, image processing and 3-D terrain modeling, along with 16 years of targeted research at the ERDC, to develop the now-patented process. In fiscal year 2016, Massaro won the Maj. Gen. Harold J. Greene Award for Innovation in the individual-civilian category for his program, “Tactical Full Motion Video to 3D,” benefiting the Army and DOD by providing Soldiers and deployed forces with highly-detailed geospatial data delivered by manned and unmanned aircraft systems (UAS).
He said this invention was primarily designed for special operations forces, intelligence community analysis, terrain specialists and anyone dealing with geospatial intelligence. The invention is a method for photogrammetric analysis of full motion video (FMV), which converts FMV to image files, extracts metadata and produces accurate 2-D and 3-D geospatial images in real time.
“I became aware of the multitude of FMV datasets when I was working at the National Geospatial-Intelligence Agency for a short time,” Massaro said. “Around that time, computational techniques for photogrammetry and small UAS platforms were greatly advancing. So I thought, ‘wouldn’t it be great to just convert the FMV into photogrammetric datasets so the way it is used requires very minimal intervention?’ Basically, the user/operator just selects the FMV source and hits ‘Go.’”
We are also working to use this as just a service, so that 3-D products are simply a byproduct of the FMV collection,” Massaro said.
The patent application stated that photogrammetry is the science of making measurements from photographs and for the recovering the exact positions of surface points. Photogrammetric computational methods draw upon optics, mathematics and projective geometry. The U.S. Army Corps of Engineers is conducting advanced research to meet the challenge of using photogrammetric techniques to transform petabytes of full motion video data from unmanned vehicles, wireless cameras and other sensors into usable 2-D and 3-D maps and models.
Invention development, deliverables
The information Massaro gathered in 2015 spurred on his team’s interest in finding terrain data solutions.
“We learned that the 2nd Brigade Combat Team, 101st Airborne Division, was being sent to Iraq to liberate Iraq from ISIS. In many cases, they were being sent into a hostile environment with old terrain data, but had full motion video assets at their disposal,” he said. “Rather quickly, it was shown that 3-D terrain datasets could be created from the FMV being collected 24/7 by airborne assets. It took quite a bit more time to fully automate that procedure. The automation allowed the user to step away and have the FMV-to-3D process run in the background while the user performed their many other tasks.”
“The development was heavily performed in the 2015-2017 timeframe. We’re still improving it and adding additional functionality. But, it has been used operationally in many scenarios already where users needed rapidly produced 3-D data and only had FMV assets at their disposal,” Massaro said.
He explained that it was it was mainly created and tested at the GRL. “However, we have tested it at many Department of Defense demonstrations/exercises and instances of it are currently being investigated by U.S. intelligence agencies,” Massaro said.
How it works
Outlining the invention’s operations, the patent application stated, “To apply photogrammetric computational models, FMV data gathered from airplanes, satellites, and other sources must be parsed into still frame images. The still frame images contain coordinates and metadata which are extracted for processing by computational model.
“The exterior orientation of the camera or sensor defines its location in space and its view direction. The inner orientation defines the geometric parameters of the imaging process. The focal length of the imaging lens is a component of these geometric parameters. By feeding the measurements from each individually processed image frame into a computational model, it is possible to rapidly and accurately estimate 3-D relative motions and other measurements for each frame.”
For those Soldiers using this invention, Massaro said, “This process generates those high-resolution terrain datasets but with existing, unmanned platforms and at an extremely small cost.”
Massaro and his co-inventors will be recognized in a patent plaque presentation ceremony to be scheduled later this year by ERDC’s Office of Research and Technology Transfer, which processes patents for the center.