ERDC researchers combine robotics, imagery technology to solve problems

U.S. Army Engineer Research and Development Center
Published March 18, 2021
A U.S. Army Engineer Research and Development Center (ERDC) prototype site characterization vehicle demonstrates its amphibious capabilities during testing at the ERDC Levee Breach Model in Vicksburg, Mississippi, Oct. 7, 2020. (U.S. Army Corps of Engineers photo)

A U.S. Army Engineer Research and Development Center (ERDC) prototype site characterization vehicle demonstrates its amphibious capabilities during testing at the ERDC Levee Breach Model in Vicksburg, Mississippi, Oct. 7, 2020. (U.S. Army Corps of Engineers photo)

The U.S. Army Engineer Research and Development Center demonstrates unmanned capabilities for combat engineers during the Maneuver Support, Sustainment, Protection Integration Experiment-2020 (MSSPIX20) at Fort Leonard Wood, Missouri, Sept. 16, 2020. During the MSSPIX20 training exercise, these two systems demonstrated a near real-time 3D mapping environment and unmanned reduction of an obstacle with the teleoperated 8-ton hydraulic excavator and the Robotics for Engineer Operations fully autonomous site characterization and mapping platforms. (U.S. Army Corps of Engineers photo)

The U.S. Army Engineer Research and Development Center demonstrates unmanned capabilities for combat engineers during the Maneuver Support, Sustainment, Protection Integration Experiment-2020 (MSSPIX20) at Fort Leonard Wood, Missouri, Sept. 16, 2020. During the MSSPIX20 training exercise, these two systems demonstrated a near real-time 3D mapping environment and unmanned reduction of an obstacle with the teleoperated 8-ton hydraulic excavator and the Robotics for Engineer Operations fully autonomous site characterization and mapping platforms. (U.S. Army Corps of Engineers photo)

A U.S. Army officer utilizes a remote-control station to operate an 8-ton excavator from beyond visual line of sight at a distance of approximately 200 m during the Maneuver Support, Sustainment, Protection Integration Experiment (MSSPIX20) demonstration at Fort Leonard Wood, Missouri, Sept. 16, 2020. The U.S. Army Engineer Research and Development Center researchers demonstrated unmanned capabilities for combat engineers during the MSSPIX20 training exercise. (U.S. Army Corps of Engineers photo)

A U.S. Army officer utilizes a remote-control station to operate an 8-ton excavator from beyond visual line of sight at a distance of approximately 200 m during the Maneuver Support, Sustainment, Protection Integration Experiment (MSSPIX20) demonstration at Fort Leonard Wood, Missouri, Sept. 16, 2020. The U.S. Army Engineer Research and Development Center researchers demonstrated unmanned capabilities for combat engineers during the MSSPIX20 training exercise. (U.S. Army Corps of Engineers photo)

Computer display of the Robotics for Engineer Operations (REO) researchers’ workstation during the Maneuver Support, Sustainment, Protection Integration Experiment demonstration at Fort Leonard Wood, Missouri, Sept. 16, 2020. On the left side of the screen is the 360-degree camera angle and feedback from the site characterization platform. The right side of the screen shows the near real-time 3D model of the environment. The U.S. Army Engineer Research and Development Center created this capability to capture terrain information, obstacles and other relevant engineering data that is utilized to launch and monitor unmanned missions. (U.S. Army Corps of Engineers photo)

Computer display of the Robotics for Engineer Operations (REO) researchers’ workstation during the Maneuver Support, Sustainment, Protection Integration Experiment demonstration at Fort Leonard Wood, Missouri, Sept. 16, 2020. On the left side of the screen is the 360-degree camera angle and feedback from the site characterization platform. The right side of the screen shows the near real-time 3D model of the environment. The U.S. Army Engineer Research and Development Center created this capability to capture terrain information, obstacles and other relevant engineering data that is utilized to launch and monitor unmanned missions. (U.S. Army Corps of Engineers photo)

A robotic system known as the Dambot operates near closure gates at Blue Mountain Dam, Arkansas in October 2020. Dambot takes the human element out of a dangerous but necessary U.S. Army Corps of Engineers (USACE) maintenance task. The cutting-edge technology has  been successfully tested and stands poised to change the course of closure gate assessments, while also safeguarding USACE team members. (U.S. Army Corps of Engineers photo)

A robotic system known as the Dambot operates near closure gates at Blue Mountain Dam, Arkansas in October 2020. Dambot takes the human element out of a dangerous but necessary U.S. Army Corps of Engineers (USACE) maintenance task. The cutting-edge technology has been successfully tested and stands poised to change the course of closure gate assessments, while also safeguarding USACE team members. (U.S. Army Corps of Engineers photo)

CHAMPAIGN, Ill. – The U.S. Army Engineer Research and Development Center’s (ERDC) Construction Engineering Research Laboratory’s Robotics for Engineer Operations (REO) team is combining efforts with the Information Technology Laboratory’s Robotic Assessment of Dams (DamBot) team to tackle challenges posed by both combat engineer operations abroad and infrastructure at home.

Both teams have the goal of removing personnel from high-risk environments using robotic and autonomous systems. REO has developed unmanned engineer support capabilities to allow operations in global navigation satellite system (GNSS)-denied, unpredictable and challenging environments at beyond-visual-line-of-sight standoff distances. DamBot is developing remote, semi-autonomous inspection systems for the nation’s 91,000 dams’ outlet works. These are two very pressing needs that cannot easily be met using current autonomous driving technologies, which are reliant on well-structured road networks, satellite infrastructure and reliable communication networks.

The ERDC REO multidisciplinary research team is currently mapping and localizing robotic platforms in extremely challenging environments where GNSS is not available and unique visual and physical cues — such as dense forests, open fields and concrete conduits ― are limited or are in constant motion.

“Before we put an unmanned capability on the ground, we have to understand what type of environment we are dealing with,” said Dr. Ahmet Soylemezoglu, REO project lead and a Construction Engineering Research Laboratory systems engineer. “It’s critical for navigation purposes but becomes even more critical if we are going to do any terrain shaping.

“The robotic systems we are testing are kitted with different sensing modalities like Light Detection and Ranging, also known as LiDAR, stereo cameras, and so forth, to collect information about the environment. The systems then turn that information into a site model on the computer. We’re gathering information like the terrain angles or slopes, soil types, obstacles, trafficability, and we’re also learning whether the vehicle can traverse the specified area. Once we have that site model, we have situational awareness and planning tools to decide how to launch unmanned engineer operations in that environment.”

REO is ensuring Army engineers can support the future force by enabling unmanned engineer operations such as obstacle reduction and removal, route maintenance/repair and fighting position preparation. REO adapts currently available engineer platforms, such as bulldozers and excavators, and improves their capabilities by implementing multimodal sensing, model-driven machine controls and artificial intelligence/machine learning-based autonomy for use in the operational environment. The engineer can manually operate the heavy equipment, or earth movers, from distances where they don’t have a direct line of sight. The lack of environmental feedback during remote operations means that a knowledge of detailed engineering characteristics of the operational area — such as soil conditions, slopes and the presence of obstacles ― is necessary. REO is developing a system for multimodal site mapping and semantic segmentation of environmental features to aid the operator in the remote performance of these combat engineering tasks. The next phase is to gradually build autonomous task execution capabilities so the machines can complete the initial tasks, or simple engineering tasks, on their own.

“This is an extremely challenging problem,” said Soylemezoglu. “The current technology in autonomous vehicle driving is a completely different problem set than what the Army encounters.”

He described how autonomous vehicle technology companies are currently investing billions of dollars in well-structured road networks, road signs, driving rules and other support, and how the research does not always translate into what the Army needs to confront dynamic and challenging terrain and unknown environments.

“We must also factor in that we have people who are actively trying to derail the mission by either projecting force or utilizing disruption to deny our communication and global positioning systems—we’re working in a completely different environment,” Soylemezoglu said.

The DamBot robotic platform is being developed with close collaboration between ERDC and personnel who work on hydraulic steel structures. This amphibious robotic system is capable of preforming initial assessments of conduits and dam gates, along with gathering photographic documentation required for dam inspection reports. The DamBot Unmanned Ground Vehicle (UGV) is equipped to relay a full 360° live video stream and generate a 3D map with a resolution greater than 0.5 cm of the conduit and the gate as it traverses through the tunnel. Unlike the regular inspection process where safety personnel use flashlights to illuminate areas of interest and cameras to document areas of special note, the reconstructed map is complete and evenly illuminated for easy location and identification of flaws.

These high accuracy 3D representations of infrastructure are stored at the ERDC High Performance Computing Center for further analysis by hydraulic steel structure safety personnel. Stored inspection records can be used to perform remote virtual inspections using virtual reality glasses or standard screens. The training of new personnel can be done virtually by comparing the flaws detected by experts and trainees. The temporal information collected through multiple scans of the same structure can inform inspectors on the speed of growth of any detected flaws. Availability of such inspection systems and data will enable repeatable and reliable inspections of our nation’s infrastructure and will provide the opportunity in the future to implement artificial intelligence/ machine learning algorithms to aid in detection and tracking of potential flaws.

“USACE has a portfolio of over 700 dams, many of which are beyond or nearing end-of-design life,” explained Dr. Anton Netchaev, an Information Technology Laboratory research computer scientist and DamBot project lead. “The outlet works of these dams require regular inspection, which currently involves personnel entering the outlet works and documenting structural deficiencies. This introduces risk to the inspection teams and potential for human errors in data collection.” 

Collaboration across the ERDC is rapidly accelerating the development of robotic and autonomous systems, such as Dambot and REO. As these technologies mature, they will continue to enhance the nation’s ability to respond to engineering challenges related to natural disaster response efforts, infrastructure assessments, civil works and other combat engineer support tasks.