Digging in the dirt is hard work. An excavator will speed up the work, but operating one is difficult and requires years of experience. One that can be operated remotely, and comes with an augmented reality system, could make the process easier, more convenient, safer – and possibly more fun.
“We’ve built a system that can take someone who doesn’t have real prospecting skills and turn them into an expert operator, way faster than would normally be possible,” says Robin Brewer, the roboticist who led the project. His team at SRI International, a California-based not-for-profit research institute, has turned an impractical excavator into a smart, granular excavator.
Conventional excavators have unintuitive controllers. To move the scoop up and down, the operator needs to perform a right-to-left movement on the control levers and levers from inside the vehicle. “There are a lot of joints, and they don’t really draw in the right direction,” Brewer says. “It becomes very confusing.” Operators also need extensive training to learn how to avoid buried gas lines, water lines and internet cables to make tiny holes in the ground.
Therefore, researchers at SRI International have upgraded the robots to excavators. Their smart excavator can now be operated with more intuitive controls, and the operator doesn’t even need to be in the driver’s seat. Users can maneuver these excavators right next to the vehicle, if they wish, or they can manipulate the machinery from home or anywhere around the world, as long as they are connected to the internet.
This “bot kit,” as Pryor calls it, can be installed into any existing hand excavator to make it operable as if it were straight out of a video game. The researchers turned on all of the levers and pedals in the machine, all of which would be synced to an online handheld console. Six overhead cameras above the rig hood provide remote users with a 360-degree view of the field site. The Oculus headset reconstructs the rig’s circumference for wearers from a live camera feed, imbuing them with depth perception so they can precisely switch arm or drill from a distance. The connected console allows remote users to perform the required drilling movements, and advanced software tracks the position of the console in real time and simulates the user’s movements with the excavator arm. For experienced users who prefer the more popular controls, there is an option to try the excavator with a wireless joystick. Augmented reality helps [making] “You really feel like you’re sitting in the cab driving the excavator,” Brewer says.
He adds that this is not the first excavator that can be operated from afar. In 2015, Volvo introduced a similar concept. But his company’s version added security controls that make it more practical. These safety features can freeze shifting motions or force the excavator to slow down when overhead cams detect a human nearby. These commands run automatically even when a remote user is behind the virtual steering wheel. These design elements are important for carrying out construction work productively and safely in crowded urban areas with noisy human traffic.
Unlike some previous efforts to automate excavation work, the project’s vision is not to eliminate the human driver, Brewer says (though the company has also programmed the excavators to be able to stop themselves). After all, the consequences of breaking a pipe or damaging infrastructure while drilling can be costly, so a certain level of human judgment is still required while drilling. Instead, the researchers hope to give users “superpowers,” says Breuer, such as the ability to see around and behind them, ideally eliminating common blind spots.
Researchers are currently partnering with construction companies to get user feedback on their robotic excavator. In the future, based on interest, the researchers say they could expand their robotics range to include other woodworking machines.
“It’s broadly generalizable to any manually operated construction equipment,” Brewer says. “Adjust the exact geometry of where we put the engine… And suddenly, instead of just doing the digging, we’re doing forklifts or front loaders or steam wagons. It’s very easily adaptable.”
Watch the video of the smart excavator here: