In 2018, I watched my buddy Allan engage the Mad Max autopilot mode on his Tesla Model S, cuing the car to switch lanes aggressively on Interstate 95. While the experience as a human was unnerving, the car leveraged cameras, sensors and artificial intelligence to maneuver safely.
Months later, I rode on a Boston Whaler 330 Outrage fitted with Mercury Marine’s Advanced Pilot Assist and Raymarine’s DockSense systems. As we approached the boat’s slip, the preproduction system used cameras, AI and the outboard engines to maintain a 3-foot safety buffer.
At the 2022 Fort Lauderdale International Boat Show, I saw these ideas meld in Avikus’ prototype NeuBoat autonomous operations system. The boat, with a human-in-the-loop operator, navigated itself out of its slip, up a river and around a lake before reversing course and docking itself.
Ready or not, autonomous technology is coming. This is likely good news for novice boaters—and for boaters who hate docking—because some of the marine industry’s smartest minds have been combining sensors and AI to smooth out boating’s rough corners. One example is NeuBoat (neuron plus boat), which Avikus is developing in partnership with Raymarine.
While experts say the sensors and software already exist to enable fully autonomous docking and navigation, Avikus and Raymarine foresee a road map to autonomy that earns trust with boaters while buying time for engine manufacturers to integrate the technology, and for agencies and organizations to create regulations.
“We’re intentionally paralleling the automotive market,” says Jamie Cox, Raymarine’s senior global product manager. “But I think we will beat automotive.”
Others agree. Sangwon Shin, Avikus’ director of strategic planning and business development, says: “In our view, the boating environment is less complicated than the car environment. So, we expect a little bit faster adoption rate.”
For boaters who are ready to start now, Avikus and Raymarine are releasing NeuBoat Dock this year. The assisted-docking system includes at least six self-calibrating, 360-degree cameras; a Raymarine multifunction display; an Avikus object-recognition unit; camera control boxes; and Avikus’ AI to provide bird’s-eye views and distance guides. (Garmin’s Surround View camera system provides similar capabilities.)
NeuBoat Dock is a level-one autonomous navigation system, which means it serves as a virtual assistant to human operators who remain in control. Level-two systems provide partial driving automation but still require a human operator. Level-three systems have conditional driving automation, requiring some human oversight, while level four has zero expectations of driver involvement. Level five is full driving automation.
Avikus, which is a spin-off of HD Hyundai, began developing NeuBoat in 2019. The resulting level-three-plus black-box prototype, which I got aboard in 2022, used the global navigation satellite system and vector cartography to establish position. The local device didn’t require internet connectivity. Instead, it employed daylight cameras and lidar (light detection and ranging) sensors to detect objects, measure distances, and scan and map berths. It also used Avikus’ AI to detect and classify nearby objects and vessels, assist with route planning, and suggest navigable courses.
This latter information was presented as screen views showing vector cartography with recommended courses, head-up displays and live camera views with augmented-reality data tags.
While impressive, the prototype didn’t use radar or the automatic identification system, so its range of object detection was limited to lidar’s 400-foot-range capacity. This range worked at our 6-knot speed, giving us 39 seconds of reaction time, but it wouldn’t work at 25 knots, only allowing for nine seconds.
Enter Raymarine, which integrated its own radar technology with Avikus’ AI. This combination extended NeuBoat’s detection range from 400 feet to 1.5 nautical miles. Shin says Avikus plans to integrate radar, sonar and infrared cameras within five years.
While extra range is important for recreational users, it’s critical for letting Avikus develop autonomous systems on large ships. “We use the same technology and the same algorithms for commercial and recreational, but the hardware specs are different,” Shin says.
In addition to radar expertise, Raymarine has amassed experience using computer vision from its DockSense and ClearCruise AR products. The latter places augmented-reality tags atop a video feed. Computer vision is a branch of AI that lets computers recognize, categorize and identify objects and people in digital images or video feeds; as such, it is critical to autonomous operations.
Looking ahead, Shin says, commercial ships and recreational vessels will first use autonomous navigation with human-in-the-loop operators, followed by autonomous operations. This isn’t a hypothetical; in 2022, Avikus’ commercial version of NeuBoat autonomously guided an LNG tanker across an ocean with human-in-the-loop oversight.
“The technology is there today,” Cox says. “We need to make sure that people are ready to use the technology responsibly and that regulations are there.”
When asked what milestones need to be met for autonomous operations aboard recreational yachts, Cox and Shin made clear they aren’t talking about distant horizons. “None are 10 years out,” Cox says, adding that by mid-2024, Avikus and Raymarine expect to have achieved sensor fusion, where the system can combine data from the vessel’s AIS, cameras, GNSS, lidar and radar. “In two years, on the control side, boats will be docking and driving themselves.”
Shin agrees: “In five years, we’re expecting lots of the boating community to accept the possibility of autonomous navigation or partial assistance on their boat.”
Before this can happen, however, Cox and Shin point to two technical complexities: networking with autopilots and engines. As with radars, Raymarine has decades of experience manufacturing autopilots, so engine interfaces could prove to be the sticky wicket. “Engine manufacturers need to become more progressive,” Shin says. “They are the powerful guys.” Cox says the goal is to integrate NeuBoat with every major engine manufacturer.
Cox and Shin also point to a need for regulations to govern autonomous vessels. This is already happening; in 2022, the American Bureau of Shipping published a white paper that detailed 10 points—from maintaining propulsion to maintaining communications—intended to create a structure for autonomous-vessel design and operations. The US Coast Guard also published guidelines on testing remote- and autonomously controlled vessels.
Convincing experienced boaters that autonomous technology is the path forward could be a hard sell for some, but this is where Avikus and Raymarine plan to parallel the automotive world. Most contemporary cars have adaptive cruise control, making these types of assistance features feel familiar. Many boaters also own cars with an autopilot feature.
But driving to work is different than taking the boat out for a spin. Here, Cox says NeuBoat isn’t going to take away boating’s joys. Instead, the idea is to reduce stress. For example, Cox describes allowing the boat to navigate autonomously to the fishing grounds or home from a cruise.
Cox also says autopilots have served boaters for decades, and that autonomous navigation is an extension of this capability, combined with the ability to avoid collisions autonomously.
For newer boaters, autonomous technology is an easier proposition. “I’m a new boater, and I get nervous a lot,” Shin says. “We target new boaters. We want more people to enjoy boating.”
Then there is boating’s greatest equalizer. “People don’t like docking,” Cox says. “We’re never going to stop you from driving your boat, but it might be nice, if you’re coming into a dock and are getting stressed out, to switch it on.”
The wait won’t be long, either. While Avikus is paralleling the automotive sector, Cox and Shin expect NeuBoat technology to navigate and dock recreational vessels sooner than cars. “People will be surprised with how quickly we will get to market,” Cox says.
Having experienced Tesla’s Mad Max mode and Avikus’ level-three-plus sea trials,
I can say that far less adrenaline is involved watching a demonstration boat dock itself than when I pawed for a nonexistent passenger-side brake pedal in my buddy Allan’s Tesla.
Better Optics
While NeuBoat Dock uses six 360-degree cameras, they only work for daytime operations. The obvious move is to add thermal-imaging cameras, and Raymarine’s parent company, Teledyne, owns FLIR. Thermal-imaging cameras would add cost, but Cox says these sophisticated optical sensors could be included aboard higher-end NeuBoat installations.
