A crab-inspired artificial vision system for terrestrial and aquatic environments

To navigate efficiently through real-world environments, robots typically analyze images captured by imaging devices built into their bodies. Therefore, to improve the performance of robots, engineers have tried to develop various types of high-performance cameras, sensors, and artificial vision systems.

Many artificial vision systems developed to date are inspired by the eyes of humans, animals, insects and fish. These systems have different characteristics and properties depending on the environment in which they are intended to be used.

Most existing sensors and cameras are designed to work either on the ground (ie in terrestrial environments) or in water (ie in aquatic environments). On the other hand, bio-inspired artificial vision systems that can function in both terrestrial and aquatic environments remain scarce.

Researchers from the Institute for Basic Science (IBS), Seoul National University, the Gwangju Institute of Science, the Massachusetts Institute of Technology (MIT) and Technology, and the University of Texas at Austin recently developed a new crab-inspired vision system that can work Both on the ground and in the water. This amphibian system featured in an article published in nature electronicsgives robots a 360° panoramic view of their surroundings, allowing them to recognize obstacles and navigate environments more effectively.

“Previous work (including our group’s research) on wide field of view (FoV) cameras was always less than 180°, which is insufficient for the ‘full’ panoramic view, and they were not suited to changing external environments,” Young Min Song, one of the researchers who conducted the study recounted TechXplore. “We wanted to develop a 360° FoV camera that could capture both air and water.”

The artificial vision system developed by this research team is inspired by the eyes of fiddler crabs. Also known as the caller crab, this unique species can get a full panoramic view of its surroundings without having to move its eyes and body. To artificially recreate the fiddler crab’s eyes, Min and his colleagues used a flat camera lens.

“If you use a conventional lens with a curvature for imaging, its focus will change when you immerse the lens in water,” Song explained. “On the other hand, if you use a lens with a flat surface, you can see a clear image regardless of the environmental conditions. The intertidal fiddler crab has such a flat surface on its lens, and we simply mimicked that crab – eye lens.”

To create their sophisticated vision system, the researchers integrated an array of flat, index-graded microlenses and an array of flexible, comb-shaped silicon photodiodes onto a spherical structure. The microlenses used can maintain their focal length regardless of changes in the external refractive index between air and water.

“To the best of our knowledge, this is the first time that amphibious and panoramic vision systems will be demonstrated anywhere in the world,” Song said. “Our vision system could pave the way for omnidirectional 360° cameras with applications in virtual or augmented reality, or all-weather vision for autonomous vehicles.”

Song and his colleagues tested their system in a series of optical simulations and imaging demonstrations, taking into account the properties of both terrestrial and aquatic environments. So far, they found that it produced very promising results, so it could soon be tested and implemented on several different hybrid and amphibious robots.

“In our next studies, we will make further technical developments to achieve higher resolution and superior imaging performance,” Song added. “In addition, we are still interested in developing a new type of camera with unique imaging capabilities inspired by other animal eyes.”

© 2022 Science X Network