The research team proposes non-clonable, invisible machine vision markers using cholesteric spherical reflectors

Over the past three decades, the digital world we access through smartphones and computers has become so rich and detailed that much of our physical world has a corresponding life in this digital reality. Today, physical and digital reality are steadily merging as robots, augmented reality (AR) and wearable digital devices invade our physical world, and physical objects receive their digital twin computer representations in the digital world.

Thanks to crypto technologies such as blockchains, these digital twins can be clearly identified and protected against manipulation. The trust these technologies provide is extremely strong, helping to fight counterfeiting, increase supply chain transparency and enable the circular economy. A weakness, however, is that there is no versatile and universally applicable identifier of physical objects that is as trusted as a blockchain. This breaks the connection between physical and digital twins, thereby limiting the potential of technical solutions.

In a new article published in Light: Science & Applicationsan interdisciplinary team of scientists led by Professors Jan Lagerwall (physics) and Holger Voos (robotics) from the University of Luxembourg, Luxembourg, and Prof. Mathew Schwartz (architecture, design of the built environment) from the New Jersey Institute of Technology, US, propose an innovative solution to this problem by giving physical objects unique and non-clonable fingerprints, realized using cholesteric spherical reflectors, or CSRs for short.

Lagerwall explains: “The unique feature of CSRs is that they are selective retroreflectors that return light to a source in any direction, but only within a narrow wavelength band and only with a specific circular polarization. The narrow wavelength band allows us to make CSRs that are undetectable to humans by locating the reflections in the near-infrared or near-ultraviolet regions, which are easily read by robots and AR devices but invisible to the human eye .

“Circular polarization allows us to separate only the CSR signals from any complex background, making any message written with CSRs stand out with exceptional contrast.”

By coating surfaces with CSRs arranged in a specific pattern, the team “fingerprints” physical objects. Because robots and AR devices can read the fingerprints with exceptional clarity and reliability, their operation becomes much more reliable when the CSR fingerprints encode information about the objects, giving the devices in question a much better understanding of their surroundings.

The ideal pattern in which the CSRs are arranged in the fingerprints are so-called fiducial markers, as Voos explains: “Fiducial markers are square-shaped binary patterns that are very similar in appearance to QR codes, which encode black-and-white pixels an identity, for example by telling a machine that reads the pattern that it is a door, a car, or a wall, and by measuring how large the pattern is and how distorted it is from the original based on the viewing angle square shape, the machine can determine how far away the object with the pattern is and how it is oriented with respect to the machine with very little computational effort.

Traditional black and white markers are common in robotics and AR research, but their size and high-contrast appearance make them impossible to use in most human-occupied spaces. They also need to be illuminated with visible light to be useful. For these reasons, they are now banished to research laboratories or restricted areas.

By manufacturing the markers with CSRs optimized for operation in the near-IR or near-UV, their performance in supporting robots and AR devices becomes accessible anywhere, including in spaces where people work, play, or rest as such CSR markers are invisible to humans and therefore do not disturb the environment as we experience it.

At the same time, the circular polarization of the CSR reflections highlights the CSR markers against any background for the machines that need to detect them, eliminating problems with false alarms and difficulty in detection.

Omnidirectional retroreflection is key to their usefulness, and Schwartz points out that “retroreflectors are common in motion detection, whether it’s tracking humans or robots, because no matter how the object being studied is moving, that’s how the retroreflector sends the light back.” to the camera also emits the light signal. However, normal retroreflectors reflect all the light, so they are very visible, which is why they are also used in traffic signs and high-visibility clothing.”

“By making datum marks with CSRs, a robot or AR device can track objects in its environment that bear the marks from any angle, day or night, by illuminating the scene with near-IR or near-UV light that is harmless.” and invisible to humans,” Schwartz continues.

A final important advantage of markers made using CSRs is that the exact local arrangement of each CSR is unpredictable and so difficult to reproduce that any CSR marker is effectively untraceable. Furthermore, when examining a CSR marker up close, its appearance depends on how one illuminates and observes it, which means that upon closer inspection, a given marker does not reveal a pattern, but in fact an infinite series of dynamic patterns unique to each Markers are unique yet well defined.

This turns CSR markers into so-called Physical Unclonable Functions or PUFs, which are very powerful at authenticating physical objects.

This is the final key benefit of CSR markers, says Lagerwall, summarizing: “CSR markers work at multiple scales: from a distance, they reproducibly and reliably reflect a pattern we choose, specifically the pattern of fiducials that identifies, what category of object But when a robot or AR device manipulates an object, it can examine the marker up close, revealing the locations of individual CSRs and examining how they reflect, for example, light sent in different directions or over different areas by on – and turning off individual LEDs in a standard lighting ring.”

“In contrast to the far-field marker pattern, the resulting near-field patterns are unique to that particular object and thus also give it its identity. In this way, CSR markers allow robots and AR devices to securely connect the physical world with the digital twin, because they tell the machine not only what the object is, but actually what it is acts. And since the optics are outside the visible spectrum, we humans don’t notice that anything has changed.”