Machine Vision: How a Simple Hardware Hack Could Replace Thousands of Lines of Code

In an increasingly digital world, it’s small wonder that we’re constantly searching for ever-more-sophisticated ways to interact with photographs and images: designers scan a 3D prototype and import its dimensions into a computer; medical programs image an internal organ and delineate the tumor to be removed; robots avoid drop-offs by recognizing the shapes that stairs make on their detectors; teenagers transform their selfies into what appears to be a pencil sketch.

Although these tasks range widely in scope, they all rely on a computer being able to accurately judge the boundaries between different parts of an image, like the line that separates a check from its background when you use a mobile deposit app. Currently the technology exists to allow for each of these applications, but professionals say that it leaves a lot to be desired.

To begin with, most methods of edge detection use complicated computer algorithms that require large amounts of processing power, making it difficult to use the results in real time. Even devices that use physical rather than computational means to enhance the contrast between different objects, such as the Nomarski interference prism or metamaterials, rely on highly specialized (not to mention expensive) equipment that puts them out of reach for many individuals. “These methods are either bulky and complex or challenging for precise fabrication,” says Zhejiang University researcher Zhichao Ruan.

That’s why Ruan—and a network of collaborators spanning four Chinese universities—decided to take a step back from all the fancy technologies and focus instead on basic equipment found in every optics lab: mirrors, lenses and polarizers.

Although everyone's familiar with mirrors and lenses, polarizers—which selectively cut out certain types of light—are also common in daily life. To understand how they work, think about light in its wavelike form. Much like a rope that can be shaken up and down, side to side, or any combination of the two, light can also take any number of orientations. A polarizer acts something like a vertical or horizontal fence that only allows certain orientations of light to pass through. This is how polarizing sunglasses cut down on road glare without dimming everything else too much—horizontal surfaces mostly reflect horizontally-polarized light.

This diagram demonstrates the basic principle behind a polarizer. The incoming light (to the left of the image) is a jumble of many light waves oriented randomly. The polarizer (center) blocks all orientations except the vertical one, resulting in a cleaner measurement at the end. Humans don’t distinguish between different polarizations (although some animals do), but the final image will appear dimmer since so much of the signal was blocked by the polarizer.
Image Credit: Fffred~commonswiki via Wikimedia Commons

Generally speaking, if you line up two polarizers at 90° to each other, no light will pass through. To understand this, just think about the image above; the light that passes through the polarizer all fluctuates in the vertical direction. Now imagine adding a second polarizer, rotated 90°. This would only allow horizontally polarized light to pass through—but that has already been blocked out by the first polarizer! As a result, all light is completely blocked.

Two polarizers lined up perpendicularly typically block all light that tries to pass through them.
Image Credit: NielsB via Wikimedia Commons.

Well, almost. That’s certainly the case when the polarizers are stacked, as in the image above, but the research team found that if they placed a mirror between the polarizers, shown below, that wasn’t the whole story. “Despite the common knowledge in textbooks that light cannot pass through two orthogonal polarizers,” Ruan says, “there is light passing through and it corresponds to optical computation of spatial differentiation.” In other words, not only does light pass through the second polarizer, but it results in a remarkably clean edge detection!

This schematic shows how an initial image, left, passes through first a vertical polarizer and then a horizontal one, bouncing off a mirror in the middle. The output image is a neat outline of the original!
Image Credit: Zhichao Ruan

Although it might seem like magic, the key lies in that intermediate reflection step. As the light reflects off the mirror, a property known as the spin Hall effect of light forces the orientation of light to rotate slightly in a direction that depends on the specific quantum properties of each photon. This transforms what was previously a uniform polarized light source into a superposition of precisely oriented “bunches” of light that can recombine and interfere with each other as they arrive at the second polarizer.

To fully explain the outline effect, the researchers had to delve far into the mathematics of wave optics. Ultimately, they showed that for a setup like theirs, the intensity of the output light is proportional to the change in intensity at that region for the input image. In other words, if a single pixel is roughly as bright as the pixels on either side of it, the output for that pixel will be quite dim. If, on the other hand, the pixel is right on the border of a shape—meaning its leftmost neighbor is much dimmer than its rightmost—it will shine brightly on the output image.

There is a catch though: this differentiation only works in a single direction. “The edge detection can only be performed along the direction perpendicular to the incident plane,” Ruan explains. This makes it impossible to detect purely vertical and purely horizontal lines simultaneously. Nevertheless, they managed to produce some stunning results, published last week in the American Physical Society journal Physical Review Applied.

A few of the researchers' experimental results are displayed above with the input image on the left and the final result on the right. Although the outlines show up clearly, in the first image vertical lines are omitted and in the second horizontal lines are. Even so, it is easy to make out the shapes of the original images. The lowest set of images shows the impressive spatial resolution of this system; it can resolve parallel lines with a separation as small as 1.6 microns, or about 1/5th the diameter of a human red blood cell!
Image Credit: Zhu et al.

Interestingly, they found that it doesn’t really matter what the mirror is made of; in fact, the central component can even be a refractor like a prism! The important part is that it forces the light to bend in some way for the spin Hall effect to do its magic.

Although they aren’t the first to achieve edge detection by a long shot, it is incredible to think that the results of their minimalistic optics setup can rival the most sophisticated of algorithms. Given all that we’ve done with relatively clunky technology so far, who knows what could be on the horizon?

—Eleanor Hook