Cameras That Only See What You Want Them to See

UCLA scientists designed a camera that can't physically capture images of any object it hasn't been trained to image.

Cameras can do a lot of cool stuff when paired with algorithms. A camera can tell me when the apples have bloomed in the orchard and are ready to harvest or scan my license plate when I enter a parking garage, rather than making me take a ticket. But the increasing use of this kind of imaging has raised privacy concerns. Is it worth recording the image of every worker and visitor to the orchard just to see if the apples are ripe? Does the license plate scanner also need to record me and the kind of car I drive? Solutions to this usually involve blurring out items or encrypting images. But in those cases, the underlying image has still been captured, leaving a small but existent privacy risk. What if you could stop the camera from capturing anything but exactly what you wanted? What if the orchard camera saw only the apples? What if the parking garage camera could only see license plate numbers?

A paper published in the journal eLight presents a smart camera design from scientists at UCLA that only images desired objects, while instantly erasing other objects, all without any digital processing. It's done in the structure of the camera itself.

The scientists have built cameras with multiple transmissive surfaces each with tens of thousands of features that diffract light at wavelength scale. As it reflects light along through the lens, it changes the wavelengths. Some are passed as is, some are altered. The structure of the surface is designed by a deep learning algorithm to modulate the phases of light so that only desired objects get imaged.

So let's put this another way. You use a deep learning algorithm to figure out the structure that works for the thing you want to capture. Let's say it's an apple. The algorithm gets trained to know what an apple looks like. Pretty easy stuff for algorithms these days. The algorithm then designs the layers of the camera so that only wavelengths associated with what an apple looks like, get passed along unaltered. Everything else is modified. You then make those layers and put them together into a camera. When you use that camera, only apples will show up clearly in high quality, other objects just show up as noise.

This isn't just theoretical either. The UCLA team trained an algorithm to recognize a handwritten number "2." Written by anybody. Then they 3D printed diffractive layers based on that algorithm. The camera would not show any other digits. So if you wrote 2201. The image would only show 22. For their next trick, they created a lens that only showed pairs of pants. Possibly useful for a clothing store inventory system that wanted to protect the privacy of its workers and shoppers. Both cameras worked even when they changed up the lighting. And finally they built encryption into the structure of the lens. So no processing was needed to deliver an encrypted image.

One of the biggest advantages of this kind of camera is power efficiency. The camera itself is passive and requires no energy to perform its computations on the light. And because it's just reflecting light, it, by definition, works at the speed of light. The only power you need is for lighting and recording the images.

The scientists used a monochrome camera for the proof of concept but they believe the diffraction design could be scaled down enough to handle the visible spectrum as well.