Light field cameras are a new paradigm in imaging technology with the potential to greatly augment computer and robotic vision. Unlike conventional cameras that only capture an image from a single perspective, light field cameras instantaneously capture multiple images of the same scene from slightly different perspectives, which may help deal with occlusions, highly reflective surfaces, and refractive objects that often break most modern robotic vision techniques. Unfortunately, there is a scarcity of commercially-available light field cameras appropriate for robotic applications—none deliver light fields at video frame rates within a sufficiently small size and mass. Creating a full camera array comes with synchronization, bulk and bandwidth issues.
Centre Researchers were able to develop a light field camera using mirrors of different shapes and orientations to reflect the scene into an upwards-facing camera, to create an array of virtual cameras with overlapping field-of-view at specified depths. Our custom mirror-based light field camera adapter, called the MirrorCam, allows video frame rate light fields with a design that is cheap, simple in construction and accessible.
The MirrorCam was mounted onto a robotic arm and used in the first-ever visual servoing experiments with a light field camera. We showed that our method outperformed conventional monocular and stereo image-based visual servoing under field-of-view constraints and occlusions. Current research is focused on how to use these developments to servo towards, and ultimately pick up and manipulate transparent objects using robotic vision.