New Paper on Hybrid, "Bio-informed" Computing

We’re pleased to announce that our new paper that was recently accepted to be published IEEE Transactions in Pattern Analysis and Machine Intelligence, a top venue in machine learning and computer vision. This work was a collaborative effort with the lab of Ken Nakayama (from the Department of Psychology), and it exemplifies the kind of boundary-crossing, interdisciplinary work that my lab likes to do.

In this work, we describe a new machine learning technique that we call “perceptual annotation”, wherein we use carefully measured data on human perceptual performance to constrain and guide a machine learning algorithm. At a high level, our technique works by teaching machines to make mistakes more like those that a human would make, and in doing so, we produce a more robust system that is better able to generalize to new examples. As a proof of concept, we describe a face detection application of technology and show that it achieves state-of-the-art performance, especially in conditions that are traditionally difficult for machine vision algorithms (unusual angles, occluded faces, out of focus images, etc.).

While this initial foray into perceptual annotation was launched using human behavioral measurement, we think that the sky is the limit on what kinds of biological data can be used to guide machine learning. We’re excited to keep pushing this work in new directions!

In the spirit of open science, we’ve started a new website, perceptualannotation.org, where we’ll be distributing code and data related to this project.