Modeling Individual Differences in Human Object Recognition with Deep Neural Networks (DSI-SRP)
This DSI-SRP fellowship funded Samuel Lee to work in the laboratory of Professor Thomas Palmeri in the Department of Psychology during the summer of 2020; Sam has majors in French Horn Performance, Computer Science, Mathematics and a minor in Data Science.
The project funded by this fellowship aimed to understand how and why neural representations of objects might differ from person to person. Object recognition was modeled using deep convolutional neural networks (CNN) that are inspired by the structure of the primate visual system. CNNs now often match and sometimes exceed human levels of visual recognition performance and are the basis for image recognition now regularly found on web sites and in consumer electronics. CNNs are now also used to understand human and non-human primate vision. What kinds of variability in visual representations are possible within CNN models (and can the variability in these models inform our understanding of the variability in human object recognition performance)? As an initial baseline on variability in representations within CNNs, Sam used techniques like representational similarity analysis (RSA) to assess differences in network representations at different layers of the pre-trained network caused by differences in the inputs to the networks (objects varying in location, orientation, contrast, and the like). Sam also trained new networks that varied in the most minimal fashion, via differences in the randomization of the initial weights (with the same sequence of training images) and in the randomization of the order of training images (with the same initial weights). The combination of these aspects of CNN modeling allowed comparisons between representational differences caused by variation in the format of the input image and differences cause by variation in network training.
In addition to receiving support through a DSI-SRP fellowship, this project was supported and facilitated by the DSI Data Science Team through their regular summer workshops and demo sessions.