Can Deep Neural Networks Model Variability in Human Visual Cognition (DSI-SRP)
This DSI-SRP fellowship funded David Brust to work in the laboratory of Dr. Thomas Palmeri, Ph.D. in the Department of Psychological Sciences during the summer of 2023. David is a rising senior with a major in Neuroscience.
Deep neural networks (DNNs) are used in neuroscience as models of brain representations and as models of human behavior in cognitive visual tasks. It is unknown if variability in human visual cognition can be modeled through variability in DNN representations. Different DNN factors (architecture, training set, hyperparameters) could be manipulated systematically in order to see if these sources of variation would create patterns of variation that parallel those seen in humans. However, due to the vast number of possible factors and the difficulty of training DNNs, this is practically impossible. Instead, David and his mentor explored a model ‘zoo’ of over 700 DNN models collected from online. They analyzed their representational similarity in order to test how different models compare, finding distributions of model similarity. A large impact of ‘expert’ models trained from existing models showed to have a significant impact on the distribution. Further, these model representations and how they can be used to perform visual cognition tasks can be analyzed to find if differences in DNNs produce variability like that seen in human brains and minds.
In addition to receiving support through a DSI-SRP fellowship, this project was supported and facilitated by the DSI Data Science Team through their regular summer workshops and demo sessions.