Title
The convergence hypothesis: assessing (and improving) the similarity between deep nets and the human brain during language processing
Deep learning has recently allowed major progress in complex tasks such as summarization and completion. Do these models process language similarly to humans, and is this similarity driven by systematic principles? Here, we test whether the activations of artificial neural networks trained on (1) image/sound, (2) word and (3) sentence processing linearly map onto human brain responses to written words, as recorded with magnetoencephalography (MEG, n=204), functional Magnetic Resonance Imaging (fMRI, n=589), and intracranial electrodes (n=176 patients, 20K electrodes). Our results reveal that visual (or sound), word and language algorithms respectively correlate with distinct areas and dynamics of the left-lateralized cortical hierarchy of reading (or speech processing, respectively). However, only specific subsets of these algorithms converge towards brain-like representations during their training. These results start to reveal what structural and training principles lead deep nets to converge to brain-like processes. Overall, this approach allows us to precisely track and dissociate the perceptual, lexical and compositional representations generated during reading in each cortical region.
Biography
I am a CNRS researcher at Ecole Normale Supérieure, currently in detachment at Facebook AI Research. My group focuses on understanding the computational bases of human cognition. For this, we develop paradigms, methods and models to analyze brain activity.