Language and Cognitive Neuroscience

CGSC-890, Spring 1999
Instructor: Colin Phillips
Location: Gore 308 and #202, 46 E. Delaware Ave
email: colin@udel.edu
Home Page: http://www.ling.udel.edu/colin/courses/cgsc890

This course provides a survey of the state-of-the-art in the cognitive neuroscience of language, focusing on areas which directly connect with research questions in linguistics and psycholinguistics. In particular, the course will pay attention to what is known about how the human brain supports successful linguistic computation.

Requirements

• Class participation (everybody)
  Each week there will be one main article, plus additional readings. Everybody is expected to read at least the main article, and come prepared to discuss it. Individuals will also be responsible for leading discussion of the other articles.
  If you are registered for the course for credit, then you should write a brief summary/commentary on the articles that you present. This need not be long (1-2 pages), but it should be carefully written. Email the text of your commentary to colin@udel.edu before class, and bring along copies to class.
• Papers (registered students only)
  2 squibs (short written papers), one due before spring break, one due on the last day of classes
  Possible topics include (i) critical review of a series of related articles, (ii) design of an experiment or computational model to test a specific question.

Readings

There is no textbook for the course. Readings are collected from a variety of sources, particular recent journal articles. Readings will be made available for photocopying in a course mailbox in the linguistics department mail room at #46 E. Delaware Ave. If you take the readings away to photocopy, you must return them promptly. You can make the copies in the linguistics department by arranging with Jane Creswell to set up a copying account.

---

Course Outline

February 9th - Introduction: Objectives of Cognitive Neuroscience of Language

February 16th - Aphasiology: syntactic comprehension and grammaticality judgments

February 23rd - Aphasiology: automatic and controlled processing

March 2nd - Aphasiology: resource limitations

March 9th - Lesions to humans and networks

March 16th - Hemodynamic Imaging

March 23rd - Electrophysiology: violation paradigms I

March 30th - Spring break

April 6th - Electrophysiology: violation paradigms II

April 13th - Electrophysiology: beyond violations

April 20th - Implementation: neural networks

April 27th - Implementaton: The need for symbols

May 4th - Implementation: Symbolic connectionism

May 11th - (tentative) "Field Trip" to U. of Maryland Neurolinguistics Lab

May 18th - Conclusions

---

Detailed Outline

This outline is of course likely to change as the term progresses.

February 9th - Introduction: Objectives of Cognitive Neuroscience of Language

Suppose that we were not limited by time, technology or money ... what would we count as having answered the main questions for a Cognitive Neuroscience of Language?

Neurolinguistics Background

• "Language and the brain." Chapter 8 of M. Gazzaniga, R. Ivry & G. Mangun. (1998). Cognitive Neuroscience: the Biology of the Mind. New York: Norton.

Neuroscience Background

• Chapters 1-3 of Gazzaniga, Ivry & Mangun (especially chapter 2).

It's hard to provide any general linguistics background material that would be of general use for the course. Fortunately, though, most of the papers that we will be reading fill in necessary linguistic background.

Specific Articles Discussed

• Sakai, K. & Y. Miyashita. (1991). Neural organization for the long-term memory of paired associates. Nature, 354, 152-155.
• Steinschneider, M, C. Schroeder, J. Arezzo & H. Vaughan. (1995). Physiologic correlates of the voice onset time boundary in primary auditory cortex (A1) of the awake monkey: temporal response patterns. Brain and Language, 48, 326-340.
• Singer, W. (1995). Time as coding space in neocortical processing. In M. Gazzaniga (ed.), The Cognitive Neurosciences. Cambridge, MA: MIT Press, pp.91-104.
• Ullman, M., S. Corkin, M. Coppola, G. Hickok, J. Growdon, W. Koroshetz & S. Pinker. (1997). A neural dissociation within language: evidence that the mental dictionary is part of declarative memory, and that grammatical rules are processed by the procedural system.

February 16th - Aphasiology: syntactic comprehension and grammaticality judgments

Syntactic Comprehension in Aphasia

• Main article: Zurif, E. (1995). Brain regions of relevance to syntactic comprehension. In L.Gleitman & M. Liberman (eds.), An Invitation to Cognitive Science, Vol. 1, 2nd edn. Cambridge, MA: MIT Press.
• Grodzinsky, Y. (1995). A restrictive theory of agrammatic comprehension. Brain and Language, 50, 27-51.
• Hickok, G. & S. Avrutin. (1995). Representation, referentiality and processing in agrammatic comprehension: two case studies. Brain and Language, 50, 10-26.
• Blumstein, S., G. Byma, K. Kurowski, J. Hourihan, T. Brown & A. Hutchinson. (1998). On-line processing of filler-gap constructions in aphasia. Brain and Language, 61, 149-168.
• Balogh, J., E. Zurif, P. Prather, D. Swinney, L. Finkel. (1998). Gap-filling and end-of-sentence effects in real-time language processing: Implications for modeling sentence comprehension in aphasia. Brain and Language, 61, 169-182.

Grammaticality Judgments

A classic article by Linebarger and colleagues claimed that agrammatic aphasics are able to make grammaticality judgments using precisely the kinds of linguistic knowledge that they apparently fail to use in comprehension and production. This is controversial.

• Grodzinsky, Y. & L. Finkel. (1998). The neurology of empty categories: aphasics' failure to detect ungrammaticality. Journal of Cognitive Neuroscience, 10, 281-292.
• Linebarger, M., M. Schwartz & E. Saffran. (1983). Sensitivity to grammatical structure in so-called agrammatic aphasics. Cognition, 13, 361-393.
• Shapiro, L., B. Gordon, N. Hack & J. Killackey. (1993). Verb argument structure processing in complex sentences in Broca's and Wernicke's aphasia. Brain and Language, 45, 423-447.

February 23rd - Aphasiology: automatic and controlled processing

• Main article: Blumstein, S. (1997). A perspective on the neurobiology of language. Brain and Language 60, 335-346.
• Milberg, W. & S. Blumstein. (1981). Lexical decision and aphasia: evidence for semantic processing. Brain and Language 14, 371-385.
• Blumstein, S., M. Burton, S. Baum, R. Waldstein & D. Katz. (1994). The role of lexical status on the phonetic categorization of speech in aphasia. Brain and Language 46, 181-197.
• Milberg, W., S. Blumstein, D. Katz, F. Gershberg & T. Brown. (1995). Semantic facilitation in aphasia: effects of time and expectancy. Journal of Cognitive Neuroscience 7, 33-50.
• Hagoort, P. (1993). Impairments of lexical-semantic processing in aphasia: evidence from the processing of lexical ambiguities.
• Hagoort, P. (1997). Semantic priming in Broca's Aphasics at a short SOA: no support for an automatic access deficit. Brain and Language, 56, 287-300.
• Tyler, L.K., R. Ostrin, M. Cooke & H. Moss. (1995). Automatic access of lexical information in Broca's aphasics: against the automaticity hypothesis. Brain and Language, 48, 131-162.

March 2nd - Aphasiology: resource limitations

• Haarmann, H. & H. Kolk. (1991). A computer model of the temporal course of agrammatic sentence understanding: the effects of variation in severity and sentence complexity. Cognitive Science 15, 49-87.
• Cornell, T. (1995). On the relation between representational and processing models of asyntactic comprehension. Brain and Language, 50, 304-324.
• Blackwell, A. & E. Bates. (1995). Inducing agrammatic profiles in normals: evidence for the selective vulnerability of morphology under cognitive resource limitation. Journal of Cognitive Neuroscience, 7, 228-257.
• Swaab, T., C. Brown & P. Hagoort. (1997). Spoken sentence comprehension in aphasia: event-related potential evidence for a lexical integration deficit. Journal of Cognitive Neuroscience 9, 39-66.
• Swaab, T., C. Brown & P. Hagoort. (1998). Understanding ambiguous words in sentence contexts: electrophysiological evidence for delayed contextual selection in Broca's aphasia. Neuropsychologia, 36, 737-761.

March 9th - Lesions to humans and networks

There have been a number of recent attempts to simulate aphasic symptoms by lesioning connectionist networks. We will discuss a couple of articles in this area, and also experiment with this ourselves, using material from Plunkett & Elman's recent set of connectionist simulations "Exercises in Rethinking Innateness".

• Main article: Dell, G., M. Schwartz, N. Martin, E. Saffran & D. Gagnon. Lexical access in aphasic and nonaphasic speakers. Psychological Review, 104, 801-838.

March 16th - Hemodynamic Imaging

• Bavelier, D., D. Corina, P. Jezzard et al. (9 other authors). (1997). Sentence reading: a functional MRI study at 4 tesla. Journal of Cognitive Neuroscience, 9, 664-686.
• Just, M., P. Carpenter, T. Keller, W. Eddy & K. Thulborn. (1996). Brain activation modulated by sentence comprehension. Science, 274, 114-116.
• J. Binder, J. Frost, T. Hammeke, S. Rao & R. Cox. (1996). Function of the left planum temporale in auditory and linguistic processing. Brain, 119, 1239-1247.
• K. Pugh, B. Shaywitz, S. Shaywitz, R. Constable, P. Skudlarski, R. Fulbright, R. Bronen, D. Shankweiler, L. Katz, J. Fletcher & J. Gore. (1996). Cerebral organization of component processes in reading. Brain, 119, 1221-1238.
• Neville, H. & D. Bavelier. (1998). Neural organization and plasticity of language. Current Opinion in Neurobiology, 8, 254-258.
• J. Binder, J. Frost, T. Hammeke, R. Cox, S. Rao & T. Prieto. (1997). Human brain language areas identified by functional magnetic resonance imaging. Journal of Neuroscience, 17, 353-362.
• H. Neville, D. Bavelier, D. Corina, J. Rauschecker, A. Karni, A. Lalwani, A. Braun, V. Clark, P. Jezzard & R. Turner. (1998). Cerebral organization for language in deaf and hearing subjects: biological constraints and effects of experience. Proceedings of the National Academy of Sciences of the USA, 95, 922-929.
• B. Rosen, R. Buckner & A. Dale. (1998). Event-related functional MRI: past, present and future. Proceedings of the National Academy of Sciences of the USA, 95, 773-780.
• M. D'Esposito, E. Zarahn & G. Aguirre. (1999). Event-related functional MRI: implications for cognitive psychology. Psychological Bulletin, 125, 155-164.

March 23rd - Electrophysiology: violation paradigms I

Semantics/lexical integration:

• Main article: Chwilla, D., C. Brown & P. Hagoort. (1995). The N400 as a function of the level of processing. Psychophysiology, 32, 274-285.
• Kutas, M. (1997). Presidential Address, 1996 - Views on how the electrical activity that the brain generates reflects the functions of different language structures. Psychophysiology, 34, 383-398.
• Nigam, A., J. Hoffman & R. Simons. (1992). N400 to semantically anomalous pictures and words. Journal of Cognitive Neuroscience, 4, 15-22.
• Helenius, P., R. Salmelin, E. Service & J. Connolly. (1998). Distinct time courses of word and context comprehension in the left temporal cortex. Brain, 121, 1133-1142.
• Van Petten, C. & Kutas, M. (1991). Influences of semantic and syntactic context on open- and closed-class words. Memory & Cognition 19, 95-112.

April 6th - Electrophysiology: Phonology

Phonology

• Näätänen, R., A. Lehtoskoskl, M. Lennes, M. Cheour, M. Huotilainen, A. Ilvonen, M. Vainio, P. Alku, R. Ilmoniemi, A. Luuk, J. Allik, J. Sinkkonen & K. Alho. (1997). Language-specific phoneme representations revealed by electric and magnetic brain responses. Nature, 385, 432-434.
• Dehaene-Lambertz, G. (1997). Electrophysiological correlates of categorical phoneme perception in adults. Neuroreport, 8, 919-24.
• Phillips, C., A. Marantz, E. Yellin, T. Pellathy, M. McGinnis, K. Wexler, D. Poeppel & T. Roberts. (1999). Auditory cortex accesses phonological categories: an MEG mismatch study. Submitted.

April 13th - Electrophysiology: Morphology & Syntax

Morphology

• Münte, T., T. Say, H. Clahsen, K. Schiltz & M. Kutas. (1999). Decomposition of morphologically complex words in English: evidence from event-related brain potentials. Cognitive Brain Research, 7, 241-253.
• Osterhout, L, M. Bersick & R. McKinnon. (1997). Brain potentials elicited by words: word length and frequency predict the latency of an early negativity. Biological Psychology, 46, 143-168.

Syntax

• Main article: Friederici, A.D. (1995). The time course of syntactic activation during language processing: a model based on neuropsychological and neurophysiological data. Brain and Language, 50, 259-281.
• Coulson, S., J. King & M. Kutas. (1998). Expect the unexpected: event-related responses to morphosyntactic violations. Language and Cognitive Processes, 13, 21-58.
• Osterhout, L. & P. Hagoort. (1999). A superficial resemblance does not necessarily mean you are part of the family: counterarguments to Coulson, King & Kutas (1998) in the P600/SPS-P300 debate. Language & Cognitive Processes 14, 1-14.
• Coulson, S., J. King & M. Kutas. (1998). ERPs and domain specificity: beating a straw horse. Language & Cognitive Processes 13, 653-672.
• Gunter, T. & A. Friederici. (1999). Concerning the automaticity of syntactic processing. Psychophysiology, 36, 126-137.
• Steinhauer, K., K. Alter & A. Friederici. (1999). Brain potentials indicate immediate use of prosodic cues in natural speech processing. Nature Neuroscience, 2, 191-196.

April 20th - Alternatives to 'surprise' paradigms in electrophysiology

Although most work using electrophysiological paradigms has been based on different measures of 'surprise' such as the N400 and the mismatch response, a growing body of work has tried to use more powerful methods of analyzing brainwaves to investigate language processing.

• Main article: Levelt, W., P. Praamstra, A. Meyer, P. Helenius & R. Salmelin. (1998). An MEG study of picture naming. Journal of Cognitive Neuroscience, 10, 553-567.
• Salmelin, R., R. Hari, O. Lounasmaa & M. Sams. (1994). Dynamics of brain activation during picture naming. Nature, 368, 463-465.
• Eulitz, C., B. Maess, C. Pantev, A. Friederici, B. Feige & T. Elbert. (1996). Oscillatory neuromagnetic activity induced by language and non-language stimuli. Cognitive Brain Research, 4, 2748-2755.
• Sarnthein, J., P. Rappelsberger, G. Shaw & A. von Stein. (1998). Synchronization between prefrontal and posterior association cortex during working memory tasks in humans. PNAS.
• van Turennout, M., P. Hagoort & C. Brown. (1998). Brain activity during speaking: from syntax to phonology in 40 milliseconds. Science, 280, 572-574.

April 27th - Implementation: Neural Networks

Simple recurrent networks have received a good deal of attention in recent connectionist accounts of language structure. We will read a couple of articles and try out a demonstration from the "Exercises in Rethinking Innateness" book.

• Main article: Elman, J. (1993). Learning and development in neural networks - the importance of starting small. Cognition, 48, 71-99.
• Elman, J. (1990). Finding structure in time. Cognitive Science, 14, 179-211.
• Christiansen, M. & N. Chater. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science.
• Plunkett, K. & J. Elman. (1997). Exercises in Rethinking Innateness. Cambridge, MA: MIT Press (Chapters 8, 12)

May 4th - Implementation: Arguments for Symbolic Computation

• Main article: Marcus, G.F. 1998. Rethinking eliminative connectionism. Cognitive Psychology, 37, 243-282.
• Marcus, G.F. 1999. The Algebraic Mind. MIT Press. (draft excerpts, with response by Elman)

May 11th - Combining Symbolic and Connectionist Representations

• Main article: Holyoak, K. & J. Hummel. (1997). Distributed representations of structure: a theory of analogical access and mapping. Psychological Review 104, 427-466.
• Smolensky, P. (1988). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11, 1-23.
• Shastri, L. & V. Ajjaganadde. (1993). From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony. Behavioral and Brain Sciences 16, 417-494.
• Steedman, M. (1999). Connectionist sentence processing in perspective. In: Special issue of Cognitive Science on Connectionist Approaches to Sentence Processing.

May 18th - Conclusions