CC BY-NC-ND
0Cognitive Brain Research, 1 (1993) 183-192
© 1993 Elsevier Science Publishers B.V. All rights reserved 0926-64I0/93/S06.00
BRESC 30013
Event-related brain potentials during natural speech processing: effects of semantic, morphological and syntactic violations
Angela D. Friederici, Erdmut Pfeifer and Anja Hahne
Cognitive Science Lab Berlin, Institul fur Psychologic, Freie Unirersiial Berlin, Berlin (FRO
(Accepted 20 July 1993)
Key words: Event-related potential; N400; Auditory word processing; Semantic priming; Syntactic priming
The present study investigated different aspects of auditory language comprehension. The sentences which were presented as connected speech were either correct or incorrect including a semantic error (selectional restriction), a morphological error (verb inflection), or a syntactic error (phrase structure). After each sentence, a probe word was presented auditorily, and subjects had to decide whether this word was part of the preceding sentence or not. Event-related brain potentials (ERPs) were recorded from 7 scalp electrodes. The ERPs evoked by incorrect sentences differed significantly from the correct ones as a function of erroT type. Semantic anomalies evoked a 'classical' N400 pattern. Morphological errors elicited a pronounced negativity between 300 and 600 ms followed by a late positivity. Syntactic errors, in contrast, evoked an early negativity peaking around 180 ms followed by a negativity around 400 ms. The early negativity was only significant over the left anterior electrode. The present data demonstrate that linguistic errors of different categories evoke different ERP patterns. They indicate that with using connected speech as input, different aspects of language comprehension processes cannot only be described with respect to their temporal structure, but eventually also with respect to possible brain systems subserving these processes.
INTRODUCTION
An adequate description of language comprehension requires not only a definition of the different subsystems involved in this process, but also a specification of the temporal parameters under which these subsystems are active. Most comprehension theories agree that various functionally distinct components responsible for the processing of phonological, syntactic and semantic aspects are engaged in language comprehension. They differ, however, with respect to their assumption about whether and how these different subsystems interact in time.
Two distinct views have been formulated with respect to how these different components are activated during language comprehension. One position claims that the different components are activated in a serial fashion with the phonological and the syntactic analysis preceding the semantic interpretation1114-47. The alternative view holds that the different subcomponents, in
particular syntactic analysis and interpretation, interact on-line as the linguistic information enters the system31-34. The debate between these positions can thus be reduced to the question of when during language processing information from the different subsystems come together.
The specification of the temporal parameters under which the different subsystems interact has been subject of a number of recent behavioral studies. The existence of functionally distinct subsystems of the language system has been supported by reaction time experiments conducted with normal subjects10-15-41-45, as well as by those involving subjects with circumscribed brain lesions17-49. Ferreira and Clifton10, using measures of eye movements during reading, for example, provided evidence in support of distinct processing components for purely structural and interpretative processes. They found that syntactic processing strategies were applied even when those resulted in themati-cally based anomaly or when they conflicted with se-
Correspondence: A. Friederici. Institut für Psychologie, Freie Universität Berlin, Habelschwerdter Allee 45, D-14195. Berlin, FRG. Fax: (49) (30) 838-3824.
mantic discourse biases. These data are compatible with the comprehension account proposed by Fra-zier'2,14 assuming two temporally distinct processing stages. On-line comprehension studies conducted with subjects suffering from specific left hemisphere brain lesions suggest that structural processes are selectively affected by left anterior brain lesions161K. Friederici16, for example, has shown that patients with lesions in the Sroca's area, but not those with lesions in the Wernicke's area, are selectively slowed down when required to monitor function words in auditorily presented sentences.
Thus, it seems that a 'fine-grained look' at the temporal structure of the language comprehension process is compatible with a view proposed by Frazier12,14 assuming an initial stage during which the parser identifies the structure of the input on the basis of syntactic category information and a late stage during which final integration of thematic and structural information takes place. At this latter stage reanalysis of the initial parse may turn out to be necessary in case of a mismatch between the initial parse and the final interpretation is detected.
The critical aspect to be taken into account when modelling language comprehension, it appears, is the temporal structure of this process. The characterization of the temporal structure can be done most directly when investigating spoken language comprehension on-line, as speech unrolls as a sequence in time. The on-line measure, usually employed in psycholin-guistics, is the registration of reaction times either in a lexical decision, a word-monitoring or in a speech-shadowing task. These tasks, however, require a person's reaction to some given input and the measures are, therefore, separated in time from the actual perception process.
One way to investigate the time course of semantic and syntactic processes more directly is to register brain potentials. A number of recent studies have done so, mostly using a word-by-word visual presentation mode with pauses of different length between each word (for a review see refs. 28 and 48). Only a few studies used auditory language material1,21,32 or natural speech7,22 while registrating event-related brain potentials. None of these auditory studies, however, investigated syntactic aspects of language processing, they rather focussed on semantic aspects.
Event-related brain potentials
The electrical activity of the brain time-locked to the presentation of a stimulus, the so-called event-related potential (ERP), has been shown to be sensitive to a variety of sensory and cognitive processes includ-
ing language comprehension92"2*4*. With respect to language processes a specific negative component peaking around 400 ms (N400) after the onset of the target word presentation has been identified to vary as a function of the context25. It was demonstrated that the amplitude of the N400 for a sentence final word was a monotonic function of the cloze probability, i.e. the N400 was greater in amplitude for less predictable words than for more predictable words. The variation of the semantic context was shown to influence the N400 in word context21,35,43,44 and sentence context studies25,26,48. It was suggested that the amplitude of the N400 is sensitive to the build up of semantic constraints upon recognition/processing of the succeeding word2ti,M.
The attempts to find an equally specific correlate for syntactic processes have been less successful. The first studies that investigated the influence of syntactic structure on auditory word recognition using ERP measures were those by Brown and colleagues2-5. They presented subjects identical word forms which had two readings, a noun reading and a verb reading (e.g. fire). This form was presented in two different disambiguating contexts, a noun context (e.g. sit by the fire) or a verb context (e.g. ready, aim, fire). The identical word forms which were presented repeatedly in each of these two contexts produced different ERP patterns as a function of the preceding syntactic context. A principal component analysis revealed an early negativity at around 150 ms over the left hemisphere with a larger amplitude for the noun than for the verb context condition. This difference in amplitude was most prominent at left anterior sites.
Kutas and Hillyard26 set out to investigate the processing of morphosyntactically incorrect sentences during reading using event-related potential measures. With English as the experimental language, their incorrect sentences contained errors in number marking. They reported a tendency for these errors to be associated with increased negativity between 200 ms and 500 ms post stimulus with a slightly more frontally located maximum than the classical N400 effect. The effect, however, failed to reach the required reliability.
Miinte et al.36 investigated syntactic and mor-phosyntactic violations during reading using German as the experimental language. As morphosyntactic violations, they introduced case marking errors. Syntactic violations were realized in sentences containing phrasal continuations which were incorrect with respect to the word's syntactic class (noun vs. conjunction). Both types of violations were correlated with a negativity around 400 ms.
Two more recent studies analyzed ERP responses
with respect to the processing of syntactic information encoded in verbs, i.e. subcategorization information4042. Rosier et al.42 investigated the processing of correct and incorrect German sentences containing verbs which cannot be passivized (e.g. 'Der Lehrer wurde gesehen' (The teacher was seen) vs. 'Der Lehrer wurde geweint' (The teacher was cried)). Sentences were presented using a visual word-by-word presentation with interword intervals of 100 ms and 200 ms, respectively. Targets preceded by the syntactically incorrect context elicited a negative going wave peaking around 400 ms post-stimulus which was significant over the left and frontal electrode sites. The topography of this wave clearly differed from that elicited by a seman-tically incorrect context for a given past participle form (selectional restriction violation, e.g. The president was murdered vs. The milk was murdered) which was similar to those reported in the literature for semantic anomalies. Interpreting these data with respect to their temporal and topographic characteristics, the negativity peaking around 400 ms was taken to reflect the stage of full lexical access, as both types of violations elicited such a waveform in this time range. The particular topography was viewed to reflect the processing of semantic versus syntactic information encoded in the verb.
Osterhout and Holcomb40 studied violations of a verb's subcategorization frame (e.g. 'The banker persuaded to seli the stock') iit comparison to correct sentences (e.g. 'The banker decided to sell the stock'). They found a late (about 600 ms) positive deflection over parietal areas of the scalp in correlation with such violations. Note, that this component may not only be a reflection of a syntactic error, but also be correlated with a so-called garden-path effect, i.e. requirement of a structural reanalysis in these sentences which becomes evident only at the end of the sentence. A similarly distributed late positivity has also been observed for other types of syntactic anomalies, as for example, sentences containing the incorrect order of words in a phrase or inflectionally marked incorrect subject-verb agreement in Dutch sentences19. Given the onset polarity and topography, the late positivity could be viewed as a member of the P300 components usually elicited by unexpected events8-9.
Finally, Neville et al.37 studying different types of syntactic violations found each to be correlated with a different ERP pattern. Phrase structure violation produced a biphasic effect with a left frontal negativity peaking at around 400 ms and a parietal positivity peaking at around 700 ms. Specificity constraint violations evoked a sustained left frontal negativity emerging immediately after the critical word and peaking at
around 700 ms. Subjacency constraint violations produced a broad parietal late positivity similar to that observed by the studies mentioned above.
Thus, taken together, the data at hand suggest at least three different waveforms to be related to syntactic processing.
First, Brown and colleagues5 using auditory language material found an early left anterior negativity peaking at around 150 ms as a correlate for the processing of syntactic word category information in a principal component analysis.
Second, a number of studies report a left anterior negativity peaking at around 400 ms in correlation with a variety of different aspects of syntactic processing during reading for the processing of lexically bound syntactic information42, the processing of those elements that carry syntactic information, i.e. the closed class elements38, or the processing of syntactically licensed filler-gap dependencies24.
Third, some studies found a late parietal positivity at around 600 ms and later in relation to the processing of syntactic aspects during reading. Such a late positivity was observed for sentences in which the preferred syntactic reading did not allow plausible interpretations and required reanalyses (ref. 40; see also ref. 19).
These ERP data in their heterogeneity cannot easily be connected to current psycholinguistic models of language comprehension. Whether the heterogeneity is due to differences in the various aspects of syntactic processing tested, the different presentation modes (auditory vs. visual) or even languages used (English, German, Dutch) cannot be decided on the basis of the data at hand. However, when considering the temporal distribution of the reported effects across the different studies, we can state a difference between lexical-semantic effects clearly located around 400 ms and syntactic processes either being correlated with early negativities (around 150 ms and 400 ms) or with a late positivity (around 600 ms and later).
The present study
The present study was designed to address the question whether different aspects of language parsing become manifest in temporally and topographically distinct patterns of event-related potentials in the same subjects. In particular, we investigated the processing of three different types of information, semantic information, morphological information and syntactic information. Given the assumptions about early fast and automatic syntactic processes involved in the initial parse, we reasoned that these may be more likely to be observed when presenting the language material online. We therefore decided to present the stimulus
material not in a word-by-word manner but as running speech.
The prediction for the semantic violation condition was that it should elicit a classical N400 wave. The predictions for the other two types of violations were less straight forward, given the state of the art discussed above. Moreover, as we were interested in studying the processing of the different types of violation under the condition of natural speech input, the basis for clear predictions was even more sparse. The only study using natural speech as input while recording ERPs had only investigated semantic aspects of sentence processing32.
MATERIALS AND METHODS
Subjects
Sixteen young adults (eight male, eight female, mean age = 25.9 years» were paid DM 10.- per hour to serve as subjects. All but one were right handed as evaluated by the Edinburgh Inventory (Old-field, 1971). They were all native speakers of German with no known hearing deficit.
correct sentence
incorrect, semantic
: Der Finder]
(the Under waa rowanM)
: Die Wnikntowaflbegraben. (IM cloud watbuHed)
In the result fIgur*«)
belohnt
incorrect morphological : Das PafkBttWBfflbohnera.
(Ihe parquet waa poUsh)
incorrect syntactic : Der Freund
im besucht
(Uio Mend waa in №• visited)
Fig. 1. Examples of stimulus items for each condition. It displays one example of a sentence as an oscillogram, in which the first critical point, i.e. the beginning of the auxiliary 'wurde' and the second critical point, i.e. the beginning of the main verb are marked.
Materials
A set of 160 experimental sentences was constructed. These fell into 4 different categories: (I) 40 sentences were correct *, (2) 40 were semantically incorrect containing a selectional restriction violation, (3) 40 were morphologically incorrect containing a verb which was inflected incorrectly, and (4) 40 were syntactically incorrect containing a violation of the phrase structure. Examples of each sentences type are displayed in Fig. 1.
The selectional restriction error in sentence type (2) concerned a mismatch between the preceding noun and the sentence final verb. The morphological error (3) was caused by a mismatch between the preceding auxiliary marking passive voice and the sentence final verb form (1st person singular instead of a past participle form). In this condition phrase structure assignment is possible as a main verb is required in this position, however, in a different inflectional form. The violation of the phrase structure in sentence type (4) was due to a mismatch between the preceding head of the phrase (here the preposition indicating the beginning of a prepositional phrase requiring a noun or pronoun as the next word) and the following word in sentence final position (here an inflected verb form, a category which is syntactically illegal as successor after a preposition). All 40 exemplars of each sentence category were of the same type (i.e. article-noun-auxiliary-(preposition)-main verb) ranging in length between 6 and 9 syllables (article = 1 syllable, noun = 1 -3 syllables, auxiliary = 2 syllables, main verb = 2-3 syllables) or 10 syllables in case of those sentences containing a preposition. This corresponds to a mean sentence length of 1.7 s (correct sentences), 1.8 s (semantic condition), 1.7 s (morphological errors) and 1.9 s (syntactic condition). In all the sentences, the sentence final main verb form was the critical word. These 160 critical words were all two- or three-syllable words. The 80 two-syllable and the 80 three-syllable words were equally distributed ovet the different conditions. We chose to control for the critical words' and the sentences' length on the basis of a number of syllables, as a syllable can be considered to be a critical unit in speech perception4'01.
In addition to these experimental sentences, 80 correct filler sentences were added to the stimulus list with the following composi-
tion: 40 sentences contained a complete prepositioral phrase, i.e. preposition plus noun phrase (20 with full nouns, 20 wilh pronouns), and 40 sentences with the structure art-noun-auxiliary-past participle.
Each sentence was spoken by a female speaker. The speaker was trained to produce correct and incorrect sentences with normal intonation or a best approach. These were produced in analogy to correct sentences containing a complete prepositional phrase with a pronoun (e.g. incorrect: Der Freund wurde im besucht/The friend was in the visited; correct: Der Freund wurde von ihm besucht/The friend was by him visited)4"*. Sentences were first recorded on analogue tape and were then digitized (20 kHz, 12 bit resolution).
As the experiment used a probe verification paradigm with trials in which each sentence was followed by a pause of 800 ms after which a probe word was presented auditorily, the following trials were constructed. Half of the probe words were so-called true probes, i.e. words that had occurred in the sentence immediately preceding the probe. Words from each position could serve as a probe. This word position-probe relation was equally distributed over the entire set of the true probes for the correct sentences and the semantically and the morphologically incorrect sentences with a quarter each. For the syntactically incorrect sentences three quarters were filled with nouns, auxiliaries and main verbs. The fourth quarter consisted of articles and prepositions to equal parts. Half of the probe words were so-called false probes, as they occurred not in the previously heard sentence. False probes were morphological variants of words that had occurred in the sentence, e.g. an article with a different case or gender marking, a noun with a different number marking, an auxiliary with a different tense or number marking, and a main verb with a different tense or number marking. We chose to use morphological variants as probes in order to keep the auditory attention at a high level. Pretests had shown the present procedure to be successful. Again these four categories were equally distributed over the false probe set for the correct sentences and the semantically and morphologically incorrect sentences. For the syntactically incorrect sentences, three quarters of the probes fell in the three
* Thirty-seven of these sentences were of an average cloze probability of 0.43 (ranging from 0.10 to 0.90). As three of the original sentences had a cloze probability of zero, these were excluded from later analysis.
** Attemps were made to splice the auditory material of correct sentences and use these parts for the incorrect sentences. This procedure, however, resulted in most unnatural sounding material, we therefore decided to use the procedure described in the method section.
categories of noun, auxiliary and main verb. The last quarter consisted of articles and prepositions to equal parts.
These probe words were spoken as individual words by the same female speaker who had spoken the sentence material on analogue tape. Like the sentences, probe words were digitized and were stored in separate disk flies. Using this technique, trials with standard intcrstimulus intervals of 800 ms between sentence offset and probe word onset were constructed. This was true for all critical trials as well as for the filler trials. All sentences and probes were reassembled and output in a quasi random order on an analogue tape together with 10 practice trials preceding the experimental trials. Two differently randomized analogue audio tapes were constructed from the same material (set A and set B). Set A and set B differed in their combination of sentences and probe types: those sentences that were combined with a true probe in set A were combined with a false probe in set B, and vice versa.
Procedure
The probe verification paradigm required the subjects to listen to the auditorily presented sentence and a target following after a pause of 800 ms. They were asked to indicate whether or not the probe word had appeared in the preceding sentence. This response was delayed by the introduction of an interstimulus interval of 800
ms between the offset of the sentence and the onset of the probe preventing the motor response from contaminating the ERP to the final word.
Subjects were instructed to give their response as fast and as accurately possitle. After a silent pause ot 2(100 ms the next trial began. Throughout the task subjects wcte picst.iieii v.irh an oar'rr;? of a small black rectangle in front of him/her. They were told that the fixation of this rectangle would ease the required restriction of eye movements during the task. All subjects were confronted with the two tapes containing material set A and material set B in a fixed order.
ERP recording
The EEG was recorded from 7 scalp electrodes including 3 places of the standard international 10-20 system locations at the midline (Fz, Cz, Pz) and 4 non-standard locations: Broca's region (Bl was defined in the fronto-temporal region as crossing point between T3-Fz and F?-Cz) and the right hemisphere homologue (Br, i.e. crossing point betweci) T4-Fz and F8-Cz) and the Wernicke's region (Wl was defined in the posterior-temporal region as crossing point between T3-P3 and C3-P5) and the right hemisphere homologue (Wr, i.e. crossing point between T4-P4 and C4-P6). All scalp electrodes were referenced to linked mastoids. Eye movements and
Fig. 2. Grand averages of ERPs in the semantic violation condition. Superimposed are the ERPs obtained for the sentence final main verb (the baseline consists of the first 250 ms of the auxiliary 'wurde' not displayed here) in correct sentences (solid line) and in semantically incorrect (broken line) sentences. The panels with event-related activity are arranged according to the placement of the electrodes: left panels are 'Broca'-left (Bl), 'Wernicke'-left (Wl), middle panels are midline frontal (Fz), midline central (Cz), midline parietal (Pz); right panels are 'Broca'-right (Br) and 'Wernicke'-right (Wr). The vertical line indicates the beginning of the main verb 3. Negativity is up in this and the
following figures.
blink artifacts were monitored by the electrooculogram (EOG) recorded from electrodes placed above and beneath the left eye. All electrodes used were Ag/AgCI electrodes. All impedances were maintained below 5 kii. The EEG and EOG channels were amplified by ESMED amplifiers with 1.6 s time constant and a low pass filter at 70 Hz. -3 Db/octave roll-off. The EEG and EOG were recorded continuously for each block and were A/D converted with 12-bit resolution at a rale of 256 Hz. Data collection was controlled by a 386-1BM compatible computer. The EEG was digitized on-line and stored for later analysis.
Off-line separated ERPs were averaged for each subject at each electrode site from trials free of EOG artifact for the sentence final words in each condition. The rejected trials due to artefacts were distributed equally over the different conditions with a mean of 24.75rr (S.D. = 12.27) for the correct sentences, a mean of 25.00Tr (S.D. = 12.94) for the semantically incorrect sentences, a mean of 25.31*2 <S.D. = 12.11) for the morphologically incorrect sentences and a mean of 26.41'* (S.D. = 12.66) for the syntactically incorrect sentences.
For each sentence, two critical points were used to allow time-locked EEG averages for the sentence final words: the first being at the onset of the auxiliary 'wurde' and the second being at the onset of the main verb. The 250 ms following the first critical point were used to calculate the prcstimulus baseline. We choose these 250 of the auxiliary 'wurde' as the word which was closest in time to the critical word and present in the sentences of each condition. The auxiliary directly preceded the main verb in the correct, the semantically incorrect and the morphologically incorrect condition. In the
syntactically incorrect condition, however, the word directly preceding the main verb was always a preposition. In order to allow a direct comparison over the different conditions, prestimulus baseline was taken for the same word type, here the auxiliary 'wurde' (see Fig. 1).
The second critical point marked the onset of the sentence final word. ERPs to the sentence final words were quantified in three different latency windows (100-250 ms, 250-700 ms and /00-1200 ms). These latency windows were defined after visual inspection of the grand averages of the four different conditions, and the visual identification of the main differences in amplitudes.
Repeated measures analyses of variance (ANOVAs) were used to analyze these data. Factors in the analysis included electrode site (Fz, Cz. Pz, Bl, Bt, Wl, Wr), time epoch (100-250 ms, 250-700 ms, 700-1200 ms) and condition (correct, semantic violation, morphological violation, syntactic violation). In cases where specific predictions were made for final words, the overall ANOVA was followed by planned pairwise comparisons.
The general linear model procedure of SAS Institute Inc. was used for the statistical analyses. All tests were adjusted for violations of sphericity which is inherent in repeated measure analysis according to the formulas of Huynh and Feldt23.
RESULTS
As behavioral data, error rates were computed separately for each condition and probe type (true vs. false
Grand average: riORPHOSYNTAX (n-16)
- Correct
tlorph. tnc.
1 »1 I I ■ I 11 >U l I > f ■ ■ 11 I I I J n I I ! I II I I s
o 0.5 1 1.5
III | I I I l| I I I I |l I I I |l I I I | II I I I III I |5
0 0.5 I 1.5
£;0_A,CSU LUN ttxwn
1111111 111II1111
■ 11 ■ 1111 1111 s 1 1.5
11 11 ii 0
"I""!""!......."I5
0.5 1 1.5
I I I |l I I I |l I I I | II I I | I II I | 5
0.5 I 1.5
Fig. 3. Grand averages of ERPs in the morphological violation. Superimposed are ERPs obtained from the sentence final main verb (the baseline consists of the first 250 ms) of the auxiliary 'wurde' (not displayed here) in correct sentences (solid line) and in morphologically incorrect (broken
line) sentences. Layout as Fig. 2.
probe). Descriptive analyses showed that overall error rates amounted to 1.1% and distributed equally over conditions. Error rates were too low to be analyzed by means of inferential statistics.
The average voltage in the 250 ms following the first critical point (i.e. the onset of the auxiliary 'wurde') was examined for significant differences as a function of experimental condition. Since no systematic effects were found in an ANOVA analysis, this interval was used as a baseline for all further ERP amplitude measures.
Visual inspection of the grand average ERPs elicited by the final words per violation condition against the correct condition plotted it; Figs. 2-4 indicates different waveforms for each of the incorrect condition, with respect to their temporal and topographic distribution.
A more detailed inspection of the time course of the ERPs revealed a very early negativity peaking at 180 ms only for the syntactic condition. Although this negativity can be seen at Bl, Br and Fz, it was largest at BI
and virtually non-existent at the posterior electrodes (Fig. 4). In the time domain between 250-700 ms we observed a negative going wave for the semantically incorrect condition which had the clear distribution of the classical N400 (Fig. 2). Morphological errors also elicited a negativity around 400 ms, however, with a smaller amplitude than the 'semantic N400*, peaking somewhat earlier and merging into a positivity at around 600 ms (Fig. 3). The syntactically incorrect condition also elicited a negativity in this time domain which, however, was most prominent at anterior and frontal electrodes and basically absent at posterior electrodes (Fig. 4). Given these observations we chose three different time windows (i.e. time epochs 100-250 ms/250-700 ms/700-1200 ms) for the following calculations.
An overall ANOVA with the factors Condition (correct/ semantic/ morphological/ syntactic violation) x Time epoch (100-250 ms/250-700 ms/700-1200 ms) X Electrode (Fz, Cz, Pz, B!, Br, Wl, Wr) revealed a
line) sentences. Layout as Fig. 2,
main effect of Condition (F,45 = 3.71, P<0.02), a main effect of Time epoch (¿,,„=10.25, P< 0.001), and a main effect of Electrode (Fhgo = 17.42, P < 0.0001). All interactions were significant (Time epoch x Condition: Fft-W = 6.58, P< 0.0001; Time epoch x Electrode: Fl2 mi = 14.49, P < 0001; Condition X Electrode: F1X'270 = 2.71, P < 0.01; Time epoch X Condition X Electrode: F3iK540 = 4,34, P < 0.0001).
The local ANOVA for the time epoch 100-250 ms with the factor Condition (correct vs. syntactically incorrect) by Electrode revealed a significant main effect of Electrode (F6-W = 5.20, P < 0.003) and a significant interaction (F6go = 3.53, P < 0.03). Planned f-tests showed that the difference between the correct and the syntactically incorrect condition was only significant at B1 (fl5 = -2.21, P < 0.05).
The local ANOVA for the time epoch 250-700 ms with the factor Condition (correct, semantically incorrect, morphologically incorrect and syntactically incorrect) and Electrode (Bl, Br, Fz, Cz, Pz, Wl, Wr) revealed a significant main effect of Condition (F, 45 = 8.68, P< 0.0004), a main effect of Electrode (FhW = 8.54, P < 0.001) and a significant interaction (FlK 27„ -3.75, P< 0.001). This interaction was analyzed further by conducting planned f-tests. The difference between the'correct and the incorrect condition was significant for the semantically incorrect sentences at all electrodes (all P's <0.01, except for Wl with a P< 0.05). For the morphologically incorrect sentences this difference was reliable at the fronto-temporal electrodes (Bl, Br) and at Pz with f <0.05. For the syntactically incorrect sentences this negativity was only significant at the fronto-temporal electrodes (Bl, Br).
In order to approach the specific question whether the observed negativities for the syntactic violation condition in the first and in the second tine epoch (100-250 ms and 250-700 ms) are to be attributed to different generators, an additional analysis was conducted on normalized data. The normalization was carried out according to McCarthy and Wood33. This analysis over Time epoch (2 levels) and Electrodes (7 levels) revealed a significant interaction (FftW = 2.79, P < 0.05).
The local ANOVA for the time epoch 700-1200 ms with the factor Condition (correct vs. morphologically incorrect) and Electrode revealed no significant main effect or interaction. A look at the visually displayed waveforms suggested that this may be due to the fact that the negativity in the preceding time window only spanned between 250 and 600 ms and that the observable positivity started at 600 ms. Therefore an additional ANOVA was calculated using the time epoch 600-1200 ms. This ANOVA revealed a significant main
effect of Electrode (F6 go = 24.47, P < 0.001) and Condition by Electrode interaction (F6qo = 5.63, P < 0.001).
DISCUSSION
The three functionally different types of linguistic violations presented in connected speech each elicited ERPs which were clearly different in their temporal and topographic parameters. Semantic selectional restriction errors became manifest in a negative going effect in the time domain at around 400 ms with a broad distribution over both hemispheres similar to the classical N400 effect. The syntactic phrase structure errors realized as a violation of syntactic word category, in contrast, evoked an early negativity peaking around 180 ms with a maximum over frontal and anterior lateral electrode sites, most salient over the electrode 'Broca left'. This early negativity was followed by a second negative going wave peaking at around 400 ms which, however, was less widely distributed than the semantic N400 effect and had its maximum at frontal and anterior sites. The detection of a morphological violation evoked a negativity at around 400 ms with a maximum over anterior sites, followed by a weak late positivity with parietal and posterior lateral distribution.
The topography and the timing of the semantic ERP-effect as observed in the present study using auditory language input is in line with previous studies using visual and auditory presentation modes22,48.
Compared to the N400 effect as a correlate for semantic processes, syntactic effects are much less established. In the Introduction we discussed effects reported for each of the time domains under analysis in the present study. In the time domain between 100 and 250 ms we found an early negativity left anterior for the phrase structure violation induced by an incorrect word category. A similar early negativity was observed by Neville et al.37 in correlation with phrase structure violations. However, a negative going wave as early as 50-100 ms at anterior sites and most prominent in the left hemisphere was also found in correlation with semantic anomalies when presented in a connected speech mode22. The authors interpret their finding to reflect an interaction between the semantic and prosodic or coarticulatory cues. Connolly, Stewart and Phillis7 investigating the processing of high and low probability words in spoken sentences, found both an early negativity ('N200') and the N400 for the low probability in contrast to high probability sentence-final words, interpreting the first negativity to reflect the acoustic analysis and the second the semantic analysis. Given these data from the auditory language process-
ing domain, the possibility must be considered that the early negativity observed in the present study using connected natural speech may not solely be due to the syntactic violation introduced, but may be influenced by hidden prosodic cues. Interestingly, in the present study no early difference between the correct and incorrect sentence was observed for the semantic and the morphological condition suggesting that possible prosodic influences must have been tieti to incorrect phrase structure, and not to incorrectness per se. This makes it most likely that the observed effect is indeed due to the syntactic violation introduced.
For the morphosyntactic violation induced by an incorrect verb inflection we found a late positivity at the central posterior site and a negativity around 400 ms preceding it. The observed negativity may be similar to the negativity reported by Kutas and Hillyard26 for morphosyntactic number marking. The late positivity may be connected to the 'late positive shift' reported by Hagoort et al.19 in correlation with verb inflection errors marking subject-verb-agreement errors in Dutch. These authors, however, found a late positivity also in correlation with a particular phrase structure violation requiring a reanalysis of a preferred syntactic reading of a given sentence. Given its distribution, this late positivity may be a member of the P300 component, often found following unexpected stimuli which require an updating of preceding information8,20.
The data of the present study evaluating three different aspects of language processing in the same subjects suggest that these aspects are processed at different points in time during comprehension. Processing of word category information, as in the phrase structure violation condition, seems to precede lexical processes in auditory language comprehension^ as indicated by the early negativity evoked by a word category error preceding the negativity at 400 ms correlated with lexical processes. The late positivity observed in some studies19,40 may reflect the detection of the need for structural reanalyses in a given sentence. Once these effects observed in correlation with syntactic processing, i.e. the early negativities and the late positivity, are validated by further experiments, they may be taken to support a two-stage module of syntactic parsing, with a first stage during which a structure-driven parser assigns an initial structure to the input based on major category information, and a second stage during which structural and semantic aspects are made available before final interpretation takes place10,13_1S.
Further research is clearly needed before we will be able to adequately describe the underlying temporal and neurotopological structure of the language comprehension process. The present study using connected
speech as input combined with ERP measures clearly indicates that there are ways to identify different temporal stages during language comprehension and to connect these to distinct neural systems in the human brain.
Acknowledgements. This work was supported by the Alfried Krupp von Bohlen und Halbach Science Award assigned to the first author and by a grant from the German Research Foundation (DFG-FR 519/12-1) assigned to the first author and to Axel Mecklinger. The authors wish to thank Burkhard MaeB for his advice and support concerning the hardware and software used for the present experiments, and Axel Mecklinger for his advice in analyzing the data and his comments on the manuscript.
REFERENCES
1 Bentin, S., Kutas, M. and Hillyard, S.A., Electrophysiological evidence for task effects on semantic priming in auditory word processing, Psychophysiology, 30 (1993) 161-169.
2 Brown, S.S., Lehmann, D. and Marsh, J.T., Linguistic meaning-related differences in evoked potential topography: English, Swiss German and imagined, Brain Lang., 11 (1980) 340-353.
3 Brown, W.S., Marsh, R.E. and Smith, J.C.. Contextual meaning effects on speech-evoked potentials, Behar. Biol., 9 (1973) 755761.
4 Brown, W.S., Marsh, R.E. and Smith, J.C., Evoked potential wave-form differences produced by the perception of different meanings of ambiguous phrase, Electroencephalogr. Clin. Neuro-physiol., 41 (1976) 113-123.
5 Brown, W.S., Marsh, R.E. and Smith, J.C., Principal component analysis of ERP differences related to the meaning of an ambiguous word, Electroencephalogr. Clin. Neurophysiol., 46 (1979) 709761.
6 Clifton, C., Speer, S. and Abney, S„ Parsing arguments: phrase structure an argument structure as determinants of initial parsing decisions, J. Mem. Lang., 30 (1991) 251-271.
7 Connolly, J.F., Stewart, S.H. and Phillis, NA, The effects of processing requirements on neurophysiological responses to spoken sentences. Brain Lang.. 39 (1990) 302-318.
8 Donchin, E., Surprise! ... surprise!, Psychophysiology, 18 (1981) 493-513.
9 Donchin, E. and Coles, M.G.H., On the conceptual foundations of cognitive psychophysiology, Behav. Brain Sci., 11 (1988) 408418.
10 Ferreira, F. and Clifton, C., The independence of syntactic processing, J. Mem. Lang., 25 (1986) 348-368.
11 Forster, K.I., Levels of processing and the structure of the language processor. In W.E. Cooper and E. Walker (Eds.), Sentence Processing: Psycholinguists Studies Presented to Merrill Garrett, Erlbaum Associates, Hillsdale, NJ, 1979, pp. 27-85.
12 Frazier, L., On comprehending sentences: syntactic parsing strategies, University of Connecticut, Doctoral Dissertation, 1978.
13 Frazier, L., Sentence processing: a tutorial review. In M. Cblt-heart (Ed.), Attention and Performance XII, Erlbaum Associates, Hillsdale, NJ, 1987, pp. 559-586.
14 Frazier, L., Exploring the architecture of the language-processing system. In G.T.M. Altmann (Ed.), Cognitive Models of Speech Processing, MIT Press, Cambridge, MA, 1990, pp. 409-433.
15 Frazier, L. and Rayner, K., Making and correcting errors during sentence comprehension: eye movements in the analysis of structurally ambiguous sentences, Cogn. Psychol., 14 (1982) 178-210.
16 Friederici, A.D., Aphasics' perception of words in sentential context: some real-time processing evidence, Neuropsychologia, 21 (1983) 351-358.
17 Friederici, A.D., Levels of processing and vocabulary types: evidence from on-line comprehension in normals and agrammatics, Cognition, 19 (1985) 133-166.
IK Fricdcrici. A.D. and Kilhorn, K.. Temporal constraints on language processing: syntactic priming in Broca's aphasia. J. Cogn. Neurtmi., 1 (1989) 262-272.
19 Hagoort. P.. Brown. C. and Groothusen. J., The syntactic positive shift as an ERP-measure of syntactic processing, Lang. Cogn. Process., in press.
20 Hillyard, S.A. and Picton, T.W., Electrophysiology of cognition. In V.B. Mountcastle, F. Plum and S.R. Geiger (Eds ), Handbook of Physiology, Vol. V. Higher Functions of the Brain, Part 2, American Physiological Society, Bethesda, 1987, pp. 519-584.
21 Holcumb. P.J. and Neville. H., Auditory and visual semantic priming in lexical decision: a comparison using event-related brain potentials, Lang. Cogn. Process.. 5 (1990)281-312.
22 Holcomb, P.J. and Neville, K.J., Natural speech processing: an analysis using event-related brain potentials, Psychobiology, 19 (1991)286-300.
23 Huynh, H. and Feldt. L.A.. Conditions under which mean square ratios in repeated measurement designs have exact /^distributions, J. Am. Stat. Assoc., 65 (1970) 1582-1589.
24 Kluender, R. and Kutas, M., Bridging the gap: evidence from ERPs on the processing of unbounded dependencies. / Cogn. Neurosci.. 5 (1993) 196-214.
25 Kutas. M. and Hillyard. S.A.. Reading between the lines: event-related brain potentials during natural sentence processing. Brain Lang., 11 (1980)354-373.
26 Kutas. M. and Hillyard, S.A., Event-related brain potentials to grammatical errors and semantic anomalies, Mem. Cogn.. 11 (1983)539-550.
27 Kutas. M„ Lindamood. T. and Hillyard. S.A., Word expectancy and event-related potentials during sentence processing. In S. Kornbli n and J. Requin (Eds.). Preparatory States and Processing, Erlb„um Press, New Jersey, 1984, pp. 217-237.
28 Kutas, M. and Van Petten, C„ Event-related potential studies of language. In P.K. Ackles. J.R. Jennings and M.G.H. Coles (Eds.), Advances in Psychophysiologic Vol. 3. JAI Press, Greenwich, 1988, pp. 139-187.
29 Kutas, M., Van Petten. C. and Besson, M., Event-related potential asymmetries during reading sentences. Electrorncephalogr. Clin. Neurophysiot, 29 (1988) 218-233.
30 Marslen-Wilson. W.D., Functional parallelism in spoken word recognition. Cognition, 25 (1987) 71-102.
31 Marslen-Wilson, W.D. and Tyler. L.K., The temporal structure of spoken language understanding. Cognition, 8 (1980) 1-71.
32 McCallum. W.C.. Fanner, S.F. and Pocock, P.K., The effect of physical and semantic incongruities on auditory event-related potentials, Electroenceplialogr. Clin. Neurophysiot., 59(1984)447-488.
33 McCarthy, G. and Wood, C.C., Scalp distributions of event-related potentials: an ambiguity associated with analysis of variance models, Electroenceplialogr. Clin. Neurophysiol., 62 (1985 ) 203208.
34 McClelland, J.L., St. John, M. and Taraban, R„ Sentence comprehension: a parallel distributed processing approach, Lang. Cogn. Process , 4 (1989) 287-336.
35 Mecklinger, A., Kramer, A.F. and Strayer, D., Event-related potentials and EEG components in a semantic memory search task, Psychophysiology. 29(1992) 104-119.
36 Münte. Th.F.. Heinze. H.-J. and Prevedel, H., Ereigniskorrelierte Hirnpotentiale reflektieren semantische und syntaktische Fehler bei Sprachverarbeitung, Z. EEG EMG verwandte Gebiete, 21
(1990) 75-81.
37 Neville. HJ., Nicole. J., Barss, A., Forster, K. and Garret, M„ Syntactically based sentence processing classes: evidence from event-related brain potentials, J. Cogn. Neurosci., 3 (1991) 155170.
38 Neville, H.J.. Mills. D.L. and Lawson, D.L., Fractionating language: different neural subsystems with different sensitive periods, Cerebral Cortex, 2 (1992) 244-258.
39 Oldfield. R.C., The assessment and analysis of handedness: The Edinburgh Inventoiy, Neuropsychology, 9 (1971) 97-113.
40 Osterhout, L. and Holcomb, P.J., Event-related brain potentials elicited by syntactic anomaly, J. Mem. Lang., 31 (1992) 785-804.
41 Rayner, K., Carlson, M. and Frazier, L., The interaction of syntax and semantics during sentence processing: eye movement in the analysis of semantically biased sentences, J. Verbal Learn. Verbal Behav., 22 (1983) 358-374.
42 Rosier, F., Friedend, A.D., Pütz, P. and Hahne, A., Event-related brain potentials while encountering semantic and syntactic constraint violations, J. Cogn. Neurosci., 5 (1993)345-362.
43 Rugg, M.D., The effects of semantic priming and word repetition on event-related potentials. Psychophysiology, 22 (1985) 642-647.
44 Rugg, M.D., Dissociation of semantic priming, word and non-word repetition effects by event-related potentials. Quart. J. Exp. Psychol., 39 (1987) 123-148.
45 Segui. J„ Dupoux, E. and Mehler, J., The role of the syllable in speech segmentation, phoneme identification and lexical access. In G.T.M. Altmann (Ed.), Cognitive Models of Speech Processing, MIT Press, Cambridge, MA, 1990, pp. 263-280.
46 Schriefers, H., Fricdcrici, A.D. and Kühn, K., The processing of local ambiguous relative clauses in German, submitted.
47 Swinney, A.D., Lexical access during sentence comprehension: (re)consideration of context effects, J. Verbal Leam. Verbal Behav., 18(1979)645-659.
48 Van Petten, C. and Kutas, M., Influences of semantic and syntactic context on open and closed class words, Mem. Cogn., 19
(1991)95-112.
49 Zwitserlood, P., The locus of the effects of sentential-semantic context in spoken-word processing. Cognition, 32 (1989) 25-64.
50 Zwitserlood, P., Schriefers, H., Lahiri, A. and van Donselaar, W„ The role of syllables in the perception of spoken Dutch, J. Exp. Psych. Learning Memory Cognition, 19 (1993) 260-271.
51 Zurif, E., Swinney, D. and Garrett, M., Lexical processing and syntactic comprehension in aphasia. In A. Caramazza (Ed.), Cogn. Neuropsychol. Neumiinguistics, Hillsdale, NJ, Lawrence Erlbaum, 1990, pp. 123-146. - '
