Scholarly article on topic 'Brain connections of words, perceptions and actions: A neurobiological model of spatio-temporal semantic activation in the human cortex'

Brain connections of words, perceptions and actions: A neurobiological model of spatio-temporal semantic activation in the human cortex Academic research paper on "Clinical medicine"

Share paper
Academic journal
OECD Field of science
{"Word acquisition" / "Semantic grounding" / "Hebbian cell assembly" / "Biologically inspired neural network" / "Word recognition EEG-MEG responses" / "Cortical connectivity"}

Abstract of research paper on Clinical medicine, author of scientific article — Rosario Tomasello, Max Garagnani, Thomas Wennekers, Friedemann Pulvermüller

Abstract Neuroimaging and patient studies show that different areas of cortex respectively specialize for general and selective, or category-specific, semantic processing. Why are there both semantic hubs and category-specificity, and how come that they emerge in different cortical regions? Can the activation time-course of these areas be predicted and explained by brain-like network models? In this present work, we extend a neurocomputational model of human cortical function to simulate the time-course of cortical processes of understanding meaningful concrete words. The model implements frontal and temporal cortical areas for language, perception, and action along with their connectivity. It uses Hebbian learning to semantically ground words in aspects of their referential object- and action-related meaning. Compared with earlier proposals, the present model incorporates additional neuroanatomical links supported by connectivity studies and downscaled synaptic weights in order to control for functional between-area differences purely due to the number of in- or output links of an area. We show that learning of semantic relationships between words and the objects and actions these symbols are used to speak about, leads to the formation of distributed circuits, which all include neuronal material in connector hub areas bridging between sensory and motor cortical systems. Therefore, these connector hub areas acquire a role as semantic hubs. By differentially reaching into motor or visual areas, the cortical distributions of the emergent ‘semantic circuits’ reflect aspects of the represented symbols’ meaning, thus explaining category-specificity. The improved connectivity structure of our model entails a degree of category-specificity even in the ‘semantic hubs’ of the model. The relative time-course of activation of these areas is typically fast and near-simultaneous, with semantic hubs central to the network structure activating before modality-preferential areas carrying semantic information.

Academic research paper on topic "Brain connections of words, perceptions and actions: A neurobiological model of spatio-temporal semantic activation in the human cortex"

Author's Accepted Manuscript

Brain connections of words, perceptions and actions: A neurobiological model of spatiotemporal semantic activation in the human cortex

Rosario Tomasello, Max Garagnani, Thomas Wennekers, Friedemann Pulvermüller'locate/neiiropsychologia

PII: S0028-3932(16)30242-1

DOI: http://dx.doi.Org/10.1016/j.neuropsychologia.2016.07.004

Reference: NSY6057

To appear in: Neuropsychologia

Received date: 15 February 2016 Revised date: 28 May 2016 Accepted date: 3 July 2016

Cite this article as: Rosario Tomasello, Max Garagnani, Thomas Wennekers anc Friedemann Pulvermuller, Brain connections of words, perceptions and actions A neurobiological model of spatio-temporal semantic activation in the huma cortex, Neuropsychologic

This is a PDF file of an unedited manuscript that has been accepted fo publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, an< review of the resulting galley proof before it is published in its final citable form Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain

Brain connections of words, perceptions and actions: A neurobiological model of spatio-temporal semantic activation in the human cortex

Rosario Tomasello1,2*, Max Garagnani1,2, Thomas Wennekers2, Friedemann Pulvermüller1,3*

1Brain Language Laboratory, Department of Philosophy and Humanities, WE4 Freie Universität Berlin Habelschwerdter Allee 45, 14195 Berlin, Germany

Centre for Robotics and Neural Systems (CRNS), University of Plymouth, A311 Portland Square Building, PL4 8AA Plymouth, Devon, United Kingdom

Berlin School of Mind and Brain, Humboldt Universität zu Berlin, Luisenstraße 56, 10117 Berlin, Germany

Corresponding author. Address for correspondence: Brain Language Laboratory, Department of Philosophy and Humanities, WE4, Freie Universität Berlin, Habelschwerdter Allee 45, 14195 Berlin, Germany. Tel.: +49 (0) 30 838 51984.


Neuroimaging and patient studies show that different areas of cortex respectively specialize for general and selective, or category-specific, semantic processing. Why are there both semantic hubs and category-specificity, and how come that they emerge in different cortical regions? Can the activation time-course of these areas be predicted and explained by brain-like network models? In this present work, we extend a neurocomputational model of human cortical function to simulate the time-course of cortical processes of understanding meaningful concrete words. The model implements frontal and temporal cortical areas for language, perception, and action along with their connectivity. It uses Hebbian learning to semantically ground words in aspects of their referential object- and action-related meaning. Compared with earlier proposals, the present model incorporates additional neuroanatomical links supported by connectivity studies and downscaled synaptic weights in order to control for functional between-area differences purely due to the

number of in- or output links of an area. We show that learning of semantic relationships between words and the objects and actions these symbols are used to speak about leads to the formation of distributed circuits, which all include neuronal material in connector hub areas bridging between sensory and motor cortical systems. Therefore, these connector hub areas acquire a role as semantic hubs. By differentially reaching into motor or visual areas, the cortical distributions of the emergent 'semantic circuits' reflect aspects of the represented symbols' meaning, thus explaining category-specificity. The improved connectivity structure of our model entails a degree of category-specificity even in the 'semantic hubs' of the model. The relative time-course of activation of these areas is typically fast and near-simultaneous, with semantic hubs central to the network structure activating before modality-preferential areas carrying semantic information.

Keywords: word acquisition, semantic grounding, Hebbian cell assembly, biologically inspired neural network, word recognition EEG-MEG responses, cortical connectivity

1. Introduction

The human brain is able to acquire and store knowledge about people, facts, objects, actions, and culture through experiences in everyday life. Much of this knowledge comes in units, as 'conceptual' or 'semantic representations', and carries symbolic linguistic labels in language, whereby the relationships between word-forms and semantic meaning appears as arbitrary. When semantic functions are damaged, serious consequences in daily cognitive activity can arise, being manifest as impairments of language and verbal communication and in some cases extending to domains such as planning, object recognition, or goal directed action such as drinking a glass of water (Bak and Chandran, 2012; Damasio et al., 1996; Gainotti, 2010; Kemmerer et al., 2012; Pulvermüller and Fadiga, 2010). Given the centrality of semantics in human life, it is crucial to understand the neural mechanisms underlying the nature of semantic knowledge in the brain, which, despite decades of research, is still one of the most controversial issues among cognitive neuroscientists, who propose quite diverging perspectives on this issue.

One view puts forth that one or more area(s) is/are active during meaning processing in the brain, which appear to function as general convergence zones or semantic hubs and process the meaning of all types of signs and symbols. 'Semantic hubs' have been proposed to be situated in the frontal, temporal and parietal cortices, especially in the left language dominant hemisphere (Bookheimer, 2002; Patterson et al., 2007; Price, 2000; Pulvermüller, 2013). For example, evidence for a multimodal semantic hub in anterior-inferior temporal cortex comes from patients suffering from semantic dementia, because damage in this region seems to be the best predictor of their semantic deficit (Mion et al., 2010). Although there is strong evidence for semantic hub areas, that is, for cortical regions which are generally important for meaning processing, an explanation of why several regions seem to play a role as semantic hubs and, especially, why they are localised in their specific cortical areas, is necessary.

A second important observation is that some additional cortical areas contribute to semantic processing in a more selective fashion, being particularly relevant for specific semantic categories, such as words typically used to speak about animals, tools, or actions and their related concepts. Some evidence also indicates that when recognising a word such as run, activity in motor cortex, and even more specifically in leg-motor cortex, emerges, whereas, when hearing an object- and visually-related word such as sun, activity in visual areas is relatively more pronounced (Boulenger et al., 2009; Damasio et al., 1996; Gainotti, 2010; Hauk et al., 2004; Pulvermüller et al., 2009). Support for category-specific semantic processes is provided by a number of neurocognitive empirical studies that have focused on the importance of the motor and premotor cortex during conceptual processing, demonstrating for example that perceiving action words and sentences evokes activity in motor and premotor cortices (Boulenger et al., 2009; Hauk and Pulvermüller, 2004; Hauk et al., 2008, 2004; Pulvermüller, 2001, 1999; Rüschemeyer et al., 2007; Shtyrov et al., 2004). Furthermore, activation in the premotor and motor cortex is so fine grained that we can differentiate semantic subcategories of action-related words somatotopically (Grisoni et al., 2016; Hauk and Pulvermüller, 2004; Hauk et al., 2004). Category-specific effects have also been seen in the visual areas, especially in the ventral temporal-occipital areas, when visually-related words are being processed (e.g. animal, colour or object-related words) (Chao et al., 1999; Kiefer, 2005; Sim and Kiefer, 2005). Importantly, category-specific semantic effects are also documented in the lesion literature, where sometimes rather small lesions in modality-preferential areas can selectively impair the processing of specific semantic categories (Dreyer et al., 2015; Warrington and Shallice, 1984). A neurobiological explanation of category-specificity has been proposed, which relates the differential activation patterns and lesion signatures to the functional level of cortical circuits with different distributions across areas. Accordingly, widely distributed cortical circuits for word forms

carried by neuronal assemblies in the perisylvian language areas are linked with neuronal ensembles storing semantic information. These semantic circuits reach into modality-preferential motor and/or sensory areas depending on whether the perceptual or action-related information is relevant for grounding the meaning of the words (Barsalou, 2008; Martin, 2007; Pulvermüller and Fadiga, 2010; Pulvermüller et al., 2005; Pulvermüller, 2001). The different distribution of the semantic circuits across the cortex, therefore, explains aspects of category-specificity. Notably, some studies reported that both category-general and category-specific semantic activation in the brain has been found to emerge rather fast, i.e. within ~200 ms after a meaningful symbol can be recognized (Hoenig et al., 2008; Penolazzi et al., 2007; Pulvermüller et al., 2005b, 2004, 2000; Shtyrov et al., 2014). For example, Moseley et al. (2013) recorded brain signals using magnetoencephalography (MEG) and found different responses for action-related, object-related and abstract written words already at 150 ms after their onset, with gradually stronger activations for the action/object items in motor/visual regions, respectively. An explanation of category-specificity has been offered in terms of neurobiological principles. However, in order to integrate theory and data about semantic hubs with established knowledge about category-specificity, it is necessary to develop formal models of cortical structure and function that explain the presence of both.

An effort towards such explanation was recently made by Garagnani and Pulvermüller (2016), who used a network implementation of cortical areas and their connectivity to mimic the function of the perisylvian language cortex, in particular inferior frontal and superior temporal cortex, along with general visual and motor areas function in order to simulate the binding of phonological/lexical and semantic information. Using Hebbian mechanisms for synaptic modification, this model was used to simulate the emergence of neuronal circuits that process information about word forms and their related action- vs. object-related meanings. However, the model used a simplified connectivity structure, and was applied to make predictions about magnitude and topography of brain activation, but not its time course. Here, we improve on this earlier architecture by incorporating additional cortico-cortical connections documented by neuroanatomical studies. This neuroanatomically more appropriate model was used, as in the earlier version, to predict the cortical distribution of the memory circuits for words with object- and action-related meaning. However, this type of model can be used to predict not only where in the brain linguistic and semantic brain activity occurs, but also when these processes take place, i.e., the time course of such activation. Although the spatiotemporal dimension was already present in the previous network architecture (Garagnani and Pulvermüller, 2016), we provide here, for the first time, a precise activation time course analysis of different areas of the network. Furthermore, the previous model included connector hub areas, which exhibited increased numbers of links compared with other areas. To make sure that the

specific activation signatures that we observed there - in particular, the generally strong activation seen in connector hub areas - were not just a result of an increased weighted sum of incoming and outgoing synaptic connections to and from neighbouring areas ('more and stronger links, more semantics'), an in-degree normalization across areas was used here to balance the overall input across areas and emphasise the role of network topology (or connection structure) as a factor influencing circuit topographies (or cell assembly distributions).

To investigate word meaning processing in the human brain, we used a neural network model implementing realistic anatomical and physiological features of the human cortex. The model simulates primary and secondary sensorimotor areas in frontal, temporal and occipital cortex along with 'connector hub' areas interfacing between different sensory and motor systems (Garagnani and Pulvermüller, 2016, 2013, 2011; Garagnani et al., 2009, 2008; Pulvermüller and Garagnani, 2014). The short and long distance connections between model areas are based on existing neuroanatomical evidence. Functionally, the model takes advantage of realistic Hebbian learning mechanisms (Hebb, 1949). The network was trained with repeatedly presented specific sensorimotor patterns coding for the articulatory and acoustic phonological structure of single words and some of their action- or perception-related semantic features. As a result of learning and area/connectivity structure, distributed 'semantic circuits' emerged in the network, spanning different areas. Importantly, the topographies of these circuits showed similarities and differences between semantic types (action vs. object words), which can be related to the semantic information stored. We document circuit distributions and their dynamic activation and discuss the results in the context of specific model features, existing experimental evidence, and novel predictions for future research.

2. Materials and Methods

We applied a neurobiologically grounded computational model replicating structure and functional properties of the human cortex to investigate the neural mechanisms underlying word meaning acquisition and processing in the perception and action systems of the mind and brain. The model's architecture mimics the left perisylvian cortex involved in spoken word processing, corresponding to articulatory and acoustic phonological word forms (Fadiga et al., 2002; Fry, 1966; Pulvermüller and Fadiga, 2010; Pulvermüller, 1999; Zatorre et al., 1996), areas outside the perisylvian cortex involved in processing visual object identity (Ungerleider and Haxby, 1994), and the execution of manual actions (Deiber et al., 1991; Dum and Strick, 2005, 2002; Lu et al., 1994). The model mimics a range of biologically realistic properties of the human cortex including the following features:

1. Area structure: 12 cortical areas were modelled, including modality-preferential sensory and motor ones as well as connector hub areas interlinking sensory and motor systems.

2. Between-area connectivity: different areas were linked based on neuroanatomical principles and data, realising sparse, random, initially weak and topographic connectivity.

3. Within-area connectivity: similarly sparse, random and initially weak connectivity was implemented locally, along with a neighbourhood bias towards local links (Braitenberg and Schüz, 1998; Kaas, 1997).

4. Local lateral inhibition and area-specific global regulation mechanisms (local and global inhibition) (Braitenberg, 1978; Palm et al., 2014; Yuille and Geiger, 2003).

5. Synaptic modification by way of Hebbian type learning, including both long-term potentiation and depression (LTP, LTD) (Artola and Singer, 1993).

6. Neurophysiological dynamics of single cells including temporal summation of inputs, nonlinear transformation of membrane potentials into neuronal outputs, and adaptation (Matthews, 2001).

7. Constant presence of uniform uncorrelated white noise in all neurons during all phases of learning and retrieval, and additional noise added to the stimulus patterns to mimic realistic noisy input conditions during retrieval (Rolls and Deco, 2010).

Word learning processes in the model are based entirely on mechanisms of Hebbian plasticity, often summarized by the phrase "cells that fire together, wire together", although the learning rule applied (see above and Appendix A) implements 'anti-Hebb' learning too, colloquially described by the phrase "cells out of sync delink" (for discussion, see Garagnani et al., 2009). Accordingly, within a network of interconnected neurons, repeatedly and consistently co-active sub-populations of cells strengthen their connections, forming the so called cell assemblies (CAs) (Hebb, 1949). According to Hebb (1949), assemblies can be considered functional units in the brain representing the building blocks of cognitive functions, including language (Braitenberg, 1978; Palm et al., 2014; Pulvermüller, 1996). In principle, the emerging neuronal assemblies can be local, that is, restricted to a small area or even cortical column of a fraction of a cubic millimetre or, alternatively, be spread out across wide cortical regions, and it is not clear a priori whether a given network and input pattern leads to the formation of local or distributed circuits. Different cortical distributions, or topographies, of cell assemblies have been postulated for symbols with different meaning. Standard postulates are that words related to actions include neurons in the motor cortex - which control the movements a word such as run is typically used to speak about - while words referring to objects (such as sun) will include neurons in areas along the ventral visual stream of object processing

(Huyck and Passmore, 2013; Pulvermüller and Preissl, 1991; Pulvermüller, 1999). Previous simulation studies have already shown the formation of distributed neuronal assemblies exhibiting differential cortical distributions as a result of repeated concomitant presentation of activation patterns and Hebbian plasticity mechanism (Garagnani and Pulvermüller, 2016, 2013, 2011; Garagnani et al., 2009, 2008; Wennekers et al., 2006).

2.1 Model architecture

The model consists of 12 cortical areas of artificial neurons with area-intrinsic connections and mutual connections between them. In the left perisylvian language cortex, we identify six cortical areas divided into two sub-systems: auditory and articulatory systems (areas highlighted in blue and red in Fig. 1. A). The auditory system includes the primary auditory cortex (A1), auditory belt (AB), and parabelt areas (PB) - whereas the articulatory system includes the primary articulatory motor cortex (inferior part of primary motor cortex, M1i), inferior premotor (PMi) and prefrontal motor cortex (PFi). Six additional areas outside the perisylvian cortex (which we call 'extrasylvian') were included to model the ventral visual stream and dorsolateral motor system (green and yellow highlighted areas). The ventral visual system is relevant for processing visual object identity and includes, apart from primary visual cortex (V1), temporo-occipital (TO) and anterior-temporal (AT) areas. Finally, the motor system which, for example, is relevant for the execution of manual actions, includes the dorsolateral fronto-central motor (M1L), premotor (PML), and prefrontal cortices (PFL).

Each model area consists of two layers of 25 x 25 excitatory and inhibitory artificial neurons (e- and /-cells) (see Fig. 1. C). Each e-cell represents a cluster of excitatory pyramidal cells, and the underlying /-cell models represent the cluster of inhibitory interneurons, situated within the same cortical column (Eggert and van Hemmen, 2000; Wilson and Cowan, 1972). As it is typical for the mammalian cortex, the connectivity between and within model areas is sparse, patchy and topographic (Amir et al., 1993; Braitenberg and Schüz, 1998; Gilbert and Wiesel, 1983). To regulate and control activity in the network, local and area-specific inhibition is implemented (Bibbig et al., 1995; Palm, 1982; Wennekers et al., 2006). Details of the model functions and of the Hebbian learning mechanism (including LTD and LTP) are summarized in previous works (Garagnani and Pulvermüller, 2016, 2013, 2011; Garagnani et al., 2009, 2008). For completeness, we recapitulate them in Appendix A.

-----------------------------Please, insert FIGURE 1 about here---------------------

Neuroanatomical and imaging studies have demonstrated the existence of next-neighbour between-area connectivity, which functionally binds adjacent cortical areas together (Pandya and Yeterian, 1985; Young et al., 1995, 1994). These functional links are modelled within each triple of areas forming the four domain-specific sub-systems in the model (see black arrows Fig. 1. B). In the perisylvian system, next-neighbour connections between locally adjacent areas are implemented within the auditory sub-system (A1, AB, PB) (Kaas and Hackett, 2000; Pandya, 1995; Rauschecker and Tian, 2000), as well as within the articulatory (PFi, PMi, M1i) sub-system (Pandya and Yeterian, 1985; Young et al., 1995). Similarly, local next neighbour links are also realised in the extrasylvian system, between adjacent ventral visual (V1, TO, AT) (Bressler et al., 1993; Distler et al., 1993), and dorsolateral motor areas (PFL, PML, M1L) (Arikuni et al., 1988; Dum and Strick, 2005, 2002; Lu et al., 1994; Pandya and Yeterian, 1985; Rizzolatti, G. Luppino, 2001).

Long distance cortico-cortical links between sub-systems (see purple arrows Fig. 1. B) are realised between all pairs of multimodal hub areas (PB, PFi, AT and PFL). This is motivated by evidence for neuroanatomical connections between inferior prefrontal (PFi) and auditory parabelt (PB) areas, carried by the arcuate and the uncinated fascicles (Catani et al., 2005; Makris and Pandya, 2009; Meyer et al., 1999; Parker et al., 2005; Paus et al., 2001; Rilling et al., 2008; Lizabeth M. Romanski et al., 1999) and, in the extrasylvian system connections between anterior-temporal (AT) and lateral prefrontal (PFL) areas, carried by the uncinate fascicle (Bauer and Jones, 1976; Chafee and Goldman-Rakic, 2000; Eacott and Gaffan, 1992; Fuster et al., 1985; Parker, 1998; Ungerleider et al., 1989; Webster et al., 1994). The peri- and extrasylvian systems are also linked by means of long distance cortico-cortical connections across the central hub areas; likewise parabelt (PB) and lateral prefrontal cortex (PFL) are reciprocally connected (Pandya and Barnes, 1987; L. M. Romanski et al., 1999; Lizabeth M. Romanski et al., 1999) as well as the anterior/middle-temporal (AT) and inferior prefrontal (PFi) areas (Pandya and Barnes, 1987; Petrides and Pandya, 2009; Rilling, 2014; Romanski, 2007; Ungerleider et al., 1989; Webster et al., 1994). A recent simulation study adopting a similar network architecture did not implement connections between inferior and superior prefrontal or between auditory parabelt and anterior temporal cortex (Garagnani and Pulvermüller, 2016). We added both links because of the evidence for reciprocal connectivity between anterior-temporal (AT) and parabelt (PB) areas (Gierhan, 2013) and between inferior and lateral prefrontal (PFi, PFl) areas (Yeterian et al., 2012). This also led to a more symmetric network structure. The asymmetries in the earlier network may account for some of its functional properties, which, as we discuss below, were not seen in the present network based on a (slightly) more realistic structure (see Discussion).

The previous study (Garagnani and Pulvermüller, 2016) found that semantic circuits included a massively enhanced number of neurons in connector hub areas compared with primary or secondary areas, which was seen as an explanation of semantic hub status. However, there are different mechanisms that could underlie the observation: One way to explain it is by way of topological network structure, especially the fact that 'connector hub' areas hold a central role in interlinking sub-systems. At the same time, and partly independent from their role as connector hubs, the same areas are also the targets and origins of an increased number of connections to other areas (i.e. a higher 'degree' of connectivity). In the case of our present model, 2 between-area connections exist for most areas (primary ones have input plus 1 connection), but connector hubs have 4 of them, thereby entailing larger amounts of activation reaching these areas when activity waves spread through the network from its different ends during learning. Any specific functional properties of hub areas, including their great involvement in carrying semantic circuit members, may therefore, result either from network topology, or from number of area input connections from other areas, or from both. If it is just the number of inputs to and thus amount of activation in an area - their 'in-degree' - that is relevant for an increased importance in semantics, the explanation of semantic hubs may trivially be based on the formula 'what activates most, is most relevant for cognition'. However, an explanation based on network topology and connectivity structure per se becomes plausible if general semantic relevance can be documented for hubs that have an overall input comparable to that of other areas. Therefore, we normalized the overall amount of input of all (equal-sized) areas by dividing the contribution of all long-distance connections (all links among the 'rich club' of connector hubs, central quadruplet in Fig. 1. B) by 3. After this in-degree normalization (which in the present symmetric architecture also implies out-degree normalization), each of the 12 areas receives two equal quantities of inputs (either 1*1 or 3*1/3), one from the left and one from the right side of the model. This procedure preserved differences in topology while normalising for amount of input activation per area.

2.2 Simulations

The simulations were carried out in two steps. After learning the semantic relationships between articulatory and acoustic information about the word form (perisylvian activity patterns in M1i and A1) and 'grounding' action or object information (extrasylvian activity pattern either in M1l or in V1) (section 2.2.1), the network was used to simulate the neurophysiological correlates of word recognition and understanding (section 2.2.2).

2.2.1 Learning phase

The network architecture described above (Fig. 1. B) was initialized at random before the learning phase began (see appendix A): 12 different, randomly initialized networks were created, each with 12 different sets of sensorimotor patterns representing object- and action-related words. These 'word-learning patterns' represented six object-related and six action-related words. Each pattern consisted of a fixed set of 19 cells chosen at random from the 25 x 25 cells of an area (ca. 3% of the cells) which were simultaneously presented to the primary areas of the network. At the linguistic and semantic levels, the cells in M1i and A1 represented articulatory and acoustic phonetic features and their values (e.g., [+labial]) and those in M1L and V1 action-related and visually-related semantic features plus values of the words (e.g., [+LEG ACTION], [+ROUND SHAPE]). Each word in our training set was grounded in input to three of the four primary areas of the model: apart from perisylvian A1 and M1i activity, object-related words received concordant visual (V1) and action words lateral motor area (M1i) grounding activity. This mimics a typical situation of object-related word learning, whereby the word is uttered while the referent object is present (Vouloumanos and Werker, 2009) or the relevant action is being performed (Tomasello and Kruger, 1992). Note that white noise was always present and overlaid all learning patterns (in addition to that already present in all areas of the network). This was implemented to account for variability of perceptions and actions of the same type. The model was set up to learn the correlation between word and referential semantic information; the critical question was which type of representations develops in the network as a consequence of learning.

Each word-learning pattern of 3 x 19 activated cells (57 cells in total) was simultaneously presented to the respective primary areas for 3000 times. Some trial-to-trial variability of patterns was due to noise overlay (see below). The number of presentation was chosen on the basis of previous simulations (Garagnani and Pulvermüller, 2016). While three primary areas were directly activated by each learning pattern, the fourth non-relevant area (M1i for object- and V1 for action-related words) received additional variable noise input, i.e. a further pattern, consisting of 19 randomly chosen cells that changed inconsistently over learning episodes, was presented to the respective primary areas. This was done to make sure that the correlation of the word-form activity in the perisylvian cortex with that of the semantic information was high in one modality for action and object words in motor and visual systems, but low in the non-relevant one. A learning trial involved presentation of a word pattern for 16 time-steps, followed by a period during which no input (inter stimulus interval - ISIs) was given. The next stimulus was presented to the network only when the global inhibition of the PFi and PB areas decreased below a specific fixed threshold; this allowed the activity in the network to return to a predefined baseline value, so as to minimize the

possibility of one trial affecting the next one. During each ISIs, only the inherent baseline noise (simulating spontaneous neuronal firing) was present in the neural-network.

2.2.2 Cell assembly definition

During the learning phase, we noticed the gradual formation of cell assembly circuits with different assemblies responding to different input patterns. After 3000 presentations in which three of the four sub-systems were co-activated by stimulating specific neurons in their respective primary cortex, distributed neuronal circuits spontaneously emerged within the network areas, linking up word-form in the perisylvian language areas (auditory and articulatory sub-systems) with referential-semantic information in the sensorimotor areas (visual and motor sub-systems) (this is further explained in section 3.1).

To identify and quantify the neurons forming the 12 CA circuits across the network areas, we computed the average firing rate of each excitatory cell (7500 e-cells) over the 15 time-steps subsequent to a single presentation of the learned sensorimotor patterns (no semantic input was provided in the primary areas of the extrasylvian system). An e-cell was defined as a member of a given CA circuits, only if its time-averaged rate (output value or "firing rate") reached a threshold 6 which was area- and cell-assembly specific, and defined as a fraction y of the maximal single-cell's time-averaged response in that area to pattern w. More formally,

6 = 6a(w) = y max O(x, t)

where O(x, t)w is the estimated time-averaged response of cell x to word pattern w (see Eq. (A4.1)

in Appendix A) and ye [0,1] is a constant (we used y = 0.5 on the basis of previous simulation results (see Garagnani et al., 2009, 2008)). This was computed for each of the 12 trained networks and the number of CA cells per area was averaged over the six object- and six action-related words. CA distributions across areas were analysed statistically as described in section 2.3.

2.2.3 Neurophysiological word recognition simulations

After training, we used the network to simulate the process of perceiving, recognizing and understanding object- and action-related words and the neurophysiological mechanisms underlying these processes. To this end, each 'testing trial' started with primary auditory area (A1) stimulation using only the A1 component of the learning pattern of one learnt 'word'. Stimulation was for 2 time-steps, followed by 50 time-steps during which no input was provided and another 10 used as a

baseline for the subsequent trial. To ensure that all testing trials started from analogous baselines, network activity was reset before the baseline. In order to obtain better signal-to-noise ratios, each of the auditory patterns was presented in 12 different testing trials. Results for each CA were obtained by averaging the 12 "trials" of its sensorimotor pattern presentation.

During word recognition, we recorded the area-specific "within-cell assemblies (CA) activity" per simulation time-step during the 10 time-steps preceding the stimulus onset and the 50 time-steps following offset. The within-CA activity was computed as the sum of the output values (cumulative firing rates, CFRs) of the emerging CA cells in each area produced by stimulation of area A1 as a function of time. By "CA cells", we mean here the cells forming the CA (as defined in Sec. 2.2.2 above); through Hebbian learning, these cells become strongly and reciprocally connected, forming the CA circuits. After this, we identified the "peak amplitude" as the maximum value reached by the CA's cumulative firing rates during the 50 post-stimulus time-steps, and the "peak delay", the latency of the peak upon stimulation. These values were computed for each of the 12 learned networks, averaged over the two word-types and across network areas: results were submitted to statistical analysis as described below.

2.3 Statistical analysis

Statistics were performed on the six object- and six action-related words learnt by one network and across the 12 different network instances. To statistically test for the presence of significant differences in the topographical CA distribution and activation dynamics, we performed repeated-measures Analyses of Variance (ANOVAs). A 4-way ANOVA was run with factors WordType (two levels: Object vs. Action), PeriExtra (two levels: Perisylvian = {A1, AB, PB, M1i, PMi, PFi}, Extrasylvian cortex = {V1, TO, AT, M1L, PML, PFL}), TemporalFrontal (TempFront)" (2 levels: temporal areas = {A1, AB, PB, V1, TO, AT}, frontal areas ={M1l, PMl, PFl, M1i, PMi, PFi}) and Areas (three levels: Primary = {A1, V1, M1L, M1i}, Secondary = {TO, AB, PML, PMi} and Central = {PB, AT, PFl, PFi} areas). We further performed a second statistical analysis on the data of the two systems separately, six perisylvian and six extrasylvian areas with factors "WordType", "TempFront", "Areas", as described above. Analysis was performed on 3 different sets of data: (i) on CA cells distributions emerged from word acquisition, on the (ii) peak amplitudes, and (iii) peak delays during word recognition simulations. Finally, we performed Bonferroni-corrected planned comparison tests (24 comparisons, corrected critical p < .0020) to further explore the significant differences in CA cells distributions and peak delay data across the four sub-system areas.

3. Data analysis & Results

3.1 Learned CA topographies for object- and action-related words

Fig. 2 illustrates six of the twelve CA-cell distributions for object- (A) and action-related (B) words, as they spontaneously emerged during simulated word learning (the other CAs produced similar results). Each set of 12 squares is a snapshot of the CA distribution of a specific word across the network, and each white pixel in the squares represents a cell.

-----------------------------Please, insert FIGURE 2 about here---------------------

The emerging CA circuits are spread out to the same degree across the perisylvian language areas for object- and action-related words, whereas motor and visual areas of the extrasylvian cortex seem to exhibit different CA cell distributions. These distributions indeed appear to show a double dissociation. Object-related words extend more into the visual (V1, TO) areas, whereas they extend only minimally into the extrasylvian motor (PML, M1L) areas; the reverse pattern emerges for the action-related words.

-----------------------------Please, insert FIGURE 3 about here---------------------

Fig. 3 illustrates the distribution of the CA circuits, given as the number of CA cells per areas averaged across 12 trained networks, for object- (light grey) and action-related words (dark grey). The extrasylvian system involved in processing visual-object identity and motor action seems to exhibit a double dissociation between the two word types, as already noted above and in Fig. 2. The perisylvian language cortices seem to show no significant differences between the circuits for the two word types. Note also that there is a larger number of CA cells in the multimodal hub areas (PB, PFi, AT, and PFL) than in the secondary areas (AB, PMi, TO, PML), where there are more cells than in primary areas (A1, M1i, V1, M1L). This appears independent of whether an object- or action-related word is represented.

The observations described above were confirmed by the 4-way ANOVA. A main effect of Areas (F2,24 = 1226.424, p < .0001) emerged, which confirms that the CA cell densities differed across areas, with CA cell densities being higher in hub than in secondary areas (p < .0001), and higher in secondary than in primary areas (p < .0001). In addition, we found a significant interaction between the factors WordType, PeriExtra, TempFront and Areas (F2,24 = 130.795, p < .0001), indicating that the distributions of the two types of word-related CA circuits across the network differed. Because the interaction also demonstrates that CA-distribution differed between

perisylvian and extrasylvian systems, we ran further statistical analyses on the data from the two systems separately, now using 3-way ANOVAs. We found a main effect of Areas for both perisylvian (F2,24 = 2091.116, p < .0001) and extrasylvian systems (F2,24 = 3959.92, p < .0001), as revealed by the 4-way ANOVA analysis. As expected, the perisylvian system did not show any significant differences between CA distributions of the two word types across the 6 areas (F2,24 = 0.38, p = 0.68). In contrast, the extrasylvian system revealed a highly significant interaction of all three factors WordType, TempFront and Areas (F2,24 = 156.555, p < .0001), confirming the word category differences in the CA topographies and local cell-density distributions across visual, motor and multimodal areas as suggested by Fig. 2 and 3. To further investigate the differences between CA types across the network, we ran Bonferroni-corrected planned comparison tests (24 comparisons, corrected critical p < .0020); these confirmed the presence of a larger number of CA cells in visual (V1, TO and AT) than in motor (M1L, PML, and PFL) areas for object- (p < .001), and the opposite for action-related words (p < .001). Post-hoc analysis of the data from the connector hubs (AT, PFl) also showed a significant difference between the two word types there, i.e. stronger action-related word CA cell densities in PFL compared to AT (p <.0001), and the opposite for object-related words (p <.001). Differences in CA-cell densities between word types and pairs of areas in the semantic systems were all significant (p < .002), as described in Fig. 2. In contrast, no significant differences emerged in the perisylvian system (p >.87).

3.2 Neurophysiological word recognition results

To obtain a simulation of spoken word recognition and comprehension processes, we analysed the time-course of the network's response to presentation of the learned auditory word-form patterns to area A1. To this end, we computed the sum of all CA cell activity values (quantified as the cumulative firing rates, CFRs, see Sec. 2.2.3) as a function of time across the entire network or for specific areas. Activation time courses showed an initial "ignition" of CA circuits, a strong activation, which peaked at time-step ~ 16 and included a majority of the circuits' neurons (Fig. 4). Replicating, in part, the structural distributions of semantic circuits depicted in Fig. 3, both types of circuits were similarly spread out across all perisylvian areas of the model; by contrast, differences between semantic circuit types were present in extrasylvian cortex: object-related words (blue pixels) elicited activation in the visual system and less in the motor system, while the reverse happened for the action-related words (red pixels). Note also the low degree of overlap between CAs of the two different word types (yellow pixels) for these two specific CAs instances.

-----------------------------Please, insert FIGURE 4 about here---------------------

In extrasylvian areas, maximal area-specific activation levels significantly differed between the circuits carrying the two semantic word-types. A significant double dissociation showed that circuits for object-related words produced higher amplitude in the visual (cumulative firing rates (CFRs) = 9.10) sub-system than in the lateral (hand) motor system (CFRs = 5.23), and, vice versa, action-related words activated the lateral motor system (CFRs = 8.38) more strongly than the visual system (CFRs = 4.86, see Fig. 5. B - Bar plot left-hand side). As visual inspection indicates, the auditory and articulatory motor sub-systems (see Fig. 5. A - Bar plot left-hand side) did not show any differences in activity levels between semantic word types. Furthermore, comparing activity levels between areas of the network (see Fig. 6. A-B & 7.A-B), multimodal hub areas (AT, PFl, PB, PFi) seemed to show the strongest activation dynamics (CFRs ~ 15) in comparison with secondary (CFRs ~ 10) and primary areas (CFRs ~ 5).

-----------------------------Please, insert FIGURE 5 about here---------------------

The statistical analyses of the dynamic functional activation of the circuits confirmed these observations, which are in line with the CA-distribution results described in Section 3.1. In particular, the 4-way ANOVA performed on peak activation levels per area and word type revealed a main effect of Areas (F2,22 = 630.246, p < .001), again with maximal CA activation in 'central' connector hub areas. In addition, a significant interaction of factors WordType, PeriExtra, TempFront and Areas (F2,22 = 137,433, p < .001) emerged, confirming different activation levels between word type circuits across the network's areas. Because of the differences between the peri-and the extrasylvian systems, we also ran a second statistical analysis on each of the two systems separately. The 3-way ANOVA revealed a main effect of Areas on both perisylvian (F2,22 = 667.146, p < .001) and extrasylvian (F2,22 = 268.1345, p < .001) systems. Whereas the perisylvian areas did not show any significant differences in peak amplitude between the two circuit types (Fi,n = 0. 98, p =.76), the extrasylvian system revealed significant interactions of factors WordType, and TempFront (F1,11 = 518.7315, p < .001), and of WordType, TempFront and Areas (F2,22 = 109.3367, p < .001), showing different activation dynamics across the extrasylvian areas between the circuits of the two word categories (Fig. 5 - left-hand side).

3.2.2 Area-specific activation time-course - peak delay results

Fig. 6 and 7 delineate the area-specific activation time courses of semantic circuits of object- (A) and action-related words (B) across the network. The activation in different areas peaked at different times and showed different maximal amplitudes. The schematic brains at the top of each panel illustrate the area-specific peak delay and the boxplots indicate the latency of maximal activation together with their standard errors (boxes) and standard deviations (whiskers).

-----------------------------Please, insert FIGURE 6-7 about here---------------------

The activation time-courses in the perisylvian language areas exhibited a similar, cascade-like time-course for both object- and action-related CA circuits (see Fig. 6 A-B). Area A1 peaked at an early time (2 time-steps) after stimulus onset because it was driven by the sensorimotor pattern presented there. The auditory-belt (AB) area peaked at ~ 6 time-steps, and shortly followed by the parabelt (PB ~ 7) and inferior prefrontal (PFi ~ 10) areas, and finally the premotor (PMi ~ 12) and primary motor (M1i ~ 13) areas. This time-course was the same for both circuit types. By contrast, the extrasylvian semantic system (Fig. 7 A-B) seemed to exhibit different temporal activation patterns for the two types of semantic circuits. The extrasylvian connector hub areas (PFL, AT) peak activated at similar latencies as the perisylvian hubs (PFi, PB) central to the network structure (1213 time-steps). Interestingly, the multimodal prefrontal area (PFL) revealed a similar activation dynamics (~ 13 simulation time-steps) for both word types, whereas the anterior-temporal hub area (AT) peaked 1 time-step earlier for action-related words (~ 12) than for object-related ones (~ 13). Massive activation time-course differences were apparently present in non-central extrasylvian areas, i.e. in the primary and secondary visual and dorsolateral motor areas of the network. Object-related words activated their lateral premotor and temporo-occipital area shortly after the connector hubs (PML ~ 15, TO ~ 15), closely followed by the primary visual (V1 ~ 16) area. In contrast, the circuits underpinning action-related words first activated the lateral premotor (PML) area (~ 15), closely followed by temporo-occipital (TO) and lateral primary motor (M1L) areas (~ 16). Both object- and action-related words activated the primary areas of the relevant system approximate ~ 15 time-steps after word onset and at the end of the activation cascade. As visible in Fig. 7. A-B, different activation dynamics can be observed for object- and action-related words in the secondary areas of the non-relevant system (PML for object-related words and TO for action-related words). However, we note that the activation peaks were quite flat in these cases, thus leading to some variance in latencies.

To confirm these observations about the activation time-course across areas for the different word-related CAs, we ran the same 4-way ANOVA as in the previous sections, but not using peak activation latencies. The statistical analysis revealed a significant interaction of factors WordType, PeriExtra, TempFront and Areas (F2,22 = 3615.08, p < .0001), which confirms the different area-specific activation time-courses between the two word type circuits. Once again, the perisylvian cortex showed no significant differences between circuit types across the six areas (F2,22 = 0.4, p = .68). The extrasylvian cortex revealed a significant interaction of the factors WordType, TempFront and Areas (F2,22 = 4791.15, p < .0001), which confirms a different activation time-course of the extrasylvian areas for object- and action-related words, as described above.

We further ran a Bonferroni-corrected planned-comparison test (24 comparisons, corrected p < .0020) to investigate the possible difference in temporal activation between the two word-types across the neural-network. Similar activation time-course for the two word types/circuits were found across the network areas, except for the temporo-occipital (TO, p = 0.001) and the anterior-temporal (AT, p = 0.0002) visual areas. Activation times for each word/circuit type showed no significant differences between the extrasylvian connector hub areas (AT, PFL: p > 0.0080), which, however, activated significantly earlier than the modality-preferential ones (p < 0.001). Intriguingly, comparisons between modality-preferential cortices showed significant differences, expect between TO and PML (p = 0.66) for object-related word circuits and between TO and M1L (p = 0.77) for action-related ones. In the perisylvian language cortex, all comparisons between area-peak activation times showed significant differences (p < 0.001) (see Fig. 6 & 7, i.e. brain/boxplot).

For putative comparison of model data with experimental data (see also Discussion below), a further analysis of the activation dynamics was performed. Activation to both word types across sub-systems unfolded symmetrically in the perisylvian and extrasylvian cortex ("Motor"-then-"Visual" vs. "Visual"-then-"Motor" - see Fig. 5., right-hand side). These observations were fully confirmed by the 2-way ANOVA run on the data of the two systems separately (i.e. peri- and extrasylvian systems), with factors WordType (2 levels: object vs. action) and TempFront (2 levels: temporal areas vs. frontal areas). The statistical analysis showed a significant interaction of WordType and TempFront (F1,11 = 24.52, p < .0004; action words, dorsal motor sub-system: 25 simulation time-steps, ventral visual sub-system: 24.37, object words, dorsal motor sub-system: 24.14, ventral visual sub-system: 25.27) in the extrasylvian systems, confirming the symmetrical time-course of activation of the two word types, with no differences in the perisylvian language cortex (F1,11 = 0.6, p < 0.46.). Notably, the significant interaction was due to slower average activation times in the relatively more relevant semantic system (dorsal action sub-system for action

words, ventral visual sub-system for object words) compared with the less relevant sub-systems, a feature due to the absence of (slow) activation in the respective primary areas (see Fig. 5.).

4. Discussion

A neurocomputational model implementing a range of cortical areas in frontal, temporal and occipital lobes along with main features of their connectivity structure and neurophysiologically realistic learning mechanisms offers an explanation of known facts about the cortical basis of meaning processing, in particular, the fact that some areas serve a general role in semantic processing, whereas others primarily take a category-specific role. When the model was used to mimic semantic grounding of word forms in action and perceptual information in motor and visual cortex, distributed neuronal assemblies developed, which functioned as 'semantic circuits' insofar as they interlinked information about word form and meaning. Intriguingly, these semantic circuits showed different distributions across extrasylvian modality-preferential areas, as already found in a previous simulation study (Garagnani and Pulvermüller, 2016). This replicates the category-specificity of action and object words, which, in a range of neuroimaging studies, more strongly activated dorsolateral motor and ventral-stream visual areas, respectively. In contrast to the category-specific behaviour of modality-preferential areas outside the perisylvian domain, substantial amounts of neuronal machinery in connector hub areas in prefrontal and anterior temporal cortex were involved to similar degrees in both kinds of cell assemblies, consistent with a role of these connector hubs as 'semantic hubs'. As in-degree normalisation was used in the present simulations, we argue below that this functional segregation into general and category-specific semantic areas resulted from connectivity structure and especially the high 'degree' of connector hubs, rather than from overall strength of the input. In fact, in contrast to earlier work (Garagnani and Pulvermüller, 2016), area function only gradually changed from category-specificity towards a category-general role, with even connector hubs exhibiting a degree of category-specificity, a feature which may be due, in part, to the inclusion of additional connections based on neuroanatomical evidence - we return to this issue below. Finally, the novel analysis of the time courses of activation indicated that in word recognition and comprehension, auditory areas are (trivially) activated first, closely followed by connector hub and modality-preferential frontal and temporal areas. Another intriguing observation was that the extrasylvian sub-systems carrying category-specific semantic information about a given word type (i.e., the dorsolateral motor subsystem for action words and the ventral visual sub-system for object words) showed a tendency toward delayed activation relative to the other areas. Moreover, a direct comparison of the

activation dynamics of the model with real cortical activations observed during spoken word processing exhibit a degree of consistency (see section 4.2 Fig. 8). Below we discuss these findings in light of empirical data, previous neurocomputational work, and future research. It needs also to be emphasized that the present model testes, and demonstrates the validity of a neurobiological theory of language, which claims that semantic content is stored in the brain by distribution of the cell assembly circuits (CAs) spread out across cortical areas, and that the specific cortical distribution (topography) of these circuits across the network reflects semantic information, in particular, semantic category-specificity (see, for example, Pulvermuller, 1999). The semantic models most popular at present still stipulate semantic hubs as the main seat of conceptual and semantic processing without providing neurobiological explanations for such hubs, nor for their specific cortical locations. A purely verbal description of a distributed semantic circuits theory - in terms of "what fires together must also bind together" - would already provide some plausibility, but one might still object that a working model of relevant cortical areas might give rise to entirely different mechanisms, for example to the emergence of local semantic processing in a single 'interface system' rather than distributed circuits that bind semantic information. Similarly, even if one is inclined to accept that distributed circuits reach into specific sensory and/or motor cortices, it would still be unclear - solely on the basis of a logical argument - whether such 'category-specific' distribution is restricted to primary areas, should include primary and secondary ones, or whether semantic specificity - as indicated by the present results - reaches the highest level of connection hubs, which, as most models postulate, are category-general and relevant for all semantic categories to the same degree.

4.1 Semantic hubs vs. category-specificity in the human brain: explaining both by a neuromechanistic circuit-level model

Diverging theories of semantic representation have been proposed to explain the extensive empirical findings about the brain basis of meaning processing revealed by neuropsychological and neurophysiological/imaging studies in patients and healthy subjects. As mentioned in the introduction, cognitive neuroscience has posited the existence of several convergence areas or "semantic hubs" that enable associating different aspects of conceptual and semantic knowledge. These areas have been located in the inferior and dorsolateral prefrontal, inferior parietal, superior temporal and anterior ventral temporal cortex, and postulated to equally process the meaning of all types of signs and symbols (Bookheimer, 2002; McCrory et al., 2000; Patterson et al., 2007; Pulvermuller, 2013). A complementary position emphasizes the importance of other cortical regions

for semantic processing which are particularly relevant for specific word types related to specific semantic categories, such as animals, tools or actions. A range of relevant neuroimaging studies have shown the relevance of the motor cortex during conceptual processing of action-related words (Dreyer et al., 2015; Grisoni et al., 2016; Hauk and Pulvermüller, 2004; Hauk et al., 2004; Shtyrov et al., 2014) and of the sensory cortex during conceptual processing of visually related words (e.g. colours, animals or object-related words) (Damasio et al., 1996; Tranel et al., 1997). Furthermore, recent neurophysiological studies (EEG-MEG) show early (<200 ms) and automaticity brain activation reflecting semantic differences (e.g., Moseley et al., 2013; Pulvermüller et al., 2005b). This evidence, which we discussed extensively in the introduction above, is consistent with the claim that semantic processing is distributed across, and divided up between, category-general hubs and category-specific areas. The frequently emphasized need for an integrative explanation of both general and semantic areas along with their location (Binder and Desai, 2011; Pulvermüller, 2013) is now being answered by results from the network simulations we report here.

The explanation of hubs and category-specificity requires reference to an intermediate level of computational simulation of neuronal circuits which bind together specific word forms and their semantic, meaning-related features (Pulvermüller et al., 2014). The formation of these semantic circuits results from (i) the correlation structure of 'grounding' sensorimotor semantic information and co-occurring word forms, (ii) the neurobiologically realistic learning and therefore mapping of the correlations on neuronal connection strengths and (iii) the structural information immanent to the neuroanatomy of cortical areas and their connectivity. As these circuits map sensorimotor correlations, they bridge between those neurons in sensory and motor areas where information -and thus correlated activation - is present during learning. This leads to category-specificity of circuit topographies, with action words such as "run" yielding cell assemblies reaching into motor systems and object words such as "sun" being implemented as circuits strongly linking up with neurons in visual cortices (Kiefer et al., 2008; Pulvermüller, 2013). These distributed word-related CA circuits did not extend into the non-relevant sub-systems (M1L for object- and V1 for action-related words) because neural activity of these areas presented a low degree of correlation. This is because during training these areas were stimulated with random patterns that changed in every learning episode (see Materials and Methods). Consequently, following the correlation based learning rule, object-related CA circuits exhibited a larger density in the visual (V1, TO, AT) than in the motor areas (M1L, PML, PFL) and vice versa for action-related words (Fig. 3).

It should be clarified here that the presence of a random-noise pattern to the non-relevant subsystems was necessary to prevent the extensions of the semantic circuits into motor areas for object-related and visual areas for action-related words. In fact, in an additional set of word learning

simulations, network training without the random noise pattern being present in the non-relevant sub-systems failed to produce a category-specific distribution. This observation further documents the important function of neuronal noise in the brain and in brain-like networks (Doursat and Bienenstock, 2006), which prevents excessive CA growth. We conclude that noise in primary areas is critical for obtaining semantic cortical circuits with category-specific signatures. In essence, as it is important to learn that the word "run" relates to certain motor patterns, it is likewise important to learn that variable visual inputs ('noise') typically occur during running so that specific visual features are de-correlated from the word form. We note that under deprived conditions, for example in blind language learners, this type of de-correlating sensory-related noise is missing in the deprived primary cortex. Resultant CA growth into the ventral stream may explain why blind individuals activate visual areas in linguistic and semantic processing (see Bedny et al., 2011; Neville and Bavelier, 2002).

In order to connect information about actions and perceptions available in the primary cortices, activity must run through connector hub areas. Therefore, neurons in multimodal cortices are included in all types of semantic circuits to a similar degree. This explains the existence and cortical location of semantic hubs in inferior and dorsolateral prefrontal cortex and in anterior and superior temporal cortex. Our model did not include areas of the parietal cortex, but if it did, it is foreseeable that the same localisation mechanisms will apply to the additional lobar system so that an additional 'semantic hub' in posterior parietal cortex (posterior supramarginal gyrus, intraparietal sulcus and angular gyri) might emerge. A new finding of the present work is the emergence of a degree of category-specificity also in extrasylvian hub areas. Earlier simulations by Garagnani and Pulvermuller (2016) had found no category differences in any of the hub areas. This may have been due, in part, to the reduced input to extrasylvian hub areas implicated by the absence of connections between ventral and dorsolateral prefrontal cortex and likewise between anterior inferior and posterior superior temporal cortex. As these connections have meanwhile been documented by anatomical studies (Gierhan, 2013; Yeterian et al., 2012), they were included in the present simulations and a small but significant degree of category-specificity in these hub areas was the consequence.

A fruitful target for future research will be to investigate the possibility of category-specific semantic deficits after lesions in anterior temporal and dorsolateral prefrontal cortex. In this context, a closer look at patients in early stages of semantic dementia may be crucial, because these patients sometimes show lesions restricted to anterior and inferior temporal areas (Patterson et al., 2007). Some work in this field suggests no differences in processing different semantic categories (Lambon Ralph et al., 2007), but other studies have reported some differences, for example between

colour- and form-related words (Gainotti, 2012; Pulvermüller et al., 2010). Stroke- and encephalitis-induced lesions of the multimodal parts of the left temporal lobe (corresponding to area AT in the network) have also been found to cause category-specific word processing deficits for animals, persons, and living things (Damasio et al., 1996; Gainotti, 2012; Pulvermüller et al., 2010; Warrington and Shallice, 1984). Thus, it seems that there is at least some evidence for category-specificity in the extrasylvian anterior-temporal connector hubs. Only future research can validate or falsify the model's prediction about a slight but significant category difference between object and action-related words after focal anterior-temporal and dorsolateral prefrontal damage.

There is quite a bit of debate about the prominence of different areas for semantic processing. Some approaches hold that true semantic processing is only present in the multimodal hubs, and modality-preferential areas only serve an optional, 'enriching' or 'colouring' function (Mahon and Caramazza, 2008). Although the network model we present here offers no justification for such a view - because all parts of the distributed semantic circuits contribute to their function and there is no basis for excluding circuit parts when it comes to function - the model offers an explanation of why some areas across which the circuits are distributed are functionally more important than others. Factors which come in here are the general location of an area's neurons with respect to the network's connectivity structure (topology) - with gradually more functional contributions of 'central' areas than 'peripheral' ones - and, importantly, the relative CA neuron density a circuit shows across areas. In this context, the generally observed main effects of the level of area, with relatively more CA neurons in secondary than primary and also much more neurons in connector hub areas than in secondary ones, is of critical importance. In the previous study (Garagnani and Pulvermüller, 2016), it was not entirely clear whether the relatively high number of CA neurons (and thus circuit neuron densities) in connector hub areas was due to the stronger input these areas generally received (higher 'in-degree') or to the network topology, or both. Here, we performed in-degree normalization (see Methods) and thus excluded the sheer amount of activity entering an area as explanatory factor. In spite of in-degree normalization across sub-systems, which ensured that all network areas received (on average) equal quantities of inputs, circuit cell density was still higher in the connector hub areas in the centre of the network architecture, where phonological and semantic word circuits converge. This result is consistent with the statement that network topology plays a major role in determining the prominence of connector hubs for general semantic processing. However, we note that larger circuit densities in the 'centre' of networks have also been observed with next neighbour between-area connections only, suggesting that, apart from its 'degree' and resultant hub status as such, the 'centrality' of an area within the network is a relevant factor (Garagnani et al., 2008).

In sum, the present neural network simulations exhibit the spontaneous formation of semantic CA circuits distributed over modality-preferential and "higher" multimodal convergence areas and mechanistically explain the emergence in the cortex of both category-specific and general semantic processes. In addition, the use of a more realistic architecture leads to the presence of moderate category-specificity in connector hub areas outside the perisylvian region. The spontaneous formation of these semantic circuits is based on, and explained by, well-documented learning mechanisms of Hebbian synaptic plasticity and cortical area and connectivity structure. These simulation results explain why modality-preferential areas are activated relatively more strongly by specific semantic categories and why the connector areas become semantic hubs and to a degree similarly great, relevance for processing all kinds of meanings.

4.2 Neurophysiological mechanisms underlying word recognition and understanding: simulating the time-course of semantic activation

The semantic circuits that had formed as a consequence of correlation learning were reactivated from the acoustic phonological end to simulate the area-specific cortical activation dynamics of spoken word understanding and to provide a functional estimate of category-general and -specific semantic activation strength, topography, and timing. Comparison of maximum circuit activity levels per area and word type revealed a dissociation similar to that found in the structural analysis of circuit topographies reported above. In particular, object-related words activated the visual system (V1, TO) more strongly than the motor system (M1l, PMl) and, for action-related words, motor system peak activation was relatively stronger (Fig. 5 B - left-hand side). As before, the perisylvian auditory and articulatory sub-systems did not show any significant difference in amplitude between word-types (see Fig. 5. A - left-hand side). Stronger activation in the connector hubs (AT, PFl, PB, PFi) than in secondary (TO, AB, PML, PMi) areas, and stronger activation in secondary than in primary (A1, V1, M1L, M1i) areas, was found. The word-category dissociation and the different activity levels predicted during simulated word-recognition processes is a direct consequence of the distinct cortical topographies of object- and action-related semantic circuits, which emerged in the model during learning, with more CA cells leading to correspondingly more activity during CA circuit ignition.

The area-specific activation time-course of the multi-area network illustrated in Fig. 6 and 7 (Brain and boxplot upper part) showed similar activation dynamics for object- and action-related words. For both word types, the perisylvian language system exhibited a cascade of activations whose peaks unfold (in a sequential manner) over a period of approximately 12 simulation time-

steps. Activation was first present in the primary auditory areas A1, driven by the external stimulus, and then spread across the perisylvian areas, terminating in the primary articulatory areas (M1i). In contrast, activation in the sensorimotor semantic areas is near-simultaneous, with all peaks concentrated within a period of just 5 simulation time-steps (hub areas activate first, regardless of word type). The "near-simultaneous" effect of the CA cells activation processes in sensorimotor areas is caused by the rich neuroanatomical connections of the convergence hub areas, which link together the different modality-preferential cortices. Therefore, upon reaching the language hubs (PB-PFi), activity leads to the simultaneous "ignition" of the CA cells present in the anterior -temporal (AT) and dorsolateral prefrontal (PFL) hub cortices, which, in turn, quickly activate the modality-preferential CAs. Thus, the inherent connectivity structure of the model leads to a near-simultaneous activation of the most richly connected hub areas as compared to the primary and secondary cortices. The multimodal hubs can be seen as a "crossroad" where information from different modality-preferential systems converges; after full ignition, CA activity gradually disappears in the multi-area network (see Fig. 4), ending in the modality-preferential areas - i.e. primary hand-motor area (M1L) for action-, and primary visual cortex (V1) for object-related words. In other words, the modality-preferential cortices (for object words V1 and TO areas and for action words M1L and PML areas) activate after all other areas. Hence, on the basis of the activation dynamics exhibited by the present model, we would predict that during semantic information retrieval, activation should spread in a cascade-like fashion across the perisylvian language areas; sensorimotor areas should then activate near-simultaneously, with semantic hubs activating before the modality-preferential areas, where additional semantic information is held.

In a recent simulation study (McNorgan et al., 2011), on the basis of within- and cross-modality feature- and concept-relatedness judgment data the authors argue that 'deep' models of semantic grounding (i.e., which involve several processing steps between sensory, and between sensory and motor components) are necessary to explain their results. Because our model is neuroanatomically realistic and, as such, it incorporates indirect multi-step links between modality-preferential sensorimotor regions, it can be considered a neurobiologically motivated 'deep' semantic model in the sense of McNorgan et al. Therefore, we conjecture that it might also be compatible with their results, although, as our present focus was on modelling neurophysiological mechanisms, we have not attempted to replicate the outcome of their specific simulations.Experimental studies analysing the latency of semantic processes in language perception suggest that semantic information provided by words is already retrieved within ~200 ms after stimulus presentation (Brown and Lehmann, 1979; Hauk et al., 2008; Preissl et al., 1995; Pulvermüller et al., 1999). Moreover, recent MEG-EEG recordings have shown that different semantic categories (visually presented) activated

different cortical areas within ~150 ms; at this point in time, action words activated mostly the motor system and object words activated the visual system (Moseley et al., 2013). However, these neuroimaging techniques with high temporal resolution (such as MEG and EEG) do not offer a sufficiently high spatial resolution to detect fine-grained differences between multimodal semantic hubs and modality-preferential areas implemented in the neural network (for example, between premotor and prefrontal areas). Therefore, we further investigated the activation dynamics of the four sub-systems, i.e. auditory, articulatory, visual and motor sub-systems implemented in the model, and compared their respective average activation time courses with each other and with real cortical activations observed during spoken word processing. Fig. 8 reports results from a Magnetoencephalography (MEG) study investigating the temporal activation dynamics evoked by action-related words (Pulvermuller et al., 2005b) and relates them to the activation time courses obtained from our model after stimulating area A1 with the 'acoustic patterns' of action- and object-related 'words'. Although the alignment of simulation time-steps and real time is always to a degree tentative, the near-simultaneous but still fast-cascading activation from superior temporal to inferior frontal and finally dorsal action-related areas exhibited by the cortical sources estimated from the MEG recordings is paralleled by the model results. Note, however, that the delay between superior temporal and inferior frontal activations is relatively longer in the simulations than in the MEG sources, thus also indicating a discrepancy. For relating simulation results more directly to empirical data, it might be advantageous to perform analogous semantic learning experiments in healthy subjects and then compare the brain and network responses of the processing of the learnt items (see also below).

Please, insert FIGURE 8 about here

In sum, the model shows a "near-simultaneous" activation time-course of the semantic areas; the semantic hubs, anterior-temporal (AT) and dorsolateral prefrontal areas (PFl), activate first, and are then followed by the modality-preferential areas carrying category-specific semantic information. The perisylvian language areas exhibited a cascade of activations, with no word type effects. Most of the empirical studies about semantic processing performed in the past used words from real natural language, making it impossible to control the way these words have been learned, or to isolate the relevant semantic features from the many other putatively confounding psycholinguistic and psychological features distinguishing the different lexical classes between each other (Kemmerer, 2014; Pulvermüller, 1999; Vigliocco et al., 2011). A well-designed word learning experiment employing neuroimaging methods with high spatial and temporal resolution

(EEG/MEG and fMRI) is needed to test the validity of the present model's results and predictions, and identify where the neural correlates of novel object- and action-related words emerge in the brain, and at which point in time of the recognition process their activation occurs.

5. Summary and Conclusions

Current neurosemantic theories still diverge about the role of category-specific and category-general semantic mechanisms and about the contribution of modality-preferential and multimodal ('amodal') brain systems in semantic processing (Barsalou, 2008; Bookheimer, 2002; Devlin et al., 2003; Gallese and Lakoff, 2005; Martin and Chao, 2001; Patterson et al., 2007; Pulvermüller, 2005; Warrington and McCarthy, 1987). Here we applied a neural-network model replicating anatomical and physiological features of a range of cortical areas including sensorimotor, multimodal and language areas to investigate the neurobiological mechanisms underlying conceptual semantic grounding of words in action- and object-related information. The word learning simulations documented the spontaneous emergence of word/symbol-specific, tightly interconnected cell assemblies within the larger networks, each binding articulatory-acoustic word-forms to sensorimotor semantic information. Due to network structure, connectivity, and Hebbian associative learning, which maps neuronal correlations, the emerging 'semantic circuits' for object- and action-related words exhibited category-specificity primarily in modality-preferential areas; the "higher" multimodal connector hub areas central to the network architecture showed only moderate category-specificity (Fig. 3 and 4). Due to their central position in the model architecture, connector hubs showed highest cell densities of both types of semantic circuits, therefore acting as 'semantic hubs'. Word category dissociations were confirmed by the reactivation of the cell assembly circuits during simulated word recognition and comprehension processes. The model's results, which can be compared with real experimental data (see Fig. 8), predict a symmetrical temporal activation for object- and action-related words, with the semantic hub areas activating first and modality-preferential ones slightly later (Fig. 6 and 7). Interestingly, extrasylvian systems relevant for semantic processing of a given word category activated with a delay upon the relevant system, whereby strong dorsal motor systems activation were preceded by weak ventral visual system activation to action words, while strong ventral visual activations to objects words were preceded by weak dorsal motor processes (Fig. 5). This observation (prediction) also calls for future experimental testing. The present simulations demonstrate that realistic neurocomputational models can elucidate aspects of semantic processing in the cortex and integrate findings from neuroimaging studies. In sum, the model illustrates the spontaneous emergence of both category-specific and

general semantic hub areas and, on the basis of well-established neuroscience principles, offers a mechanistic explanation of where and when meaning is processed in the brain.


Supported by the Freie Universität Berlin, Deutsche Forschungsgemeinschaft (Pu 97/16-1), and the EPSRC and BBSRC, UK (project on Brain-inspired architecture for brain embodied language, BABEL (grant no. EP/J004561/1)). The authors would also like to thank Valerie Keller for suggestions and comments on an earlier version of the manuscript and the HPC Service of ZEDAT, Freie Universität Berlin, for computing time.

APPENDIX A - Full model specification

Each of the 12 simulated areas (see Fig. 1.B) was implemented as two layers of artificial neuronlike elements ("cells"), 625 excitatory and 625 inhibitory, thus resulting in 15,000 cells in total. Each excitatory cell "e" can be considered the network equivalent of a local cluster, or column, of approximately 25,000 real excitatory cortical neurons, that is pyramidal cells, while its twin inhibitory cell "i" (see Fig. 1.C) models the cluster of inhibitory interneurons situated within the same cortical column (Eggert and van Hemmen, 2000; Wilson and Cowan, 1972). The activity state of each cell e is uniquely defined by its membrane potential V(e,t), representing the average of the sum of all (excitatory and inhibitory) postsynaptic potentials acting upon neural pool (cluster) e at time t, and governed by the following equation:

T_ dn^i) = _V(e,t) + k(V (e,t) + k2q(e,t)) (A1) dt

where VIn (e, t) is the net input to cell e at time t (sum of all inhibitory and excitatory postsynaptic potentials - I/EPSPs; inhibitory synapses are given a negative sign - plus a constant baseline value Vb), t is the membrane's time constant, k1, k2 are scaling constants and 7/(-,t) is a white noise process with uniform distribution over [-0.5,0.5]. Note that noise is an inherent property of each model cell, intended to mimic the spontaneous activity (baseline firing) of real neurons. Therefore, noise was constantly present in all areas, in equal amounts (inhibitory cells have k2=0, i.e., the noise is generated just by the excitatory cells, for simplicity).

Cells produce a graded response that represents the average firing rate of the neuronal cluster; in particular, the output (transformation function) of an excitatory cell e at time t is:

O (e, t) =

0 ifV (e, t )< p

V(e,t)-j) if0 < (V(e,t)-j)< 1

1 otherwise

O(e,t) represents the average (graded) firing rate (number of action potentials per time unit) of cluster e at time t; it is a piecewise-linear sigmoid function of the cell's membrane potential V(e,t), clipped into the range [0, 1] and with slope 1 between the lower and upper thresholds p and p +1. The output O(i,t) of inhibitory cell i is 0 if V(i,t)<0, and V(i,t) otherwise. In excitatory cells, the value of the threshold p in Eq. (A2) varies in time, tracking the recent mean activity of the cell so as to implement neuronal adaptation (Kandel et al., 2000). Thus, stronger activity leads to a higher threshold in subsequent time-steps. More precisely,

p(e, t) = a -w(e,t) (A3)

where co(e,t) is the time-average of cell e's recent output and a is the "adaptation strength" (see below for the exact parameter values used in the simulations). For an excitatory cell e, the approximate time-average w(e,t) of its output O(e,t) is estimated by integrating the linear differential equation Eq. (A4.1) below with time constant ta, assuming initial average &>(e,0)=0:

_ . t) = -W(e, t) + O(e, t) (A4.1) A dt

Local (lateral) inhibitory connections (see Fig. 1.C) and area-specific inhibition are also implemented, realising, respectively, local and global competition mechanisms (Duncan, 2006, 1996) and preventing activation from falling into non-physiological states (Braitenberg and Schuz, 1998). More formally, in Eq. (A1) the input VIn(e,t) to each excitatory cell of the same area includes

an area-specific ("global") inhibition term kS • coS(e,t), which is subtracted from the total sum of the I/EPSPs postsynaptic potentials Vn in input to the cell, with (Ds(e,t) defined by:

^. ^eH ^ (e, t) + £ O(e, t) (A4.2)


The low-pass dynamics of the cells (Eq. (A1), (A2), (A4.1-2)) are integrated using the Euler scheme with step size At, where At = 0.5 ms.

Excitatory links within and between (possibly non-adjacent) model areas are established at random and limited to a local (topographic) neighbourhood; weights are initialised at pattern, in the range [0, 0.1]. The probability of a synapse to be created between any two cells falls off with their distance (Braitenberg and Schuz, 1998) according to a Gaussian function clipped to 0 outside the chosen neighbourhood (a square of size n =19 for excitatory and n=5 for inhibitory cell projections). This produces a sparse, patchy and topographic connectivity, as typically found in the mammalian cortex (Amir et al., 1993; Braitenberg and Schuz, 1998; Douglas and Martin, 2004; Kaas, 1997).

The Hebbian learning mechanism implemented simulates well-documented synaptic plasticity phenomena of long-term potentiation (LTP) and depression (LTD), as implemented by Artola, Brocher and Singer (Artola and Singer, 1993; Artola et al., 1990). This rule, which covers both "true" Hebbian co-occurrence ("what fires together wires together") as well as decorralative "anti-Hebb" ("neurons out of sync delink") plasticity, provides a realistic approximation of known experience-dependent neuronal plasticity and learning (Finnie and Nader, 2012; Malenka and Bear, 2004; Rioult-Pedotti et al., 2000). In the model, we discretized the continuous range of possible synaptic efficacy changes into two possible levels, +Aw and _Aw (with Aw<<1 and fixed). Following Artola et al., we defined as "active" any link from an excitatory cell x such that the output O(x,t) of cell x at time t is larger than 6pre, where 6pre e]0,1] is an arbitrary threshold representing the minimum level of presynaptic activity required for LTP (or LTD) to occur. Thus, given any two cells x and y connected by a synaptic link with weight wt(x,y), the new weight wt+1(x,y) is calculated as follows:

' w (x, y) + Aw ifO (x, t) 30preandV (y, t) 30+ (LTP)

W (x, y)_ Aw ifO(x, t) 30preandO_ < V (y, t) < 0+ (LTD)

wt+1 (x y)

w (x, y)_Aw ifO(x, t)<0preandV (y, t) >0+ (LTD)

\( x, y ) otherwise

Parameter values used during the simulations

Eq. (A1) Time constant (excitatory cells): Time constant (inhibitory cells): Scaling factor: Baseline potential Noise scaling factor Global inhibition during training (during word recognition: Eq. (A3) Adaptation:

Eq. (A4.1-2) Time-average constant for CA definition: (time constant for adaptation mechanism: Global inhibition time constant: Eq. (A5) Postsynaptic potential thresholds for LTP/LTD:

0_=0.15 #+=0.15

Presynaptic output activity required for any synaptic change:

t = 2.5 (simulation time-steps) t = 5 (simulation time-steps) k1 = 0.01 Vb=0

k2= 27-V48 kS= 95 ks= 75) a= 0.01

ta = 3 (simulation time-steps) ta = 15)

TS = 12 (simulation time-steps)



Amir, Y., Harel, M., Malach, R., 1993. Cortical hierarchy reflected in the organization of intrinsic connections in macaque monkey visual cortex. J. Comp. Neurol. 334, 19-46. doi:10.1002/cne.903340103

Arikuni, T., Watanabe, K., Kubota, K., 1988. Connections of area 8 with area 6 in the brain of the macaque monkey. J. Comp. Neurol. 277, 21-40. doi:10.1002/cne.902770103

Artola, A., Brocher, S., Singer, W., 1990. Different voltage-dependent thresholds for inducing long-term depression and long-term potentiation in slices of rat visual cortex. Nature 347, 69-72. doi: 10.1038/347069a0

Artola, A., Singer, W., 1993. Long-Term depression of excitatory synaptic transmission and its relationship to long-term potentiation. Trends Neurosci. 16, 480-487.

Bak, T.H., Chandran, S., 2012. What wires together dies together: Verbs, actions and neurodegeneration in motor neuron disease. Cortex 48, 936-944. doi: 10.1016/j.cortex.2011.07.008

Barsalou, L.W., 2008. Grounded cognition. Annu. Rev. Psychol. 59, 617-645. doi:10.1146/annurev.psych.59.103006.093639

Bauer, R.H., Jones, C.N., 1976. Feedback training of 36-45 Hz EEG activity in the visual cortex and hippocampus of cats: evidence for sensory and motor involvement. Physiol.Behav. 17, 885-890.

Bedny, M., Pascual-Leone, A., Dodell-Feder, D., Fedorenko, E., Saxe, R., 2011. Language processing in the occipital cortex of congenitally blind adults. Proc. Natl. Acad. Sci. 108, 4429-4434.

Bibbig, A., Wennekers, T., Palm, G., 1995. A neural network model of the coritco-hippocampal interplay and the representation of contexts. Behav. Brain Res. 66, 169-175.

Binder, J.R., Desai, R.H., 2011. The neurobiology of semantic memory. Trends Cogn. Sci. doi:10.1016/j.tics.2011.10.001

Bookheimer, S., 2002. Functional MRI of language: new approaches to understanding the cortical organization of semantic processing. Annu. Rev. Neurosci 25, 151-88. doi:10.1146/annurev.neuro.25.112701.142946

Boulenger, V., Hauk, O., Pulvermüller, F., 2009. Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension. Cereb. Cortex 19, 1905-1914. doi: 10.1093/cercor/bhn217

Braitenberg, V., 1978. Cell assemblies in the cerebral cortex, in: Heim, R., Palm, G. (Eds.), Theoretical Approaches to Complex Systems. Springer, Berlin, pp. 171-188.

Braitenberg, V., Schüz, A., 1998. Cortex: Statistics and Geometry of Neuronal Connectivity, in: Springer. Springer, Berlin. doi:10.1007/978-3-662-03733-1_27

Bressler, S.L., Coppola, R., Nakamura, R., 1993. Episodic multiregional cortical coherence at multiple frequencies during visual task performance. Nature 366, 153-156. doi:10.1038/366153a0

Brown, W.S., Lehmann, D., 1979. Verb and noun meaning of homophone words activate different cortical generators: a topographic study of evoked potential fields. Exp. Brain Res. 2, 159168.

Catani, M., Jones, D.K., Ffytche, D.H., 2005. Perisylvian language networks of the human brain. Ann. Neurol. 57, 8-16. doi:10.1002/ana.20319

Chafee, M. V, Goldman-Rakic, P.S., 2000. Inactivation of parietal and prefrontal cortex reveals interdependence of neural activity during memory-guided saccades. J. Neurophysiol. 83, 1550-1566.

Chao, L.L., Haxby, J. V, Martin, A., 1999. Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects. Nat. Neurosci. 2, 913. doi:10.1038/13217

Damasio, H., Grabowski, T.J., Tranel, D., Hichwa, R.D., Damasio, A.R., 1996. A neural basis for lexical retrieval. Nature 380, 499-505. doi:10.1038/380499a0

Deiber, M.P., Passingham, R.E., Colebatch, J.G., Friston, K.J., Nixon, P.D., Frackowiak, R.S.J., 1991. Cortical areas and the selection of movement: a study with positron emission tomography. Exp. Brain Res. 84, 393-402. doi:10.1007/BF00231461

Devlin, J.T., Matthews, P.M., Rushworth, M.F., 2003. Semantic processing in the left inferior prefrontal cortex: a combined functional magnetic resonance imaging and transcranial magnetic stimulation study. J. Cogn. Neurosci. 15, 71-84. doi:10.1162/089892903321107837

Distler, C., Boussaoud, D., Desimone, R., Ungerleider, L.G., 1993. Cortical connections of inferior temporal area TEO in macaque monkeys. J. Comp. Neurol. 334, 125-150. doi:10.1002/cne.903340111

Douglas, R.J., Martin, K.A.C., 2004. Neuronal Circuits of the Neocortex. Annu. Rev. Neurosci. 27, 419-451. doi: 10.1146/annurev.neuro.27.070203.144152

Doursat, R., Bienenstock, E., 2006. Neocortical self-structuration as a basis for learning, in: ... on Development and Learning (ICDL ... Indiana University, Bloomington, Indiana, pp. 1-6.

Dreyer, F.R., Frey, D., Arana, S., von Saldern, S., Picht, T., Vajkoczy, P., Pulvermüller, F., 2015. Is the motor system necessary for processing action and abstract emotion words? Evidence from focal brain lesions. Front. Psychol.

Dum, R.P., Strick, P.L., 2005. Frontal lobe inputs to the digit representations of the motor areas on the lateral surface of the hemisphere. J.Neurosci. 25, 1375-1386. doi:10.1523/JNEUR0SCI.3902-04.2005

Dum, R.P., Strick, P.L., 2002. Motor areas in the frontal lobe of the primate. Physiol. Behav. 77, 677-682.

Duncan, J., 2006. EPS Mid-Career Award 2004: brain mechanisms of attention. Q. J. Exp. Psychol. (Hove). 59, 2-27. doi:10.1080/17470210500260674

Duncan, J., 1996. Competitive brain systems in selective attention. Int. J. Psychol. 31, 3343.

Eacott, M.J., Gaffan, D., 1992. Inferotemporal-frontal disconnection: The uncinate fascicle and visual associative learning in monkeys. Eur. J. Neurosci. 4, 1320-1332. doi: 10.1111/j.1460-9568.1992.tb00157.x

Eggert, J., van Hemmen, J.L., 2000. Unifying framework for neuronal assembly dynamics. Phys. Rev. E. Stat. Phys. Plasmas. Fluids. Relat. Interdiscip. Topics 61, 1855-1874. doi: 10.1103/PhysRevE.61.1855

Fadiga, L., Craighero, L., Buccino, G., Rizzolatti, G., 2002. Speech listening specifically modulates the excitability of tongue muscles: A TMS study. Eur. J. Neurosci. 15, 399-402. doi:10.1046/j.0953-816x.2001.01874.x

Finnie, P.S.B., Nader, K., 2012. The role of metaplasticity mechanisms in regulating memory destabilization and reconsolidation. Neurosci. Biobehav. Rev. 36, 1667-1707. doi: 10.1016/j.neubiorev.2012.03.008

Fry, D., 1966. The development of the phonological system in the normal and deaf child, in: Smith, F., Miller, G.A. (Eds.), The Genesis of Language. MIT Press, Cambridge, MA, pp. 187-206.

Fuster, J.M., Bauer, R.H., Jervey, J.P., 1985. Functional interactions betwene inferotemporal and prefrontal cortex in a cognitive task. Brain Res. 330, 299-307.

Gainotti, G., 2012. The format of conceptual representations disrupted in semantic dementia: A position paper. Cortex 48, 521-529. doi:

Gainotti, G., 2010. The influence of anatomical locus of lesion and of gender-related familiarity factors in category-specific semantic disorders for animals, fruits and vegetables: A review of single-case studies. Cortex 46, 1072-1087. doi:10.1016/j.cortex.2010.04.002

Gallese, V., Lakoff, G., 2005. The Brain's concepts: the role of the Sensory-motor system in

conceptual knowledge. Cogn. Neuropsychol. 22, 455-79. doi:10.1080/02643290442000310

Garagnani, M., Pulvermüller, F., 2016. Conceptual grounding of language in action and

perceptions neurocomputational model of the emergence of category specificity and semantic hubs. Eur. J. Neurosci. in press. doi:10.1111/ejn.13145

Garagnani, M., Pulvermüller, F., 2013. Neuronal correlates of decisions to speak and act:

Spontaneous emergence and dynamic topographies in a computational model of frontal and temporal areas. Brain Lang. 127, 75-85. doi:10.1016/j.bandl.2013.02.001

Garagnani, M., Pulvermüller, F., 2011. From sounds to words: A neurocomputational model of adaptation, inhibition and memory processes in auditory change detection. Neuroimage 54, 170-181. doi: 10.1016/j.neuroimage.2010.08.031

Garagnani, M., Wennekers, T., Pulvermüller, F., 2009. Recruitment and consolidation of cell assemblies for words by way of hebbian learning and competition in a multi-layer neural network. Cognit. Comput. 1, 160-176. doi:10.1007/s12559-009-9011-1

Garagnani, M., Wennekers, T., Pulvermüller, F., 2008. A neuroanatomically grounded Hebbian-learning model of attention-language interactions in the human brain. Eur. J. Neurosci. 27, 492-513. doi:10.1111/j.1460-9568.2008.06015.x

Gierhan, S.M.E., 2013. Connections for auditory language in the human brain. Brain Lang. 127, 205-221. doi: 10.1016/j.bandl.2012.11.002

Gilbert, C.D., Wiesel, T.N., 1983. Clustered intrinsic connections in cat visual cortex. J. Neurosci. 3, 1116-1133.

Grisoni, L., Dreyer, F., Pulvermüller, F., 2016. Somatotopic semantic priming and prediction in the motor system. Cereb. Cortex in press.

Hauk, O., Johnsrude, I., Pulvermüller, F., 2004. Somatotopic Representation of Action Words in Human Motor and Premotor Cortex. Neuron 41, 301-307. doi:10.1016/S0896-6273(03)00838-9

Hauk, O., Pulvermüller, F., 2004. Neurophysiological Distinction of Action Words in the Fronto-Central Cortex. Hum. Brain Mapp. 21, 191-201. doi:10.1002/hbm.10157

Hauk, O., Shtyrov, Y., Pulvermüller, F., 2008. The time course of action and action-word

comprehension in the human brain as revealed by neurophysiology. J. Physiol. Paris 102, 5058. doi: 10.1016/j .jphysparis.2008.03.013

Hebb, D.O., 1949. The organization of behavior, John Wiley. New York.

Hoenig, K., Sim, E.-J., Bochev, V., Herrnberger, B., Kiefer, M., 2008. Conceptual flexibility in the human brain: dynamic recruitment of semantic maps from visual, motor, and motion-related areas. J. Cogn. Neurosci. 20, 1799-1814. doi:10.1162/jocn.2008.20123

Huyck, C.R., Passmore, P.J., 2013. A review of cell assemblies. Biol. Cybern. 107, 263-288. doi:10.1007/s00422-013-0555-5

Kaas, J.H., 1997. Topographic Maps are Fundamental to Sensory Processing. Brain Res. Bull. 44, 107-112. doi:

Kaas, J.H., Hackett, T.A., 2000. Subdivisions of auditory cortex and processing streams in primates. Proc. Natl. Acad. Sci. U. S. A. 97, 11793-9. doi:10.1073/pnas.97.22.11793

Kandel, E.R., Schwartz, J.H., Jessell, T.M., 2000. Principles of neural science. McGraw-hill New York.

Kemmerer, D., 2014. Word classes in the brain: Implications of linguistic typology for cognitive neuroscience. Cortex 58, 27-51. doi:

Kemmerer, D., Rudrauf, D., Manzel, K., Tranel, D., 2012. Behavioral patterns and lesion sites associated with impaired processing of lexical and conceptual knowledge of actions. Cortex 48, 826-848. doi:10.1016/j.cortex.2010.11.001

Kiefer, M., 2005. Repetition-priming modulates category-related effects on event-related potentials: further evidence for multiple cortical semantic systems. J. Cogn. Neurosci. 17, 199-211. doi:10.1162/0898929053124938

Kiefer, M., Sim, E.-J., Herrnberger, B., Grothe, J., Hoenig, K., 2008. The sound of concepts: four markers for a link between auditory and conceptual brain systems. J. Neurosci. 28, 1222412230. doi:10.1523/JNEUR0SCI.3579-08.2008

Lambon Ralph, M.A., Lowe, C., Rogers, T.T., 2007. Neural basis of category-specific semantic deficits for living things: evidence from semantic dementia, HSVE and a neural network model. Brain 130, 1127-1137.

Lu, M.T., Preston, J.B., Strick, P.L., 1994. Interconnections between the prefrontal cortex and the premotor areas in the frontal lobe. J. Comp. Neurol. 341, 375-392. doi:10.1002/cne.903410308

Mahon, B.Z., Caramazza, A., 2008. A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. J. Physiol. Paris 102, 59-70. doi: 10.1016/j.jphysparis.2008.03.004

Makris, N., Pandya, D.N., 2009. The extreme capsule in humans and rethinking of the language circuitry. Brain Struct. Funct. 213, 343-358. doi:10.1007/s00429-008-0199-8

Malenka, R.C., Bear, M.F., 2004. LTP and LTD: An embarrassment of riches. Neuron 44, 5-21.

Martin, A., 2007. The Representation of Object Concepts in the Brain. Annu. Rev. Psychol. 58, 2545. doi:10.1146/annurev.psych.57.102904.190143

Martin, A., Chao, L.L., 2001. Semantic memory and the brain: Structure and processes. Curr. Opin. Neurobiol. 11, 194-201. doi:10.1016/S0959-4388(00)00196-3

Matthews, G.G., 2001. Neurobiology: molecules, cells and systems. Blackwell Sci.

McCrory, E., Frith, U., Brunswick, N., Price, C., 2000. Abnormal functional activation during a simple word repetition task: A PET study of adult dyslexics. J. Cogn. Neurosci. 12, 753-62. doi:10.1162/089892900562570

McNorgan, C., Reid, J., McRae, K., 2011. Integrating conceptual knowledge within and across representational modalities. Cognition 118, 211-233. doi:10.1016/j.cognition.2010.10.017

Meyer, J.W., Makris, N., Bates, J.F., Caviness, V.S., Kennedy, D.N., 1999. MRI-Based topographic parcellation of human cerebral white matter. Neuroimage 9, 1-17. doi:S1053811998903834 [pii]

Mion, M., Patterson, K., Acosta-Cabronero, J., Pengas, G., Izquierdo-Garcia, D., Hong, Y.T., Fryer, T.D., Williams, G.B., Hodges, J.R., Nestor, P.J., 2010. What the left and right anterior fusiform gyri tell us about semantic memory. Brain 133, 3256-3268. doi: 10.1093/brain/awq272

Moseley, R.L., Pulvermüller, F., Shtyrov, Y., 2013. Sensorimotor semantics on the spot: brain activity dissociates between conceptual categories within 150 ms. Sci. Rep. 3, 1928. doi: 10.1038/srep01928

Neville, H., Bavelier, D., 2002. Human brain plasticity: Evidence from sensory deprivation and altered language experience. Prog. Brain Res. 138, 177-188. doi:10.1016/S0079-6123(02)38078-6

Palm, G., 1982. Neural assemblies. Springer, Berlin.

Palm, G., Knoblauch, A., Hauser, F., Schüz, A., 2014. Cell assemblies in the cerebral cortex. Biol. Cybern. 108, 559-572. doi:10.1007/s00422-014-0596-4

Pandya, D.N., 1995. Anatomy of the auditory cortex. Rev. Neurol. (Paris). 151, 486-494. doi:10.1016/B978-0-323-05283-2.00129-4

Pandya, D.N., Barnes, C.L., 1987. Architecture and connections of the frontal lobe, in: Perecman, E. (Ed.), The Frontal Lobes Revisited. The IRBN Press, New York, pp. 41-72.

Pandya, D.N., Yeterian, E.H., 1985. Architecture and Connections of Cortical Association Areas, in: Peters, A., Jones, E. (Eds.), Association and Auditory Cortices SE - 1, Cerebral Cortex. Springer US, pp. 3-61. doi:10.1007/978-1-4757-9619-3_1

Parker, A., 1998. Interaction of frontal and perirhinal cortices in visual object recognition memory in monkeys. Eur. J. Neurosci. 10, 3044-3057. doi:10.1046/j.1460-9568.1998.00306.x

Parker, G.J.M., Luzzi, S., Alexander, D.C., Wheeler-Kingshott, C.A.M., Ciccarelli, O., Lambon Ralph, M.A., 2005. Lateralization of ventral and dorsal auditory-language pathways in the human brain. Neuroimage 24, 656-666. doi:10.1016/j.neuroimage.2004.08.047

Patterson, K., Nestor, P.J., Rogers, T.T., 2007. Where do you know what you know? The

representation of semantic knowledge in the human brain. Nat. Rev. Neurosci. 8, 976-87. doi: 10.1038/nrn2277

Paus, T., Castro-Alamancos, M.A., Petrides, M., 2001. Cortico-cortical connectivity of the human mid-dorsolateral frontal cortex and its modulation by repetitive transcranial magnetic stimulation. Eur. J. Neurosci. 14, 1405-1411. doi:10.1046/j.0953-816X.2001.01757.x

Penolazzi, B., Hauk, O., Pulvermüller, F., 2007. Early semantic context integration and lexical access as revealed by event-related brain potentials. Biol. Psychol. 74, 374-388. doi: 10.1016/j.biopsycho.2006.09.008

Petrides, M., Pandya, D.N., 2009. Distinct parietal and temporal pathways to the homologues of Broca's area in the monkey. PLoS Biol. 7, e1000170. doi:10.1371/journal.pbio.1000170

Preissl, H., Pulvermüller, F., Lutzenberger, W., Birbaumer, N., 1995. Evoked potentials distinguish between nouns and verbs. Neurosci. Lett. 197, 81-83. doi:10.1016/0304-3940(95)11892-Z

Price, C.J., 2000. The anatomy of language: contributions from functional neuroimaging. J. Anat. 197 Pt 3, 335-359. doi:10.1046/j.1469-7580.2000.19730335.x

Pulvermüller, F., 2013. How neurons make meaning: Brain mechanisms for embodied and abstract-symbolic semantics. Trends Cogn. Sci. 17, 458-470. doi:10.1016/j.tics.2013.06.004

Pulvermüller, F., 2005. Brain mechanisms linking language and action. Nat. Rev. Neurosci. 6, 576582. doi:10.1038/nrn1706

Pulvermüller, F., 2001. Brain reflections of words and their meaning. Trends Cogn. Sci. doi:10.1016/S1364-6613(00)01803-9

Pulvermüller, F., 1999. Words in the brain's language. Behav. Brain Sci. 22, 253-336. doi: 10.1017/S0140525X9900182X

Pulvermüller, F., 1996. Hebb's concept of cell assemblies and the psychophysiology of word

processing. Psychophysiology. doi:10.1111/j.1469-8986.1996.tb01057.x

Pulvermüller, F., Cooper-Pye, E., Dine, C., Hauk, O., Nestor, P.J., Patterson, K., 2010. The word processing deficit in semantic dementia: all categories are equal, but some categories are more equal than others. J. Cogn. Neurosci. 22, 2027-2041. doi:10.1162/jocn.2009.21339

Pulvermüller, F., Fadiga, L., 2010. Active perception: sensorimotor circuits as a cortical basis for language. Nat. Rev. Neurosci. 11, 351-360. doi:10.1038/nrn2811

Pulvermüller, F., Garagnani, M., 2014. From sensorimotor learning to memory cells in prefrontal and temporal association cortex: A neurocomputational study of disembodiment. Cortex 57, 121. doi:10.1016/j.cortex.2014.02.015

Pulvermüller, F., Garagnani, M., Wennekers, T., 2014. Thinking in circuits: toward neurobiological explanation in cognitive neuroscience. Biol. Cybern. 108, 573-593. doi:10.1007/s00422-014-0603-9

Pulvermüller, F., Härle, M., Hummel, F., 2000. Neurophysiological distinction of verb categories. Neuroreport 11, 2789-2793. doi:10.1097/00001756-200008210-00036

Pulvermüller, F., Hauk, O., Nikulin, V. V., Ilmoniemi, R.J., 2005a. Functional links between motor and language systems. Eur. J. Neurosci. 21, 793-797. doi:10.1111/j.1460-9568.2005.03900.x

Pulvermüller, F., Kherif, F., Hauk, O., Mohr, B., Nimmo-Smith, I., 2009. Distributed cell

assemblies for general lexical and category-specific semantic processing as revealed by fMRI cluster analysis. Hum. Brain Mapp. 30, 3837-3850. doi:10.1002/hbm.20811

Pulvermüller, F., Lutzenberger, W., Preissl, H., 1999. Nouns and verbs in the intact brain: evidence from event-related potentials and high-frequency cortical responses. Cereb. Cortex 9, 497-506. doi:10.1093/cercor/9.5.497

Pulvermüller, F., Preissl, H., 1991. A neuronal model of aphasic syndromes, in: Elsner, N., Penzlin, H. (Eds.), Synapse - Transmission - Modulation. Contributions to the 19th Göttingen Neurobiology Conference. Thieme, Stuttgart, New York, p. 548.

Pulvermüller, F., Shtyrov, Y., Ilmoniemi, R., 2005b. Brain signatures of meaning access in action word recognition. J. Cogn. Neurosci. 17, 884-892. doi:10.1162/0898929054021111

Pulvermüller, F., Shtyrov, Y., Kujala, T., Näätänen, R., 2004. Word-specific cortical activity as revealed by the mismatch negativity. Psychophysiology 41, 106-112. doi:10.1111/j.1469-8986.2003.00135.x

Rauschecker, J.P., Tian, B., 2000. Mechanisms and streams for processing of "what" and "where" in auditory cortex. Proc. Natl. Acad. Sci. U. S. A. 97, 11800-6. doi:10.1073/pnas.97.22.11800

Rilling, J.K., 2014. Comparative primate neuroimaging: Insights into human brain evolution. Trends Cogn. Sci. 18, 46-55. doi:10.1016/j.tics.2013.09.013

Rilling, J.K., Glasser, M.F., Preuss, T.M., Ma, X., Zhao, T., Hu, X., Behrens, T.E.J., 2008. The evolution of the arcuate fasciculus revealed with comparative DTI. Nat. Neurosci. 11, 426428. doi:10.1038/nn2072

Rioult-Pedotti, M.-S.S., Friedman, D., Donoghue, J.P., 2000. Learning-Induced LTP in Neocortex. Science (80-. ). 290, 533-536. doi:10.1126/science.290.5491.533

Rizzolatti, G. Luppino, G., 2001. The cortical motor system. Neuron 31, 889-901.

Rolls, E.T., Deco, G., 2010. The Noisy Brain: Stochastic Dynamics as a Princple of Brain Function. Oxford University Press, Oxford.

Romanski, L.M., 2007. Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. Cereb. Cortex 17, i61-9. doi:10.1093/cercor/bhm099

Romanski, L.M., Bates, J.F., Goldman-Rakic, P.S., 1999. Auditory belt and parabelt projections to the prefrontal cortex in the rhesus monkey. J. Comp. Neurol. 403, 141-157. doi:10.1002/(SICI)1096-9861(19990111)403:2<141::AID-CNE1>3.0.CO;2-V

Romanski, L.M., Tian, B., Fritz, J., Mishkin, M., Goldman-Rakic, P.S., Rauschecker, J.P., 1999. Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nat. Neurosci. 2, 1131-6. doi:10.1038/16056

Rüschemeyer, S.-A., Brass, M., Friederici, A.D., 2007. Comprehending Prehending: Neural Correlates of Processing Verbs with Motor Stems. J. Cogn. Neurosci. 19, 855-865. doi:10.1162/jocn.2007.19.5.855

Shtyrov, Y., Butorina, A., Nikolaeva, A., Stroganova, T., 2014. Automatic ultrarapid activation and inhibition of cortical motor systems in spoken word comprehension. Proc. Natl. Acad. Sci. U. S. A. 111, E1918-23. doi:10.1073/pnas.1323158111

Shtyrov, Y., Hauk, O., Pulvermüller, F., 2004. Distributed neuronal networks for encoding category-specific semantic information: The mismatch negativity to action words. Eur. J. Neurosci. 19, 1083-1092. doi:10.1111/j.0953-816X.2004.03126.x

Sim, E.J., Kiefer, M., 2005. Category-related brain activity to natural categories is associated with the retrieval of visual features: Evidence from repetition effects during visual and functional judgments. Cogn. Brain Res. 24, 260-273. doi:10.1016/j.cogbrainres.2005.02.006

Tomasello, M., Kruger, A.C., 1992. Joint attention on actions: acquiring verbs in ostensive and non-ostensive contexts. J. Child Lang. 19, 311-333. doi:10.1017/S0305000900011430

Tranel, D., Damasio, H., Damasio, A.R., 1997. A neural basis for the retrieval of conceptual knowledge. Neuropsychologia 35, 1319-1327. doi:10.1016/S0028-3932(97)00085-7

Ungerleider, L.G., Gaffan, D., Pelak, V.S., 1989. Projections from inferior temporal cortex to prefrontal cortex via the uncinate fascicle in rhesus monkeys. Exp. Brain Res. 76, 473-484. doi: 10.1007/BF00248903

Ungerleider, L.G., Haxby, J. V., 1994. "What" and "where" in the human brain. Curr. Opin. Neurobiol. 4, 157-165. doi:10.1016/0959-4388(94)90066-3

Vigliocco, G., Vinson, D.P., Druks, J., Barber, H., Cappa, S.F., 2011. Nouns and verbs in the brain: A review of behavioural, electrophysiological, neuropsychological and imaging studies. Neurosci. Biobehav. Rev. 35, 407-426. doi:

Vouloumanos, A., Werker, J.F., 2009. Infants' learning of novel words in a stochastic environment. Dev. Psychol. 45, 1611-7. doi:10.1037/a0016134

Warrington, E.K., McCarthy, R.A., 1987. Categories of knowledge: further fractionations and an attempted integration, Brain. doi:10.1093/brain/110.5.1273

Warrington, E.K., Shallice, T., 1984. Category specific semantic impairments. Brain 107 ( Pt 3, 829-854. doi: 10.1016/j.neuropsychologia.2007.11.018

Webster, M.J., Bachevalier, J., Ungerleider, L.G., 1994. Connections of inferior temporal areas TEO and TE with parietal and frontal cortex in macaque monkeys. Cereb. cortex 4, 470-483. doi:10.1093/cercor/4.5.470

Wennekers, T., Garagnani, M., Pulvermüller, F., 2006. Language models based on Hebbian cell assemblies. J. Physiol. Paris 100, 16-30. doi:10.1016/j.jphysparis.2006.09.007

Wilson, H.R., Cowan, J.D., 1972. Excitatory and inhibitory interactions in localized populations of model neurons. Biophys. J. 12, 1-24. doi:10.1016/S0006-3495(72)86068-5

Yeterian, E.H., Pandya, D.N., Tomaiuolo, F., Petrides, M., 2012. The cortical connectivity of the prefrontal cortex in the monkey brain. Cortex 48, 68-81. doi:10.1016/j.cortex.2011.03.004

Young, M.P., Scannell, J.W., Burns, G., 1995. The analysis of cortical connectivity. Springer, Heidelberg.

Young, M.P., Scannell, J.W., Burns, G.A.P.C., Blakemore, C., 1994. Analysis of connectivity: Neural systems in the cerebral cortex. Rev. Neurosci. 5, 227-250. doi: 10.1515/REVNEUR0.1994.5.3.227

Yuille, A.L., Geiger, D., 2003. Winner-Take-All Mechanisms, in: Arbib, M. (Ed.), The Handbook of Brain Theory and Neural Networks. MIT Press, Cambridge, MA, pp. 1056-1060.

Zatorre, R.J., Meyer, E., Gjedde, A., Evans, A.C., 1996. PET Studies of Phontic Processing of Speech: Review, Replication, and Reanalysis. Cereb. Cortex 6, 21-30. doi:10.1093/cercor/6.1.21

Figure 1. Model of lexical and semantic mechanisms: The 12 cortical areas modeled (A), their global connectivity architecture (B), and aspects of the micro-structure of their connectivity (C) are illustrated. (A) Six perisylvian (i) and six extrasylvian (ii) model areas are shown, each including a dorsolateral (frontal) and a ventral (temporal) part: (i) perisylvian cortex include an articulatory system (red colors), including inferior-prefrontal (PFi), premotor (PMi) and primary motor cortex (M1i) and auditory system (areas in blue), including auditory parabelt (PB), auditory belt (AB) and primary auditory cortex (A1). These areas can store correlations between neuronal activations carrying articulatory-phonological and corresponding acoustic-phonological information, for example when phonemes, syllables and spoken word forms are being articulated (activity in M1i) and acoustic features of these spoken words are simultaneously perceived (stimulation of primary auditory cortex, A1). (ii) Extrasylvian areas include a motor system (yellow to brown), including dorsolateral prefrontal (PFL), premotor (PML) and primary motor cortex (M1L) and a "what" visual stream of object processing (green), including anterior-temporal (AT), temporo-occipital (TO) and early visual areas (V1). Together with the perisylvian areas, these extrasylvian areas can store correlations between neuronal activations carrying semantic information, for example when words are used (activity in all perisylvian areas) to speak about objects present in the environment (activity in V1, TO, AT) or about actions that the individual engages in (activity in

M1L, PML, PFl). Numbers indicate Brodmann Areas (BAs). (B) Schematic illustration of all 12 model areas and the known between-area connections implemented. The colours indicate correspondence between cortical and model areas. (C) Micro-connectivity structure of one of the 7,500 single excitatory neural elements modelled (labeled "e"). Within-area excitatory links (in grey) to and from "cell" e are limited to a local (19x19) neighbourhood of neural elements (light-grey area). Lateral inhibition between e and neighbouring excitatory elements is realized as follows: the underlying cell ' i' inhibits e in proportion to the total excitatory input it receives from the 5x5 neighbourhood (dark-purple shaded area); by means of analogous connections (not depicted), e inhibits all of its neighbours. Each pair (e,i) of model cells is taken to represent an entire cluster or column (grey matter under approximately 0.25 mm2 of cortical surface) of pyramidal cells and the inhibitory interneurons therein.

Figure 2. Distributions of cell-assemblies (CAs) emerging in the 12 area network during simulation of word learning in the semantic context of visual (A) and action (B) perceptions. Results of one typical instantiation of the model in Fig. 1 are shown, using the same area labels. Each set of 12 squares (in black) illustrates the distribution of "cells" of one specific CA across the 12 network areas. Each white pixel in a square indexes one CA cell. CAs for object-related words extend into higher and primary visual cortex (V1, TO, but not M1L), linking information about spoken word forms (perisylvian pattern) with information from the visual modality (neural pattern in V1). Network correlates of action-related words extend into lateral motor cortex (M1L, PML, but not V1), thus semantically grounding words in information about actions. For convenience, the area structure of the network is repeated at the top.

Figure 3. Average distributions of CAs emerging in 12 instantiations of the 12 area network architecture during simulation of word learning in the semantic context of actions and visual perceptions. Bars show average numbers of CA neurons per area for object- (dark grey) and action-related (light grey) word representations; error bars show standard errors over networks. (A) Data from the six perisylvian areas whose cells can be seen as circuit correlates of spoken word forms do not show category-specific effects. (B) The extrasylvian areas whose cells can be seen as circuit correlates of word meaning show a double dissociation, with relatively more strongly developed CAs for object- than for action-related words in primary and secondary visual areas (V1, TO), but stronger CAs for action-related than for object-related words in dorsolateral primary motor and pre-motor cortices (PML, M1L). Asterisks indicate that, within a given area, the number of CA neurons significantly differed between the circuits of action and object words (Bonferroni-corrected planned comparison tests, 24 comparisons; critical thresholdp < .0020).

Figure 4. Activation spreading in the 12 area network showing the simulated recognition of object- (blue pixels) and action-related (red pixels) words. Yellow pixels illustrate the overlap between the two word-related CAs. Network responses to stimulation of A1 with the "auditory" patterns of two of the learned words; each set of 12 "squares" depicts a selected snapshot of the entire network's activity (as in Fig. 2). Cell activity levels are indicated by brightness of pixels; snapshot numbers indicate simulation time-steps of the network output. See main text for details.

Figure 5. Bar plots illustrating the amount of activity - "peak amplitude" (left hand side) and the activation time-course - "peak delay" (right hand side) of auditory and articulatory (A) and visual and motor (B) areas for object- and action-related words during auditory word recognition.

Figure 6. Spatio temporal activation patterns in the network: All curves (bottom part of each panel) illustrate area-specific activation dynamics plotted against time during the neurophysiological word recognition processes (time is in simulation time steps). A & B shows simulation results for the six perisylvian model areas and C & D shows them for the six extrasylvian areas. Brain schematics (at the top of each panel) highlight the cortical locations of the areas for each specific activation curve and peak. Two or more areas are plotted into the same brain schematic if there were no significant delay differences between their peak activations (Bonferroni-corrected for 24 comparisons; critical threshold p < .0020). The latency of maximal activation together with standard errors (boxes) and standard deviations (whiskers) are illustrated by a given boxplot. The small horizontal segment indicates stimulus onset and offset. Averages and statistics are calculated across 12 different networks.

Figure 7. Spatio temporal activation patterns in the network: All curves (bottom part of each panel) illustrate area-specific activation dynamics plotted against time during the neurophysiological word recognition processes (time is in simulation time steps). Fig 6 shows simulation results for the six perisylvian model areas and Fig 7 shows them for the six extrasylvian areas (Fig. 7). Brain schematics (at the top of each panel) highlight the cortical locations of the areas for each specific activation curve and peak. Two or more areas are plotted into the same brain schematic if there were no significant delay differences between their peak activations (Bonferroni-corrected for 24 comparisons; critical threshold p < .0020). The latency of maximal activation together with standard errors (boxes) and standard deviations (whiskers) are illustrated by a given boxplot. The small horizontal segment indicates stimulus onset and offset. Averages and statistics are calculated across 12 different networks.

Figure 8. Comparison of real and simulated brain activations elicited by specific semantic word categories. (A) Time course of activation of cortical areas elicited by passive presentation of spoken action words and determined using magnetoencephalography (MEG) and distributed source localizations. Action words elicited sequential but near-simultaneous activations in left superior temporal, inferior frontal and superior central cortex. The average latency of maximal activation in the four ROIs is reported together with the standard errors (boxes; bars indicate 1.96 SE, data adapted from Pulvermüller et al., 2005b). The boxplots in panels B & C illustrate results from the corresponding simulated activation time-courses. The point in time at which stimulus-evoked activity is peaking in each of the modelled four sub-systems (auditory, articulatory, visual and motor systems) is plotted against time given in simulation time-steps. Boxes give standard errors and whiskers standard deviations. The average was computed across the 12 different networks and calculated separately for (B) Action and (C) Object-related words. Notice that the respective non-relevant sub-systems (Visual for action- and motor for object-related words) are not illustrated here, as the activation levels are relatively low.


Articulatory areas

to/from AT and PB

• • O /o O • •

Figure 1

Visual & Motor areas

A1 AB PB_PFj PMj M1j

A. Auditory & Articulatory areas B.

Object-related words Action-related words

CA 6 CA 12




Figure 2



Model AREA


o 10 ÛÉ


Model AREA

Figure 3

PM-i Ml-i

2. 12.


4. 16.


A1 AB PB PFi PMi Mli A1 AB PB PFi PMi Mli

8. 20.


A1 AB PB PFi PMi Mli A1 AB PB PFi PMi Mli

Object-related word ■ Action-related word

Overlap Figure 4



Amount of activity - "Peak amplitude"

Auditory & Articulatory areas

Activation time course - "Peak delay"

Auditory & Articulatory areas


Motor areas


& Motor areas

Figure 5

Activation time course - EXTRASYLVIAN AREAS

Object-related words

c/) CD

26 ¿7 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

14 15 16 17 18 19 20 21 22

23 24 25

14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

time (simulation steps) Action-related words

± Stdev ± Standard Error + Mean

C/) 14 15 16 17 18 19 20 21 ¿2 ¡23 24 25: CD

"£15 -AT:

26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

time (simulation steps)

Figure 6

Figure. 7

Activation time course - EXTRASYLVIAN AREAS

Object-related words

14 15 16 17 18 19 20 21 22 ¡23 24 25 !26 tl 28 29 30 31 32 33 34 35 36 37 38 39 AO 41 42 43

14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

time (simulation steps)

± Stdev

± Standard Error

+ Mean

Action-related words

C/) 14 15

P 0. 1— 14 15

time (simulation steps)

Figure 7

superior temporal

Action-related words - MEG activation

inferior superior



± 1.96 Std Error ± Standard Error Mean

170 180

time (msec)

Action-related words - simulation results

Auditory Articulatory Motor systems

17 18 19 20 21

time (simulation steps)

Object-related words - simulation results "

Auditory Articulatory Visual systems



14 15 16 17 18 19 20 21 22

time (simulation steps)

Figure 8


• A neurobiological model of language acquisition is proposed

• Cell assembly circuits emerged linking word-form and semantic information

• Connectivity and learning explain category-specific and -general semantic areas

• Near-simultaneous activation dynamics across network-areas are documented