CC BY
0Hindawi Publishing Corporation Neural Plasticity
Volume 2011, Article ID 203462, 11 pages doi:10.1155/2011/203462
Review Article
Reactivation, Replay, and Preplay: How It Might All Fit Together
Laure Buhry,1,2 Amir H. Azizi,1,2,3 and Sen Cheng1,2,3
1 Mercator Research Group "Structure of Memory", Ruhr-University Bochum, Universitaetsstrafie 150, 44801 Bochum, Germany
2 Faculty of Psychology, Ruhr-University Bochum, Universitaetsstrafie 150, 44801 Bochum, Germany
3International Graduate School of Neuroscience, Ruhr-University Bochum, Universitaetsstrafie 150, 44801 Bochum, Germany Correspondence should be addressed to Sen Cheng, sen.cheng@rub.de Received 7 April 2011; Revised 13 June 2011; Accepted 15 June 2011 Academic Editor: Christian Leibold
Copyright © 2011 Laure Buhry et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Sequential activation of neurons that occurs during "offline" states, such as sleep or awake rest, is correlated with neural sequences recorded during preceding exploration phases. This so-called reactivation, or replay, has been observed in a number of different brain regions such as the striatum, prefrontal cortex, primary visual cortex and, most prominently, the hippocampus. Reactivation largely co-occurs together with hippocampal sharp-waves/ripples, brief high-frequency bursts in the local field potential. Here, we first review the mounting evidence for the hypothesis that reactivation is the neural mechanism for memory consolidation during sleep. We then discuss recent results that suggest that offline sequential activity in the waking state might not be simple repetitions of previously experienced sequences. Some offline sequential activity occurs before animals are exposed to a novel environment for the first time, and some sequences activated offline correspond to trajectories never experienced by the animal. We propose a conceptual framework for the dynamics of offline sequential activity that can parsimoniously describe a broad spectrum of experimental results. These results point to a potentially broader role of offline sequential activity in cognitive functions such as maintenance of spatial representation, learning, or planning.
1. Introduction
Reactivation of neural activity in the hippocampus was first studied in 1989 by Pavlides and Winson [1]. The authors recorded spiking activity of place cells, hippocampal neurons that are selectively active in restricted regions of space [2]. After determining the so-called place field of a particular neuron, they either allowed the animal to run through the place field (exposure condition) or prevented the access to it (nonexposure condition). As a consequence, the place cell was active in the exposure condition, but inactive in the nonexposure condition. Intriguingly, the activity level of cells during subsequent sleep reflected their earlier activity level during exploration, showing that place cells are reactivated during sleep. Following this finding, investigators first studied pairs [3, 4] and then larger ensembles of place cells [5, 6]. These studies reveal that place cells are activated in a consistent sequential order both during rapid eye movement (REM) sleep [5] and during slow-wave sleep (SWS) [6]. Moreover, the order during sleep matches the order in which
the same cells were active during the preceding run on a linear track. This phenomenon is therefore called replay. During SWS sleep, reactivation largely co-occurs with hippocampal sharp-waves/ripples (SWR), brief (-80ms) high-frequency bursts (100-250 Hz) that appear in the local field potential (LFP) in the hippocampus [7-9]. Nevertheless, it remains unclear whether all SWRs are accompanied by replay events and vice versa.
Since the rodent hippocampus is required for learning about new places [10], as well as sequences of nonspatial items [11], it was suggested early on that reactivation might be involved in learning. Specifically, reactivation is thought to be a neural mechanism for consolidation, a process through which memories gradually become independent of the hippocampus [12]. It has been suggested that memories are initially stored in the hippocampus and then gradually transferred to neocortical areas through reactivation in a two-stage process [13, 14].
In this paper, we discuss three classes of experimental findings and their associated functions. First, mounting
evidence suggests that reactivation during sleep plays a functional role in consolidation. Second, more recently, reactivation has been observed during the awake state, which differs in intriguing ways from sleep reactivation. Third, recent results suggest that sequential neuronal activation during offline states is not always a replay of sequences previously driven by sensory stimuli [15, 16]. We introduce the term "offline sequential activity (OSA)" to refer to sequential neural activation that is not apparently driven by sensory stimuli. "OSA" is meant to be a catch-all term that potentially includes several types of activity such as replayed sequences, sequences that are not repetition of previously experienced sequences, sequences with a functional relevance, and those without one. We suggest a conceptual framework for the dynamics of OSA that can explain many experimental findings in the three classes of the reviewed phenomena.
2. The Link between Reactivation during Sleep and Consolidation
Consolidation can be defined as "a process that transforms new and initially labile memories encoded in the awake state into more stable representations that become integrated into the network of pre-existing long-term memories. Consolidation involves the active re-processing of fresh memories within the neuronal networks that were used for encoding them. It seems to occur most effectively off-line, [■■■ ] so that encoding and consolidation cannot disturb each other and the brain does not hallucinate during consolidation" [17]. This view of consolidation would strongly suggest that it takes place during sleep, as others have suggested before [18, 19]. Here, we review the mounting experimental evidence that reactivation during sleep plays an important role in the process of consolidation.
2.1. Memory Performance Is Linked with SWRs and Reactivation. The most direct lines of evidence for a role of reactivation in consolidation are the correlations between memory performance, on the one hand, and SWRs and reactivation, on the other hand. Axmacher et al. [20] presented epilepsy patients with sequences of pictures (landscapes and houses) while recording LFP in the hippocampus and rhinal cortex with standard macro electrodes. The number of successfully recalled items after an «1 h nap is correlated with the number of rhinal, though not hippocampal, SWRs. Dupret et al. [21] showed in rats that the number of reactivation events is correlated with the number of new goal locations that animals can successfully retrieve. These results provide only a correlation between memory performance and reactivation or SWRs, so they do not show that reactivation and/or SWRs are causally driving consolidation.
To show a causal relationship, Girardeau et al. [22] and Ego-Stengel and Wilson [23] interrupted reactivation in rats during sleep. The authors detected the onset of SWRs in the LFP and, upon detection, stimulated the commissural fibers, which bilaterally connect the hippocampi. Such stimulation can suppress CA3 activity, and thus the propagation of
replay sequences. Stimulated animals show a small, but significant, memory deficit relative to controls. The residual learning can be explained perhaps by incomplete SWR detection (detection rate approx. 85%), thus allowing some replay events to occur and to potentially drive residual consolidation [22]. It is also possible that some replay events are not accompanied by SWRs and, therefore, cannot be detected by the methods used in these studies. A careful study of the exact relationship between replay events and SWRs has yet to be done.
Even though the deficit in stimulated animals is somewhat small, the results show a clear impairment of memory consolidation due to the suppression of SWRs.
2.2. Spiking during Reactivation Can Drive Synaptic Plasticity. We next turn our attention to the mechanism by which reactivation might drive consolidation. The most discussed possibility is that experience of a sequence leads to replay them then drives plasticity, which in turn strengthens the memory of the replayed experience. This chain of arguments can be evaluated at several points. For one, we would expect that more experience (either recent [24, 25] or total experience [3, 13]) leads to more replay. Indeed, sleep replay seems to depend on repeated sequential experiences [25]. The rates of SWRs and reactivation increase with the number of repetitions and the regularity of the behavior in both CA1 and CA3. O'Neill et al. [26] showed that the more time animals spent within the overlap oftwo place fields, the more the corresponding place cell pair are reactivated during sleep.
Next, we consider whether reactivation could drive synaptic plasticity. Spiking in CA1 pyramidal cells during SWRs is locked to the cycle of the ripple [7], and thus highly similar to tetanic stimuli frequently used to induce long-term potentiation (LTP) in experimental settings [27]. Moreover, spikes fired by neuron pairs in a SWR fall within a time window of tens of milliseconds, consistent with spike-timing-dependent plasticity (STDP) [28-30]. Hence, spiking during reactivation clearly has the properties to drive LTP and STDP.
It has been claimed that replayed sequences are compressed in time by a factor of about 20 relative to the sensory-driven sequences [6, 31]. This apparent compression is, however, an analysis artifact, when spiking during SWRs is compared to the average time it takes the animal to move between the place fields. A better reference are perhaps the firing sequences generated during exploration due to theta phase precession [32]. Phase precession has been observed in individual passes through a cell's place field [33] and, therefore, can generate neural sequences on the order to tens of milliseconds.
2.3. Reactivation Occurs in Several Networks throughout the Brain. Since consolidation presumably involves the transfer of information to neocortical areas, reactivation should be observed in brain regions outside of the hippocampus. While most work on reactivation was done in the hippocampus, reactivation was also observed in several other brain structures. First evidence of neocortical reactivation
was demonstrated in pairwise correlations during SWS [34]. In the primary visual cortex (V1) and the neighboring secondary visual cortex, Ji and Wilson [35] found that firing sequences evoked by awake experience are replayed during SWS. Similarly to hippocampal place cells, the firing activity of V1 cells is tied to specific locations within the environment; however, spiking of V1 cells is probably driven by local visual, rather than spatial, cues.
Replay was observed in prefrontal cortex (PFC) in transient episodes during SWS [36, 37]. These sequences are compressed in time compared to the average activity during behavior as it is in the hippocampus, but with a factor of6 to 7 [36]. However, it remains unclear whether a phenomenon similar to phase precession in the hippocampus could explain the apparent compression in prefrontal cortex.
Furthermore, SWR and reactivation were also observed in the ventral striatum during SWS after a task on a track (T maze or triangular track) where rewards were present [3840]. The striatal reactivation does not seem to decay across 40 min ofsleep after the task and is associated with short time intervals after ripple onset [38, 39]. However, in contrast to replay in the hippocampus, this reactivation does not appear during REM sleep and seems to be a sparse phenomenon in the sense that a minority of firing units (coding for the track exploration) are reactivated.
Finally, SWR-like events were recorded in the rhinal cortex of humans by [20], in the entorhinal cortex of rats [41], and in the LFP in cortical sensorimotor and association areas 3, 4, 5, 7, 17, 18, and 21 of cats [42].
2.4. SWRs and Reactivation Are Coordinated between Neocortex and Hippocampus. To transfer information from the hippocampus to neocortical areas, activity in different brain regions has to be coordinated. Hippocampal SWRs are presumed to be means for information exchange between the hippocampus and neocortical areas during consolidation [13]. While a coherent picture ofthese interactions is lacking, many pieces of the puzzle have emerged. For instance, the occurrence of prefrontal sleep spindles and that of SWRs are correlated [43], as are SWR-like events in rhinal cortex and hippocampus [20].
A number of studies have focused on the neocortical up and down states that occur during SWS and in anesthetized animals. Up states are characterized by depolarized membrane potentials and high neural activity and down states by hyperpolarized membrane potentials with little spiking activity [44, 45]. SWRs in the hippocampus apparently appear more frequently during down states, particularly near the transition from the down to the up state [46-48]. A potential coupling mechanism might be the strong synchronization between the membrane potential of hippocampal interneurons and up-down transitions in neocortical LFP [49]; however, up-down states have not been found in the hippocampus itself [35, 50]. Consistent with the hypothesis that different neocortical areas need to be coordinated during consolidation, up-down transitions are correlated across different neocortical sites [51] as well as between cortical areas and hippocampal subareas [50].
The synchronization across brain regions extends to spiking activity as well. Ji and Wilson [35] observed that during SWS there are distinct periods of low and high spiking activity in both hippocampus and V1. The authors called periods of high activity frames and suggested that frames correspond largely to cortical up states although they could not be sure due to the lack ofmembrane potential recordings [35]. The results suggest that frames in the hippocampus and V1 are synchronized as well as replay sequences in the two areas. Frames appear to be involved in the generation of SWRs since hippocampal frames are frequently followed by SWRs about 30 ms later.
Furthermore, hippocampal SWRs are associated with reactivation of neurons in PFC [37, 52] and in ventral striatum [38, 40], which receives direct inputs from CA1 and subiculum [53]. Hoffman and McNaughton [54] studied cell pairs in different neocortical areas of nonhuman primates during a visual task and subsequent sleep. They found that the patterns of correlations during the task phase is reactivated during sleep. Finally, the number of reactivation events is correlated with the density of up-down states [55].
In addition to suggesting correlations between areas, the simple information transfer view implies that information flows from the hippocampus to the other brain regions. Consistent with this expectation, the hippocampus appears to lead the ventral striatum [40] and prefrontal cortex [37]. Some evidence, however, point to a flow of information in the opposite direction. Sirota et al. [46] showed that neuronal bursts in somatosensory cortex trigger SWRs in the hippocampus, Hahn et al. [49] suggested that prefrontal up-down states influence the probability of replay events in the hippocampus, and Ji and Wilson [35] showed that frames in V1 lead the hippocampal frames by 50 ms. These experimental observations suggest that the interactions between hippocampus and other brain regions during consolidation is more complex than the simple view. Interactions might be bidirectional and the direction might change dynamically throughout the consolidation process [37].
2.5. Reactivation Occurs in Several Different Species. Since memory consolidation is believed to be a general principle of memory storage, we would expect that a similar mechanism underlies consolidation in different species. Indeed, there is much evidence for SWRs and reactivation in species other than rodents.
Recordings obtained from epileptic patients reveal SWR-like events in the LFP in the human parahippocampus [56], hippocampus, and rhinal cortex [20]. In these studies, SWRs occur only during slow wave sleep or quiet rest with eyes closed and are not observed during the active awake state. Interestingly, single units in the human hippocampus appear to represent spatial locations not unlike place cells in rodents
[57]. In nonhuman primates (rhesus macaques), SWRs were observed in the hippocampus and neighboring structures
[58]. The authors suggest that SWRs originate mainly in CA1, like they do in rats. Reactivation was observed in the macaque motor, somatosensory, and posterior parietal cortices but not in prefrontal cortex [54]. SWR-like events
were detected in the LFP of other mammalian brain regions such as cat cortex [42] and rabbit hippocampus [59].
Finally, replay has been observed in at least one nonmam-malian system: area RA of songbirds [60]. The sequential activity of sensorimotor neurons during sleep matches their activity during daytime singing when they drive song production.
3. Reactivation during the Awake State and Its Enhancement
Consolidation occurs during sleep, so the popular argument goes, because then the brain is not burdened by stimulus-driven neural activity that could interfere with the network restructuring during consolidation. Similarly, the initial assumption was that reactivation only occurs during sleep [3, 4, 8]. However, Kudrimoti et al. [61] later reported that reactivation occurred in the awake state, too. O'Neill et al. [62] showed that the hippocampal network during the awake state is simultaneously driven by both internal dynamics and sensory inputs. The latter result was found in exploratory SWRs (eSWRs), which the authors defined as SWR that occur during brief periods of rest, that is, within 2.4 s of theta activity, which ceases when the animal remains stationary [62].
3.1. Accounting for the Special Properties of Awake Reactivation. During the awake state, SWRs are also frequently accompanied by sequential activation of place cells [24, 63, 64]. However, these sequences exhibit a fascinating difference to reactivation during sleep. Some sequences of activation during quiescent periods are reversed as compared to the sequence during run [24]. It is remarkable that this stunning experimental observation had been predicted by Buzsaki [13] based on a moving-threshold model. In this model, place cells receive subthreshold inputs even when the animal is located far away from its apparent place field. During SWRs, the threshold for spiking is lowered gradually. As the threshold drops, one place cell fires spikes first, the cell that receives the strongest subthreshold excitation, that is, the one with a place field closest to the animal's current location. As the threshold is lowered further, cells with progressively weaker excitation, that is, cells with place fields further and further away, fire spikes, generating a neuronal sequence. When place fields are unidirectional, the moving-threshold model predicts sequences in both the forward and reverse order.
While Foster and Wilson [24] originally reported awake replay in the reverse order, later studies reported awake replay in both the forward and reverse directions on a linear track [64, 65], and in an open environment [63]. When the animal pauses in-between runs along the linear track, forward replay is observed mostly just before a run, suggesting an anticipation of the run [64]. On the other hand, reverse replay is observed just after a run was completed, suggesting a mechanism that uses the information about the outcome of the run to correct the preceding action.
The moving-threshold model made a second important prediction: replay is initiated at the animal's current location. There is indeed a tendency for awake replay to be initiated at or modulated by the animal's current location [62-64]. However, it seems that awake replay can be initiated at remote locations on the track [65] and may correspond to a different track altogether [66]. Furthermore, the fundamental assumption of the moving-threshold model are challenged by recent intracellular recordings [67, 68]. These studies suggest that there is no subthreshold excitatory inputs to place cells distant from their spiking place field.
The experimental evidence on awake replay is not unambiguous. While reactivation is observed in PFC, it apparently does not appear in rest periods that are not classified as SWS [36, 37]. It also remains controversial whether to include SWRs-like events that are observed while the animal is running at significant speeds [62, 69].
What might be the functional role of awake reactivation? This role could be very different from sleep reactivation and possibilities include planning, modifying neural representations, attention, motion, and memory retrieval. Alternatively, awake and sleep replay might serve a similar function: consolidation. If consolidation set in only after the animal had fallen asleep, perhaps hours later, intervening neural activity might overwrite the information about the cells' activity during behavior. This problem could be circumvented by starting the process of consolidation immediately after the experience [70]. If this were the case, we would expect that reactivation was increased by factors that are known to correlate with memory demand or performance. We briefly review the available evidence in the following, and refer the reader to Carr et al. [70] for more detail.
3.2. Ripple-Associated Activity Enhanced by Novelty. To study memory formation and/or consolidation we need to examine neural activity when new associations are learned. While some earlier studies found SWR-associated reactivation following exposure to a novel environment [6, 61], they did not compare the neural representations of novel and familiar locations. Foster and Wilson [24] reported that it is easier to observe reactivation of novel linear tracks than of familiar tracks; that is, P values indicating the statistical significance of replay were lower after exposures to novel environments than after exposure to familiar environment. This result suggests that there is a difference between the reactivation of novel and of familiar environments, but it does not point to the source of the difference. We, therefore, studied SWR-associated neural activity, while rats alternated between a familiar and a novel arm in an eight-arm maze [69]. Place cells that represent the novel arm show significantly increased activity during SWRs as compared to cells representing the familiar arm. Furthermore, spiking of novel arm cells during SWRs have a higher temporal precision and significantly more SWRs occur when the animal is located in the novel arm. At the same time, spiking activity as a function of spatial location and theta phase is less regular in the novel arm. We, therefore, proposed that enhanced SWR-associated activity drives the formation
Day 1 Day 2 Day 3 Day n Sleep
(a) (b)
Figure 1: The dynamics of offline sequential activity (OSA) as a function of exposure. This schematic summarizes the results of many reactivation studies in novel and familiar environments. (a) The number of OSA within a day increases with the length of exposure (solid line segments). At the same time, the level of OSA decreases across days as the animal becomes familiar with a novel environment (dashed line). Ar represents the amount of intrinsic OSA (Ar = 0 implies that OSA are purely reactivation of prior sensory-driven sequences). The grey-shaded regions are compressed, and the dynamics within these regions is omitted here for clarity. They are shown in the next panel. (b) Dynamics of OSA within one day. The solid lines are based on experimental observations. The dashed lines represent different hypotheses about the unknown dynamics of OSA between the end of the exposure and the beginning of the sleep phase.
of precise spatiotemporal representations [69]. Interestingly, in this context, the number of goal-associated eSWRs in a spatial task is correlated with memory performance [21].
Enhanced replay of memory traces during SWS due to learning was also observed in the hippocampus, neo-cortex, putamen and thalamus [71] although there are methodological concerns with their data analysis [72]. Other studies also reported an increase in SWRs during sleep after exposure to novelty in an association task [73] and a spatial discrimination experiment [74]. Here, learning seems to increase the number of hippocampal SWRs during the first hour of postlearning SWS. Rats that did not learn the discrimination during the training session do not show any change in the number of SWRs.
At this point, one might wonder how it is possible that reactivation increases with time spent in the environment, as discussed in the previous section, and decreases with familiarity, as discussed here. We illustrate how both trends can occur at the same time in a schematic representation of the potential dynamics of reactivation (Figure 1). Each solid line segment in Figure 1(a) shows that reactivation within a day increases with the duration of exposure; however, the heights of the solid line segments decrease with each day of exposure before reaching an asymptote. To be consistent with our argument in the next section, we already use the more general term offline sequential activity (OSA), which also includes replay (Figure 1).
3.3. Reactivation Is Enhanced by Reward and Affective State. Reward and affective state are known to influence memory formation [75-78]. We would, therefore, expect that reward
and affective state also change reactivation. In most experiments, animals are rewarded for their performance to drive learning. Once the task is learned, the reward is obtained consistently, leaving few unrewarded trials to analyze. By switching the reward contingency mid-session without a signal to the animal, recent experiments were able to induce a period with a significant number of mistakes leading to unrewarded trials [39, 79]. CA3 place cells are more active during SWRs following rewarded trials as compared to unrewarded trials [79]. The enhancement is associated with the reward location: cells with place fields near the reward locations have more enhanced reactivation than others [79]. This enhancement could allow the animal to learn the relationship between the path and the outcome. The goal (or a reward location) is a particularly important location. Indeed, the formation of the goal's spatial representation and its reactivation is correlated with learning [21]. These results are compatible with an earlier observation that place fields gradually shift towards prospective reward locations over multiple trials of a T maze alternation task [80].
As during sleep, reactivation associated with reward during quiet wakefulness is not only observed in the hippocampus, but also in the ventral striatum [39], known to be involved in reward anticipation [81]. The striatal reactivation generally occurs after the hippocampal replay [40].
Together, these observations show that factors known to be correlated with memory performance also influence OSA in different brain regions, underscoring the potential role of awake reactivation in memory consolidation.
4. A New Type of Offline Sequential Activity (OSA)?
In the following, we discuss sequential neural activity that occurs during offline states, but that is not necessarily replay of previously experienced sequences. To emphasize this possibility, we use the term "offline sequential activity (OSA)" for internally generated neuronal sequences during offline states, that may or may not be replay of previously sensory-driven sequences.
4.1. Offline Sequences That Do Not Seem to Be Reactivation. Gupta et al. [15] investigated the relationship between experience and content of OSA in CA1 cells. They trained rats to run in a two-choice T maze with two return loops. After the second choice, the animals either turned left or right and then completed the loop to returned to the starting location. Animals had to run forward on the left or the right loop or alternate between the two in a block design. At any given stage of the experiment, animals had, therefore, experienced certain parts of the maze more or less recently. Gupta et al. [15] then analyzed the distribution of intervals between experience and replay events. The distribution is inconsistent with replay of the most recent or the accumulated experience, but is consistent with experience-independent replay. To explain the discrepancy with previous studies, the authors suggested that "... the increase in replay with experience
seen in these earlier studies [25, 26] may be due to general experience in the environment rather than the experience of particular trajectories." In other words, the x-axis in Figure 1 is "time spent in the test environment", not the amount of specific experience such as the number of repetitions of a particular trajectory.
A further decoupling between OSA and experienced sequences emerged. Although the animals were mostly prevented from running in the reverse direction, a similar number of forward and reverse replay events was found
[15]. Moreover, shortcut sequences were observed during replay that do not correspond to any previously experienced sequence. The animals were prevented from crossing directly between the left and right loops, yet, a significant number of such OSA occurred.
One issue with this study is that animals had experienced the environment in its entirety, even if they had not experienced those particular trajectories that were replayed, thus making it at least possible that the animal had mentally explored all trajectories that were replayed [82]. These objections were overcome by a recent experiment in mice that found preplay [16]. CA1 firing sequences recorded during periods of awake rest are correlated with sensory-induced sequences in an environment experienced only later. The OSA were observed both in forward and in reverse order.
A potential interpretation is that preplay emerges from the network structure, that is, sequences preexist in the network structure and are recruited for encoding new memories
[16]. This hypothesis is supported by a recent suggestion that the cellular properties of hippocampal cells predict which cells will become active in a novel environment [67]. The Ar in Figure 1 denotes the amount of intrinsic OSA. Ar = 0 would imply that there are no intrinsic OSA, that is, that OSA are always reactivation of prior sensory-driven sequences. The results of Dragoi and Tonegawa [16] indicate a significant nonzero Ar. Figure 1 also illustrates how this preplay, or intrinsic OSA, might have been missed so far, even though several studies have looked at reactivation before and after novel experience [6,24, 61,64]. These studies presumed that any correlation between run and presleep are spurious and thus have to be subtracted from the correlation between run and postsleep [72]. For instance, in the exploration of a familiar environment, Kudrimoti et al. [61] found that the pairwise correlations during SWS preceding run are significantly related to those during run. They then proceeded to substract this presleep-run correlation from the postsleep-run correlation for the analysis of reactivation. In addition, correlation-based measures probably lack the statistical power to detect the effect of the relatively small number of significant preplay events [16]. In summary, the most recent results suggest that the level of OSA is not driven by the specific sensory experience and OSA does not correspond to previously experienced sequences.
4.2. Possible Functions for Awake Offline Sequential Activity. We next turn to the question what function OSA might serve, when the OSA is not a repetition of previous activity. We previously suggested that SWR-associated activity might
drive the formation of spatial representation in novel environments, since SWR-associated activity is enhanced while spatio-temporal spiking is less organized [69]. Consistent with this view, Gupta et al. [15] suggest that the function of forward and reverse replay is to establish and maintain representations of the environment, rather than to replay specific recent experience, since the level of OSA appear to be driven by the familiarity to the environment, and not by the amount of sensory-driven neural activity during behavior. These suggestions are consistent with the idea of maintaining cognitive maps [83, 84].
As preplay occurs before the exploration task, we believe it to be consistent with the notion of planning [65, 85]. Hopfield [82] proposed a model that could support this function. This model, based on the continuous attractor model of Samsonovich and McNaughton [86], includes a hippocampus-like network and allows for mental exploration of trajectories never actually experienced. The goal of this mental exploration could be trajectory planning, that is, finding the optimal path between two locations.
OSA are indeed often observed before choosing a trajectory in spatial tasks or before obtaining a reward; it might, therefore, play a role in predicting the reward location and ingestion [38], possibly through a reverse replay phenomenon [64]. Johnson and Redish [87] suggest that the sequential activity of hippocampal neurons seem to represent future situation rather than recent experience.
Other suggestions focus on the function of OSA in memory and are not explained by the above-cited studies. Dragoi and Tonegawa [16] propose that preplay may facilitate future learning when "a new experience is introduced with multiple steps of increasing novelty". The presence of OSA could allow for the integration of novel information into a network of older memories. In other words, the sequences would be naturally present in the neural network and would be utilized during a learning experience to store new memory traces.
4.3. Sleep Offline Sequential Activity Revisited. The recent results on awake OSA, and the wide range of possible functions that they suggest, call for a reconsideration of sleep OSA. We began our paper with a review of the mounting evidence for the link between reactivation and consolidation. In our view, this link is still the best supported by experimental evidence. However, new possibilities for sleep OSA have opened up and a few nagging question have not been answered so far.
For example, we do not know if sleep OSA is only replaying previous experience. Dragoi and Tonegawa [16] showed preplay during "sleep/rest" periods. We do not know whether preplay occurs during the awake or sleep state or both. Given that many results on awake OSA were only revealed through careful analysis after they were initially overlooked, it seems worthwhile to reexamine sleep OSA in similar ways. Analyses of OSA in both sleep and the awake state in one and the same experiment are needed to determine whether sleep and awake OSA are fundamentally different, aside from the fact that the latter occurs in the reverse order, too. In this vein, the conflicting reports that
replay is independent of experience [15] and that replay reflects the amount of previous exposure [25, 26] might be explained by the fact that the first study was conducted in the awake state and the other two in sleep.
While there is much evidence that sleep OSA is involved in consolidation, the link is not exclusive in either direction. On the one hand, sleep OSA in the hippocampus might not be the sole, or even the main, mechanism of consolidation. In fact, the number of rhinal, but not hippocampal, SWRs is correlated with subsequent memory performance in a study conducted in humans [20]. In rats, consolidation is only partially impaired when reactivation in the hippocampus is suppressed by stimulation [22, 23]. In addition to technical explanations for the residual learning discussed above, it is also possible that replay is only one of several mechanisms involved in consolidation. Other potential mechanisms for consolidation include neuronal synchronization [88, 89], and nonsynaptic plasticity phenomena [90, 91].
In the other direction, consolidation might not be the only function of sleep OSA. For example, a computational modeling study of neocortical-hippocampal interaction found that replay might aid the formation of semantic memories and be necessary for the continued maintenance of episodic memories [92]. This result lends support to the multiple memory trace hypothesis that suggests that episodic memories never become fully independent of the hippocampus [93].
Finally, it remains unclear how exactly replay drives consolidation. We do not know, for instance, whether replay strengthens synaptic weights or weakens recently established synaptic connections. Rasch and Born [94] proposed, for instance, that memory traces were transiently destabilized by reactivation to allow for their stabilization and integration into preexisting long-term memories. Similarly, Mehta [95] suggested that reactivation erases memory traces to create a clean slate for future memories. In rat hippocampal slices that are able to spontaneously produce SWRs, Colgin et al. [96] observed that LTP is impaired probably due to the presence of SWRs. These suggestions are consistent with the broader hypothesis that the main purpose of sleep is synaptic downscaling to restore encoding capabilities of the network [97].
5. Conclusion
We have reviewed many of the exciting and important findings about reactivation and replay and proposed that they are part of a larger class of phenomena, which we termed OSA. Recent findings suggest that some instances of awake OSA do not repeat sequences that were previously driven by sensory inputs, suggesting that awake OSA is not necessarily a memory trace. We proposed a conceptual framework that parsimoniously accounts for the major known features of the dynamics of OSA. Mounting evidence suggests that sleep reactivation is involved in consolidation although more work is needed to establish the precise mechanisms and to what extend consolidation is driven by OSA versus other mechanisms. By contrast, the functional role of awake OSA and nonreplay OSA are much less clear.
The ubiquity of OSA highlights the ability of biological neural networks to internally generate sequential activity. As such, the mechanism that generates OSA might be related to mechanisms that generate internal dynamics on different timescales and in different neural systems. We name only a few examples here: internally generated sequences at time scales of seconds in the hippocampus [98], free recall of movie sequences [99], working memory maintenance [100], and internal dynamics in sensorimotor learning [101, 102].
We close this paper by pointing out significant gaps in our understanding of OSA dynamics. The schematic shown in Figure 1(a) is only a rough guess. In particular, it is unknown how strongly the network is driven by novelty (dashed curve) versus exposure time in one session (solid line segments), since these two effects have never been studied in the same experiment. Moreover, OSA during the period after exposure, but before sleep sets in, remains unexplored (dashed curves, Figure 1(b)). Activity during this intervening period could simply store the accumulated memory for later consolidation or already be part of the consolidation process. In conclusion, we think that we have only seen the proverbial tip of the iceberg when it comes to OSA. So stay tuned.
Glossary
Consolidation:
Frames:
Exposure:
Offline state:
OSA (offline sequential activity):
Reactivation:
REM: Replay:
Memory process by which memories gradually become independent of the hippocampus
Periods of high activity in V1 and the hippocampus believed to correspond largely to cortical up states [35] Exploratory sharp wave/ripple. Occurs during brief periods of rest, within 2.4s of theta activity [62] Occurs when the animal is placed in an environment and is allowed to explore the environment Behavioral state during which the animal is asleep or resting quietly Sequential activation of neurons during offline states. A catch-all term that includes activity with or without a functional relevance Offline activation of neurons that were active together during earlier exploration
Sleep stage characterized by rapid eye movements
Sequential activation of (>2) neurons during offline states in the same order as previously observed during active exploration Sharp wave/ripples. Sharp waves: large deflections in the unfiltered LFP; ripples: brief (-80 ms) high-frequency (100-250 Hz) bursts in the LFP
SWS: Slow wave sleep. Sleep stage
characterized by slow oscillations in the EEG.
Acknowledgments
The authors thank K. P. Hoffmann for helpful comments on the manuscript. This work was funded by a grant (SFB 874, Project B2) from the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) and a grant from the Stiftung Mercator to S. Cheng.
References
[1] C. Pavlides and J. Winson, "Influences of hippocampal place cell firing in the awake state on the activity of these cells during subsequent sleep episodes," Journal of Neuroscience, vol. 9, no. 8, pp. 2907-2918, 1989.
[2] J. O'Keefe and J. Dostrovsky, "The hippocampus as a spatial map: preliminary evidence from unit activity in the freely-moving rat," Brain Research, vol. 34, no. 1, pp. 171-175, 1971.
[3] M. A. Wilson and B. L. McNaughton, "Reactivation of hippocampal ensemble memories during sleep," Science, vol. 265, no. 5172, pp. 676-679, 1994.
[4] W. E. Skaggs and B. L. McNaughton, "Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience," Science, vol. 271, no. 5257, pp. 18701873, 1996.
[5] K. Louie and M. A. Wilson, "Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep," Neuron, vol. 29, no. 1, pp. 145-156, 2001.
[6] A. K. Lee and M. A. Wilson, "Memory of sequential experience in the hippocampus during slow wave sleep," Neuron, vol. 36, no. 6, pp. 1183-1194, 2002.
[7] G. Buzsaki, L. Leung, and C. Vanderwolf, "Cellular bases of hippocampal EEG in the behaving rat," Brain Research, vol. 287, no. 2, pp. 139-171, 1983.
[8] G. Buzsaki, "Hippocampal sharp waves: their origin and significance," Brain Research, vol. 398, no. 2, pp. 242-252, 1986.
[9] G. Buzsaki, Z. Horvat, R. Urioste, J. Hetke, and K. Wise, "High-frequency network oscillation in the hippocampus," Science, vol. 256, no. 5059, pp. 1025-1027, 1992.
[10] R. G. M. Morris, P. Garrud, J. N. P. Rawlins, and J. O'Keefe, "Place navigation impaired in rats with hippocampal lesions," Nature, vol. 297, no. 5868, pp. 681-683, 1982.
[11] N. J. Fortin, K. L. Agster, and H. B. Eichenbaum, "Critical role of the hippocampus in memory for sequences of events," Nature Neuroscience, vol. 5, no. 5, pp. 458-462, 2002.
[12] J. J. Kim and M. S. Fanselow, "Modality-specific retrograde amnesia of fear," Science, vol. 256, no. 5057, pp. 675-677, 1992.
[13] G. Buzsaki, "Two-stage model of memory trace formation: a role for "noisy" brain states," Neuroscience, vol. 31, no. 3, pp. 551-570, 1989.
[14] J. L. McClelland, B. L. McNaughton, and R. C. O'Reilly, "Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory," Psychological Review, vol. 102, no. 3, pp. 419-457, 1995.
[15] A. S. Gupta, M. A. A. van der Meer, D. S. Touretzky, and A. D. Redish, "Hippocampal replay is not a simple function of experience," Neuron, vol. 65, no. 5, pp. 695-705, 2010.
[16] G. Dragoi and S. Tonegawa, "Preplay of future place cell sequences by hippocampal cellular assemblies," Nature, vol. 469, pp. 397-401,2010.
[17] S. Diekelmann and J. Born, "The memory function of sleep," Nature Reviews Neuroscience, vol. 11, no. 2, pp. 114-126, 2010.
[18] D. Marr, "Simple memory: a theory for archicortex," Philosophical Transactions of the Royal Society of London, Series B, vol. 262, no. 841, pp. 23-81, 1971.
[19] F. Crick and G. Mitchison, "The function of dream sleep," Nature, vol. 304, no. 5922, pp. 111-114, 1983.
[20] N. Axmacher, C. E. Elger, and J. Fell, "Ripples in the medial temporal lobe are relevant for human memory consolidation," Brain, vol. 131, part 7, pp. 1806-1817, 2008.
[21] D. Dupret, J. O'Neill, B. Pleydell-Bouverie, and J. Csicsvari, "The reorganization and reactivation of hippocampal maps predict spatial memory performance," Nature Neuroscience, vol. 13, no. 8, pp. 995-1002, 2010.
[22] G. Girardeau, K. Benchenane, S. I. Wiener, G. Buzsaki, and M. B. Zugaro, "Selective suppression of hippocampal ripples impairs spatial memory," Nature Neuroscience, vol. 12, no. 10, pp. 1222-1223, 2009.
[23] V. Ego-Stengel and M. A. Wilson, "Disruption of ripple-associated hippocampal activity during rest impairs spatial learning in the rat," Hippocampus, vol. 20, no. 1, pp. 1-10, 2010.
[24] D. J. Foster and M. A. Wilson, "Reverse replay of behavioural sequences in hippocampal place cells during the awake state," Nature, vol. 440, no. 7084, pp. 680-683, 2006.
[25] J. C. Jackson, A. Johnson, and A. D. Redish, "Hippocampal sharp waves and reactivation during awake states depend on repeated sequential experience," Journal of Neuroscience, vol. 26, no. 48, pp. 12415-12426, 2006.
[26] J. O'Neill, T. J. Senior, K. Allen, J. R. Huxter, and J. Csicsvari, "Reactivation of experience-dependent cell assembly patterns in the hippocampus," Nature Neuroscience, vol. 11, no. 2, pp. 209-215, 2008.
[27] T. V. P. Bliss and T. Lomo, "Long lasting potentiation of synaptic transmission in the dentate area ofthe anaesthetized rabbit following stimulation of the perforant path," Journal of Physiology, vol. 232, no. 2, pp. 331-356, 1973.
[28] W. Gerstner, R. Kempter, J. L. van Hemmen, and H. Wagner, "A neuronal learning rule for sub-millisecond temporal coding," Nature, vol. 383, no. 6595, pp. 76-78, 1996.
[29] H. Markram, J. Lubke, M. Frotscher, and B. Sakmann, "Regulation of synaptic efficacy by coincidence of postsynaptic APs andEPSPs," Science, vol. 275, no. 5297, pp. 213-215, 1997.
[30] J. C. Magee and D. Johnston, "A synaptically controlled, associative signal for Hebbian plasticity in hippocampal neurons," Science, vol. 275, no. 5297, pp. 209-213, 1997.
[31] Z. Nadasdy, H. Hirase, A. Czurko, J. Csicsvari, and G. Buzsaki, "Replay and time compression of recurring spike sequences in the hippocampus," Journal of Neuroscience, vol. 19, no. 21, pp. 9497-9507, 1999.
[32] J. O'Keefe and M. L. Recce, "Phase relationship between hippocampal place units and the EEG theta rhythm," Hippocampus, vol. 3, no. 3, pp. 317-330, 1993.
[33] R. Schmidt, K. Diba, C. Leibold, D. Schmitz, G. Buzsaki, and R. Kempter, "Single-trial phase precession in the hippocampus," Journal of Neuroscience, vol. 29, no. 42, pp. 1323213241,2009.
[34] Y. L. Qin, B. L. McNaughton, W. E. Skaggs, and C. A. Barnes, "Memory reprocessing in corticocortical and hippocam-pocortical neuronal ensembles," Philosophical Transactions of the Royal, Society B, vol. 352, no. 1360, pp. 1525-1533, 1997.
[35] D. Ji and M. A. Wilson, "Coordinated memory replay in the visual cortex and hippocampus during sleep," Nature Neuroscience, vol. 10, no. 1, pp. 100-107, 2007.
[36] D. R. Euston, M. Tatsuno, and B. L. McNaughton, "Fast-forward playback of recent memory sequences in prefrontal cortex during sleep," Science, vol. 318, no. 5853, pp. 11471150, 2007.
[37] A. Peyrache, M. Khamassi, K. Benchenane, S. I. Wiener, and F. P. Battaglia, "Replay of rule-learning related neural patterns in the prefrontal cortex during sleep," Nature Neuroscience, vol. 12, no. 7, pp. 919-926, 2009.
[38] C. M. A. Pennartz, E. Lee, J. Verheul, P. Lipa, C. A. Barnes, and B. L. McNaughton, "The ventral striatum in off-line processing: ensemble reactivation during sleep and modulation by hippocampal ripples," Journal of Neuroscience, vol. 24, no. 29, pp. 6446-6456, 2004.
[39] C. S. Lansink, P. M. Goltstein, J. V. Lankelma, R. N. J. M. A. Joosten, B. L. McNaughton, and C. M. A. Pennartz, "Preferential reactivation of motivationally relevant information in the ventral striatum," Journal of Neuroscience, vol. 28, no. 25, pp. 6372-6382, 2008.
[40] C. S. Lansink, P. M. Goltstein, J. V. Lankelma, B. L. McNaughton, and C. M. A. Pennartz, "Hippocampus leads ventral striatum in replay of place-reward information," PLoS Biology, vol. 7, no. 8, Article ID e1000173, 2009.
[41] A. Bragin, J. Engel, C. L. Wilson, I. Fried, and G. Buzsaki, "High-frequency oscillations in human brain," Hippocampus, vol. 9, no. 2, pp. 137-142, 1999.
[42] F. Grenier, I. Timofeev, and M. Steriade, "Focal synchronization of ripples (80-200 Hz) in neocortex and their neuronal correlates," Journal of Neurophysiology, vol. 86, no. 4, pp. 1884-1898, 2001.
[43] A. Siapas and M. Wilson, "Coordinated interactions between hippocampal ripples and cortical spindles during slow-wave sleep," Neuron, vol. 21, no. 5, pp. 1123-1128, 1998.
[44] M. Steriade, A. Nunez, and F. Amzica, "Intracellular analysis of relations between the slow (< 1 Hz) neocortical oscillation and other sleep rhythms of the electroencephalogram," Journal ofNeuroscience, vol. 13, no. 8, pp. 3266-3283, 1993.
[45] R. L. Cowan and C. J. Wilson, "Spontaneous firing patterns and axonal projections of single corticostriatal neurons in the rat medial agranular cortex," Journal of Neurophysiology, vol. 71, no. 1,pp. 17-32, 1994.
[46] A. Sirota, J. Csicsvari, D. Buhl, and G. Buzsaki, "Communication between neocortex and hippocampus during sleep in rodents," Proceedings of the National Academy of Sciences of the United States ofAmerica, vol. 100, no. 4, pp. 2065-2069, 2003.
[47] F. P. Battaglia, G. R. Sutherland, and B. L. McNaughton, "Hippocampal sharp wave bursts coincide with neocortical "up-state" transitions," Learning and Memory, vol. 11, no. 6, pp. 697-704, 2004.
[48] M. Molle, O. Yeshenko, L. Marshall, S. J. Sara, and J. Born, "Hippocampal sharp wave-ripples linked to slow oscillations
in rat slow-wave sleep," Journal of Neurophysiology, vol. 96, no. 1, pp. 62-70, 2006.
[49] T. T. G. Hahn, B. Sakmann, and M. R. Mehta, "Phase-locking of hippocampal interneurons' membrane potential to neocortical up-down states," Nature Neuroscience, vol. 9, no. 11, pp. 1359-1361,2006.
[50] Y. Isomura, A. Sirota, S. Ozen et al., "Integration and segregation of activity in entorhinal-hippocampal subregions by neocortical slow oscillations," Neuron, vol. 52, no. 5, pp. 871882, 2006.
[51] M. Volgushev, S. Chauvette, M. Mukovski, and I. Timofeev, "Precise long-range synchronization of activity and silence in neocortical neurons during slow-wave sleep," Journal of Neuroscience, vol. 26, no. 21, pp. 5665-5672, 2006.
[52] C. M. Wierzynski, E. V. Lubenov, M. Gu, and A. G. Siapas, "State-dependent spike-timing relationships between hippocampal and prefrontal circuits during sleep," Neuron, vol. 61, no. 4, pp. 587-596, 2009.
[53] H. Groenewegen, E. Derzee, A. Tekortschot, and M. Witter, "Organization of the projections from the subiculum to the ventral striatum in the rat: a study using anterograde transport of Phaseolus vulgaris leucoagglutinin," Neuroscience, vol. 23, no. 1, pp. 103-120, 1987.
[54] K. L. Hoffman and B. L. McNaughton, "Coordinated reactivation of distributed memory traces in primate neocortex," Science, vol. 297, no. 5589, pp. 2070-2073, 2002.
[55] L. A. Johnson, D. R. Euston, M. Tatsuno, and B. L. McNaughton, "Stored-trace reactivation in rat prefrontal cortex is correlated with down-to-up state fluctuation density," Journal of Neuroscience, vol. 30, no. 7, pp. 2650-2661, 2010.
[56] Z. Clemens, M. Molle, L. Eross, P. Barsi, P. Halasz, and J. Born, "Temporal coupling of parahippocampal ripples, sleep spindles and slow oscillations in humans," Brain, vol. 130, no. 11, pp. 2868-2878,2007.
[57] A. D. Ekstrom, M. J. Kahana, J. B. Caplan et al., "Cellular networks underlying human spatial navigation," Nature, vol. 425, no. 6954, pp. 184-188, 2003.
[58] W. E. Skaggs, B. L. McNaughton, M. Permenter et al., "EEG sharp waves and sparse ensemble unit activity in the macaque hippocampus," Journal of Neurophysiology, vol. 98, no. 2, pp. 898-910, 2007.
[59] M. S. Nokia, M. Penttonen, and J. Wikgren, "Hippocam-pal ripple-contingent training accelerates trace eyeblink conditioning and retards extinction in rabbits," Journal of Neuroscience, vol. 30, no. 34, pp. 11486-11492, 2010.
[60] A. S. Dave and D. Margoliash, "Song replay during sleep and computational rules for sensorimotor vocal learning," Science, vol. 290, no. 5492, pp. 812-816, 2000.
[61] H. S. Kudrimoti, C. A. Barnes, and B. L. McNaughton, "Reactivation of hippocampal cell assemblies: effects of behavioral state, experience, and EEG dynamics," Journal of Neuroscience, vol. 19, no. 10, pp. 4090-4101, 1999.
[62] J. O'Neill, T. Senior, and J. Csicsvari, "Place-selective firing of CA1 pyramidal cells during sharp wave/ripple network patterns in exploratory behavior," Neuron, vol. 49, no. 1, pp. 143-155, 2006.
[63] J. Csicsvari, J. O'Neill, K. Allen, and T. Senior, "Place-selective firing contributes to the reverse-order reactivation of CA1 pyramidal cells during sharp waves in open-field exploration," European Journal of Neuroscience, vol. 26, no. 3, pp. 704-716, 2007.
[64] K. Diba and G. Buzsáki, "Forward and reverse hippocampal place-cell sequences during ripples," Nature Neuroscience, vol. 10, no. 10, pp. 1241-1242, 2007.
[65] T. J. Davidson, F. Kloosterman, and M. A. Wilson, "Hippocampal replay of extended experience," Neuron, vol. 63, no. 4, pp. 497-507, 2009.
[66] M. P. Karlsson and L. M. Frank, "Awake replay of remote experiences in the hippocampus," Nature Neuroscience, vol. 12, no. 7, pp. 913-918, 2009.
[67] J. Epsztein, M. Brecht, and A. K. Lee, "Intracellular determinants of hippocampal CA1 place and silent cell activity in a novel environment," Neuron, vol. 70, no. 1, pp. 109-120, 2011.
[68] C. D. Harvey, F. Collman, D. A. Dombeck, and D. W. Tank, "Intracellular dynamics of hippocampal place cells during virtual navigation," Nature, vol. 461, no. 7266, pp. 941-946, 2009.
[69] S. Cheng and L. M. Frank, "New experiences enhance coordinated neural activity in the hippocampus," Neuron, vol. 57, no. 2, pp. 303-313,2008.
[70] M. F. Carr, S. P. Jadhav, and L. M. Frank, "Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval," Nature Neuroscience, vol. 14, no.
2, pp. 147-153, 2011.
[71] S. Ribeiro, D. Gervasoni, E. S. Soares et al., "Long-lasting novelty-induced neuronal reverberation during slow-wave sleep in multiple forebrain areas," PLoS Biology, vol. 2, no. 1, article e24, 2004.
[72] M. Tatsuno, P. Lipa, and B. L. McNaughton, "Methodological considerations on the use of template matching to study long-lasting memory trace replay," Journal of Neuroscience, vol. 26, no. 42, pp. 10727-10742, 2006.
[73] O. Eschenko, W. Ramadan, M. Molle, J. Born, and S. J. Sara, "Sustained increase in hippocampal sharp-wave ripple activity during slow-wave sleep after learning," Learning and Memory, vol. 15, no. 4, pp. 222-228, 2008.
[74] W. Ramadan, O. Eschenko, and S. J. Sara, "Hippocampal sharp wave/ripples during sleep for consolidation of associative memory," PLoS ONE, vol. 4, no. 8, Article ID e6697,2009.
[75] B. F. Skinner, The Science of Learning and the Art of Teaching, Cambridge, Mass, USA, 1954.
[76] G. H. Bower, "Mood and memory," The American Psychologist, vol. 36, no. 2, pp. 129-148, 1981.
[77] B. Roozendaal, "Stress and memory: opposing effects of glucocorticoids on memory consolidation and memory retrieval," Neurobiology of Learning and Memory, vol. 78, no.
3, pp. 578-595, 2002.
[78] R. C. Grider and K. J. Malmberg, "Discriminating between changes in bias and changes in accuracy for recognition memory of emotional stimuli," Memory and Cognition, vol. 36, no. 5, pp. 933-946, 2008.
[79] A. C. Singer and L. M. Frank, "Rewarded outcomes enhance reactivation of experience in the hippocampus," Neuron, vol. 64, no. 6, pp. 910-921,2009.
[80] I. Lee, A. L. Griffin, E. A. Zilli, H. Eichenbaum, and M. E. Hasselmo, "Gradual translocation of spatial correlates of neuronal firing in the hippocampus toward prospective reward locations," Neuron, vol. 51, no. 5, pp. 639-650, 2006.
[81] W. Schultz, P. Apicella, E. Scarnati, and T. Ljungberg, "Neuronal activity in monkey ventral striatum related to the expectation of reward," Journal of Neuroscience, vol. 12, no. 12, pp. 4595-4610, 1992.
[82] J. J. Hopfield, "Neurodynamics of mental exploration," Proceedings of the National Academy of Sciences of the United States of America, vol. 107, no. 4, pp. 1648-1653, 2010.
[83] E. C. Tolman, "Cognitive maps in rats and men," Psychological Review, vol. 55, no. 4, pp. 189-208, 1948.
[84] J. O'Keefe and L. Nadel, The Hippocampus as a Cognitive Map, Clarendon Press, Oxford, UK, 1978.
[85] R. L. Buckner, "The role of the hippocampus in prediction and imagination," Annual Review of Psychology, vol. 61, no. 1,pp. 27-48, 2010.
[86] A. Samsonovich and B. L. McNaughton, "Path integration and cognitive mapping in a continuous attractor neural network model," Journal of Neuroscience, vol. 17, no. 15, pp. 5900-5920, 1997.
[87] A. Johnson and A. D. Redish, "Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point," Journal of Neuroscience, vol. 27, no. 45, pp. 1217612189,2007.
[88] N. Axmacher, F. Mormann, G. Fernandez, C. E. Elger, and J. Fell, "Memory formation by neuronal synchronization," Brain Research Reviews, vol. 52, no. 1, pp. 170-182, 2006.
[89] J. Fell and N. Axmacher, "The role of phase synchronization in memory processes," Nature Reviews Neuroscience, vol. 12, no. 2, pp. 105-118, 2011.
[90] G. Daoudal and D. Debanne, "Long-term plasticity ofintrin-sic excitability: learning rules and mechanisms," Learning and Memory, vol. 10, no. 6, pp. 456-465, 2003.
[91] N. Axmacher, A. Draguhn, C. E. Elger, and J. Fell, "Memory processes during sleep: beyond the standard consolidation theory," Cellular and Molecular Life Sciences, vol. 66, no. 14, pp. 2285-2297, 2009.
[92] S. Kali and P. Dayan, "Off-line replay maintains declarative memories in a model of hippocampal-neocortical interactions," Nature Neuroscience, vol. 7, no. 3, pp. 286-294, 2004.
[93] L. Nadel and M. Moscovitch, "Memory consolidation, retrograde amnesia and the hippocampal complex," Current Opinion in Neurobiology, vol. 7, no. 2, pp. 217-227, 1997.
[94] B. Rasch and J. Born, "Maintaining memories by reactivation," Current Opinion in Neurobiology, vol. 17, no. 6, pp. 698-703, 2007.
[95] M. R. Mehta, "Cortico-hippocampal interaction during up-down states and memory consolidation," Nature Neuroscience, vol. 10, no. 1, pp. 13-15, 2007.
[96] L. L. Colgin, D. Kubota, Y. Jia, C. S. Rex, and G. Lynch, "Long-term potentiation is impaired in rat hippocampal slices that produce spontaneous sharp waves," Journal ofPhysiology, vol. 558, no. 3, pp. 953-961, 2004.
[97] G. Tononi and C. Cirelli, "Sleep function and synaptic homeostasis," Sleep Medicine Reviews, vol. 10, no. 1, pp. 4962, 2006.
[98] E. Pastalkova, V. Itskov, A. Amarasingham, and G. Buzsaki, "Internally generated cell assembly sequences in the rat hippocampus," Science, vol. 321, no. 5894, pp. 1322-1327, 2008.
[99] H. Gelbard-Sagiv, R. Mukamel, M. Harel, R. Malach, and I. Fried, "Internally generated reactivation of single neurons in human hippocampus during free recall," Science, vol. 322, no. 5898, pp. 96-101,2008.
[100] L. Fuentemilla, W. D. Penny, N. Cashdollar, N. Bunzeck, and E. Duzel, "Theta-coupled periodic replay in working memory," Current Biology, vol. 20, no. 7, pp. 606-612, 2010.
[101] S. Cheng and P. N. Sabes, "Modeling sensorimotor learning with linear dynamical systems," Neural Computation, vol. 18, no. 4, pp. 760-793, 2006.
[102] S. Cheng and P. N. Sabes, "Calibration of visually guided reaching is driven by error-corrective learning and internal dynamics," Journal of Neurophysiology, vol. 97, no. 4, pp. 3057-3069, 2007.
