Excitatory/inhibitory balance of serotonergic axon connectivity in the brain

Arnauld Belmer1*, Vanessa Lanoue3, Omkar L. Patkar1, and Selena E. Bartlett1,2

1Translational Research Institute, Queensland University of Technology (QUT), Brisbane, Australia
2Institute of Health and Biomedical Innovation (IHBI), Queensland University of Technology (QUT), Brisbane, Australia
3Queensland Brain Institute (QBI), The University of Queensland, Brisbane, Australia


Serotonin neurons originate from the brainstem raphe nuclei and innervate the entire brain to regulate mood, emotion, sleep, appetite and aggression. Previous electron microscopy (EM) studies have revealed that 5-HT boutons directly contact several neuronal populations via asymmetrical (excitatory) or symmetrical (inhibitory) synapses. Additionally, 5-HT boutons sometimes form “triads” with the pre and postsynaptic components of asymmetrical or symmetrical synapses to modulate their activity. However, the exact proportion and distribution of 5-HT excitatory/inhibitory synapses and triads within the entire brain remains poorly described. Recently, we have published a novel semi-quantitative approach which combines fluorescent confocal microscopy and 3D reconstruction of 5-HT fibers apposed to excitatory and inhibitory neurochemical synapses (triads). Here, we review the similarities and differences in the distribution of 5-HT asymmetrical/symmetrical synapses observed in EM and the distribution of 5-HT excitatory/inhibitory triads quantified in our recent study. We further put into perspective the possible physiological role played by 5-HT triads in the regulation of glutamate and GABA signaling in these various brain regions.

Serotonin (5-Hydroxytriptamine, 5-HT) neurons have been extensively involved in the regulation of mood1, sleep2, learning3, memory4, cognition5 and impulsivity6. As such, the functional connectivity of 5-HT neurons within the brain is complex and remains poorly understood. Pioneer work from electron microscopy (EM) studies have unraveled the heterogeneous connectivity of 5-HT neurons throughout the rodent brain, revealing that 5-HT neurons send axons to almost all brain regions and modulate various populations of neurons by synaptic contacts that are either asymmetrical (excitatory) or symmetrical (inhibitory)7. Similarly, some of these electron microscopy studies have also allowed the observation of 5-HT-containing boutons that synapse to the pre or the postsynaptic components of asymmetrical or symmetrical synapses to form “triads”. These triads have been observed in the cortex8, the nucleus accumbens9, the hippocampus10, the amygdala11, the VTA12 and the striatum7. However, the complex and time consuming processing/serial sectioning of samples required for high-quality EM have not allowed the aforementioned studies to quantify these triads in the corresponding brain regions.

Therefore, we have recently developed a semi-quantitative method combining high-resolution confocal microscopy and 3D reconstruction of 5-HT transporter SERT-positive axons to map the distribution of 5-HT boutons apposed to either excitatory or inhibitory neurochemical synapses in the mouse limbic brain13. Briefly, we labeled serotonin transporter (SERT) immunoreactive axons and reconstruct in 3D their distribution within limbic brain regions. We also labelled key pre- (synaptophysin) and postsynaptic components of excitatory (PSD95) or inhibitory (gephyrin) synapses. Using the masking function in IMARIS software, we were able to isolate the synaptophysin punctate labelling inside the SERT-positive fibers (SYNin) to identify 5-HT boutons. In parallel, we used the same masking function to isolate the synaptophysin puncta outside SERT fibers (SYNout) and their engagement into excitatory (SYNout/PSD95 spot pairs) or inhibitory (SYNout/GEPH spot pairs) neurochemical synapses. Finally, we quantified the density of 5-HT boutons forming “triads” with either excitatory or inhibitory neurochemical synapses in various limbic regions of the mouse brain13.

Importantly, the distribution of excitatory vs inhibitory triads in our study was quite similar to the distribution of direct asymmetrical (excitatory) vs symmetrical (inhibitory) synapses made by 5-HT boutons, as observed in previous electron microscopy studies. This suggests that the excitatory/inhibitory balance of 5-HT axon connectivity is a common structural feature shared by both 5-HTergic direct synapses and synaptic triads in the rodent brain. Here, we review the similarities and differences in the distribution of 5-HT synapses and triads throughout various brain regions where serotonin is known to play an important regulatory role, with an emphasis on the physiological significance previously demonstrated in functional studies.

Previous electron microscopy studies in rat motor, visual and somatosensory cortices8 found that only 20-40% of 5-HT-immunolabelled varicosities (boutons) were involved in synaptic contacts. These contacts were essentially asymmetric on dendritic spines and branches. Similar results were observed in rat frontoparietal cortex using Tryptophan hydroxylase (TPH) immunolabelling14 or in the prefrontal cortex using SERT immunolabelling15. In the prefrontal cortex of monkeys, only 23% of labelled 5-HT boutons were engaged in synaptic contacts, mostly asymmetrical and formed on dendritic shafts16. However, in the auditory and sensorimotor cortices of cats and monkeys, respectively, a relatively low synaptic incidence (2-3%) was demonstrated, but again synaptic contacts were all asymmetrical17,18. Additionally, in the upper layers of various cortical regions, 5-HT terminals were often apposed to non-5-HT axons engaged in asymmetrical synapses in a triadic formation8. This suggests a potential role of cortical 5-HT in the regulation of neurotransmitter release. In line with this, we showed that 5-HT boutons essentially form synaptic triads with excitatory synapses in the upper layer of the mouse prefrontal cortex and were located closer to the presynaptic than the postsynaptic specializations, also suggesting a role of 5-HT in the regulation of excitatory neurotransmitter (glutamate) release13. The potential role of 5-HT in the regulation of excitatory transmission has been demonstrated in electrophysiology studies showing a 5-HT-mediated increase in glutamate release and amplitude of glutamatergic EPSCs at the apical dendrites of pyramidal neurons19. Blockade of 5-HT receptors (5-HT2A) was also shown to prevent local glutamate release in the medial prefrontal cortex (mPFC) in response to NMDA glutamate receptor antagonist20.

In the rat stratum oriens layer of CA3, a region known to have the highest density of 5-HT fibers in the hippocampus21, about 20% of immunolabelled 5-HT boutons were involved in synaptic contacts, with a greater proportion of asymmetrical synapses, only on dendritic shafts. We found that 5-HT boutons form almost twice as much triadic contacts with excitatory than inhibitory neurochemical synapses in the CA3 region of the mouse hippocampus, suggesting that 5-HT signaling is more involved in the regulation of excitatory transmission in this brain region. Similarly, strong evidence has suggested a preferential involvement of 5-HT in the negative regulation of glutamate signaling in other hippocampal regions including CA1 and CA222–26. This was further shown to play an important role in the regulation of long term potentiation (LTP)27,28. However, functional studies in the CA3 region have demonstrated that 5-HT both exerts a direct positive modulation of glutamate receptors29 and depress the GABABreceptor component of GABA-mediated IPSPs30–32, suggesting a 5-HT-mediated regulation of inhibitory synapses as well.

In the NAC core and shell, 5-HT labelled boutons were shown to have a relatively high synaptic incidence (39 and 46%, respectively9) as compared to other brain regions. The great majority of these 5-HT boutons were in apposition with terminals that often form asymmetric contacts with dendrites, suggesting that 5-HT boutons likely modulate the activity of excitatory axons9. We also found a higher proportion of triadic contacts with excitatory synapses (21%) compared to inhibitory synapses (3%) in the NAC core. This result suggests that 5-HT signaling may preferentially modulate the activity of excitatory synapses in this brain region. However, in the NAC shell, we observed an equal distribution of 5-HT triads onto neurochemical excitatory and inhibitory synapses. The NAC shell was previously shown to be innervated by two functionally different types of 5-HTergic axons that either contain or lack the serotonin transporter SERT33. Therefore, our method of labeling 5-HT fibers with an antibody directed against the SERT may have revealed a specific subtype of 5-HTergic axons that is equally engaged in triads with excitatory and inhibitory synapses. Similar differences have been observed in electron microcopy studies where the use of a 5-HT antibody allows for the labelling of 5-HT boutons forming mostly asymmetrical synapses9, whereas the use of a SERT antibody allowed for the labelling of 5-HT boutons forming both symmetrical and asymmetrical synapses in the rat NAC shell34. Interestingly, we found a significantly higher proportion of 5-HT boutons located closer to the presynaptic component of putative inhibitory synapses within the NAC shell, suggesting that 5-HT could have a modulatory effect on GABA release from GABAergic synapses in this region. In addition, functional studies have also revealed a 5-HT-mediated control of glutamate release in rat NAC core and shell slices via activation of presynaptic 5-HT1B receptors35, however, we did not observe any preferential distribution of 5-HT boutons to the pre- or the postsynaptic specialization of excitatory triads. Whether 5-HT modulates only glutamate release from presynaptic terminals or also postsynaptic activity of glutamatergic synapses in the NAC still needs to be determined.

Serotonin axons are abundant in the VTA, and it is likely that half of the EM-labelled 5-HT boutons are involved in synaptic contacts, almost exclusively on symmetrical (inhibitory) synapses, as evaluated by quantification extrapolated from a single thin section study in the rat brain36. Importantly, we found that the 5-HT triads were exclusively formed with inhibitory neurochemical synapses in our study, which suggests that 5-HT plays a pivotal role in the regulation of inhibitory transmission in the VTA. In line with these findings, limited evidence supports a role 5-HT in the regulation of glutamate signaling in the VTA, while there is an extensive literature showing a 5-HT-mediated regulation of GABAergic neurotransmission. For instance, in rat VTA slices, 5-HT1Breceptor activation was shown to inhibit GABA release37,38 as well as GABAB-mediated IPSCs39. Furthermore, the reduction in GABAB inhibitory postsynaptic potentials in dopamine neurons of rat VTA slices induced by cocaine was found to be mediated by 5-HT1B receptor39 which, in turn, facilitates cocaine-induced increases in dopamine levels in the NAC core40,41. This suggests that in the VTA, 5-HT signalling is principally involved in the control of inhibitory transmission.

Serotonin axon connectivity is highly heterogeneous along the rostro-caudal axis and, interestingly, the balanced distribution of asymmetrical/symmetrical synapses observed in electron microscopy studies follows a pattern similar to the distribution of 5-HT excitatory/inhibitory triads in our recent study (Figure 1). Although asymmetrical and symmetrical synapses have been shown to express PSD9542 and gephyrin43 respectively, there is so far no evidence that 5-HT asymmetrical or symmetrical synapses are excitatory or inhibitory, or whether 5-HT postsynaptic specializations express PSD95 or gephyrin. Further studies are therefore needed to identify specific markers of 5-HT synapses and provide a more comprehensive understanding of 5-HT axon connectivity.


Figure 1: Distribution of 5-HT synaptic contacts and triads in the rodent brain. Cx: cortex; mPFC: medial prefrontal cortex; NAC: nucleus accumbens; BNST: bed nucleus of the stria terminalis; TH: thalamus; CeA: central nucleus of the amygdala; BLA: basolateral nucleus of the amygdala; VTA: ventral tegmental area; DR: dorsal raphe. Color coded from blue to red represents the heterogeneous distribution of 5-HT axons connectivity (blue: asymmetrical synapses/excitatory triads; red: symmetrical synapse/inhibitory triads. Adapted from8–41.

  1. Strasser B, Gostner JM, Fuchs D. Mood food and cognition role of tryptophan and serotonin. Curr Opin Clin Nutr Metab Care 19. 2016: 55–61.
  2. Monti JM. The structure of the dorsal raphe nucleus and its relevance to the regulation of sleep and wakefulness. Sleep Med Rev. 2010; 14: 307–317 .
  3. Meneses A, Liy Salmeron G. Serotonin and emotion learning and memory. Rev Neurosci. 2012; 23: 543–553.
  4. Seyedabadi M, Fakhfouri G, Ramezani V. The role of serotonin in memory interactions with neurotransmitters and downstream signaling. Exp Brain Res. 2014; 232: 723–738.
  5. Cowen P, Sherwood AC. The role of serotonin in cognitive function evidence from recent studies and implications for understanding depression. J Psychopharmacol Oxf Engl. 2013; 27: 575–583.
  6. Miyazaki K. Miyazaki KW, Doya K. The role of serotonin in the regulation of patience and impulsivity. Mol Neurobiol. 2012; 45: 213–224.
  7. Descarries L. in Handbook of Behavioral Neuroscience ed Jacobs C P M and B L.2010; 21: 65–101 Elsevier.
  8. Séguéla P, Watkins KC, Descarries L. Ultrastructural relationships of serotonin axon terminals in the cerebral cortex of the adult rat. J Comp Neurol. 1989; 289: 129–142.
  9. Van Bockstaele EJ, Pickel VM. Ultrastructure of serotonin immunoreactive terminals in the core and shell of the rat nucleus accumbens cellular substrates for interactions with catecholamine afferents. J Comp Neurol. 1993; 334: 603–617.
  10. Umbriaco D, Garcia S, Beaulieu C. Relational features of acetylcholine, noradrenaline serotonin and GABA axon terminals in the stratum radiatum of adult rat hippocampus CA1. Hippocampus. 1995; 5: 605–620.
  11. Muller JF, Mascagni F, McDonald AJ. Serotonin immunoreactive axon terminals innervate pyramidal cells and interneurons in the rat basolateral amygdala. J Comp Neurol. 2007; 505: 314–335.
  12. Van Bockstaele EJ, Cestari DM, Pickel VM. Synaptic structure and connectivity of serotonin terminals in the ventral tegmental area potential sites for modulation of mesolimbic dopamine neurons. Brain Res. 1994; 647: 307–322.
  13. Belmer A, Klenowski PM, Patkar OL. Mapping the connectivity of serotonin transporter immunoreactive axons to excitatory and inhibitory neurochemical synapses in the mouse limbic brain. Brain Struct Funct. 2016. doi:10.1007/s00429-016-1278-x
  14. Cohen Z, Ehret M, Maitre M. Ultrastructural analysis of tryptophan hydroxylase immunoreactive nerve terminals in the rat cerebral cortex and hippocampus their associations with local blood vessels. Neuroscience. 1995; 66: 555–569.
  15. Miner LH, Schroeter S, Blakely RD. Ultrastructural localization of the serotonin transporter in superficial and deep layers of the rat prelimbic prefrontal cortex and its spatial relationship to dopamine terminals. J Comp Neurol. 2000; 427: 220–234.
  16. Smiley JF, Goldman Rakic PS. Serotonergic axons in monkey prefrontal cerebral cortex synapse predominantly on interneurons as demonstrated by serial section electron microscopy. J Comp Neurol.1996; 367: 431–443.
  17. DeFelipe J, Jones EG. A light and electron microscopic study of serotonin-immunoreactive fibers and terminals in the monkey sensory motor cortex. Exp Brain Res. 1988; 71: 171–182.
  18. DeFelipe J, Hendry SH, Hashikawa T. Synaptic relationships of serotonin-immunoreactive terminal baskets on GABA neurons in the cat auditory cortex. Cereb Cortex N Y N .1991; 1: 117–13.
  19. Aghajanian GK, Marek GJ. Serotonin induces excitatory postsynaptic potentials in apical dendrites of neocortical pyramidal cells. Neuropharmacology. 1997; 36: 589–599.
  20. Ceglia I, Carli M, Baviera M. The 5-HT receptor antagonist M100907 prevents extracellular glutamate rising in response to NMDA receptor blockade in the mPFC. J Neurochem. 2004; 91: 189–199.
  21. Oleskevich S, Descarries L, Watkins KC. Ultrastructural features of the serotonin innervation in adult rat hippocampus: an immunocytochemical description in single and serial thin sections. Neuroscience. 1991; 42: 777–791.
  22. Aït Amara D, Segu L, Naïli S. Serotonin 1B receptor regulation after dorsal subiculum deafferentation. Brain Res Bull. 1995; 38: 17–23.
  23. Schmitz D, Empson RM, Heinemann U. Serotonin and 8-OH-DPAT reduce excitatory transmission in rat hippocampal area CA1 via reduction in presumed presynaptic Ca2+ entry. Brain Res. 1995; 701: 249–254.
  24. Boeijinga PH, Boddeke HW. Activation of 5-HT1B receptors suppresses low but not high frequency synaptic transmission in the rat subicular cortex in vitro. Brain Res. 1996; 721: 59–65.
  25. Dawson LA, Nguyen HQ, Li P. The 5-HT(6) receptor antagonist SB-271046 selectively enhances excitatory neurotransmission in the rat frontal cortex and hippocampus. Neuropsychopharmacol Off Publ Am Coll Neuropsychopharmacol. 2001; 25: 662–668.
  26. Mlinar B, Falsini C, Corradetti R. Pharmacological characterization of 5-HT 1B receptor-mediated inhibition of local excitatory synaptic transmission in the CA1 region of rat hippocampus. Br J Pharmacol. 2003; 138: 71–80.
  27. Mlinar B, Stocca G, Corradetti R. Endogenous serotonin facilitates hippocampal long-term potentiation at CA3/CA1 synapses. J Neural Transm Vienna Austria 1996. 2015; 122: 177–185.
  28. Staubli U, Otaky N. Serotonin controls the magnitude of LTP induced by theta bursts via an action on NMDA receptor mediated responses. Brain Res. 1994; 643: 10–16.
  29. Mennini T, Miari A. Modulation of 3H glutamate binding by serotonin in the rat hippocampus an autoradiographic study. Life Sci. 1991; 49: 283–292.
  30. Andrade R, Malenka RC, Nicoll RA. A G protein couples serotonin and GABAB receptors to the same channels in hippocampus. Science, 1986; 234: 1261–1265.
  31. Okuhara DY, Beck SG. 5-HT1A receptor linked to inward-rectifying potassium current in hippocampal CA3 pyramidal cells. J Neurophysiol. 1994; 71: 2161–2167.
  32. Oleskevich S, Lacaille JC. Reduction of GABAB inhibitory postsynaptic potentials by serotonin via pre- and postsynaptic mechanisms in CA3 pyramidal cells of rat hippocampus in vitro. Synap N Y N; 12: 173–188.
  33. Brown P, Molliver ME. Dual serotonin 5-HT projections to the nucleus accumbens core and shell: relation of the 5-HT transporter to amphetamine-induced neurotoxicity. J Neurosci Off J Soc Neurosci. 2000; 20: 1952–1963.
  34. Pickel VM, Chan J. Ultrastructural localization of the serotonin transporter in limbic and motor compartments of the nucleus accumbens. J Neurosci Off J Soc Neurosci. 1999; 19: 7356–7366.
  35. Muramatsu M, Lapiz MD, Tanaka E. Serotonin inhibits synaptic glutamate currents in rat nucleus accumbens neurons via presynaptic 5-HT1B receptors. Eur J Neurosci. 1998; 10: 2371–2379.
  36. Hervé D, Pickel VM, Joh TH. Serotonin axon terminals in the ventral tegmental area of the rat fine structure and synaptic input to dopaminergic neurons. Brain Res. 1987; 435: 71–83.
  37. Johnson SW, Mercuri NB, North RA. 5-hydroxytryptamine1B receptors block the GABAB synaptic potential in rat dopamine neurons. J Neurosci Off J Soc Neurosci. 1992; 12: 2000–2006.
  38. Yan QS, Yan SE. Serotonin-1B receptor-mediated inhibition of 3H GABA release from rat ventral tegmental area slices. J Neurochem. 2001; 79: 914–922.
  39. Cameron DL, Williams JT. Cocaine inhibits GABA release in the VTA through endogenous 5-HT. J Neurosci Off J Soc Neurosci. 1994; 14: 6763–6767.
  40. O’Dell LE, Parsons LH. Serotonin1B receptors in the ventral tegmental area modulate cocaine-induced increases in nucleus accumbens dopamine levels. J Pharmacol Exp Ther. 2004; 311: 711–719.
  41. Parsons LH, Koob GF, Weiss F. RU 24969 a 5-HT1B/1A receptor agonist, potentiates cocaine-induced increases in nucleus accumbens dopamine. Synap N Y N. 1999; 32: 132–135.
  42. El-Husseini AE, Craven SE, Chetkovich DM. Dual palmitoylation of PSD-95 mediates its vesiculotubular sorting postsynaptic targeting and ion channel clustering. J Cell Biol. 2000; 148: 159–172.
  43. Giustetto M, Kirsch J, Fritschy JM. Localization of the clustering protein gephyrin at GABAergic synapses in the main olfactory bulb of the rat. J Comp Neurol. 1998; 395: 231–244.

Article Info

Article Notes

  • Published on: December 23, 2016


  • Serotonin

  • Connectivity
  • Triads
  • Excitatory
  • Inhibitory
  • Synapse


Dr. Arnauld Belmer PhD
Translational Research Institute, Institute for Health and Biomedical Innovation, Queensland University of Technology, Brisbane, Qld 4059, Australia Telephone:+61 7 34437291
Email: arnauld.belmer@qut.edu.au