This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Phattanasri, P. Articles by Beer, R. D. Search for Related Content Social Bookmarking                         What's this?

The Dynamics of Associative Learning in Evolved Model Circuits

Department of Electrical Engineering and Computer Science, Case Western Reserve University, Cleveland, OH 44106, phattanardp{at}fabrinet.co.th

Hillel J. Chiel

Departments of Biology, Neurosciences, and Biomedical Engineering Case Western Reserve University Cleveland, OH 44106, hjc{at}case.edu

Randall D. Beer

Cognitive Science Program, Department of Computer Science, Department of Informatics, Indiana University, Bloomington, IN 47406, rdbeer{at}indiana.edu

In this article, we evolve and analyze continuous-time recurrent neural networks capable of associating the smells of different foods with edibility or inedibility in different environments. First, we present an in-depth analysis of this task, highlighting the evolutionary challenges it poses and how these challenges informed our experimental design. Next, we describe the evolution of nonplastic neural circuits that can solve this food edibility learning problem. We then show that the dynamics of the best evolved nonplastic circuits instantiate finite state machines that capture the combinatorial structure of this task. Finally, we demonstrate that successful circuits with Hebbian synaptic plasticity can also be evolved, but that such circuits do not utilize their synaptic plasticity in a traditional way.

Key Words: evolution of learning • evolutionary robotics • dynamical systems theory • synaptic plasticity • neuroscience

References

• Abbott, L.F., & Regehr, W.G. (2004). Synaptic computation. Nature 431: 796—803.[CrossRef][Medline] [Order article via Infotrieve]
• Baxter, D.A., & Byrne, J.H. (1993). Learning rules from neurobiology. In D. Gardner (Ed.), The neurobiology of neural networks (pp. 71—105). Cambridge, MA: MIT Press.
• Beer, R.D. (1997). The dynamics of adaptive behavior: A research program. Robotics and Autonomous Systems 20: 257—289.
• Beer, R.D., & Gallagher, J.C. (1992). Evolving dynamical neural networks for adaptive behavior. Adaptive Behavior 1: 91—122.[Abstract/Free Full Text]
• Bi, G-Q., & Rubin, J. (2005). Timing in synaptic plasticity: From detection to integration. Trends in Neurosciences 28: 222—228.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
• Blynel, J., & Floreano, D. (2003). Exploring the T-maze: Evolving learning-like robot behaviors using CTRNNs. In C. Ryan, T. Soule, M. Keijzer, E. Tsang, R. Poli, & E. Costa (Eds.), Applications of evolutionary computing (Lecture Notes in Computer Science 2611) (pp. 593—604). Berlin: Springer.
• Boal, J.G., Dunham, A.W., Williams, K.T., & Hanlon, R.T. (2000). Experimental evidence for spatial learning in Octopuses (Octopus bimaculoides). J. Comp. Psych. 114: 246—252.[CrossRef]
• Bullock, T.H., & Horridge, G.A. (1965). Structure and function in the nervous systems of invertebrates. W.H. Freeman.
• Casey, M. (1996). The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Neural Computation 8: 1135— 1178.[Web of Science][Medline] [Order article via Infotrieve]
• Chalmers, D.J. (1991). The evolution of learning: An experiment in genetic connectionism. In D. S. Touretzky, J. L. Elman, T. J. Sejnowski & G. E. Hinton (Eds.), Connectionist models: Proceedings of the 1990 Summer School (pp. 81—90). San Mateo: Morgan Kaufmann.
• Chiel, H.J., & Susswein, A.J. (1993). Learning that food is inedible in freely-behaving Aplysia californica, Behavioral Neuroscience 107: 327—338.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
• Cotter, N.E., & Conwell, P.R. (1990). Fixed-weight networks can learn. In Proceedings of the International Joint. Conference on Neural Networks (pp. 11553—11559). IEEE Press.
• Dan, Y., & Poo, M-M. (2004). Spike timing-dependent plasticity of neural circuits. Neuron 44: 23—30.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
• Dan, Y., & Poo, M-M. (2006). Spike timing-dependent plasticity: From synapse to perception. Physiological Review 86: 1033—1048.[Abstract/Free Full Text]
• Destexhe, A., & Marder, E. (2004). Plasticity in single neuron and circuit computations. Nature 431: 789—795.[CrossRef][Medline] [Order article via Infotrieve]
• Feldkamp, L.A., Puskorius, G.V., & Moore, P.C. (1996). Adaptation from fixed weight dynamic networks. In Proceedings of the IEEE International Conference on Neural Networks (pp. 155—160). IEEE Press.
• Floreano, D., & Mondada, F. (1996). Evolution of plastic neurocontrollers for situated agents. In P. Maes, M. Mataric, J. Meyer, J. Pollack, & S. Wilson (Eds.), From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior (pp. 402— 410). Cambridge, MA: MIT Press.
• Floreano, D., & Urzelai, J. (2001). Evolution of plastic control networks. Autonomous Robotics 11: 311—317.[CrossRef]
• Funahashi, K.I., & Nakamura, Y. (1993). Approximation of dynamical systems by continuous time recurrent neural networks. Neural Networks 6: 801—806.[CrossRef][Web of Science]
• Izquierdo-Torres, E., & Harvey, I. (2006). Learning on a continuum in evolved dynamical node networks. In L. Rocha et al. (Eds.), Artificial Life X: Proceedings of the Tenth International Conference on the Simulation and Synthesis of Living Systems (pp. 507—512). Cambridge, MA: MIT Press.
• Kandel, E.R., Kupfermann, I., & Iverson, S. (2000). Learning and memory. In E. R. Kandel, J. H. Schwartz, & T. M. Jessell (Eds.), Principles of neural science, 4th ed. (pp. 1227—1246). McGraw-Hill.
• Kandel, E.R., & Siegelbaum, S.A. (2000). Transmitter release. In E. R. Kandel, J. H. Schwartz, & T. M. Jessell (Eds.), Principles of neural science, 4th ed. (pp. 253—279). McGraw-Hill.
• Kelly, A.M.C., & Garavan, H. (2005). Human functional neuroimaging of brain changes associated with practice. Cerebral Cortex 15: 1089—1102.
• Kimura, M., & Nakano, R. (1998). Learning dynamical systems by recurrent neural networks from orbits. Neural Networks 11: 1589—1599.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
• Llinas, R.R. (1988). The intrinsic electrophysiological properties of mammalian neurons: Insights into central nervous system function. Science. 242: 1654—1664.[Abstract/Free Full Text]
• Lockery, S.R., & Sejnowski, T.J. (1993). Voyages through weight space: Network models of an escape reflex in the leech. In R. D. Beer, R. E. Ritzmann & T. McKenna (Eds.), Biological neural networks in invertebrate neuroethology and robotics (pp. 251—266). San Diego: Academic Press.
• Major, G., & Tank, D. (2004). Persistent neural activity: prevalence and mechanisms. Current Opinion in Neurobiology 14: 675—684.
• Marder, E., Abbott, L.F., Turrigiano, G.G., Liu, Z., & Golowasch, J. (1996). Memory from the dynamics of intrinsic membrane currents. Proc. Nat. Acad. Sci. USA 93: 13481— 13486.[Abstract/Free Full Text]
• Margulies, C., Tully, T., & Dubnau, J. (2005). Deconstructing memory in Drosophila. Current Biology 15: R700—R713.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
• McFarland, D. (Ed.) (1981). The Oxford companion to animal behavior. Oxford University Press.
• McIntosh, A.R. (1999). Mapping cognition to the brain through neural interactions. Memory 7: 523—548.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
• Miller, G.F., & Todd, P.M. (1991). Exploring adaptive agency: I. Theory and methods for simulating the evolution of learning. In D. S. Touretzky, J. L. Elman, T. J. Sejnowski & G. E. Hinton (Eds.), Connectionist models: Proceedings of the 1990 summer school (pp. 65—80). San Mateo: Morgan Kaufmann.
• Münte, T.F., Altenmüller, E., & Jäncke, L. (2002). The musician's brain as a model of neuroplasticity. Nature Reviews Neuroscience 3: 473—478.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
• Phattanasri, P. (2002). Associative learning in evolved dynamical neural networks. Ph.D. Dissertation, Department of Electrical Engineering and Computer Science, Case Western Reserve University.
• Susswein, A.J., Schwarz, M., & Feldman, E. (1986). Learned changes of feeding behavior in Aplysia in response to edible and inedible foods. Journal of Neuroscience 6: 1513— 1527
• Sutton, M.A., & Carew, T.J. (2002). Behavioral, cellular and molecular analysis of memory in Aplysia I: Intermediate-term memory. Integrative and Comparative Biology 42: 725—735.
• Thorndike, E.L. (1898). Animal intelligence: An experimental study of the associative processes in animals. The Psychological Review, Series of Monographs Supplements (Supplement 2) 4: 1—109.
• Todd, P.M., & Miller, G.F. (1991a). Exploring adaptive agency II: Simulating the evolution of associative learning. In J.A. Meyer & S.A. Wilson (Eds.), From Animals to Animats 1: Proceedings of the First International Conference on Simulation of Adaptive Behavior (pp. 306—315). Cambridge, MA: MIT Press.
• Todd, P.M., & Miller, G.F. (1991b). Exploring adaptive agency III: Simulating the evolution of habituation and sensitization. In H. P. Schwefel & R. Manner (Eds.), Proceedings of the First International Conference on Parallel Problem Solving From Nature (pp. 307—313). Berlin: Springer.
• Toledo-Rodriguez, M., El Manira, A., Wallen, P., Svirskis, G., & Hounsgaard, J. (2005). Cellular signaling properties in microcircuits. Trends in Neurosciences 28: 534—540.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
• Trewavas, A. (2003). Aspects of plant intelligence. Annals of Botany 92: 1—20.[Abstract/Free Full Text]
• Tuci, E., Quinn, M., & Harvey, I. (2002). An evolutionary ecological approach to the study of learning behavior using a robot-based model. Adaptive Behavior 10: 201—221.[Abstract/Free Full Text]
• Wood, D.C. (1969). Parametric studies of the response decrement produced by mechanical stimuli in the protozoan Stentor coeruleus. J. Neurobiology 1: 345—360.[CrossRef]
• Yamauchi, B., & Beer, R.D. (1994a). Sequential behavior and learning in evolved dynamical neural networks. Adaptive Behavior 2: 219—246.[Abstract/Free Full Text]
• Yamauchi, B., & Beer, R.D. (1994b). Integrating reactive, sequential and learning behavior using dynamical neural networks. In D. Cliff, P. Husbands, J. Meyer & S. Wilson (Eds.), From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior (pp. 382—391). Cambridge, MA: MIT Press.
• Younger, A.S., Conwell, P.R., & Cotter, N.E. (1999). Fixed-weight on-line learning. IEEE Transactions in Neural Networks 10: 272—283.[CrossRef]
• Zeevi, D. Wu, J-Y., Cohen, L.B., London, J.A., Höpp, H-P., & Falk, C.X. (1989). Hundreds of neurons in the Aplysia abdominal ganglion are active during the gill-withdrawal reflex. J. Neuroscience 9: 3681—3689.[Abstract]
• Zhang, Y., Lu, H., & Bargmann, C.I. (2005). Pathogenic bacteria induce averse olfactory learning in Caenorhabditis elegans. Nature 438: 179—184.[CrossRef][Medline] [Order article via Infotrieve]

Adaptive Behavior, Vol. 15, No. 4, 377-396 (2007)
DOI: 10.1177/1059712307084688


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				E. Izquierdo, I. Harvey, and R. D. Beer Associative Learning on a Continuum in Evolved Dynamical Neural Networks Adaptive Behavior, December 1, 2008; 16(6): 361 - 384. [Abstract] [PDF]

This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Phattanasri, P. Articles by Beer, R. D. Search for Related Content Social Bookmarking                         What's this?