Neuroevolution: Harnessing Creativity in AI Agent Design

• Overview
• Authors
• Table of Contents
• Demos
• Exercises
• Course and Tutorials
• For Instructors
• Community
• Cite

Neuroevolution is a population-based search method that makes it possible to train neural networks when training targets are not known, and good performance requires many decisions over time, such as robotic control, game playing, and decision-making. This is the first comprehensive book on this topic: It reviews decades of work on building intelligent agents for such tasks, recent approaches for optimizing deep learning, RL, and GenAI models, experiments to gain insight into biological intelligence, and methods for optimizing neural networks for hardware implementation. In all these areas neuroevolution can be harnessed to discover surprising, effective solutions that often elude human designers. In addition to the book, the website includes animations, interactive demos, tutorials, exercises, and class materials that illustrate neuroevolution at work, and can serve as a starting point for your own explorations in creative agent design. We hope you find these resources useful!



Click to watch Sebastian, David, Yujin, and Risto discuss the book at NeurIPS 2025

Sebastian Risi

Sebastian Risi a Professor at the IT University of Copenhagen, where he directs the Creative AI Lab, and a Research Scientist at Sakana AI. He received his Ph.D. from the University of Central Florida in 2012. His research spans machine learning, artificial life, and human–computer interaction, and has been recognized with numerous international awards, including multiple best paper awards, an ERC Consolidator Grant (2022), the Distinguished Young Investigator in Artificial Life Award (2018), a Google Faculty Research Award (2019), and an Amazon Research Award (2020). His work has been published in top-tier conferences and featured in various media outlets. At Sakana AI, he is exploring how collective intelligence and generative processes can be harnessed to develop more adaptive and creative AI systems.

Yujin Tang

Yujin Tang is a research scientist at Sakana AI. He received an M.S in Engineering from Waseda University, (and a Ph.D. in Computer Science from The University of Tokyo). Before his current job, he worked at Google Brain and later Google DeepMind as a research engineer and a research scientist, during the time of which he published multiple evolutionary computing related papers at GECCO, NeurIPS, Artificial Life, earning best paper and best paper runner-up awards. He also developed and open-sourced EvoJAX – a toolkit for accelerated neuroevolution. His current research focuses on methods of neuroevolution in the applications of improving foundation models.

David Ha

David Ha is the Co-founder and CEO of Sakana AI, an AI Lab in Tokyo focused on advancing foundational AI models using the concepts of computational evolution. He previously worked as a Research Scientist at Google, leading the Google Brain Research team in Japan. His research interests include complex systems, self-organization, and creative applications of machine learning. He has published papers in the intersection of Deep Learning, Artificial Life, Neuroevolution and Complex Systems at NeurIPS, ICLR, ICML, GECCO, Artificial Life, Collective Intelligence. Prior to joining Google, he was a derivatives trader, serving as Managing Director and head of interest rates trading at Goldman Sachs in Japan. He obtained his undergraduate in engineering science from the University of Toronto, and a PhD from the University of Tokyo.

Risto Miikkulainen

Risto Miikkulainen is a Professor of Computer Science at the University of Texas at Austin and a VP of AI Research at the Cognizant AI Lab. He received an M.S. in Engineering from Helsinki University of Technology (now Aalto University) in 1986, and a Ph.D. in Computer Science from UCLA in 1990. His current research focuses on methods and applications of neuroevolution, as well as neural network models of natural language processing and vision; he is an author of over 500 articles in these research areas. At the Cognizant AI Lab he is scaling up these approaches to real-world problems. Risto is an AAAI, IEEE, and INNS Fellow; his work on neuroevolution has recently been recognized with the IEEE CIS Evolutionary Computation Pioneer Award, the Gabor Award of the International Neural Network Society, and Outstanding Paper of the Decade Award of the International Society for Artificial Life.

Foreword Online Supplement Preface 1 Introduction

• 1.1 Evolving Neural Networks
• 1.2 Extending Creative AI
• 1.3 Improving the World
• 1.4 Plan for the Book
• 1.5 Plan for Hands-on Exercises
• 1.6 Chapter Review Questions
2 The Basics
• 2.1 Evolutionary Algorithms
* 2.1.1 Representation * 2.1.2 Population-Based Search * 2.1.3 Selection * 2.1.4 Variation Operators * 2.1.5 Fitness Evaluation * 2.1.6 Reproduction and Replacement * 2.1.7 Termination
• 2.2 Types of Evolutionary Algorithms
* 2.2.1 Genetic Algorithm * 2.2.2 Evolution Strategy * 2.2.3 Covariance-Matrix Adaptation Evolution Strategy * 2.2.4 OpenAI Evolution Strategy * 2.2.5 Multiobjective Evolutionary Algorithms * 2.2.6 Further Evolutionary Computation Techniques * 2.2.7 Try These Algorithms Yourself
• 2.3 Neural Networks
* 2.3.1 Feedforward Neural Networks * 2.3.2 Training Feedforward Neural Networks with Gradient Descent * 2.3.3 Recurrent Neural Networks * 2.3.4 Long Short-Term Memory * 2.3.5 Convolutional Neural Networks * 2.3.6 Transformers
• 2.4 Neuroevolution: An Integrated Approach
• 2.5 Chapter Review Questions
3 The Fundamentals of Neuroevolution
• 3.1 Neuroevolution Taxonomy
* 3.1.1 Fixed-Topology Neuroevolution * 3.1.2 Topology and Weight Evolving Artificial Neural Networks * 3.1.3 Direct Encoding * 3.1.4 Indirect Encoding
• 3.2 Case study: Evolving a Simple Walking Agent
* 3.2.1 The Challenge * 3.2.2 Fitness Function * 3.2.3 Neural Network Architecture * 3.2.4 Evolutionary Algorithm * 3.2.5 Training for Generality
• 3.3 Neuroevolution of Augmenting Topologies
* 3.3.1 Motivation and Challenges * 3.3.2 Genetic Encoding and Historical Markings * 3.3.3 Speciation and Fitness Sharing * 3.3.4 Example: Double Pole Balancing
• 3.4 Scaling up Neuroevolution
* 3.4.1 Neuroevolution vs. Deep Learning * 3.4.2 Deep Neuroevolution * 3.4.3 Taking Advantage of Big Compute
• 3.5 Chapter Review Questions
4 Indirect Encodings
• 4.1 Why Indirect Encodings?
• 4.2 Developmental Processes
* 4.2.1 Cell-Chemistry Approaches * 4.2.2 Grammatical Encodings * 4.2.3 Learning Approaches
• 4.3 Indirect Encoding through Hypernetworks
* 4.3.1 Compositional Pattern Producing Networks * 4.3.2 Case Study: Evolving Virtual Creatures with CPPN-NEAT * 4.3.3 HyperNEAT * 4.3.4 Multiagent HyperNEAT * 4.3.5 Evolvable Substrate HyperNEAT * 4.3.6 General Hypernetworks and Dynamic Indirect Encodings
• 4.4 Self-attention as Dynamic Indirect Encoding
* 4.4.1 Background on Self-Attention * 4.4.2 Self-Attention as a Form of Indirect Encoding * 4.4.3 Self-Attention Based Agents
• 4.5 Chapter Review Questions
5 Utilizing Diversity
• 5.1 Genetic Diversity
• 5.2 Behavioral Diversity
• 5.3 Novelty Search
• 5.4 Quality Diversity Methods
* 5.4.1 Motivation and Challenges * 5.4.2 Novelty Search with Local Competition * 5.4.3 MAP-Elites * 5.4.4 Implementing and Enhancing QD Algorithms
• 5.5 Multiobjectivity
• 5.6 Ensembling
• 5.7 Utilizing Population Culture and History
• 5.8 Chapter Review Questions
6 Neuroevolution of Behavior
• 6.1 From Control to Strategy
• 6.2 Discovering Robust Control
* 6.2.1 Noise, Exploration, and Novelty * 6.2.2 Symmetry, Context, and Adaptation * 6.2.3 Transfer to Physical Robots
• 6.3 Discovering Flexible Strategies
* 6.3.1 Switching between Behaviors * 6.3.2 Evolving Cognitive Behaviors * 6.3.3 Utilizing Stochasticity, Coevolution, and Scale
• 6.4 Decision-Making
* 6.4.1 Successes and Challenges * 6.4.2 Surrogate Modeling * 6.4.3 Case Study: Mitigating Climate Change through Optimized Land Use * 6.4.4 Case Study: Optimizing NPIs for COVID-19 * 6.4.5 Leveraging Human Expertise
• 6.5 Chapter Review Questions
7 Neuroevolution of Collective Systems
• 7.1 Cooperative Coevolution
* 7.1.1 Evolving a Single Neural Network * 7.1.2 Evolving Structured Heterogeneous Networks * 7.1.3 Evolving a Team
• 7.2 Competitive Coevolution
* 7.2.1 Evolving Single Neural Networks * 7.2.2 Evolving Multiple Teams
• 7.3 Cellular Automata
* 7.3.1 Evolving Neural Cellular Automata * 7.3.2 Growing Functional Machines * 7.3.3 Case Study: Growing Game Levels with QD-Evolved NCAs * 7.3.4 Evolving Self-Assembling Neural Networks * 7.3.5 Combining Evolutionary Creativity with Gradient Descent Precision
• 7.4 Chapter Review Questions
8 Interactive Neuroevolution
• 8.1 The NERO Machine Learning Game
• 8.2 Incorporating Human Knowledge into NERO
• 8.3 Neuroevolution-enabled Collaboration
• 8.4 Case Study: Collaborative Interactive Neuroevolution Through Play
• 8.5 Making Human Contributions Practical
• 8.6 Chapter Review Questions
9 Open-ended Neuroevolution
• 9.1 Open-ended Discovery of Complex Behavior
* 9.1.1 Neutral Mutations with Weak Selection * 9.1.2 Extinction Events * 9.1.3 Evolvable Representations * 9.1.4 Expressive Encodings * 9.1.5 Major Transitions * 9.1.6 Open-ended Evolution of Intelligence
• 9.2 Cooperative Coevolution of Environments and Solutions
* 9.2.1 The Influence of Environments * 9.2.2 Body and Brain Coevolution * 9.2.3 Coevolution Driven by Interestingness
• 9.3 Competitive Coevolution of Environments and Solutions
* 9.3.1 Paired Open-Ended Trailblazer * 9.3.2 Learning to Chase-and-Escape
• 9.4 Chapter Review Questions
10 Evolutionary Neural Architecture Search
• 10.1 Neural Architecture Search with NEAT
• 10.2 NAS for Deep Learning
• 10.3 Case Studies: Improving Deep Learning SOTA
* 10.3.1 LSTM Designs * 10.3.2 CoDeepNEAT * 10.3.3 AmoebaNet
• 10.4 Multiobjective and Multitask NAS
• 10.5 Making NAS Practical
• 10.6 Beyond Neural Architecture Search
• 10.7 Chapter Review Questions
11 Optimization of Neural Network Designs
• 11.1 Designing Complex Systems
• 11.2 Bilevel Neuroevolution
• 11.3 Evolutionary Meta-learning
* 11.3.1 Loss functions * 11.3.2 Activation Functions * 11.3.3 Data Use and Augmentation * 11.3.4 Learning Methods * 11.3.5 Utilizing Surrogates * 11.3.6 Synergies
• 11.4 Case Study: Meta-learning vs. Human Design
• 11.5 Neuroevolution of Neuromorphic Systems
* 11.5.1 Neuromorphic Computation * 11.5.2 Evolutionary Optimization * 11.5.3 Examples * 11.5.4 Future Directions
• 11.6 Chapter Review Questions
12 Synergies with Reinforcement Learning
• 12.1 Reinforcement learning vs. Neuroevolution
• 12.2 Synergistic Combinations
* 12.2.1 Integrating Population-Based and Reinforcement-Based Search * 12.2.2 Evolving Value Networks for RL * 12.2.3 Evolving Starting Points for RL
• 12.3 Evolving Neural Networks to Reinforcement Learn
* 12.3.1 Evolving Hebbian Learning Rules * 12.3.2 Case Study: Hebbian Learning for Physical Robot Transfer * 12.3.3 Learning When to Learn through Neuromodulation * 12.3.4 Indirectly Encoded Plasticity * 12.3.5 Learning to Continually Learn through Networks with External Memory
• 12.4 Integrating Evolution, Learning, and Embodiment
• 12.5 Chapter Review Questions
13 Synergies with Generative AI
• 13.1 Background on Large Language Models
• 13.2 Evolutionary Computing Enhances LLMs
* 13.2.1 Evolutionary Prompt Engineering/Adaptation * 13.2.2 Evolutionary Model Merging * 13.2.3 Fine-Tuning with Evolution Strategy
• 13.3 LLMs Enhance Evolutionary Computing
* 13.3.1 Evolution through Large Models * 13.3.2 Language Model Crossover * 13.3.3 LLMs as Evolution Strategies * 13.3.4 AlphaEvolve
• 13.4 Case Studies: NE-enhanced Generative AI for Game Level Generation
* 13.4.1 MarioGAN * 13.4.2 MarioGPT
• 13.5 World Models
* 13.5.1 A Simple World Model for Agents * 13.5.2 Using the World Model for Feature Extraction * 13.5.3 Training an Agent Inside Its Own World Model
• 13.6 Chapter Review Questions
14 What Neuroevolution Can Tell Us About Biological Evolution?
• 14.1 Understanding Neural Structure
• 14.2 Evolutionary Origins of Modularity
• 14.3 Understanding Neuromodulation
• 14.4 Developmental Processes
* 14.4.1 Synergistic Development * 14.4.2 Development through Genetically Directed Learning
• 14.5 Constrained Evolution of Behavior
• 14.6 Case Study: Understanding Human-like Behavior
• 14.7 Case Study: Understanding an Evolutionary Breakthrough
• 14.8 Evolution of Language
* 14.8.1 Biology of Language * 14.8.2 Evolving Communication * 14.8.3 Evolution of Structured Language
• 14.9 Chapter Review Questions
15. Epilogue References Subject Index Author Index

Below are links to animations that illustrate the concepts in the book, as well as several interactive demos (labeled as such) that allow exploring those concepts through a web interface.

Chapter 2: The Basics

• Simple GA: Shaffer, Rastrigin
• Simple ES: Shaffer, Rastrigin
• CMA-ES: Shaffer, Rastrigin, Shaffer2, Rastrigin2
• NES: Shaffer, Rastrigin
• OpenAIES: Shaffer, Rastrigin
• Bipedal walker: stuck, successful

Chapter 3: The Fundamentals of Neuroevolution

• Bipedal walker learning: Walk, Getting unstuck
• Bipedal walker strategies: Reaching, Jumping
• Bipedal walker on other terrains: Terrain1, Terrain2
• Walking and Atari playing with Simple GAs
• NEAT-evolved Mario controller

Chaper 4: Indirect Encodings

• Evolving soft robots with CPPNs
• Evolving gaits for legged robots with HyperNEAT
• Multiagent HyperNEAT Predator-Prey Scaling
• Multi-brain HyperNEAT for team patrol
• AttentionAgent in CarRacing and VizDoom

Chapter 5: Searching for / Utilizing Diversity

• Stepping Stones in Novelty Search
• Novelty Search in Bipedal Walking
• Diversity of creatures from a single run of novelty search with local competition
• Egalitarian Social Learning

Chapter 6: Neuroevolution of Behavior

• Controlling a Finless Rocket
• Race-Car Driving
• Personal Satellite Assistant
• Evolving Modular Controllers for Multilegged Robots
• Evolving Controller Symmetry for Multilegged Robots
• Evolving Controllers for Physical Multilegged Robots
• Modeling the Environment through Context+Skill Networks
• Keepaway Soccer
• Learning in Fractured Domains
• Multimodal behavior in Ms. Pac-Man: Multilife; Single-life; Interleaved; Isolated
• Automatic Regularization with a Surrogate Model
• Automated Curricular Evolution with a Surrogate Model
• Evolution of the Pareto Front
• Interactive Demo: Land-use Optimization
• Interactive Demo: Non-Pharmaceutical Interventions in COVID-19

Chapter 7: Neuroevolution of Collective Systems

• Cooperation Based on Stigmergy vs. Communication
• Heterogeneous neural architecture training through deep innovation protection
• Adaptive Teams of Agents (Legion-II Game)
• Competitive Coevolution (Robotic Duel)
• Evolutionary Arms Race (Hyenas and Zebras)
• Severe Damage Recovery in Evolving Soft Robots through Gradient Descent
• Growing Game Levels with Neural Cellular Automata
• Hyper Neural Cellular Automata
• Neural Developmental Programs
• Growing Minecraft Machines with NCAs

Chapter 8: Interactive Neuroevolution

• NERO 1.0 Behaviors
• NERO 2.0 Videos
• Picbreeder
• Galactic Arms Race: intro video, game
• Petalz gameplay video

Chapter 9: Open-Ended Neuroevolution

• Body-Brain Coevolution
• POET
• Enhanced POET
• OMNI-EPIC
• Chaser-escapee coevolution

Chapter 10: Evolutionary Neural Architecture Search

• Backprop NEAT
• Discovering Complex LSTM Designs
• Interactive demo: LSTM Music Maker
• CoDeepNEAT in Character Recognition
• CoDeepNEAT Optimizing Accuracy and Size
• Discovering Multitask Learning Topologies in Character Recognition
• Interactive demo: Character Recognition
• Interactive demo: Celebrity Match through Multitask Evolution
• Interactive demo: Weight-Agnostic Neural Networks

Chapter 11: Evolutionary Meta-Learning

• Bilevel Optimization of Helicopter Hovering
• Neuromorphic Autonomous Racecar

Chapter 12: Synergies with Reinforcement Learning

• Walking robot with ES-MAML
• Meta-learning with Hebbian networks: Car Racing, Quadruped Locomotion
• Hebbian Learning for Physical Robot Transfer
• DERL: Embodied Intelligence via Learning and Evolution

Chapter 13: Synergies with Generative AI

• Evolutionary Model Merging
• Sodaracer: Seeds, Seeded results, Variations, Blob, Hopper, Centipede,
• Sodaracer: Lineage, Bump challenge, Solution1, Solution2, Tunnel challenge
• AlphaEvolve
• MarioGAN
• Agent Playing Level Generated by MarioGPT
• Evolution Strategy Fine Tuning an LLM
• Interactive demo: World Models

Chapter 14: What Neuroevolution Can Tell Us About Biological Evolution

• Evolutionary origins of modularity
• Evolution of Pattern Generators
• Evolved Virtual Creatures; Multitask Extension; Muscle-drive Extension
• Botprize Demo Video; Other Botprize Videos
• Complex Coordinated Behavior of Hyenas vs. Lions

Most of the exercises below use Google Colab, a free cloud-based service that allows you to write and run your Python code in a Jupyter notebook. This platform includes many python libraries such as Ternsorflow, Matplotlib, Pandas, etc. Colab also provides access to limited GPU resources so you can run your experiments right there. You can access Colab through your Google drive. Create a copy of the exercise notebook in your Google Colab account. It has a code for the general setup; the exercise consists of filling in the missing parts of the code yourself, running it, and verifying that it works. As an example of working code, here’s the Bipedal walker example used in Chapters 2 & 3.

Assignments

Chapter 2 –

• Evolution strategies (ES) and genetic algorithms (GA) on black-box optimization using Numpy Notebook
Chapter 3 –
• Part 1: ES/GA evolving a neural network for the MNIST task Notebook
• Part 2: ES/GA evolving a neural network for the CartPole task Notebook
• Part 3: Implementing NEAT for data classification task Notebook
Chapter 4 –
• Part 1: Evolving a CPPN for creative pattern generation Notebook
• Part 2: Implementing MAP-Elites for MNIST Notebook
Chapter 6 –
• COVID-19 Policy Prescriptions Assignment
Chapter 7 –
• Evolving Neural Cellular Automata (NCA) to generate a pattern Notebook
Chapter 8 –
• Part 1: Neuroevolution of a simple SlimeVolley player against a random opponent Assignment
• Part 2: Neuroevolution of a better SlimeVolley player with evolutionary arms race Assignment Notebook (for both Part1 and Part2)
• This assignment can be run as a class-wide tournament
Chapter 12 –
• Evolving Hebbian learning Notebook
Tutorial Exercises –
• This is the exercise used in some of the neuroevolution conference tutorials. It includes: pole-balancing, model merging, and MAP-Elites. Notebook

Solutions

The solutions are available upon request at the For Instructors Only page.

Course

An advanced undergraduate (or graduate) course based on the book was taught by Risto in Fall 2024. All the public materials for it (syllabus, reading, slides-4up, lecture recordings, exercises) are available at the course website. The materials intended for instructors only (Overleaf sources for the slides and the solutions for the exercises) are available upon request at the “For Instructors Only” page.

---

Tutorials

Past:

• GECCO 2024 (pre-book: slides, video; 2hrs)
• Artificial Life conference 2024 (updated for Alife 2025)
• IJCNN 2025 (updated for AAAI 2026)
• GECCO 2025 (updated for AAAI 2026)
• IJCAI 2025 (updated for AAAI 2026)
• Artificial Life conference 2025 (slides)
• ICDABI 2025 (slides, video (45mins))
• AAAI 2026 (slides, conference video (1:50hrs), website)
• GECCO 2026 (slides, prerecorded video (1:30hrs), website)
Coming up:
• IJCAI 2026
• NeurIPS2026

Public materials for a sample course on Neuroevolution (syllabus, reading, slides-4up, lecture recordings, exercises) are available at the course website. The materials intended for instructors only (slide sources and exercise solutions), are available upon request at the For Instructors Only page.

Search for:

Neuroevolution Book Discussion Have thoughts on the book? Have questions for the authors? You have come to the right place. Click here to return to the main site at any time * 1,338 * 1,338 * 5 months, 4 weeks ago richard

Found errors in the book, issues with the website, or have other suggestions? Please send them to authors@neuroevolutionbook.com. Or better yet: The book and the resources on this site can hopefully serve as a starting point for your own explorations on neuroevolution. To support such efforts, the Neuroevolution Community GitHub lists software, benchmarks, research groups, follow-up papers, and other community contributions. We invite you to engage with it!

Risi, S., Tang, Y., Ha, D., and Miikkulainen, R. (2025). Neuroevolution: Harnessing Creativity in AI Agent Design. Cambridge, MA: MIT Press. https://neuroevolutionbook.com .

BibTeX

@book{risi-et-al:book2025, title = {Neuroevolution: Harnessing Creativity in AI Agent Design}, author = {Sebastian Risi and Yujin Tang and David Ha and Risto Miikkulainen}, publisher = {MIT Press}, address = {Cambridge, MA}, url = {https://neuroevolutionbook.com}, year = {2025} }