To test your understanding of artificial neural networks (ANNs), evolutionary computation (EC), and neuroevolution, this exam requires you to design a neuroevolutionary system and experimental setup that, if implemented, could be used to study an interesting set of research questions related to topics we have covered in this class. You will need to think carefully about the interrelationships of ANN and EC design, as well as experimental design in general, in designing this neuroevolutionary system.
The goal of this assignment is to test your understanding of course concepts by requiring you to apply them to an authentic neuroevolutionary problem.
Design a neuroevolutionary system and associated experimental setup to use for studying the following. Be sure to specify and justify all aspects of this system and setup. Write up your design in the form (length, tone, content, etc.) of an experimental design section of a research paper.
Note that this design should be sufficiently detailed such that any person who is proficient in the relevant topics (neuroevolution and software development) could implement code to carry out the neuroevolutionary experiments you describe and get results that are highly likely to be in accordance with the results that would be obtained by any other person who is also proficient in the relevant topics who independently implemented code to carry out the experiments you describe. That is, you do not need to explain neuroevolution or software development in your description. However, all of the particulars of your design necessary to carry out the experiments and arrive at the results should be given.
Note also that you will not actually implement this design nor carry out the experiments. You will only be designing and writing up the system and the experiments which someone could carry out. You will not include a results section because you will have no results to report but you should explain the types of data that would be collected, if someone were to carry out these experiments, and how that data would be analyzed.
Prof~H is interested in using reinforcement learning (RL) to control real-world physical systems in environments with continuous state and action spaces. This means that traditional RL approaches such a Q-Learning that are designed for discrete states and actions will not suffice. Instead, he would like to use ANNs to learn appropriate mappings from continuous states to continuous actions. However, because Prof~H doesn't have a good way to estimate a priori the complexity of the control mapping, he would like to evolve ANNs of appropriate complexity.
Because different control problems have different complexities from very simple to very complex, Prof~H would like to use a neuroevolutionary system to evolve an appropriate ANN for each problem. He would like there to be a high degree of flexibility in the possible neural architectures evolved and, if possible, he would like to see multiple, very different architectures evolved for the same task. He also thinks that allowing for customized weight-update mechanisms (e.g., learning rules) might allow for more effective ANNs to evolve, that both topological symmetries and asymmetries might be highly useful for different parts of the system, and that environments can have a drastic effect on evolutionary outcomes.
Because Prof~H wants to deploy these control systems in the real world, he would like them to continually adapt to their environments after they are deployed. However, evolving them on real-world physical systems would be too costly, so he would like to evolve them on computer simulations of the physical systems, then transfer the evolved ANNs to the corresponding real-world systems.
In designing your neuroevolutionary system and associated experiments, here are some questions to consider:
Note that you do not have to consider all or even many possible neuroevolutionary designs and/or experiments appropriate to explore these ideas. A single system and a single experiment is sufficient.
You will turn in to the appropriate drop box in Canvas a machine readable electronic copy of your work that completes the exercise above.