CS 5073 Programming Assignment 1 — Backprop

1. Motivation

One of the most widely known, best understood, and most widely used types of artificial neural network (ANN) is a feedforward neural network (FFNN) trained using supervised learning via error backpropagation (backprop). Therefore, knowing how to design and train backprop nets for various problems are fundamental skills for ANN practitioners and understanding the functioning of these ANNs is a fundamental skill for ANN researchers. Moreover, backprop nets are quite often used for pattern classification and function approximation. As you have already worked to develop your understanding of using FFNNs for classification problems in Homeworks 1 and 2, you will here focus on function approximation.

2. Goals

The goals of this assignment are:

• to give you baseline experience with implementing an ANN,
• to give you experience with backprop learning systems for FFNNs in particular,
• to familiarize you with some of the issues facing ANN implementors,
• to give you experience with the supervised learning process.

3. Assignment Overview

You will design, program, and run a simple backprop net to learn the following simple functions:

1. f(x) = x³ + 4cos(4x); −3 ≤ x ≤ +3
   




2. f(x,y) = y³ + 6sin(3y) + 12cos(4x) + x³ + (x + 4)²; −1 ≤ x ≤ +1, −1 ≤ y ≤ +1
   

You will train, validate, and test your ANN on these functions both with and without noise. The noise for each function will be Gaussian with a mean of zero and a standard deviation of 5. You will also turn in written material regarding the design and implementation choices you made regarding the FFNN and an analysis of the results you will collected from your runs.

4. Assignment Details

Carry out the following steps. Underlined steps require a written response, those in code require you to write software, and those in italics require you to collect data. Written responses, code, and data will be turned in for grading

1. Consider the choices one needs to make regarding the design and implementation of any ANN learning system.
2. List the choices that need to be made when designing an ANN learning system that have already been made for you in this assignment. You should be able to list at least four.
3. For each of these choices, list which option I chose for you in making this assignment.
4. List the choices you need to make regarding the design of your ANN. (Note, these do not include purely implementation choices such as programming language.) You should be able to list at least four.
5. Choose an option that seems reasonable to you for each of these design choices and explain why it seems reasonable to you. If you do not have a good reason for your chosen option, say so. Note that you may want to choose different options for your networks for each of the functions you are to approximate.
6. Implement your ANN.
7. Now that you have implemented your ANN, you are likely to have recognized more choices that you needed to make along the way. List the choices you needed to make regarding the design of your ANN. Including the choices you listed previously, your list should now contain at least eight choices.
8. For each of these design choices, list the option you chose, and explain why it seems reasonable to you. If you do not have a good reason for your chosen option, say so.
9. Answer the following questions about data collection, reporting, and conclusions so that you are ensured of collecting the appropriate data. Attempt to justify your answers to these questions. As you do so, keep in mind that what you want is a data set that allows you to understand the workings of your ANN as a function approximation tool and you want to report the minimum amount that allows your reader to thoroughly understand what you have learned. Note that you may lack a justification for your answers to some of these questions at this time. That is acceptable since this is your first programming assignment in this course. However, you should keep all of these questions in your mind as the course progresses and be able to give good, justified answers to similar questions for later assignments.
   • How many data samples will you use to train your ANN? To validate it? To test it?
   • How many times will you run (train, validate, and test) your ANN in order to be able to give an accurate measure of its performance on these functions? Once? Ten times? Twenty times? 100 times?
   • If you run your ANN more than once, what will you change from run to run? What will you keep the same?
   • What data will you collect?
   • What data will you report? Everything? Best performance? Worst performance? Averages? If averages, what will be averaged together? Average error at a given time step during a single run? At the end of a run? Averages over multiple runs?
   • How will you report this data? Text? Numbers? Graphs? What form will these take?
   • What conclusions will you be able to draw from your results?
10. Run your ANN on f(x) without noise.
11. Report your results and conclusions regarding the application of your ANN to approximating this function.
12. Run your ANN on f(x) with noise.
13. Report your results and conclusions regarding the application of your ANN to approximating this function.
14. Run your ANN on f(x,y) without noise.
15. Report your results and conclusions regarding the application of your ANN to approximating this function.
16. Run your ANN on f(x,y) with noise.
17. Report your results and conclusions regarding the application of your ANN to approximating this function.
18. Compare the performance of your ANN on these two functions with and without noise and report your conclusions.

5. What to Turn In

Write Up

You will turn in an electronic copy of your write up. Your write-up should be a coherent document that covers all of the underlined steps from the assignment above. Note that selected data in a digested form (such as tables or graphs) should be included in your writeup; however, your raw data should not be included here.

Code

You will turn in an electronic copy of your code. You will turn in the source code you have written for this ANN. Your source code should be well structured and well commented. It should conform to good coding standards (e.g., no memory leaks).

Data

You will turn in an electronic copy of your data. This may be in a single file or multiple files. You will also need to include a brief writeup on how the data is organized.

6. Notes on this Assignment

You may write your program from scratch or may start from programs for which the source code is freely available (such as on the web or from friends or student organizations). If you do not start from scratch, you must give a complete and accurate accounting of where all of your code came from and indicate which parts are original or changed, and which you got from which other source.

As an alternative to writing your own code, either from scratch or building on existing code, you may complete this assignment by using one of the many ANN packages available these days. However, it has to be something freely available and needs to allow the configuration you've used to be exported, so that I can download it and try out the configuration you've submitted. In addition, if you use a third-party package, you must graph the functions learned by your ANN.

For the written components of this assignment you may follow the format or content of other written works but you must give a complete and accurate accounting of who deserves credit for all parts of your documents.

Failure to give credit where credit is due is academic fraud and will be dealt with accordingly. See the University’s web pages on academic integrity.