**NOTE: This assignment is due at the beginning of the class
period. This means that if you are even a minute late, you lose 20%. If
you are worried about potentially being late, turn in your assignments
ahead of time. Do this by submitting them through D2L before the
time they are due. Do not send assignments to me through email or leave
them in my departmental mail box.**

One of the most widely known, best understood, and most widely used types of artificial neural network (ANN) is a feedforward neural network (FFNN) trained using supervised learning via error backpropagation (backprop). Therefore, knowing how to design and train backprop nets for various problems are fundamental skills for ANN practitioners and understanding the functioning of these ANNs is a fundamental skill for ANN researchers. Moreover, backprop nets are quite often used for pattern classification and function approximation. As you have already worked to develop your understanding of using FFNNs for classification problems in Homeworks 1 and 2, you will here focus on function approximation.

The goals of this assignment are:

- to give you baseline experience with implementing an ANN,
- to give you experience with backprop learning systems for FFNNs in particular,
- to familiarize you with some of the issues facing ANN implementors,
- to give you experience with the supervised learning process.

You will design, program, and run a simple backprop net to learn the following simple functions:

*f*(*x*) = 10×sin(*x*)+*x*^{2}; −5 ≤*x*≤ +5

*f*(*x*,*y*) = 10×sin(*x*)+*y*^{2}; −5 ≤*x*≤ +5, −10 ≤*y*≤ +10

You will train, validate, and test your ANN on these functions both with and without noise. The noise for the first function will have a mean of zero and a variance of 5. The noise for the second function will have a mean of zero and a variance of 50. You will also turn in written material regarding the design and implementation choices you made regarding the FFNN and an analysis of the results you will collected from your runs.

Carry out the following steps. __Underlined__ steps require a
written response, those in `code`

require you to write
software, and those in *italics* require you to collect data.
__Written responses__, `code`

, and *data* will be
turned in for grading

- Consider the choices one needs to make regarding the design and implementation of any ANN learning system.
__List the choices that need to be made when designing an ANN learning system that have already been made for you in this assignment.__You should be able to list at least four.__For each of these choices, list which option I chose for you in making this assignment.____List the choices you need to make regarding the design of your ANN.__(Note, these do not include purely implementation choices such as programming language.) You should be able to list at least four.__Choose an option that seems reasonable to you for each of these design choices and explain why it seems reasonable to you.__If you do not have a good reason for your chosen option, say so. Note that you may want to choose different options for your networks for each of the functions you are to approximate.`Implement your ANN.`

- Now that you have implemented your ANN, you are likely to have
recognized more choices that you needed to make along the way.
__List the choices you needed to make regarding the design of your ANN.__Including the choices you listed previously, your list should now contain at least eight choices. __For each of these design choices, list the option you chose, and explain why it seems reasonable to you.__If you do not have a good reason for your chosen option, say so.__Answer the following questions about data collection, reporting, and conclusions so that you are ensured of collecting the appropriate data. Attempt to justify your answers to these questions.__As you do so, keep in mind that what you want is a data set that allows you to understand the workings of your ANN as a function approximation tool and you want to report the minimum amount that allows your reader to thoroughly understand what you have learned. Note that you may lack a justification for your answers to some of these questions at this time. That is acceptable since this is your first programming assignment in this course. However, you should keep all of these questions in your mind as the course progresses and be able to give good, justified answers to similar questions for later assignments.- How many data samples will you use to train your ANN? To validate it? To test it?
- How many times will you run (train, validate, and test) your ANN in order to be able to give an accurate measure of its performance on these functions? Once? Ten times? Twenty times? 100 times?
- If you run your ANN more than once, what will you change from run to run? What will you keep the same?
- What data will you collect?
- What data will you report? Everything? Best performance? Worst performance? Averages? If averages, what will be averaged together? Average error at a given time step during a single run? At the end of a run? Averages over multiple runs?
- How will you report this data? Text? Numbers? Graphs? What form will these take?
- What conclusions will you be able to draw from your results?

*Run your ANN on f(x) without noise.*__Report your results and conclusions regarding the application of your ANN to approximating this function.__*Run your ANN on f(x) with noise.*__Report your results and conclusions regarding the application of your ANN to approximating this function.__*Run your ANN on f(x,y) without noise.*__Report your results and conclusions regarding the application of your ANN to approximating this function.__*Run your ANN on f(x,y) with noise.*__Compare the performance of your ANN on these two functions with and without noise and report your conclusions.__

- Write Up
- You will turn in an electronic copy of your write up. Your write-up
should be a coherent document that covers all of the
__underlined__steps from the assignment above. Note that selected data in a digested form (such as tables or graphs) should be included in your writeup; however, your raw data should not be included here. - Code
- You will turn in an electronic copy of your code. You will turn in the source code you have written for this ANN. Your source code should be well structured and well commented. It should conform to good coding standards (e.g., no memory leaks).
- Data
- You will turn in an electronic copy of your data. This may be in a single file or multiple files. You will also need to include a brief writeup on how the data is organized.

You may write your program from scratch or may start from programs
for which the source code is freely available (such as on the web or
from friends or student organizations). If you do not start from
scratch, you **must** give a complete and accurate accounting of
where all of your code came from and indicate which parts are original
or changed, and which you got from which other source. Similarly, for
the written components of this assignment you may follow the format or
content of other written works but you **must** give a complete and
accurate accounting of who deserves credit for all parts of your
documents. Failure to give credit where credit is due is academic fraud
and will be dealt with accordingly. Please see the University’s web pages on academic
integrity.