# CS 5970 Programming Assignment 1 — Backprop

## Due Monday, September 21, 2016

NOTE: This assignment is due at the beginning of the class period. This means that if you are even a minute late, you lose 20%. If you are worried about potentially being late, turn in your assignments ahead of time. Do this by submitting them through D2L before the time they are due. Do not send assignments to me through email or leave them in my departmental mail box.

## 1. Motivation

One of the most widely known, best understood, and most widely used types of artificial neural network (ANN) is a feedforward neural network (FFNN) trained using supervised learning via error backpropagation (backprop). Therefore, knowing how to design and train backprop nets for various problems are fundamental skills for ANN practitioners and understanding the functioning of these ANNs is a fundamental skill for ANN researchers. Moreover, backprop nets are quite often used for pattern classification and function approximation. As you have already worked to develop your understanding of using FFNNs for classification problems in Homeworks 1 and 2, you will here focus on function approximation.

## 2. Goals

The goals of this assignment are:

• to give you baseline experience with implementing an ANN,
• to give you experience with backprop learning systems for FFNNs in particular,
• to familiarize you with some of the issues facing ANN implementors,
• to give you experience with the supervised learning process.

## 3. Assignment Overview

You will design, program, and run a simple backprop net to learn the following simple functions:

1. f(x) = 10×sin(x)+x2; −5 ≤ x ≤ +5
2. f(x,y) = 10×sin(x)+y2; −5 ≤ x ≤ +5, −10 ≤ y ≤ +10

You will train, validate, and test your ANN on these functions both with and without noise. The noise for the first function will have a mean of zero and a variance of 5. The noise for the second function will have a mean of zero and a variance of 50. You will also turn in written material regarding the design and implementation choices you made regarding the FFNN and an analysis of the results you will collected from your runs.

## 4. Assignment Details

Carry out the following steps. Underlined steps require a written response, those in `code` require you to write software, and those in italics require you to collect data. Written responses, `code`, and data will be turned in for grading

1. Consider the choices one needs to make regarding the design and implementation of any ANN learning system.
2. List the choices that need to be made when designing an ANN learning system that have already been made for you in this assignment. You should be able to list at least four.
3. For each of these choices, list which option I chose for you in making this assignment.
4. List the choices you need to make regarding the design of your ANN. (Note, these do not include purely implementation choices such as programming language.) You should be able to list at least four.
5. Choose an option that seems reasonable to you for each of these design choices and explain why it seems reasonable to you. If you do not have a good reason for your chosen option, say so. Note that you may want to choose different options for your networks for each of the functions you are to approximate.
6. `Implement your ANN.`
7. Now that you have implemented your ANN, you are likely to have recognized more choices that you needed to make along the way. List the choices you needed to make regarding the design of your ANN. Including the choices you listed previously, your list should now contain at least eight choices.
8. For each of these design choices, list the option you chose, and explain why it seems reasonable to you. If you do not have a good reason for your chosen option, say so.
• How many data samples will you use to train your ANN? To validate it? To test it?
• How many times will you run (train, validate, and test) your ANN in order to be able to give an accurate measure of its performance on these functions? Once? Ten times? Twenty times? 100 times?
• If you run your ANN more than once, what will you change from run to run? What will you keep the same?
• What data will you collect?
• What data will you report? Everything? Best performance? Worst performance? Averages? If averages, what will be averaged together? Average error at a given time step during a single run? At the end of a run? Averages over multiple runs?
• How will you report this data? Text? Numbers? Graphs? What form will these take?
• What conclusions will you be able to draw from your results?
10. Run your ANN on f(x) without noise.
11. Report your results and conclusions regarding the application of your ANN to approximating this function.
12. Run your ANN on f(x) with noise.
13. Report your results and conclusions regarding the application of your ANN to approximating this function.
14. Run your ANN on f(x,y) without noise.
15. Report your results and conclusions regarding the application of your ANN to approximating this function.
16. Run your ANN on f(x,y) with noise.
17. Report your results and conclusions regarding the application of your ANN to approximating this function.
18. Compare the performance of your ANN on these two functions with and without noise and report your conclusions.

## 5. What to Turn In

Write Up
You will turn in an electronic copy of your write up. Your write-up should be a coherent document that covers all of the underlined steps from the assignment above. Note that selected data in a digested form (such as tables or graphs) should be included in your writeup; however, your raw data should not be included here.
Code
You will turn in an electronic copy of your code. You will turn in the source code you have written for this ANN. Your source code should be well structured and well commented. It should conform to good coding standards (e.g., no memory leaks).
Data
You will turn in an electronic copy of your data. This may be in a single file or multiple files. You will also need to include a brief writeup on how the data is organized.

## 6. Notes on this Assignment

You may write your program from scratch or may start from programs for which the source code is freely available (such as on the web or from friends or student organizations). If you do not start from scratch, you must give a complete and accurate accounting of where all of your code came from and indicate which parts are original or changed, and which you got from which other source. Similarly, for the written components of this assignment you may follow the format or content of other written works but you must give a complete and accurate accounting of who deserves credit for all parts of your documents. Failure to give credit where credit is due is academic fraud and will be dealt with accordingly. Please see the University’s web pages on academic integrity.