**NOTE: This assignment, like others in this class, is due at the
beginning of the class period. This means that if you are
even a minute late, you lose 20%. If you are worried about
potentially being late, turn in your homework ahead of time.
Do this by submitting them to me during office hours or by sliding it
under my office door. Do not send assignments to me through email or
leave them in my departmental mail box.**

Artificial neurons (ANs) can be combined in many ways to compute more complex functions than could be computed by a single AN. The most fundamental way is by combining ANs into a layered, feedforward neural network (FFNN). Likewise, FFNNs can learn in many ways but the most fundamental way is supervised learning. Moreover, FFNNs may be used for many tasks but the two most fundamental are classification and function approximation, of which classification is the easier to visualize. These ANN fundamentals — FFNNs, supervised learning, and classification — are the topics of this homework.

The goal of this assignment is to give you experience with basic FFNNs, supervised learning using backpropagation of error, and classification.

- Consider a two-layer FFNN — that is, one with two layers of
computational elements (ANs) — used for classification in a 2D
space with augmented vectors. The ANs in this FFNN are all SUs and their
activation functions are identical to
*f*as given in Homework 1, 1.1. There are three ANs in the hidden layer and one in the output layer. Given the weights_{AN}*v*_{1,1}=−1.0,*v*_{2,1}=1.0,*v*_{3,1}=1.0,*v*_{1,2}=0.6,*v*_{2,2}=−0.9,*v*_{3,2}=0.6,*v*_{1,3}=−0.2,*v*_{2,3}=0.6,*v*_{3,3}=−0.3,*w*_{1}=−1.0,*w*_{2}=0.0,*w*_{3}=−0.8, and*w*_{4}=0.0,**draw**this FFNN. **Draw**the decision region encoded by this FFNN. Be sure to indicate the γ_{1}side of the region. Be sure to indicate which portion of the decision region is due to each hidden layer AN.**Add**the following points on the graph you just drew and**label**the class of each according to the AN.- (−2, −3)
- (3, −3)
- (0, 0)
- (0, −2)
- (−4, −3)
- (1, 3)
- (−3, 4)

**Explain**the significance of the value of*w*_{4}.**Explain**the significance of the value of*w*_{2}.**Explain**the significance of the relative values of*w*_{1}and*w*_{3}.**Explain**how the decision region for this FFNN would change if the value of*w*_{2}were changed to 0.3 rather than 0.0 and**explain**which points, if any, from those above would be classified differently and which would be classified the same. Be sure to discuss the relative values of*w*_{1},*w*_{2}, and*w*_{3}**Explain**how the decision region for this FFNN would change if γ_{2}were changed to −1 rather than 0 and**explain**which points, if any, from those above would be classified differently and which would be classified the same. (For this hypothetical, use a value of 0.3 for*w*_{2}.)**Explain**how the decision region for this FFNN would change if*f*for the hidden layer ANs were changed to be a linear activation function, in particular, if_{AN}*f*(_{AN}*net*)=*net*for the hidden layer ANs. (For the output AN,*f*remains unchanged in this hypothetical.)_{AN}**Explain**which points, if any, from those above would be classified differently and which would be classified the same. (For this hypothetical, use a value of 0.0 for*w*_{2}and a value of 0 for γ_{2}.)

- Consider the FFNN given above in 1.1 but with sigmoidal activation
functions with λ=1 for each AN and
*η*= 0.5. Here the target value for γ_{1}is 0.9 and for γ_{2}is 0.1.**Explain**how its weights would be updated, using the backpropagation algorithm we covered in class, if presented with the data item (−1.0, −1.0) γ_{1}.**Show your work.**Keep track of four significant digits. **Calculate**the output value of the FFNN above if, after learning on (−1.0, −1.0) γ_{1}, you were to present this data item to the FFNN again.**Show your work.**Keep track of four significant digits.**Explain**whether the error value for the input (−1.0, −1.0) γ_{1}increased or decreased due to learning.

Turn in a neatly handwritten copy of your answers to the exercises for this assignment. The diagrams should be drawn on engineering or graph paper. You may also turn in a scanned electronic copy of this assignment as a backup in case your paper copy is misplaced.