Homework 2 - More ANN Fundamentals

1. Motivation

Artificial neurons (ANs) can be combined in many ways to compute more complex functions than could be computed by a single AN. The most fundamental way is by combining ANs into a layered, feedforward neural network (FFNN). Moreover, FFNNs may be used for many tasks but the two most fundamental are classification and function approximation, of which classification is the easier to visualize. These ANN fundamentals — FFNNs and classification — are the topics of this homework.

2. Goal

The goal of this assignment is to give you experience with basic FFNNs and classification.

3. Assignment

1. Consider a two-layer FFNN — that is, one with two layers of computational elements (ANs) — used for classification in a 2D space with augmented vectors. The ANs in this FFNN are all SUs and their activation functions are identical to f_AN as given in Homework 1, Subpart 1.1. There are three ANs in the hidden layer and one in the output layer. Given the weights v_1,1=−0.1, v_2,1=0.2, v_3,1=−0.9, v_1,2=0.6, v_2,2=0.8, v_3,2=−0.1, v_1,3=−0.2, v_2,3=−0.2, v_3,3=−0.4, w₁=−0.2, w₂=0.6, w₃=0.0, and w₄=0.1, draw this FFNN.
2. Draw the decision region encoded by this FFNN. Be sure to indicate the γ₁ side of the region. Be sure to indicate which portion of the decision region is due to each hidden layer AN. (Hint: If you're having difficulty figuring out exactly how the decision region works out given the weights above, try plotting the points from Exercise 3 immediately below and calculate their classes by plugging their point values into the equations defined by the weights above in Exercise 1.)
3. Add the following points on the graph you just drew and label the class of each according to the AN.
   1. (0.9, 0.9)
   2. (−0.6, −0.5)
   3. (−0.6, 0.2)
   4. (0.8, −0.8)
   5. (−0.4, −0.3)
   6. (0.0, 0.0)
4. Explain the significance of the value of w₄.
5. Explain the significance of the value of w₃.
6. Explain the significance of the relative values of w₁ and w₂.
7. Explain how the decision region for this FFNN would change if the value of w₃ were changed to −0.5 rather than 0.0 and explain which points, if any, from those above would be classified differently and which would be classified the same. Be sure to discuss the relative values of w₁, w₂, and w₃.
8. Explain how the decision region for this FFNN would change if γ₂ were changed to −1 rather than 0 and explain which points, if any, from those above would be classified differently and which would be classified the same. (For this hypothetical, use a value of −0.5 for w₃.)
9. Explain how the decision region for this FFNN would change if f_AN for the hidden layer ANs were changed to be a linear activation function, in particular, if f_AN(net)=net for the hidden layer ANs. (For the output AN, f_AN remains unchanged in this hypothetical.) Explain which points, if any, from those above would be classified differently and which would be classified the same. (For this hypothetical, use a value of −0.5 for w₃.)

4. What to Turn In

Turn in a neatly handwritten copy of your answers to the exercises for this assignment. The diagrams should be drawn on engineering or graph paper. You may also turn in a scanned electronic copy of this assignment as a backup in case your paper copy is misplaced.