Homework 2 - More ANN Fundamentals
Due Thursday, February 20, 2014
NOTE: This assignment, like others in this class, is due at the
beginning of the class period. This means that if you are
even a minute late, you lose 20%. If you are worried about
potentially being late, turn in your homework ahead of time.
Do this by submitting them to me during office hours or by sliding it
under my office door. Do not send assignments to me through email or
leave them in my departmental mail box.
As discussed in class, artificial neurons (ANs) can be combined in many
ways to compute more complex functions than could be computed by a single
AN. The most fundamental way is by combining ANs into a layered,
feedforward neural network (FFNN). Likewise, FFNNs can learn in many ways
but the most fundamental way is supervised learning. Moreover, FFNNs may
be used for many tasks but the two most fundamental are classification and
function approximation, of which classification is the easier to
visualize. These ANN fundamentals — FFNNs, supervised learning, and
classification — are the topics of this homework.
The assignment.
Complete the following exercises:
Part 1 — FFNN Representation
- Consider a two-layer FFNN — that is, one with two layers of
computational elements (ANs) — used for classification in a 2D space
with augmented vectors. The ANs in this FFNN are all SUs and their
activation functions are identical to fAN as given in
1.1. There are four ANs in the hidden layer and one in the output
layer. Given the weights v1,1=−0.3,
v2,1=0.1, v3,1=0.4,
v4,1=−0.6, v1,2=0.2,
v2,2=−0.3, v3,2=0.9,
v4,2=0.5, v1,3=−0.2,
v2,3=−0.9, v3,3=0.2,
v4,3=−0.5, w1=0.7,
w2=1.0, w3=−0.6, and
w4=0.0, draw this FFNN.
- Draw the decision region encoded by this FFNN. Be sure to
indicate the γ1 side of the region.
- Add the following points on the graph you just drew and
label the class of each according to the AN.
- (0.7, 0.7)
- (0, 0)
- (0.1, 0.7)
- (−0.8, −0.3)
- (0.2, −0.1)
- Explain how the decision region for this AN would change if
γ2 were changed to −1, rather than 0 and
explain which points, if any, from those above would be classified
differently and which would be classified the same.
Part 2 — FFNN Learning
- Consider the FFNN given above in 1.1 but with sigmoidal activation
functions for each AN and η = 0.5. Explain how its
weights would be updated, using the backpropagation algorithm we covered
in class, if presented with the data item (1.0, 0.2)
γ1. Show your work. Keep track of four
significant digits.
- Calculate the output value of the FFNN above if, after
learning on (1.0, 0.2) γ1, you were to present this data
item to the FFNN again. Show your work. Keep track of four
significant digits.
- Explain whether the error value for the input (1.0, 0.2)
γ1 increased or decreased due to learning.
What to turn in.
Turn in a neatly handwritten copy of your answers to the exercises for
this assignment. The diagrams should be drawn on engineering or graph
paper. You may also turn in a scanned electronic copy of this assignment
as a backup in case your paper copy is misplaced.