Homework 2.5 - More ANN Fundamentals

1. Motivation

Artificial neurons (ANs) can be combined in many ways to compute more complex functions than could be computed by a single AN. The most fundamental way is by combining ANs into a layered, feedforward neural network (FFNN). Likewise, FFNNs can learn in many ways but the most fundamental way is supervised learning. Moreover, FFNNs may be used for many tasks but the two most fundamental are classification and function approximation, of which classification is the easier to visualize. These ANN fundamentals — FFNNs, supervised learning, and classification — are the topics of this homework.

2. Goal

The goal of this assignment is to give you experience with supervised learning using backpropagation of error using FFNNs for classification.

3. Assignment

Consider a two-layer FFNN—that is, one with two layers of computational elements (ANs)—used for classification in a 2D space with augmented vectors. The ANs in this FFNN are all SUs with sigmoidal activation functions with λ=1 and η = 0.5. The target values for γ₁ and γ₂ are 0.9 and 0.1, respectively. There are three ANs in the hidden layer and one in the output layer, with the weights v_1,1=−0.1, v_2,1=0.2, v_3,1=−0.9, v_1,2=0.6, v_2,2=0.8, v_3,2=−0.1, v_1,3=−0.2, v_2,3=−0.2, v_3,3=−0.4, w₁=−0.2, w₂=0.6, w₃=0.0, and w₄=0.1. (Note that this is the same network structure with the same weights as found in Homework 2.)

Complete the following exercises:

1. Explain how its weights would be updated, using the backpropagation algorithm we covered in class, if presented with the data item (−1.0, −1.0) γ₁. Show your work. Keep track of four significant digits.
2. Calculate the output value of the FFNN above if, after learning on (−1.0, −1.0) γ₁, you were to present this data item to the FFNN again. Show your work. Keep track of four significant digits.
3. Explain whether the error value for the input (−1.0, −1.0) γ₁ increased or decreased due to learning.

4. What to Turn In

Turn in a neatly handwritten copy of your answers to the exercises for this assignment. You may also turn in a scanned electronic copy of this assignment as a backup in case your paper copy is misplaced.