Homework 1  ANN Fundamentals
Due Wednesday, August 31, 2016
NOTE: This assignment, like others in this class, is due at the
beginning of the class period. This means that if you are
even a minute late, you lose 20%. If you are worried about
potentially being late, turn in your homework ahead of time.
Do this by submitting them to me during office hours or by sliding it
under my office door. Do not send assignments to me through email or
leave them in my departmental mail box.
1. Motivation
The foundational computing element for an artificial neural network
(ANN) is the artificial neuron (AN). ANs can learn in many ways but the
most fundamental way is supervised learning. Moreover, ANs may be used for
many tasks but the most basic is classification. These ANN basics —
ANs, supervised learning, and classification — are the topics of this
homework.
2. Goal
The goal of this assignment is to give you experience with basic ANs,
the basic steps involved in updating weights of an AN, and using ANs for
classification problems.
3. Assignment
Complete the following exercises:
Part 1 — AN Representation
 Consider a single AN used for classification in a 2D space with an
augmented vector as discussed in the Engelbrecht text. This AN is a
summation unit (SU) and its activation function f_{AN} is
a step function with outputs γ_{1}=1 for
f_{AN}(net) ≥ 0 and γ_{2}=0 for
f_{AN}(net) < 0. Given the weights
v_{1}=0.4, v_{2}=0.1, and
v_{3}=−0.8, draw this AN.
 Draw (on graph or engineering paper or by using software) the
decision boundary encoded by this AN. Be sure to indicate the
γ_{1} side of the boundary.
 Add the following points on the graph you just drew and
label the class of each according to the AN.
 (−0.1, −0.8)
 (0, 0)
 (0.2, −0.2)
 (1.3, 1.3)
 (−1.2, −0.4)
 Assume that the AN’s classification of each of the points above
is correct. List a new labeled data item which, if added to the
data set above, would necessarily cause the AN to misclassify at least
one data item. (That is, given this new data item, there would be no
set of possible weights that would allow this AN to correctly classify
all data items.) Explain why the AN would necessarily
misclassify at least one data item.
 Explain how the decision boundary for this AN would change if
γ_{2} were changed to −1, rather than 0.
[Note for
explanatory questions, like this one: To explain the answer, describe
what the answer is — in this case, what changes (if any) there are
to the decision boundary — and also why that is the correct
answer — in this case that answer would be based on how the value
of γ affects the decision boundary.]
 Explain how the decision boundary for this AN would change if
f_{AN} were changed to be a sigmoidal function.
 Explain how the classification of each of the points listed
above would change if f_{AN} were a sigmoidal
function.
Part 2 — AN Learning
 Consider the same AN given above in 1.1 together with the following
learning rule (and ignore the points in 1.3 and 1.4 and possible
modifications to the AN listed in 1.5 – 1.7):

v_{i}(t) = v_{i}(t−1) + (t_{p} − o_{p}) z_{i,p}
Explain how the AN weights would be updated given each of the
following labeled data points, presented in the following order:
 (0.6, 1.4) γ_{1}
 (0.8, −1.2) γ_{2}
 (0.5, −0.8) γ_{1}
 (0, 0) γ_{1}
 Draw the graph of the data and the decision boundary after
each weight update for the data given in 2.1.
 Explain which of the points in 2.1 are correctly classified by
this AN after all of the weight updates from one learning pass through
this data and explain what this tells us about the use of this
learning rule.
 Explain how many more passes through the data are needed
before all of the points from 2.1 are correctly classified by this AN.
(Note that you may answer this question on a scale of "one, two, three,
many." That is, if all of the points are not correctly classified after
three more passes through the data, you may stop at three passes and
report that it still had not correctly classified all of the points
after three passes.) Show your work.
 Explain the likely effect of introducing a learning rate
parameter (η) into the equation given.
4. What to Turn In
Turn in a neatly handwritten or typed copy of your answers to the
exercises for this assignment. The diagrams should be neatly drawn on
engineering or graph paper or by using software. You should turn in both a
paper and electronic (perhaps scanned) copy of this assignment.