Homework 1 - ANN Fundamentals
Due Thursday, February 11, 2021
1. Motivation
The foundational computing element for an artificial neural network
(ANN) is the artificial neuron (AN). ANs can learn in many ways but the
most fundamental way is supervised learning. Moreover, ANs may be used
for many tasks but the most basic is classification. These ANN basics
— ANs, supervised learning, and classification — are the
topics of this homework.
2. Goal
The goal of this assignment is to give you experience with basic ANs,
the basic steps involved in updating weights of an AN, and using ANs for
classification problems.
3. Assignment
Complete the following exercises:
Part 1 — AN Representation
- Consider a single AN used for classification in a 2D space with an
augmented vector as discussed in the Engelbrecht text. This AN is a
summation unit (SU) and its activation function fAN
is a step function with outputs γ1=1 for
fAN(net) ≥ 0 and γ2=0 for
fAN(net) < 0. Given the weights
v1=1.0, v2=−0.3, and
v3=0.5, draw this AN.
- Draw (on graph or engineering paper or by using software)
the decision boundary encoded by this AN. Be sure to indicate the
γ1 side of the boundary.
- Add the following points on the graph you just drew and
label the class of each according to the AN.
- (1.0, 0.2)
- (0.0, 0.0)
- (−0.1, −0.5)
- (1.7,0.1)
- (1.8, −1.4)
- Assume that the AN’s classification of each of the points
above is correct. List a new labeled data item which, if added
to the data set above, would necessarily cause the AN to misclassify
at least one data item. (That is, given this new data item, there
would be no set of possible weights that would allow this AN to
correctly classify all data items.) Explain why the AN would
necessarily misclassify at least one data item.
- Explain how the decision boundary for this AN would change
if γ2 were changed to −1, rather than
0.
[Note for explanatory questions, like this one: To explain the
answer, describe what the answer is — in this case, what changes
(if any) there are to the decision boundary — and also
why that is the correct answer — in this case that answer
would be based on how the value of γ affects the decision
boundary.]
- Explain how the decision boundary for this AN would change
if fAN were changed to be a sigmoidal logistic
function.
- Explain how the classification of each of the points listed
above would change if fAN were a sigmoidal logistic
function.
Part 2 — AN Learning
- Consider the same AN given above in 1.1 together with the
following learning rule (and ignore the points in 1.3 and 1.4 and
possible modifications to the AN listed in 1.5 – 1.7):
-
vi(t) =
vi(t−1) + (tp
− op) zi,p
Explain how the AN weights would be updated given each of the
following labeled data points, presented in the following order:
- (−0.1, −1.0) γ2
- (0.2, −0.9) γ2
- (0.6, 0.8) γ1
- (0.0, 0.0) γ1
- Draw the graph of the data and the decision boundary after
each weight update for the data given in 2.1.
- Explain which of the points in 2.1 are correctly classified
by this AN after all of the weight updates from one learning pass
through this data and explain what this tells us about the use
of this learning rule.
- Explain how many more passes through the data are needed
before all of the points from 2.1 are correctly classified by this
AN. (Note that you may answer this question on a scale of "one, two,
many." That is, if all of the points are not correctly classified
after two more passes through the data, you may stop at a total of
three passes and report that it still had not correctly classified all
of the points after three passes.) Show your work.
- Explain the likely effect of introducing a learning rate
parameter (η) into the equation given.
4. What to Turn In
You may write and draw your responses to this assignment neatly by hand
or type your answers and use graphing software to complete the exercises for
this assignment. If drawn, the diagrams should be neatly drawn on
engineering or graph paper. In any case, you should turn in to the
appropriate Canvas dropbox an electronic (perhaps scanned) copy of your
assignment, so that I can grade it online.