CS 5043: HW4: Convolutional Neural Networks

Assignment notes:

Data Set

The Core50 data set is a large database of videos of objects as they are being moved/rotated under a variety of different lighting and background conditions. Our general task is to classify the object being shown in a single frame of one of these videos.

Data Organization

Provided Code

We are providing the following code (posted in the git repository):

Prediction Problem

We will focus on the distinction between mugs, scissors, and glasses, for which we only have five distinct example objects (though, for each, we have many different perspectives and conditions). Our goal is to construct a model that will be generally applicable: ideally, it will be able to distinguish between any mug, any pair of scissors, and any glasses. However, given the small number of objects, this is a challenge. For the purposes of this assignment, we will use four objects from each class for training and one distinct object from each class for validation (there won't be an independent test set). For rotation 0:

Architectures

You will create two convolutional neural networks to distinguish the mug, scissors, and glasses: one will be a shallow network and the other will be a deep network. Each will nominally have the following structure:

Since the data set is relatively small, it is important to take steps to address the over-fitting problem. Here are the key tools that you have:

Experiments

Training with Mini-Batches

Loading 3 object classes x 4 object instances x 10 conditions x 30 images makes for a fairly large training set. As we have discussed, when we have such large training sets, especially when there is a lot of autocorrelation between the examples, we can get away with estimating the gradient using a small subset of the training set. To do this, we will use a python generator to produce a mini-batch of training samples for every training epoch.

There is a variety of ways to implement the generator. Here is one example that chooses a random subset of samples for every epoch: 

def training_set_generator_images(ins, outs, batch_size=10,
                          input_name='input', 
                        output_name='output'):
    '''
    Generator for producing random mini-batches of image training samples.
    
    :param ins: Full set of training set inputs (examples x row x col x chan)
    :param outs: Corresponding set of sample (examples x nclasses)
    :param batch_size: Number of samples for each minibatch
    :param input_name: Name of the model layer that is used for the input of the model
    :param output_name: Name of the model layer that is used for the output of the model
    '''
    
    while True:
        # Randomly select a set of example indices
        example_indices = random.choices(range(ins.shape[0]), k=batch_size)
        
        # The generator will produce a pair of return values: one for inputs and one for outputs
        yield({input_name: ins[example_indices,:,:,:]},
             {output_name: outs[example_indices,:]})

Then, model fitting looks like this: 

    # Training generator (only used for training data!)
    generator = training_set_generator_images(ins, outs, batch_size=args.batch)
    
    # Learn
    history = model.fit(x = generator,
                        epochs=args.epochs,
                        steps_per_epoch=2,
                        verbose=args.verbose>=2,
                        validation_data=(ins_validation, outs_validation), 
                        callbacks=[early_stopping_cb])

Notes:

Hints / Notes

What to Hand In

Hand in your notebook containing all of your code + the PDF export of the code. The PDF file must include:

Grades

andrewhfagg -- gmail.com

Last modified: Sat Mar 13 02:50:39 2021