CS 5043: HW4: Convolutional Neural Networks

Assignment notes:

Data Set

The Core50 data set is a large database of videos of objects as they are being moved/rotated under a variety of different lighting and background conditions. Our general task is to classify the object being shown in a single frame of one of these videos.

Data Organization

Prediction Problem

We will focus on the distinction between mugs and scissors, for which we only have five distinct example objects (though, for each, we have many different perspectives and conditions). Our goal is to construct a model that will be generally applicable: ideally, it will be able to distinguish any mug from any pair of scissors. However, given the small number of objects, this is a challenge. For the purposes of this assignment, we will use four objects of each class for training and one distinct object of each class for validation (there won't be an independent test set). Specifically:

Architectures

You will create two convolutional neural networks to distinguish the mugs and scissors: one be a shallow network and the other will be a deep network. Each will nominally have the following structure:

Since we are at the limits of data, it is important to take steps to address the over-fitting problem. Here are the key tools that you have:

Experiments

Training with Mini-Batches

Loading 8 objects x 10 conditions x 30 images makes for a fairly large training set. As we have discussed, when we have such large training sets, especially when there is a lot of autocorrelation between the examples, we can get away with estimating the gradient using a small subset of the training set. To do this, we will use a python generator to produce a mini-batch of training samples for every training epoch.

There is a variety of ways to implement the generator. Here is one example that chooses a random subset of samples for every epoch: 

def training_set_generator_images(ins, outs, batch_size=10,
                          input_name='input', 
                        output_name='output'):
    '''
    Generator for producing random mini-batches of image training samples.
    
    @param ins Full set of training set inputs (examples x row x col x chan)
    @param outs Corresponding set of sample (examples x nclasses)
    @param batch_size Number of samples for each minibatch
    @param input_name Name of the model layer that is used for the input of the model
    @param output_name Name of the model layer that is used for the output of the model
    '''
    
    while True:
        # Randomly select a set of example indices
        example_indices = random.choices(range(ins.shape[0]), k=batch_size)
        
        # The generator will produce a pair of return values: one for inputs and one for outputs
        yield({input_name: ins[example_indices,:,:,:]},
             {output_name: outs[example_indices,:]})

Then, model fitting looks like this: 

    # Training generator
    generator = training_set_generator_images(ins, outs, batch_size=args.batch)
    
    # Learn
    history = model.fit_generator(generator,
                                  epochs=args.epochs,
                                  steps_per_epoch=2,
                                  use_multiprocessing=True, 
                                  verbose=args.verbose>=2,
                                  validation_data=(ins_validation, outs_validation), 
                                  callbacks=[early_stopping_cb])

Notes:

Hints / Notes

What to Hand In

Hand in your notebook containing all of your code + the PDF export of the code. The PDF file must include:

Grades

andrewhfagg -- gmail.com

Last modified: Mon Mar 9 01:43:25 2020