CS 327: Artificial Intelligence

Assignment: Neural Networks and Support Vector Machines

Note to self (2/23): Assignment should be more difficult next time. Maybe provide an option to implement backprop?

Assigned on Wednesday, 2/12.
Due on paper in class on Monday, 2/17.

Written portion

1. Genetic algorithms have been proposed as a technique for learning neural networks. Why would one think about using genetic algorithms for this purposes instead of backprop? Doesn't backprop find the best possible solution? If not, why would genetic algorithms potentially do better or faster? Indicate an appropriate chromosome representation, fitness function, and operators for crossover, mutation, and inversion.

2. Textbook exercise 20.12.

Lab portion

This lab exercise will give you some practice with using neural network software in learning to fit training data.

The neural network applet is really easy to use, and highly visual. It is also somewhat unstable. If you have problems with it, shut it down, shut your web browser down, and start again.

Part 1: Boolean Warmup

  1. From the course web page, click on the "Neural Network Applet" link. Click on the "New Window" button that appears.
  2. From the File menu, choose "Load Sample Graph with Wizard," then choose "Boolean Example" from the drop down box. Click "Load".
  3. The Neural Network Construction Wizard should pop up. It might be minimized; keep an eye out for it on the task bar.Click OK.
  4. You should see a neural network with three outputs, each computing a different boolean function based on two inputs. Click on the "Solve" tab, then click the "Randomize Parameters" button.
  5. Click the "Edit Examples" button, and observe the training data. Note that there is no test data for this dataset. Close this window.
  6. Click the "Show Plot" button, and move this window off to the side so you can watch both this window and the network window simultaneously.
  7. Click the Calculate Output button, and enter in a some boolean values (0 and 1) for Input 1 and Input 2. Observe that the output values are incorrect.
  8. Click the "Step" button a few times. Observe the plot.
  9. Click "Step To Target Error" and watch the network and plot fly. When it stops, try Calculate Output again. The results should now be significantly closer to correct.
  10. Print out your network and your plot, and turn this in with your assignment.

Part 2: Bigger Example

  1. Close and restart the neural network applet. Use the wizard as above to choose the "small car database." Click OK at the neural network construction wizard.
  2. Maximize the applet window, and use the mouse to straighten out the network as best as you can.
  3. Click the "Solve" tab, then click "Randomize Parameters."
  4. Click "Edit Examples." Notice that this time there is both training and test data.
  5. Click on "Show Plot," and start training the network.
  6. When training is done, print out your network and your plot to turn in. Examine the training error and the test error. Which is worse, and why?
  7. Start over and try again with different numbers of hidden nodes (in the Construction Wizard). Find a way to get the neural network to overfit the training data. Print out your network and your plot, and indicate where overfitting began occurring.
  8. Change the learning rate to 0.1 (under Options, Learning Options) and see what happens. Print out your plot and explain.
  9. Change the learning rate to 10 and see what happens. Print out your plot and explain.