CS 327: Artificial Intelligence
Assignment: Neural Networks and Support Vector Machines
Note to self (2/23): Assignment should be more difficult next time.
Maybe provide an option to implement backprop?
Assigned on Wednesday, 2/12.
Due on paper in class on Monday, 2/17.
Written portion
1. Genetic algorithms have been proposed as a technique for learning
neural networks. Why would one think about using genetic algorithms for
this purposes instead of backprop? Doesn't backprop find the best
possible solution? If not, why would genetic algorithms potentially do
better or faster? Indicate an appropriate chromosome representation,
fitness function, and operators for crossover, mutation, and inversion.
2. Textbook exercise 20.12.
Lab portion
This lab exercise will give you some practice with using neural network
software in learning to fit training data.
The neural network applet is really easy to use, and highly visual. It
is also somewhat unstable. If you have problems with it, shut it down,
shut your web browser down, and start again.
Part 1: Boolean Warmup
- From the course web page, click on the "Neural Network Applet"
link. Click on the "New Window" button that appears.
- From the File menu, choose "Load Sample Graph with Wizard," then
choose "Boolean Example" from the drop down box. Click "Load".
- The Neural Network Construction Wizard should pop up. It might be
minimized; keep an eye out for it on the task bar.Click OK.
- You should see a neural network with three outputs, each
computing a different boolean function based on two inputs. Click on
the "Solve" tab, then click the "Randomize Parameters" button.
- Click the "Edit Examples" button, and observe the training data.
Note that there is no test data for this dataset. Close this window.
- Click the "Show Plot" button, and move this window off to the
side so you can watch both this window and the network window
simultaneously.
- Click the Calculate Output button, and enter in a some boolean
values (0 and 1) for Input 1 and Input 2. Observe that the output
values are incorrect.
- Click the "Step" button a few times. Observe the plot.
- Click "Step To Target Error" and watch the network and plot fly.
When it stops, try Calculate Output again. The results should now be
significantly closer to correct.
- Print out your network and your plot,
and turn this in with your assignment.
Part 2: Bigger Example
- Close and restart the neural network applet. Use the wizard as
above to choose the "small car database." Click OK at the neural
network construction wizard.
- Maximize the applet window, and use the mouse to straighten out
the network as best as you can.
- Click the "Solve" tab, then click "Randomize Parameters."
- Click "Edit Examples." Notice that this time there is both
training and test data.
- Click on "Show Plot," and start training the network.
- When training is done, print out your
network and your plot to turn in. Examine the training error and the
test error. Which is worse, and why?
- Start over and try again with different numbers of hidden nodes
(in the Construction Wizard). Find a way to get the neural network to
overfit the training data. Print out
your network and your plot, and indicate where overfitting began
occurring.
- Change the learning rate to 0.1 (under Options, Learning Options)
and see what happens. Print out your
plot and explain.
- Change the learning rate to 10 and see what happens. Print out your plot and explain.