Neural Networks

This is your chance to play with some neural network software to see what you can learn. All boldfaced content indicates items that you need to turn in. You should turn in all questions in some sort of word processing document, and submit electronically.

Part 1: Boolean Warmup

  1. Visit http://aispace.org/neural. Click where it says "Click here" to start the tool.
  2. From the File menu, choose "Load Sample Graph and Data," then choose "Boolean Example" from the drop down box. Click "Load".
  3. You should see a neural network with three outputs, each computing a different boolean function based on two inputs. Click on the "Solve" tab, then click the "Initialize Parameters" button.
  4. Click the "View/Edit Examples" button, and observe the training data. Note that there is no test data for this dataset. Close this window.
  5. Click the "Show Plot" button, and move this window off to the side so you can watch both this window and the network window simultaneously.
  6. Click the "Calculate Output" button, and enter in a some boolean values (0 and 1) for Input 1 and Input 2. Observe that the output values are incorrect. Close this window.
  7. Click the "Step" button a few times. Observe the plot.
  8. Select the "Neural Options" drop down menu, and choose "Stopping Conditions." Change the target error to 0.02.
  9. Click "Step 50X" and watch the network and plot fly. Repeat this until it appears that the network has stabilized, i.e. it is not making any further improvement in accuracy. When it stops, try "Calculate Output" again. The results should now be significantly closer to correct, at least for some of the outputs. Which output does the network fail to learn, and why?.
  10. Examine all the weights that your network now has. Explain why the "and" and "or" outputs yield the correct answers based on the weights that you see.
  11. Take a screenshot of your network and your plot, and turn this in with your assignment.

Part 2: Bigger Example

  1. From the File menu, choose "Load Sample Graph and Data," then choose "Mail Reading" from the drop down box. Click "Load".
  2. Click the "Solve" tab, then click "Initialize Parameters."
  3. Click "View/Edit Examples." Under the training examples, in the drop box, choose "Select % of Examples." In the "Select Data Examples" box that pops up, enter "50" in the box (for a random 50% of the data). Click OK. Back at the "Edit Data Set Examples" box, click the right arrow in the center to move these examples you just selected over to the test set on the right hand side. Close the "Edit Data Set Examples" window.
  4. Click on "Show Plot," and start training the network.
  5. When training is done, initialize the parameters again, and train again. Do this exercise a few times. You should see a general pattern emerging, though the behavior is somewhat different every time due to the randomization.
  6. After you have gotten a sense of what to expect, run it a few more times to choose a representative example to turn in. Take a screenshot of your network and your plot. Examine the training error and the test error. Which is worse, and why?
  7. See if you can find a run where overfitting occurs. Take a screenshot of your network and your plot, and explain where on the plot where overfitting began occurring.
  8. Change the learning rate to 0.01 (under Neural Options, Learning Options) and see what happens. Take a screenshot of your plot, and explain what you see.
  9. Change the learning rate to 10, then play with restarting with different random parameters and different datasets until you obtain an example where the plot shows some oscillations.Take a screenshot of your plot and explain what's going on.