Neural Networks
This is your chance to play with some neural network software to
see what you can learn. All boldfaced content indicates items that you
need to turn in. You should turn in all questions in some sort of word
processing document, and submit electronically.
Part 1: Boolean Warmup
- Visit http://aispace.org/neural. Click where it says "Click here" to start the tool.
- From the File menu, choose "Load Sample Graph and
Data," then choose "Boolean Example" from the drop down box. Click
"Load".
- You should see a neural network with three outputs, each
computing a different boolean function based on two inputs. Click on
the "Solve" tab, then click the "Initialize Parameters" button.
- Click the "View/Edit Examples" button, and observe the training data.
Note that there is no test data for this dataset. Close this window.
- Click the "Show Plot" button, and move this window off to the
side so you can watch both this window and the network window
simultaneously.
- Click the "Calculate Output" button, and enter in a some boolean
values (0 and 1) for Input 1 and Input 2. Observe that the output
values are incorrect. Close this window.
- Click the "Step" button a few times. Observe the
plot.
- Select the "Neural Options" drop down menu, and
choose "Stopping Conditions." Change the target error to 0.02.
- Click "Step 50X" and watch the network and plot
fly. Repeat this until it appears that the network has stabilized,
i.e. it is not making any further improvement in accuracy. When
it stops, try "Calculate Output" again. The results should now be
significantly closer to correct, at least for some of the
outputs. Which output does the network fail to learn, and
why?.
- Examine all the weights that your network now
has. Explain why the "and" and "or" outputs yield the correct
answers based on the weights that you see.
- Take a screenshot of your network and your plot,
and turn this in with your assignment.
Part 2: Bigger Example
- From the File menu, choose "Load Sample Graph and
Data," then choose "Mail Reading" from the drop down box. Click
"Load".
- Click the "Solve" tab, then click "Initialize
Parameters."
- Click "View/Edit Examples." Under the training
examples, in the drop box, choose "Select % of Examples." In the
"Select Data Examples" box that pops up, enter "50" in the box (for
a random 50% of the data). Click OK. Back at the "Edit Data Set
Examples" box, click the right arrow in the center to move these
examples you just selected over to the test set on the right hand
side. Close the "Edit Data Set Examples" window.
- Click on "Show Plot," and start training the
network.
- When training is done, initialize the parameters
again, and train again. Do this exercise a few times. You should see
a general pattern emerging, though the behavior is somewhat
different every time due to the randomization.
- After you have gotten a sense of what to expect,
run it a few more times to choose a representative example to turn
in. Take a screenshot of your network and your plot. Examine
the training error and the test error. Which is worse, and
why?
- See if you can find a run where overfitting
occurs. Take a screenshot of your network and your plot, and
explain where on the plot where overfitting began occurring.
- Change the learning rate to 0.01 (under Neural
Options, Learning Options) and see what happens. Take a screenshot
of your plot, and explain what you see.
- Change the learning rate to 10, then play with
restarting with different random parameters and different datasets
until you obtain an example where the plot shows some
oscillations.Take a screenshot of your plot and explain what's
going on.