Presented at NIPS=96:
Tricks of the Trade Workshop
Saturday December 6, 1996
Snowmass, CO

R. Renner*
Dept. of Computer Science
Florida State University
Tallahassee, FL

Techniques for optimization of constructive networks are proposed. Constructive networks attempt a solution to the generally recognized problem of network topology inherent in backpropagation models. Unfortunately, the network topology created is highly dependent upon paramaterization issues. The resulting network may or may not be optimal for the given application domain. Finding an optimal network topology requires choosing the Aright@ training parameters. Is it possible to find such parameters or combinations of paramaters? This issue is investigated through the analysis of networks constructed by the Cascade Correlation learning architecture on a realizable classification problem. The findings of this research suggest that multiple methods may be combined to achieve parameter optimization. Analysis of specific training parameters, their relationships and causal effect on topology are discussed. A dynamic approach to cross-validation training analysis and adaptive parameterization is proposed, as well as a combining method for an ensemble of networks.

This research project was under the direction of, and in collaboration with R.C. Lacher. Thanks to K.T. Lacher for contributions of data to this research.