Cheeseman, P., Oldford, R.W.(eds).: Selecting models from data. Springer-Verlag, New York 1994.

Farlow, S.J. (ed.): Self-Organizing methods in Modeling. GMDH Type Algorithm. Marcel Dekker. New York, Basel. 1984, ISBN: 0-8247-7161-3

Ivakhnenko, A.G., Müller, J.-A.: Selbstorganisation von Vorhersagemodellen. Verlag Technik, Berlin 1984. (in german)

Madala, H.R., Ivakhnenko, A.G.: Inductive Learning Algorithms for Complex Systems Modelling. CRC Press Inc..Boca Raton, Ann Arbor, London, Tokyo. 1994, ISBN: 0-8493-4438-7

Müller, J.-A., Lemke, F.: Self-Organising Data Mining. Libri, Hamburg, 2000, ISBN: 3-89811-861-4

a. in english

 

Barron, A.R., Barron, R.L.: Statistical learning networks: A unifying view. Proceedings of the 20th Symposium Computer Science and Statistics. (1988) pp. 192-203.

 

Hild, Ch.R., Bozdogan, H.: The use of information-based model selection criteria in the GMDH algorithm. SAMS,20 (1995), 1-2 ,pp. 29-50.

 

Ivakhnenko, A.G., Müller, J.-A.: Present state and new problems of further GMDH development . SAMS, 20 (1995), 1-2, pp. 3-16.

 

Ivakhnenko, A.G.: On the Relation of Data Mining and Knowledge Mining. Comments on the book "Self-Organising Data Mining". Kiev, 2000

 

Ivakhnenko, A. G., Müller, J.-A. : Self-organization of nets of active neurons. SAMS. vol.20 (1995), 1-2, pp.93-106.

 

Lemke, F.: SelfOrganize! - software tool for modelling and prediction of complex systems. SAMS, 20 (1995), 1-2, pp.17-28.

 

Lemke, F.: Knowledge Extraction from Data Using Self-Organizing Modeling Technologies. eSEAM'97 conference, MacSciTech organization (1997)

 

Lemke,F., Müller, J.-A.: Self-Organizing Data Mining for a Portfolio Trading System. Journal for Computational Intelligence in Finance 3 (1997).

 

Müller, J.-A.: Selection procedures and application in Ecology and Economy. In: Sydow, A. (ed) Computational Systems Analysis (1992). Elsevier Publ.. Tokyo. pp. 489-494.

 

Müller, J.-A.: Computer aided modelling and its application in economy and ecology. In "Computational Systems analysis 1992". Elsevier Publ., Tokyo, Amsterdam (1992).

 

Müller, J.-A., Lemke,F.: Self-Organizing modelling and decision support in economics. In "Proceedings of the IMACS Symposium on Systems Analysis and Simulation". Gordon and Breach Publ. (1995), pp. 135-138.

 

Müller, J.-A.: Self-organizing modelling in analysis and prediction of stock market. In: Second International Conference on Application of fuzzy systems and soft computing ICAFS-96. (with Ivachnenko, A.G.). Siegen (1996), pp-491-500.

 

Müller, J.-A., Lemke, F., Ivachnenko, A.G.: GMDH algorithms for complex systems modelling. Mathematical and Computer Modelling of Dynamical Systems 4(1998)4 pp. 275-316.

 

Elder, J. F.: Induction and Polynomial Networks. In: Fraser, M.D. (Ed.): Network Models for Control and Processing, Intellect Press (2000)

 

Elder, J. F.:Pereceptrons, Regression, and Global Network Optimization. Technical Report (1992)

 

Sarle, W.S.: Neural networks and Statistical Models. In: Proceedings of 19.th Annual SAS User Group International Conference. Dallas. (1994) pp. 1538-1549.

 

b. in german language

 

Müller, J.-A. (1993) Selbstorganisation mathematischer Modelle geoökologischer Prozesse. In "informatik aktuell. Informatik für den Umweltschutz". Springer Verlag Berlin, Heidelberg, S.101-112.

 

Müller, J.-A. (1990) Aufgaben und Probleme der wissensbasierten Modellierung. In Ehrenberg,D., Krallmann,H., Rieger,B. (eds) Wissensbasierte Systeme in der Betriebswirtschaft. E. Schmidt Verlag. Berlin S. 217-231

 

c. in russian language

 

Aksenova,T.I., Jurackovskij,Ju.P.: Charakterizacija nesmescennoj struktury i uslovija ee J-optimal'nosti. avtomatika 3(1988), 4 , pp.34-37

 

Ivachnenko, A.G. (1971) Sistemy evristiceskoj samoorganizacii v techniceskoj kibernetike. Technika. Kiew.

 

Ivachnenko, A.G., Stepasko, V.S.: Pomechoustojcivost' modelirovanija. Kiev: Naukova dumka 1985

 

Ivachnenko, A.G., Müller, J.-A.: Algoritmy vostanovlenija garmoniceskich processov iz binarnych vyborok. avtomatika 35(1992), 6, pp.34-39.

 

or have a look at http://www.gmdh.net

Statistical learning networks: A unifying view

A variety of network models for empirical inference have been introduced in rudimentary form as models for neurological computation. Motivated in part by these brain models and to a greater extent motivated by the need for general purpose capabilities for empirical estimation and classification, learning network models have been developed and successfully applied to complex engineering problems for at least 25 years. In the statistics community, there is considerable interest in similar models for the inference of high-dimensional relationships. In these methods, functions of many variables are estimated by composing functions of more traceable lower-dimensional forms. In this presentation, wedescribe the commonality as well as the diversity of the network models introduced in these different settings and point toward some new developments.

back to papers overview

The use of information-based model selection criteria in the GMDH algorithm
Cheryl Hild, Hamparsum Bozdogan

The Group Method of Data Handling (GMDH) algorithm is an elegant approach to statistical data modeling. This paper introduces and develops information-based model evaluation criteria for the use in the GMDH algorithm to study the quality of the competing off-spring models at each generation. Thus, the problem is to integrate model selection criteria into the GMDH algorithm that choose the model which "best" approximates the given data set among the set of competing alternative models with different numbers of parameters. In order to sift a model that fits the data well, a criteria is needed that evaluates each competing alternative model in terms of bias, variability, and goodness-of-fit. Also, there is a need to consider the complexity of the selected model. The general principle, known as Occam's Razor, is that a parsimonious model is preferable to a more complex one. Therefore, we propose the use of information-theoretic model selection criteria to facilitate the identification of a parsimonious model or, in other words, a model that provides the highest information gain with the least complexity in the GMDH technology. A real numerical example is shown along with an open architecture symbolic computational toolbox to illustrate the utility of the new proposed approach.

back to papers overview

At present, GMDH algorithms give us the only way to get the most accurate approximations of functions and forecasts of random processes and events in case of noised and short input sampling. Revised GMDH algorithms, recently developed, use two sorting-out criteria: the basic one and at the next stage - the discriminating criterion. There are several ways to raise accuracy and the forecasting validation lead time. The first way is to develop a set of revised GMDH algorithms with a different mathematical language of modelling to choose the description which is adequate to the objects characteristics. The second way is to use GMDH algorithms as active neurons in the neuronet. The third way is to unite the GMDH algorithm with the algorithm of mathematical programming. An example of such unification is presented in the case of the number of characteristic variables being equal to the number of output variables.

back to papers overview

Up to now the known networks have been characterized by neurons, which are very simple processing units. Such passive neurons are not able to select and estimate their own inputs. In a new approach, which corresponds in a better way to the actions of human nervous system, the connections between several neurons are not fixed but change in dependence on the neurons themselves. Such active neurons are able, during the learning or self-organizing process, to estimate which inputs are necessary to minimize the given objective function of the neuron. This is only possible on the condition that every neuron is a complicated processing unit, such as GMDH algorithm. As an application of such nets of active neurons is considered the prediction of activity of stock exchange.

back to papers overview

A software tool using the GMDH technique for modeling and prediction of complex linear or nonlinear multi-input / multi-output systems is presented. Key features of this tool necessary to make it applicable to a large spectrum of modelling tasks in economy, ecology and other fields are mentioned, like the use of the cross-validation principle, the selection procedure for selection of the intermediate input variables and the avoidance of conflicts during the synthesis of a system of equations. The results of a preliminary model of the national economy of the Federal Republic of Germany constructed by this tool are shown to give an overview of the effeciency and flexibility of GMDH.

The name "SelfOrganize!" is no longer valid. It was replaced by the name "KnowledgeMiner".

back to papers overview

Abstract

Today, knowledge extraction from data (also referred to as Data Mining) plays an increasing role in sifting important information from existing data. Commonly, regression-based methods like statistics or Artificial Neural Networks as well as rule-based techniques like fuzzy logic and genetic algorithms are used.

This paper describes two methods working on the cybernetic principles of self-organization: Group Method of Data Handling (GMDH) and Analog Complexing. GMDH combines the best of both statistics and Neural Networks and creates adaptively models from data in the form of networks of optimized transfer functions (Active Neurons) in an evolutionary fashion of repetitive generation of populations of alternative models of growing complexity and corresponding model validation and survival-of-the-fittest selection until an optimally complex model has been created. Nonparametric models obtained by Analog Complexing are selected from a given variables set representing one or more patterns of a trajectory of past behavior which are analogous to a chosen reference pattern.

Both approaches have been developed for complex systems modeling, prediction, identification and approximation of multivariate processes, diagnostics, pattern recognition and clusterization of data samples and they are implemented in the KnowledgeMiner modeling software tool. They can be applied to problems in economy (macro economy, marketing, finance e.g.), ecology (water and air pollution problems e.g.), social sciences, medicine (diagnosis and classification problems) and other fields.

complete article

back to papers overview

This paper describes the application of data mining algorithms for a portfolio trading system. The goal of data mining in this case is prediction of assets of a portfolio by means of parametric or nonparametric models. Parametric models are adaptively created from data by the Group Method of Data Handling (GMDH) in the form of networks of optimized transfer functions (Active Neurons). Nonparametric models are selected from a given variables set by Analog Complexing representing one or more patterns of a trajectory of past behavior which are analogous to a chosen reference pattern. Both approaches of self-organizing modeling include not only core data mining algorithms but also an iterative process of generation of alternative models with growing complexity, their evaluation, validation and selection of a model of optimal complexity. Therefore, these approaches are denoted in this paper as self-organizing data mining.

In a modeling/prediction module a self-organizing data mining is performed to extract and synthesize hidden knowledge from a given data set systematically, fast and explicit visible. The control module of the trading system is responsible for signals generation based on predictions provided by the modeling module.

Initial performance results of a trading system are presented. The trading system simulates trading a portfolio of diverse stocks using daily out-of-sample price data.

back to papers overview

Knowledge extraction from data using inductive methods like GMDH has advantages in modelling of rather complex and ill-defined objects with fuzzy characteristics and for noised and extremely short data samples. Using a GMDH algorithm which optimize additionally the nonlinear partial models an example for analysis of systems of characteristics will be presented.

complete article

back to papers overview

At present, GMDH algorithms give us a way to identify and forecast economic processes in case of noised and short input sampling. In contrast to neural networks, the results are explicit mathematical models, obtained in a relatively short time. For ill-defined objects with very big noises, better results are obtained by analog complexing methods. Nets with active neurons should be applied to increase accuracy. Active neurons are able, during the self-organizing process, to estimate which inputs are necessary to minimize a given objective function of the neuron. In the neuronet with such neurons, we have a twofold multilayered structure: neurons themselves are multilayered, and they will be united into a multilayered network.

KnowledgeMiner is an easy-to-use modelling tool which realizes twice-multilayered neuronets and enables the creation of time series, multi input/single output and multi input/multi output systems (system of equations). Successful applications are shown in the field of analysis and prediction of characteristics of stock markets in financial risk control modelling.

back to papers overview

A wide variety of inductive modeling algorithms exist for practical use; these range from artificial neural networks (ANNs) and decision trees, to kernel and spline techniques. The variety of induction methods indicates both the importance of the field from an application perspective and the inherent difficulty in performing induction tasks. This chapter aims to 1) survey and categorize leading techniques for inducing estimation, classification, and control models from sample data, 2) suggest fruitful enhancements to and combinations of the methods, and 3) demonstrate the power of a particular induction technique - polynomial networks. This chapter moves from a broad overview of induction techniques to general purpose enhancements to these techniques, and finally to a particularly effective but less well-known approach, polynomial networks, that developed out of the neural network and adaptive systems communities.

The remainder of this chapter is organized as follows. Section 2 provides a brief survey of the leading techniques, and organizes them into a hierarchy according to the level of "decisions" they leave to the computer. Sections 3 and 4 contain our suggested enhancements to inductive modeling. In particular, Section 3 describes some of the principle hazards of inferring useful models from data, and proposes a simple way to improve model induction by integrating visual information. Section 4 discusses model selection and demonstrates the performance of a number of complexity-penalty metrics on an example problem. The last two sections concentrate on networks of polynomial nodes, such as created by the Group Method of Data-Handling, GMDH (Ivakhnenko, 1968; Farlow, 1984), the Polynomial Network Training Algorithm, PNETTR-IV (R. Barron, Mucciardi, Cook, Craig, and A. Barron, 1984) and the Algorithm for Synthesis of Polynomial Networks, ASPN (Elder; 1985, 1989). Section 5 describes each of these approaches to the construction of polynomial networks and concludes with suggested enhancements to the methods. Section 6 provides examples of the use of polynomial networks. Finally, Section 7 contains conclusions and suggestions for future research in induction.

back to papers overview

There has been much publicity about the ability of artificial neural networks to learn and to generalize. In fact, the most commonly used artificial neural networks, called multilayer perceptrons, are nothing more than nonlinear regression and discriminant models that can be implemented with standard statistical software. This paper explains what neural networks are, translates neural network jargon into statistical jargon, and shows the relationships between neural networks and statistical models such as generalized linear models, maximum redundancy analysis, projection pursuit, and cluster analysis.

back to papers overview

Vorgestellt werden Forschungsarbeiten, die sich einordnen in die physisch-geographische Prozessforschung zur Beherrschung und planmaessigen Steuerung von Geooekosystemen sowie zur Analyse, Modellierung und Vorhersage von Stofftransporten und - umsaetzen in verschiedenen Landschaften. Auf der Grundlage von Zeitreihen die die Witterung, Bodeneigenschaften sowie das Abflussgeschehen kennzeichnen, wird die Abhaengigkeit geooekologischer Prozesse von ausgewaehlten Kenngroessen des meteorologischen Regimes sowie des oberen Bodenhorizontes und der Bodeneigenschaften untersucht. Zur Anwendung kommt neben der Korrelations- und Regressionsanalyse die Selbstorganisation mathematischer Modelle. Die erreichten Ergebnisse bestaetigen die Leistungsfaehigkeit einer auf den GMDH Algorithmen basierenden automatischen Modellgenerierung insbesondere bei ungenuegender A-priori-Information ueber das System und die wesentlichen qualitativen Einflussgroessen.

back to papers overview

Contact:
Research
Prof. J.-A. Müller, author of many papers related to self-organizing modeling. He has also developed algorithms to make Analog Complexing usable for evolutionary processes.