University of Missouri-Columbia

Links: http://myprofile.cos.com/popescum

Back to top

　

　

　

Paul J. Werbos

NSF

Vita: Paul J. Werbos is best known as the original inventor of backpropagation, as part of his Harvard PhD thesis, which was reprinted in full in his book the Roots of Backpropagation, Wiley 1994, along with his classic 1990 tutorial on backpropagation through time for Proc IEEE. Also, his 1967 article in Cybernetica first proposed the idea of approximating dynamic programming as a way to improve reinforcement learning, elaborated in many later papers. He was one of the three original two-year presidents of the International Neural Network Society, and winner of the IEEE Pioneer Award. He is Program Director for Control, Networks and Computational Intelligence at NSF, which actively seeks more proposals in this area. He has also been active in many cross-cutting funding initiatives; for example, he serves on the Working Group for energy storage and distribution of the interagency Climate Change Technology Program, and coordinated the NASA-NSF-EPRI solicitation on space solar power (NSF 02-098). He is also on the Planning Committee of the Millennium Project of the United Nations University (http://millennium-project.org), and has published a few papers on quantum foundations and technology (see arXiv.org, physics and nonlinear systems). He also has two degrees in economics from Harvard and the London School of Economics.

He is on the governing boards of INNS and of the IEEE Industrial Electronics Society.

Abstract: Neural nets are used in a growing variety of real-world applications, in a growing number of areas (including the modeling of natural intelligence). But this has been a mixed blessing, as various subcultures have not kept up with each other. For example, at http://www.fulton.asu.edu/~nsfadp/ applications of adaptive dynamic programming (aka advanced reinforcement learning) are reported which beat the previous best performance in electric power grid control, aircraft control, in large-scale logistics, etc. -- but best performance requires integrating concepts from many subcultures. Yet in some subcultures, people are still using "mappings from sensory to motor coordinates" (direct adaptive control), whose limits were clear years ago. Some are using indirect methods inspired by control theory with well-known stability theorems -- theorems whose conditions are rarely satisfied, and methods which are easily improved upon. Similarly, in prediction, some subcultures falsely believe that recurrent networks are hard to train and of marginal benefit, even as people in industry use them routinely, and report performance better than extended Kalman filters and equal (at lower cost) to elaborate particle filter methods. The foundations of learning also bear on the practical choices.

This tutorial will try to offer a kind of practical roadmap of these issues, and suggest how it points towards future functionality in tasks as large-scale as what brains can handle.

Keywords: adaptive dynamic programming (advanced reinforcement learning)

Back to top

　

　

　

 Vladimir Cherkassky

University of Minnesota

Vita: Vladimir Cherkassky is Professor of Electrical and Computer Engineering at the University of Minnesota.  He received Ph.D. in Electrical Engineering from University of Texas at Austin in 1985. His current research is on methods for predictive learning from data, and he has written a monograph Learning From Data published by Wiley in 1998. Prof. Cherkassky has served on the Governing Board of INNS. He has served on editorial boards of IEEE Transactions on Neural Networks, the Neural Networks Journal, the Natural Computing Journal and the Neural Processing Letters. He served on the program committee of major international conferences on Artificial Neural Networks. He was Director of NATO Advanced Study Institute (ASI) From Statistics to Neural Networks: Theory and Pattern Recognition Applications held in France, in 1993. He has presented numerous tutorials on neural network and statistical methods for learning from data.

Abstract: Most existing learning algorithms have been developed under standard formulations of the learning problem, such as classification or regression. However, many real-life applications motivate approaches that go beyond traditional learning methods. These new directions include new types of inference different from standard inductive inference (i.e., transduction), as well as new formulations of the learning problem. This tutorial will review standard learning formulations using the framework of Vapnik-Chervonenkis (VC) learning theory, and then discuss several possible ‘non-standard’ formulations. The tutorial consists of four parts:

(1) Motivation for new learning formulations. We provide historical, philosophical and application-driven motivations for developing new learning formulations and new types of inference (as opposed to research on new learning algorithms) under general conceptual framework of Vapnik’s Statistical Learning Theory. Then we present standard inductive learning formulations, and critically review the assumptions underlying such formulations. Relaxing some of these assumptions leads to several ‘non-standard’ formulations as discussed below.

(2) Non-inductive types of inference. These include non-inductive types of inference such as transduction and selection learning formulations. We describe these formulations, and several recent applications studies comparing inductive vs. transductive learning.

(3) Application-driven formulations. We outline general framework for mapping application requirements onto a learning problem formulation, and then provide examples of such new application-driven formulations. Such examples range from financial engineering (trading) to identity fraud detection.

(4) Multiple model estimation. All standard inductive learning formulations assume that all available (or training) data can be described by a single statistical model. For example, in classification setting the goal is to estimate a (single) decision boundary. Likewise, under regression formulation, the goal is to estimate a single target function from finite and noisy data samples. However, there are many applications which naturally lead to learning formulations where the goal is to estimate several models from available (finite) data. For example, in computer vision, a sequence of video frames may contain several moving objects (with overlapping trajectories), so the problem of multiple motion estimation can be viewed under multiple model estimation framework. I will describe a new learning formulation for such settings, and then introduce SVM-based learning algorithms for this new formulation.

Finally, I will discuss methodological issues arising in using learning methods for various real-life applications, such as:

- clear separation between the learning problem formulation and the solution approach (e.g, learning algorithm);

- importance of mapping application requirements onto an appropriate problem formulation, rather than inventing/ reinventing new learning algorithms;

- empirical comparisons of learning algorithms.

INTENDED AUDIENCE: researchers and practitioners interested in understanding new learning formulations motivated by practical application requirements, rather than standard formulations motivated by mathematical modeling considerations.

OTHER CONSIDERATIONS: the proposed tutorial is based on two plenary talks (on the same subject of New Learning Problem Formulations) given by the instructor at ICANN-01 in Vienna, Austria and ANNIE-04 in St Louis, Missouri.

Keywords: Inductive principle, inductive inference, neuroimaging, new learning formulations, Popper's falsifiability, regularization, statistical learning, support vector machines, transduction, Vapnik-Chervonenkis theory.

Back to top

　

　

　

Pierre Baldi
University of California, Irvine

Vita: Pierre Baldi is a Professor in the School of Information and Computer Sciences and the Department of Biological Chemistry at the University of California, Irvine and the Director of the Institute for Genomics and Bioinformatics at UCI. Born and raised in Europe, he received his PhD from the California Institute of Technology in 1986. From 1986 to 1988 he was a postdoctoral fellow at the University of California, San Diego. From 1988 to 1995 he held faculty and member of the technical staff positions at the California Institute of Technology and at the Jet Propulsion Laboratory. He was CEO of a startup company from 1995 to 1999 and joined UCI in 1999. He is the recipient of a 1993 Lew Allen Award at JPL and a Laurel Wilkening Faculty Innovation Award at UCI. Dr. Baldi has written over 100 research articles and four books:

• Modeling the Internet and the Web--Probabilistic Methods and Algorithms, Wiley, (2003);
• DNA Microarrays and Gene Regulation--From Experiments to Data Analysis and Modeling, Cambridge University Press, (2002);
• The Shattered Self--The End of Evolution, MIT Press, (2001);
• Bioinformatics: the Machine Learning Approach, MIT Press, Second Edition (2001).

His research focuses in AI, machine learning, computational biology, and biological and chemical informatics.

Abstract: This tutorial focuses on the application of machine learning methods, and in particular of neural networks, to problems in protein structure prediction. The tutorial first addresses the general problem of how to apply neural networks techniques to variable-size structured data, such a sequences, trees, and graphs in bioinformatics and other applications. The tutorial covers the design of feedforward and recursive neural network architectures for the prediction of protein structural features, such as secondary structures, relative solvent accessibility, disulphide bridges, and contact maps. Simulations and state-of-the-art results are presented and discussed together with more general issues, such as tradeoffs and integration with other methods, and applications to other related problems.

Keywords: neural networks, recurssive neural networks, bioinformatics, sequence analysis, protein, protein structure prediction, structural proteomic

Back to top

　

　

　

Dario Floreano
EPFL, Switzerland

Vita: DARIO FLOREANO is Professor of Intelligent Systems at the Swiss Federal Institute of Technology in Lausanne (EPFL) where he is director of the Institute of Systems Engineering. His research activities include embodied and embedded neural networks, artificial evolution, robotics, bio-mimetic electronics, self-organizing systems, and artificial life. He held senior research positions at the National Research Council in Roma, at the University of Stirling, at the Swiss Federal Institute of Technology in Lausanne, and at Sony Computer Science Laboratory in Tokyo. Dario published more than 100 peer-reviewed papers, authored 2 books, and edited 3 other books. His book with S. Nolfi, Evolutionary Robotics, was reprinted by MIT Press three times over the last four years. He co-organized 6 international conferences and joined the program committee of more than 50 other conferences. He is on the editorial board of the journals Neural Networks, Genetic Programming and Evolvable Machines, Adaptive Behavior, Artificial Life, Connection Science, IEEE Transactions on Evolutionary Computation, and Autonomous Robots. He is co-founder and member of the Board of Directors of the International Society for Artificial Life, Inc. and member of the Board of Governors of the International Society for Neural Networks.

Abstract: Evolutionary Robotics is the artificial evolution of control systems for autonomous robots. Most of the past and ongoing research in Evolutionary Robotics is concerned with evolution of neural controllers for which available learning algorithms are not suitable. The tutorial will put special emphasis on how Evolutionary Robotics can be used to study open questions in evolutionary biology, neurophysiology, and cognitive science. A short introduction to artificial evolution will allow participants to fully understand and replicate most of the experiments by using freely available software.

An highlight of the topics: Evolution of simple navigation skills. The principle of parsimony: Evolution of complex behaviours using simple reactive mechanisms. The role of fitness functions. Evolution of vision-based navigation. The role of the environment in the evolutionary process and the emergence of spatial maps. Co-evolution of active vision and feature sensitivity for image recognition, robot navigation, and car driving. Interactions between evolution and learning. Evolution of learning rules and analysis of evolved neural controllers. Competitive co-evolution in biology and robotics. The case of predator-prey robots. Introducing learning in co-evolutionary competitive systems. Evolution of spiking control networks and other exotic neural controllers.

All topics will be illustrated with several examples of robots with wheels, legs, wings.

Keywords: mobile robots, evolutionary systems, autonomous systems, time-dependent networks, control systems

Back to top

　

　

　

Nik Kasabov (nkasabov@aut.ac.nz)
Knowledge Engineering and Discovery Research Institute (KEDRI), http://www.kedri.info, Auckland University of Technology, New Zealand

Vita: Professor Nik Kasabov is the Founding Director and the Chief Scientist of the Knowledge Engineering and Discovery Research Institute KEDRI, Auckland (http://www.kedri.info/). He obtained a MSc and a PhD from the Technical University of Sofia. At present he holds a Chair of Knowledge Engineering at the School of Computer and Information Sciences at Auckland University of Technology. He is a Fellow of the Royal Society of New Zealand, Fellow of the New Zealand Computer Society, and a Senior Member of IEEE. Kasabov is the chair of the Adaptive Systems Task Force of the Neural Network Technical Committee of the IEEE Computational Intelligence Society. He is a member of the Board of Governors of the INNS and a board member of the Asia Pacific Neural Network Assembly (APNNA). His main research interests are in the areas of intelligent information systems, soft computing, neuro-computing, bioinformatics, brain study, speech and image processing, data mining and knowledge discovery, where he has published 10 books, 380 refereed papers, 25 patents and other work. He has an extensive academic experience, holding positions at the Technical University of Sofia- Bulgaria, University of Essex - UK, University of Otago -New Zealand. Kasabov is on the editorial boards of 8 international journals and has been on the PC of 50 international conferences in the last 5 years. He chaired the series of ANNES conferences (1993-2001). More information of Prof. Kasabov can be found on the Web site: http://www.kedri.info.

Abstract: Evolving connectionist systems (ECOS) is a general paradigm of neural systems that evolve their structure, their functionality and their internal knowledge representation through continuous learning from data and interaction with the environment. The learning process can be: on-line, off-line, incremental, supervised, unsupervised, active, sleep/dream, etc. ECOS are inspired by the evolving processes in the brain. These processes are self-organized and are governed by both external stimuli (data) and internal parameters - genes. The tutorial consists of three parts, each presented in 35 minutes with a break of 15 min. after the first two parts. Part one presents biological principles of evolving brains, that include: synaptic changes through learning, neuron creation, brain development, the role of genes for brain functions and malfunctions, gene interaction networks, the role of evolution.
Part two presents several ECOS models, that include both simple ECOS and computational neurogenetic models (CNGM) explained and illustrated through simulations and examples. Simple ECOS models are presented in [N.Kasabov, Evolving connectionist systems: Methods and Applications in Bioinformatics, Brain study and intelligent machines, Springer, 2002, www.springer.de)]. They are adaptive neural systems that learn on-line and facilitate rule extraction. CNGM are connectionist models that integrate neural networks and gene interaction networks. Interaction of genes in every neuron affects the dynamics of this neuron and may influence the whole neural network performance which changes as a function of gene expression. Through optimization of the gene interaction networks, the initial gene/protein expression values and the CNGM parameters, particular target states of the CNGM can be achieved. This is illustrated by means of a simple neurogenetic model of a spiking neural network (SNN). The behaviour of the SNN is evaluated by means of the local field potential, thus making it possible to attempt modelling the role of genes in different brain states, where EEG data is available to test the model.
Part three presents applications of ECOS models in bioinformatics, brain study, adaptive robot control, finance and business prediction, medical decision support, evolvable hardware, evolvable software, adaptive integrated speech, image and fingerprint recognition.
The tutorial targets computer scientists, neuroscientists, biologists, engineers and graduate students.

Keywords: Computational Intelligence, Neuroinformatics, Bioinformatics, Knowledge-based neural networks, Evolving connectionist systems, Gene regulatory networks, Computational neurogenetic modeling.

　

Back to top

　

　

Xiuwen Liu, Anuj Srivastava and  Washington Mio
Dept. of Computer Science, Dept. of Statistics and  Dept. of Mathematics, Florida State University

Vita: Dr. Xiuwen Liu is a computer scientist with research interests in computer vision, biologically plausible principles for visual modeling, and image analysis. He has developed and implemented distinctive computational texture models that provide quantitative explanations for texture perception as well as successful applications in texture synthesis, image segmentation, object recognition, and terrain classification. His work with Dr. Srivastava is the first to computationally derive sparse filters that are optimal for recognition. His expertise in efficient computing has helped translate many theoretical ideas into efficient practical algorithms. He is a co-director of the Center for Applied Vision and Imaging Sciences (CAVIS) at the Florida State University.

Dr. Anuj Srivastava has expertise in the areas of statistical image analysis, signal processing, and statistical inferences on nonlinear manifolds. He has worked on applications of automated target recognition (ATR), derived performance bounds for statistical ATR, derived new probability models for natural images, derived nonlinear filtering algorithms on Lie groups and their quotient spaces, and is currently working on algorithms for statistical shape analysis. He directs the Center for Applied Vision and Imaging Sciences (CAVIS) at the Florida State University.

Dr. Washington Mio is a geometric topologist with expertise in the areas of topology and geometry of manifolds, who is currently working on problems in computer vision and pattern recognition. Dr. Mio’s present investigations include the algorithmic study of the differential geometry of various spaces of curves, energy functionals on these spaces, and applications to shape and image analysis. His previous research includes the study of the geometric topology of manifolds and generalized manifolds, group actions on these spaces, knot theory, and submanifold and map transversality. Dr. Mio is a co-director of CAVIS.

Abstract: Many of the problems in learning can be posed as an optimization one on underlying manifolds. While Euclidean space is commonly used due to its simplicity, intrinsic constraints of problems often lead to manifolds that are nonlinear in nature. For example, common to component analysis techniques (such as PCA, ICA, FDA, and so on), one often requires the components to be orthonormal and this requirement leads to Stiefel manifold. By exploiting the geometric structure of the manifolds, one can often design and derive efficient and effective algorithms.

The main goal of this tutorial is to give a systematic coverage of nonlinear manifolds in pattern recognition and image analysis areas. A novel aspect of this tutorial is that we will introduce the tools from differential geometry that are necessary to understand existing techniques and more importantly to build new ones. Another important feature of this tutorial is that we will use live demonstrations to show the effectiveness of such techniques for pattern recognition. For example, by using optimal component analysis, we can often improve recognition performance significantly. It is widely recognized that neural network performance often depends critically on the features that are used. Compared to other areas, systematic study on feature extraction and selection is very limited. These nonlinear manifold techniques provide solutions to some of these problems by optimization in the feature space, which is often nonlinear in nature. We believe this tutorial on nonlinear manifolds is very timely and should be very useful for reseachers in pattern recognition, computer vision, machine learning, and other related areas.

This tutorial will consist of four main parts (see http://www.cavis.fsu.edu/manifold-tutorials/). In the first part, we will define nonlinear manifolds, and give examples to motivate the importance of nonlinear manifolds especially in the pattern recognition and image analysis areas. In the second part, we will cover the mathematical tools from differential geometry for optimization on nonlinear manifolds, including differentiable manifolds, examples of manifolds, tangent vectors, tangent spaces, examples of tangent vectors and tangent spaces, differential geometry of manifolds, geodesics, numerical techniques for calculating geodesics, and optimization on nonlinear manifolds. In the third part, we will extend usual techniques from statistics onto nonlinear manifolds, including probability distributions on manifolds, intrinsic means and covariances, sampling and estimation on manifolds, and learning and hypothesis testing on manifolds. In the last part, we will give two concrete examples that are important for many pattern recognition and image analysis applications: optimal component analysis, and shape inference and clustering on shape manifolds. We will give live demonstrations using techniques presented in this tutorial.

Keywords: Nonlinear manifolds, optimal component analysis, pattern recognition, feature extraction, statistics on nonlinear manifolds, image analysis, Grassman manifold, and Stiefel manifold.

Back to top

　

　

　

Joao Luıs Garcia Rosa
PUC-Campinas, Brazil

Vita: João Luís Garcia Rosa is Titular Professor of Pontifical Catholic University of Campinas (PUC-Campinas), São Paulo, Brazil. He got his doctorate in Computational Linguistics from Linguistics Department, Language Studies Institute, State University of Campinas (Unicamp), Sao Paulo, Brazil, in 1999, and his master's degree in Electrical Engineering (Artificial Intelligence), from Department of Computer Engineering and Industrial Automation, School of Electrical and Computer Engineering, State University of Campinas in 1993. He was graduated in Electrical Engineering (Automation and Electronics) from the latter department in 1983. His research interests are: Artificial Neural Networks, Natural Language Processing, Computational Psycholinguistics, and Biologically Plausible Connectionist Models. His academic experience: taught postgraduate level courses for masters level on computing systems: Intelligent Systems, Artificial Neural Networks, and Natural Language Processing disciplines, and taught graduate level courses for Computer Engineering and System Analysis: Programming Languages, Artificial Intelligence, and Formal Languages and Automata disciplines, at Pontifical Catholic University of Campinas (PUC-Campinas), Campinas, Sao Paulo, Brazil. Since 1998, he contributes to the field of Biologically Plausible Artificial Neural Networks with published papers and supervision of undergraduate and graduate students.

Abstract: The objectives of the proposed tutorial are to provide artificial neural networks notions and to show how this intelligent computational tool is related to neuroscience. Why current artificial neural network models are so distant from biological ones? Existing connectionist mathematical models are compared with biological neurons and networks and the main differences are stressed. Desired characteristics in biologically plausible artificial neural networks and their corresponding elements in cerebral cortex are discussed. Neuroscience basic concepts about neuron physiology, its connections and the physical chemistry of central nervous system are approached. Several biological features are being taken into account in order to achieve new models that restore the artificial neural systems first concerns. Connectionist models based on neuroscience are about to be considered the next generation of artificial neural networks, inasmuch as nowadays models are far from biology, mainly for mathematical simplicity reasons. The tutorial encompasses the biological neuron, chemical and electrical synapses, the roles of transmitters and receptors, and the neuromodulators of the synaptic function. How the brain became a model to such machine learning technique? Conventional and biologically plausible connectionist algorithms are presented. Why backpropagation, the most used supervised connectionist algorithm, is considered biologically implausible? Some connectionist architectures are taken into account: feedforward, recurrent, and bi-directional. The latter structure is considered more physiologically realistic. A number of applications are presented. They show that the alleged biologically plausible artificial neural networks are even more computationally efficient than conventional ones.

Keywords: Artificial neural networks, Biologically plausible connectionist architectures, Biologically plausible connectionist algorithms, Computational neuroscience.

Back to top

　

　

　

Antonino Staiano and Roberto Tagliaferri
Università di Salerno, Italia

Vita: Antonino Staiano is a post-doctoral research fellow at the Department of Mathematics and Computer Science, University of Salerno, Italy. He got his PhD in Computer Science at University of Salerno in 2003, discussing a thesis on unsupervised neural network models for the extraction of scientific information from astronomical databases. Since October 2000 he has been working within the AstroNeural collaboration project (a joint project together with the Department of Physical Science, University Federico II of Naples) to the development of neural -based tools for data visualization and clustering for astronomical and genetic data mining. He has worked at European Southern Observatory (ESO) - Space Telescope European Coordinating Facility and is currently involved in the European Virtual Observatory. Fields of interest and research: Data Mining, Data Visualization, Statistical Pattern Recognition, Bioinformatics, Machine Learning, Ensemble of learning systems, Evolutionary Algorithms.

Roberto Tagliaferri is Associate Professor in Computer Science and Neural Networks at the University of Salerno. His research covers the area of neural nets: neural dynamics, fuzzy neural nets, applications to signal and image processing with astronomical and geological data, industrial and medical computer aided diagnosis. Co-chairman of special sessions at the AMSE-ISIS’97, at IJCNN’99, IJCNN 2001, IJCNN 2003, WILF 2003, IJCNN 2004 and co-editor of a special issue of Neural Networks. He had a tutorial on Learning with multiple machines: ECOC models vs Bayesian Framework at IJCNN 2003. He is the author or the co-author of more than 80 publications in the area of neural networks. Since 1995, co-editor of the Proceedings of the Italian Workshops on Neural Nets (WIRN). Secretary of  SIREN (Società Italiana Reti Neuroniche) and vice-president of the Italian SIG of the INNS. Member of the Director Council of the IIASS (International Institute for Advanced Scientific Studies) "E. R. Caianiello". Senior member of the IEEE, member of INFN, INFM and AIIA.

Abstract: The recent technological advances are producing huge data sets in almost all fields of scientific research, from astronomy to genetics. Although each research field often requires ad-hoc, fine tuned, procedures to properly exploit all the available information inherently present in the data, there is an urgent need for a new generation of general computational theories and tools capable to boost most human activities of data analysis. Traditional data analysis methods, in fact, are inadequate to cope with such exponential growth in the data volume and especially in the data complexity (ten or hundreds of dimensions of the parameter space). Among the data mining methodologies, visualization plays a key role in developing good models for data especially when the quantity of data is large. For a scientist, i.e. the expert in a specific domain, is essential the need for a visual environment that facilitates exploring high-dimensional data dependent on many parameters.

Data visualization is an important means of extracting useful information from large quantities of raw data. The human eye and brain together make a formidable pattern detection tool, but for them to work the data must be represented in a low-dimensional space, usually two or three dimensions. Even quite simple relationship can seem very obscure when the data is presented in tabular form, but are often very easy to see by visual inspection.

Many algorithms for data visualization have been proposed by both neural computing and statistics communities, most of which are based on a projection of the data onto a two or three dimensional visualization space. This tutorial embraces a number of these visualization techniques both linear and nonlinear:

• Principal Component Analysis (PCA), a classical linear projection method for mapping the data to a lower dimensional space. PCA does this with a linear transformation that preserves as much data variance as possible.

• Probabilistic PCA. The disadvantage of PCA is that it does not define a generative model and indeed there is no principled interpretation of its error function. However classical PCA can be made into a density model by using a latent variable approach, derived from factor analysis, i.e. Probabilistic Principal Component Analysis (PPCA). PPCA is  a Gaussian mixture model in which each Gaussian has a covariance matrix which is diagonal and has a single variance parameter to describe all the variance in each component.

• Mixture of PPCA. Because PCA only defines a linear projection of data, it is a rather limited technique. One way around this is to use a collection of local linear models. The attraction of this is that each model is simpler to understand and usually easier to fit. PCA, PPCA and mixture of PPCA are appropriate when the data is linear or approximately piece-wise linear. An alternative approach is to use global nonlinear methods:

• Self Organizing Maps (SOM), a neural network algorithm based on a competitive learning which summarizes a set of data vectors in a high-dimensional space by a set of reference vectors organized on a lower dimensional sheet (usually two dimensional). SOM has been used for a wide variety of applications thanks to its simplicity and for its several plotting options. Although SOM provides easy of computation and powerful visualizations it, indeed, does not define any density model and suffers of other drawbacks which can be overcame employing nonlinear latent variable models;

• Generative Topographic Mapping (GTM), a nonlinear latent variable model in which the density is a Gaussian mixture model. The centre of each Gaussian is constrained to lie on a smooth manifold of low dimension (usually 1 or 2) but all Gaussians have a common spherical covariance;

• Probabilistic Principal Surfaces (PPS), a generalization of GTM with oriented  covariance. PPS allows a spherical manifold in 3D space which is better suitable to characterize high-dimensional data which usually is sparse and located at the periphery (curse of dimensionality). While such algorithms can usefully display the structure of simple and more complex data sets, they often prove inadequate in the face of data sets which are too complex. A single two or three dimensional projection, even if it is nonlinear, may be insufficient to capture all the interesting aspects of the data set. For example, the projection which best separates two clusters may not be the best for revealing internal structure within one of the clusters. This motivates the consideration of a hierarchical model involving multiple two or three dimensional visualization spaces. The goal is that the top-level projection should display the entire data set, perhaps revealing the presence of clusters, while lower-level projections display internal structure within individual clusters, such as the presence of subclusters, which might not be apparent in higher-level projections. Therefore, the tutorial reviewes hierarchical linear (based on mixture of PPCA) and nonlinear (based on GTM) latent variable models and concludes by illustrating a new proposed hierarchical model based on Probabilistic Principal Surfaces.

Keywords: Informatics, Data mining

Back to top

　

　

　

Harold Szu

Vita: Harold Szu received in 1971 Ph. D. in Physics/Statistical Mechanics from Prof. George Uhlenbeck at the Rockefeller University, New York, N.Y. and worked at Naval Research Lab Wash DC in three Divisions, and became the Information Science Group Leader of Naval Surface Warfare Center at White Oak (1990-1996) and at Dahlgren VA(1997-now). 

He has been a Fellow of SPIE (since 1995) for neural nets, Fellow of OSA (1996) for adaptive wavelets, Fellow of IEEE (1997) for bi-sensor fusion, Fellow of Am. Inst. Med. & Bio. Eng. 2004 for medical image diagnoses.  Dr. Szu is a Champaign of brain-style computing for decades, a founder, former president, and a current governor of International Neural Network Society (INNS), He has contributed to the establishment of 30 SIG/Chapter worldwide.  Notably, he has scientifically contributed to the unsupervised redundancy reduction of sensory pair fusion by postulating the thermodynamic free energy for learning.  He received INNS D. Gabor Award, in 1997, “for outstanding contribution to neural network applications in information sciences and pioneer implementations of fast simulated annealing search," and Italy Academy the Eduardo R. Caianiello Award, 1999, for “elucidating and implementing a chaotic neural net as a dynamic realization for fuzzy logic membership function."  Dr. Szu is a foreign academician of Russian Academy of Nonlinear Sciences in 1999 for the homeostasis unsupervised learning based on constant Cybernetic Temperature.

Abstract: The tutorial presents unsupervised learning artificial neural networks (ANN), which yields unbiased computational intelligence (CI), information acquisition, feature extraction and knowledge discovery. Their applications in bioinformatics, brain study, unmanned robot control, finance and business are briefly presented. 

The necessary and sufficient conditions are that (1) all CPU Brain is always on all the time and is kept at an isothermal equilibrium temperature 37oC and (2) each CPU Brain is equipped with power of 5 sensors pairs e.g. two eyes, two ears, etc. altogether 10 dimensional vector time series for input data space.  The overwhelming incoming excitations must not be replenished, a natural choice is letting fluctuating disagreements so-called noise decay away, and only reinforce coincident signals so-called features.  This biomimetics is our minimum free energy principle of unsupervised feature extraction & fusion methodology. The tutorial proves these conditions (1) (2) constituted the basis of unsupervised learning ANN and real world applications in remote sensing and early tumor detection are given for computer scientists, neuroscientists, biologist, engineers and graduate students. 

Keywords: Unsupervised Learning, Computational Intelligence, Knowledge-based neural networks, Computational Neurogenetic modeling.

Back to top

　

　

　

Evangelia Micheli-Tzanakou, PhD
Department of Biomedical Engineering, RUTGERS UNIV. 

Vita: Dr. Tzanakou is Professor II and director of the Computational Intelligence Laboratories (2000-present); Chair of Biomedical Engineering Dept at Rutgers University (1990-2000);established the BME Undergraduate curriculum in 1999. Has an active research group and has  published over 250 scientific papers. Author/co-author of 2 books. Has graduated 37 MS and PhDs. 

Fellowships: IEEE Fellow (1992);Founding Fellow, American Institute for Medical & Biological Eng. (AIMBE), 1993; Member of Sigma Xi and Eta Kappa Nu; Listed in American Men and Women in Science; Honorary Member of the British Brain Research Association; Honorary Member of the European Brain and Behavior Society.

Awards: NJ Women of Achievement Award, 1995; Featured in the “Notable Twentieth Century Scientists, 1994; Achievement Award Recipient, Society of Women Engineers, 1992; Outstanding IEEE Branch Advisor/Counselor Award, 1985 ; cited as a Pioneer in the IEEE Web page (http://www.ieee.org/innovators).

Editorships: Editorial Board, IEEE Trans. On Nano-BioScience (2002-present); Editorial Board Biomedical Engineering On Line (2001-present ); Associate Editor, IEEE Transactions on Neural Networks (2000-present); Book Series Editor in Biomedical Engineering, Plenum/Klewer (1999-present); Book Series Editor, Biomedical Systems, IES Book series, CRC Press (1997-present);  Editorial Board, IEEE Trans. Biomedical Information Tech.(1997-2001); Editorial Board, International J. On Advanced Computational Intelligence (1997-1999); Associate Editor, IEEE Transactions on Neural Networks (1989-1992).

Abstract: One of the major problems a researcher faces is what is learned from data obtained by various methods and different techniques. This tutorial will discuss and compare topics such as: Statistical Advances and Challenges, as well as Feature Extraction in Computational Intelligence methodologies.

Often a simple model describes the data well, simply because the S/N ratio is too small for detection of more complex structures-which for example is the case with medical data involving human subjects. One has a lot of variability both in intra- and inter-sets of data.

Some important Simple Tools that have been used for a long time are: Linear Regression, Discriminant analysis, Principal Component Analysis etc. In all of these, the size of the data set matters. Huge data sets create memory problems. The question is how do we handle different data types and how do we handle them? What if the data are correlated? What if we have complex data structures?

In this tutorial you will learn more about Computational Intelligence, how to get to know your data and how to do Feature Extraction.

Some examples of "features" will be given and different feature extraction methods will be discussed.

Back to top

　

　

　

Bernard Widrow
Stanford University, USA

Keywords: cognitive memory, pattern recognition, applications

Back to top

　

　

　

Jerry M. Mendel

Department of Electrical Engineering Systems,

University of Southern California, USA

　

Vita: Jerry M. Mendel received the Ph.D. degree in electrical engineering from the Polytechnic Institute of Brooklyn, Brooklyn, NY. Currently, he is Professor of Electrical Engineering and Associate Director for Education and Outreach of the Integrated Media Systems Center at the University of Southern California, Los Angeles, where he has been since 1974. He has published over 430 technical papers and is the author and/or editor of eight books, including Uncertain Rule-based Fuzzy Logic Systems: Introduction and New Directions (Upper Saddle River, NJ: Prentice-Hall, 2001). His present research interests include: type-2 fuzzy logic systems and their applications to a wide range of problems, including target classification and computing with words, and spatio–temporal fusion of decisions. Dr. Mendel is a Distinguished Member of the IEEE Control Systems Society. He was President of the IEEE Control Systems Society in 1986, and is presently Chairman of the Fuzzy Technical Committee and a Member of the Administrative Committee of the IEEE Neural Networks Society. Among his awards are the 1983 Best Transactions Paper Award of the IEEE Geoscience and Remote Sensing Society, the 1992 Signal Processing Society Paper Award, the 2002 TRANSACTIONS ON FUZZY SYSTEMS Outstanding Paper Award, a 1984 IEEE Centennial Medal, and an IEEE Third Millennium Medal.

Abstract: This paper makes type-2 fuzzy logic systems much more accessible to fuzzy logic system designers, because it provides mathematical formulas and computational flowcharts for computing the derivatives that are needed to implement steepest-descent parameter tuning algorithms for such systems. It explains why computing such derivatives is much more challenging than it is for a type-1 fuzzy logic system. It provides derivative calculations that are applicable to any kind of type-2 membership functions, since the calculations are performed without prespecifying the nature of those membership functions. Some calculations are then illustrated for specific type-2 membership functions.

Keywords: Derivations, fuzzy logic system, type-2 fuzzy logic system.

Back to top

　

　

　

David Fogel
Chief Executive Officer, Natural Selection, Inc.

Vita: Dr. Fogel’s experience includes over 19 years of applying computational intelligence methods and statistical experimental design to real-world problems in industry, medicine, and defense. Prior to joining Natural Selection, Inc. in 1993, he was a Systems Analyst for Titan Systems, Inc. (1984-1988), and a Senior Principal Engineer at ORINCON Corporation (1988-1993). He received the Ph.D. in engineering sciences from the University of California at San Diego in 1992, and has subsequently taught undergraduate and graduate courses in evolutionary computation, stochastic processes, and statistical process control.

Dr. Fogel has over 200 publications in the technical literature, the majority treating the science and application of evolutionary computation. He is the author of six books, including Blondie24: Playing at the Edge of AI, 2002 , Evolutionary Computation: Toward a New Philosophy of Machine Intelligence (second edition), IEEE Press, 2000, How to Solve It: Modern Heuristics (co-authored by Professor Zbigniew Michalewicz of the University of North Carolina at Charlotte), Springer, 2000, and Evolutionary Computation: Principles and Practice for Signal Processing (SPIE, 2000) as well as the editor of Evolutionary Computation: The Fossil Record, IEEE Press, 1998, and co-editor-in-chief of the Handbook of Evolutionary Computation, Oxford University Press, 1997.

Dr. Fogel served as the founding editor-in-chief of the IEEE Transactions on Evolutionary Computation (1996-2002), which has a circulation of several thousand IEEE members and distribution in libraries internationally. He was the founding president of the Evolutionary Programming Society (1991-1993) and was elected a Fellow of the IEEE in 1999. He also serves as the editor-in-chief of BioSystems, is on the editorial boards of several other technical journals, and is Vice President of Membership Activities for the IEEE Computational Intelligence Society. Dr. Fogel served as the general chairman for the 2002 IEEE World Congress on Computational Intelligence, held in Honolulu, Hawaii, May 12-17, 2002, and was program chair for the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks, May 11-12, 2000, San Antonio, TX.

Dr. Fogel has received several honors and awards, including the 2002 Sigma Xi Southwest Region Young Investigator Award, the 2003 Sigma Xi San Diego Section Distinguished Scientist Award, the 2003 SPIE Computational Intelligence Pioneer Award, and most recently the 2004 IEEE Kiyo Tomiyasu Technical Field Award.

Abstract: Intelligence may be viewed as the ability to adapt behavior to meet goals in a range of environments, yet artificial intelligence has focused traditionally on replicating human behaviors in software. This approach has achieved some very visible successes, including for example, Deep Blue, the chess machine that defeated Garry Kasparov in May, 1997. The approach is limited, however, to address problems for which people already have the answers. In contrast, computational intelligence methods, such as evolutionary computing, can afford a computer with the ability to learn how to solve complex problems without relying on human expertise. A synergistic effect can be obtained by combining simulated evolutionary learning and human learning. Examples will be given in the areas of games, including checkers and chess, and other real-world applications in industry, medicine, and defense. Speculation on the future capabilities of these combined learning mechanisms will be offered.

Links: http://www.natural-selection.com/people_dfogel.html

Back to top

　

　

　

Gábor Horváth

Abstract: System modeling, system identification is an important way of investigating and understanding the world around. Identification is a process of deriving a mathematical model of a predefined part of the world, using observations. There are several different approaches of system identification, and these approaches utilize different forms of knowledge about the system. When only input-output observations are used behavioral or black box model can be constructed. In black box modeling neural networks play an important role. The purpose of this tutorial is to give an overview of the application of neural networks in system identification.
It defines the task of system identification, shows the basic questions and introduces the different approaches can be applied. It deals with the basic neural network architectures, the capability of neural networks and shows the motivations why neural networks are applied in system identification. The tutorial presents the main steps of neural identification and details the most important special problems, which must be solved when neural networks are used in system modeling. The general statements are illustrated by a real world complex industrial application example, where important practical questions and the strength and weakness of neural identification are also discussed.

Links: http://www.mit.bme.hu/~horvath/indexe.html

Back to top

　

　

Xin Yao

Abstract: Evolution has created wonders like ourselves. There is much to be learned from Nature. Evolutionary computation is the study of computational systems which use ideas and get inspirations from natural evolution. This tutorial introduces the basic ideas and concepts behind evolutionary computation. The relationship between various evolutionary computation techniques and some well-known techniques in artificial intelligence and computer science will be discussed.  The establishment of such relationships help to put evolutionary computation in a larger picture and help to better understand when, how and why evolutionary computation techniques work. This tutorial covers optimization, learning, design and theory in evolutionary computation.

Links: http://www.cs.bham.ac.uk/~xin/

Back to top

　

　

Witold Pedrycz

Abstract: We are concerned with the fundamentals and algorithmic aspects of Granular Computing regarded as a general framework of representing and processing information granules.

The talk offers a unified view at the commonly encountered formalisms of Granular Computing (including interval analysis, fuzzy sets, rough sets, shadowed sets), identifies their main features and links those to some selected and representative application areas. A particular attention is paid to the development of information granules and here we discuss a variety of possibilities including mechanisms of fuzzy clustering. We also demonstrate the role of Granular Computing in the general setting of Computational Intelligence.

Links: http://www.ece.ualberta.ca/~pedrycz/index.html

Back to top