Determining the computational nature of animate intelligence
is perhaps the greatest single challenge facing the field
of computer science.
Many approaches are possible, and
will no doubt be necessary if this riddle is to be solved.
These range from the study of biological nervous systems, to
analysis of the tasks that creatures perform - and frequently
include a constructive component in which a machine is
built or programmed to exhibit behavior that is in some
sense intelligent.
Here a distinction emerges.
When the focus is entirely on successful performance of a
particular task, one is led to consider designs that
in all likelihood shed little light on the central question
above. In its pure form this pursuit is then the study of
*artificial intelligence*.
That is, machines and
software engaged in a masquerade - resembling in some
respects the object of their emulation, but made of essentially
different *stuff*.
By contrast the study of *synthetic intelligence* strives
towards constructions that while man-made, are nevertheless
intelligent in the natural and animate sense.
Of course in practice
the distinction between synthetic and artificial is often
hard to make, and really represents a difference in research
direction and motivation. Moreover, while we may seem to attach
a pejorative connotation to ``artificial'' in the discussion
above, it certainly may be that machines will
emerge from this work that are unquestionably intelligent in
a deeply different sense from that with which we are today familiar.
Nevertheless, our main interests lie along the synthetic direction.

Natural intelligence is impressively robust in the presence of
noise and uncertainty. One view is that such factors should
be dealt with by an *outer wrapper* of techniques,
exposing an inherently discrete and symbolic problem at the
core. Indeed there is a long history of investigators
focusing on problems such as theorem proving and logical
reasoning, while avoiding the less easily framed problem of
robustness. Our view is that the manner in which nature deals
with this more elusive problem may represent the central idea
behind intelligence, not just a front-end noise filter.

Stochastic modeling techniques represent a formal approach to the problems of noise and uncertainty - and have been somewhat successful when applied to difficult problems such as speech recognition and other signal and image processing tasks.

Chapter 2 introduces the Finite Growth Model (FGM)
framework which spans many existing model classes and opens
up important new possibilities.
Among these is the notion
of stochastic transduction in which a machine converts
one observation into another. The probability of
transduction between two objects can be thought of as
an indication of their similarity.
A characteristic of natural intelligence is its use of
nontrivial metrics, i.e. notions for similarity.
Another striking feature is that these metrics are sometimes
learned.
Both of these are possible within the stochastic transduction
paradigm. We remark that the well-known concept of string edit distance may
be viewed as a single state memoryless transducer, and our work
provides a convenient way to optimize its cost parameters.
Speech recognition may be viewed as a grand transduction from
signal to text. Chapter 3 describes experiments
that represent a first step towards approaching the problem
using this formalism.
Finite growth models also include the class of hidden Markov models
and stochastic context free grammars
which can provide a means to discover hidden structure in a
a set of observations. This corresponds to another salient
characteristic of natural intelligence.
FGMs also allow the model designer to cope with the
learning-theoretic considerations of overtraining and
generalization by building data-appropriate models resulting
from optimization criteria such as that of minimum description
length (MDL) or the maximum *a posteriori* probability estimate (MAP).

While interesting and perhaps practical, the stochastic
modeling techniques we introduce would seem to be more
*artificial* than synthetic - since a common language
for this field is that of linear algebra, an unlikely component
of our biological endowment. Beyond this observation, it
seems unlikely that nature would confine herself to the use
of strict probabalistic models.

An important contribution of chapter 2 is the presentation of FGMs, and by inclusion many specific stochastic model classes, in terms of weighted graphs and a related optimization problem. These constructions need not represent causal probability models - or probabilities at all. Nevertheless the celebrated Baum-Welch and EM algorithms are shown to still apply, and we argue that their essential message is one of decomposition. That is, breaking up a particular graph-based global optimization problem into a set of local problems such that progress on the local problems will necessarily advance the global objective. These observations paint FGMs in a far more connectionist light and it is for this reason that we see it as at least plausible that nature might employ related principles.

Chapter 4 sketches the design of a software library
and language for FGMs. Experimentation in
expedited by effective tools, and we argue that our design
should be viewed as an *assembly language* for computational
hidden-state stochastic modeling.

The material of chapters 5 and
6 is of a more esoteric and perhaps artificial
nature. Chapter 5 exposes a fascinating and
somewhat counterintuitive degeneracy in the relationship
between the prior and *a posteriori* distributions arising
from a mixture of normal densities. This degeneracy is then
exploited to prove a reparameterization theorem that provides
a modicum of theoretical justification to learning approaches
that proceed by reweighting the input pattern set.
In chapter 6 we discuss approaches
to the learning of continuous context models.

We submit that a stochastic and information-theoretic paradigm for intelligence is emerging; although it is not clear whether the insights it generates pertain to the synthetic or only to the artificial face of the problem.

2002-06-24