My Personal Research
Aspirations
I
was initially interested in finding a general approach to the solution of
difficult nonlinear real world problems, especially underwater acoustic signal processing
problems. It was this drive that motivated me to develop the Modified
Probabilistic Neural Network (MPNN). The MPNN is a very neat neural network
with a parallel architecture specially designed for the processing of spatial
and time series signals. It is a practical network which is easy to understand,
use and train whose architecture is simply constructed from the signal waveform
character or signal dynamics. The MPNN has only one tuning parameter whose
adjustment from zero to infinity introduces a continuum between a nonlinear to
linear network character. This led to the thought that there may be a general
approach to both linear and nonlinear systems where linear systems can be
treated as a subset of nonlinear systems. I became much more interested in how
(intelligent?) information is encoded in evolving and self-replication systems
and how that may have contributed to the complexity we find in the world. I
began to see how these concerns could be addressed using Intelligent Parallel
Processing Systems (IPPS) and by studying the dynamic processes and patterns of
Information in Time and Space (ITS). The ultimate goal then became to invent a
broader generic framework to my interests in signal and information processing
through the further development of MPNN theory and what I have called
hyperspace processing. Hyperspace processing is simply processing in the
hyperspace created by the MPNN architecture, which actually embodies signal
dynamics information as knowledge centres. When I came across Vapnik’s Support
Vector Machine (SVM) I immediately recognised that it was the theory that could
tie my ideas together. The SVM’s very high dimensional hyperspace was very
similar to that of the MPNN. Furthermore, in the SVM hyperspace, classification
and regression problems can be solved using optimal linear techniques, and the
set of SVM support vectors, which represent kind of essential knowledge
centres, drop out automatically. What a marvellous thing! Here was not only a
possible solution to my quest of turning nonlinear problems into optimal linear
ones, but also it may even offer a mechanism for generalisation, characteristic
of real brains.
Eventually,
I developed what I call the Integrated Sensor Intelligent System (ISIS) model,
an offshoot of my Tuneable Approximate Piecewise Linear Regression (TAPLR)
model. This was a simple extension of the MPNN made up of a set of smoothly
merged parallel piecewise linear (actually affine) models, significantly
increasing the efficiency of the MPNN structure. It then occurred to me if I
made all these parallel linear sub-space models adaptive I would have a very
nice learning structure that could model nonlinear or very complex linear
processes. I could apply linear theory to each sub-space model, and since each
of them is decoupled from the rest it also solves the stability-plasticity
dilemma. I call this the Sub-Space Adaptive Filter (SSAF) model. The SSAF has
significant application potential for underwater acoustic signal processing,
audio system modelling and dynamic equalisation.
If
you have followed this tormented lineage of wishful thinking you may well have
suspected by now that this in turn led to yet another thought. Maybe I can
build a self-organising system which "learns" from the information it
is exposed to and then continues to order itself according to the evolving
patterns it creates to higher orders of abstraction. The knowledge centres in
the MPNN architecture are true parts of the input signals but depending on what
constraints are placed on their selection it will determine the possible
patterns and configurations that they can take at higher levels. It is a bit
like when we take in information from the world and we put labels on what we
may initially consider to be significant separate parts of that world. We then
not only abstract taking those parts as valid but we are affected in our view
and interpretation of further information by those labels and resulting logical
abstractions. The problem is that in fact there are no real independent parts
in the world; we only make them so in our minds. If after we take in so much
information and then label it and we don't allow any further input it may well
be possible to develop a self-consistent system and valid abstractions. But of
course who cares, the system is closed, dead and of no interest because it is
arbitrary and finite. You may know people like that! What this indicates to me
is that although this process can be very useful it is only so in so far as the
process produces a self-consistent structure with lowest possible incoherence
to new information. There must always be some incoherence because the system
can never contain all there is to know. If there is increasing incoherence it
may be possible to reduce it by local adaptation or there may be a need to
completely re-evaluate the labelling constraints and/or the abstractions and
reconfigure itself globally. So a self-organising system to be able to
"survive" it must be able to adequately keep applying its process of
information processing as information continues to come in, even though it may
have to go through revolutionary reconfiguration from time to time. This can be
done quite well with standard adaptive engineering systems in fairly
constrained applications where the system is designed a priori to cope with the
range of expected inputs but it gets much more difficult in more complex and
open situations. I along with every other self respecting researcher in the
field believe that I may have an angle on this problem, so I feel compelled to
pursue it until the revolutionary reconfigurations become too many and too
often, indicating that I'm probably deceiving myself. At that point there are
always solitary mountain tops and flutes. It’s a pity I can't play the flute,
stand the cold or resist a willing ear so I will have to press on regardless,
hoping to blaze a path where there has never been one before. But of course one
can always philosophise endlessly giving the impression of progress and so long
as you keep moving quickly enough you may not get shot down too soon. Sigh,
where is Wonder Woman when you need her?
