Free PhD eTheses
1. A Modified Probabilistic Network for Signal
Processing and Pattern Recognition, by Anthony Zaknich, 1995. Download zipped pdf (1.7 MB)
2. A
Swept-Carrier Technique for Underwater Acoustic Communications, by Gareth J.
Cook, 1999. Download zipped
pdf (5.5 MB)
3. Approximate Dynamic Programming with Adaptive
Critics and the Algebraic Perceptron as a Fast Neural Network related to
Support Vector Machines, by Thomas Hanselmann, 2003. Download zipped pdf (4.8 MB)
4. Equalization of Nonlinear Communications
Channels with Practical Large Margin Classifiers, by James Ping Young, 2003. Download zipped pdf (1.7 MB)
A Modified Probabilistic Network for
Signal Processing and Pattern Recognition
Anthony Zaknich
The University of
This
thesis identifies some of the features and major limitations of current
artificial neural network architectures and learning laws and presents an
alternative class of practical non-parametric and semi-parametric networks which are quick and easy to train
and can be realised with present or foreseeable technology. An approach is
adopted which guarantees engineering solutions to a wide range of nonlinear
signal processing problems. This has been achieved by utilising as a basis an
artificial neural network structure which is theoretically well founded and
understood and easily realisable. The modified probabilistic neural network
structure has been invented and extended from the probabilistic neural network
classifier. The probabilistic neural network is derived from Bayes' theory and
Parzen non-parametric windowing techniques. The main problems with utilising
the probabilistic neural network are that it is only a classifier and that its
implementation requires a large memory storage and access for complex
problems. An adequate memory size can
not be predetermined for general network design. The modified probabilistic
neural network, however, is based on a semi-parametric approach and can be
implemented with a fixed memory size which is related to the resolution of the
data quantisation devices and the required accuracy. It is most similar to
Specht's general regression neural network and the network developed by Moody
and Darken and has significant similarities with Albus' cerebellar model
articulation controller and Kohonen's self-organising map. The generic aspect
of the work is shown by giving solutions to a number of applications to show
the utility and effectiveness of the modified probabilistic neural network.
Performance comparisons are made with other artificial neural networks and more
established methods including the multi-layer perceptron with
backpropagation-of-error, adaptive least mean squares linear finite impulse
response filter and nonlinear second and third order Volterra filters. These
comparisons demonstrate the applicability and implementation advantages of the
new artificial neural network architecture. Some optoelectronic and VLSI
implementation ideas are developed to show how easily these technologies can be
used to build modified probabilistic neural network hardware realisations. A
significant contribution of the thesis is the development of an effective and
practical general artificial neural network methodology and design for
application to a broad class of nonlinear time series signal processing
problems.
A Swept-Carrier Technique for Underwater
Acoustic Communications
Gareth J. Cook
The
Underwater acoustic channels present a number of challenges to
the designers of communications systems. One of the their most important
characteristics is multipath propagation, which severely limits data rate and
system stability.
This thesis addresses this problem of multipath propagation in
underwater acoustic channels through the proposal of a novel communications
signal. The signal was inspired by a spectral measurement known as time-delay spectrometry (TDS). It has a
form which identifies it with a class of signals previously defined as swept-carrier signals. The original
material presented in this thesis can be considered either as a special
application of TDS or an extension to earlier work in swept-carrier
communications.
TDS was originally devised as a means of measuring the frequency
response of loudspeakers in reverberant environments. As such, it has two
characteristics which the signals proposed in this thesis seek to exploit.
These are:
1.
multipath
suppression;
2.
system
identification
Under certain somewhat idealised conditions the problem of
multipath interference can be avoided through careful design of the
swept-carrier signal. Most of the theoretical results derived in this thesis
assume idealised suppression of multipath.
The spectral characteristics of the swept-carrier signal were
examined through use of convenient intuitive approximations and also through
rigorous mathematical analysis, resulting in expressions for the power density
spectrum. The error performance of the swept-carrier signal was determined for
a particular subclass of discrete multipath channels (which includes the additive white Gaussian noise channel).
The proposed communication technique was observed to be similar
in principle to the classical frequency
stepping approach commonly supported in many commercial acoustic modems.
The channel conditions under which the proposed technique would be considered
appropriate coincide with the conditions normally associated with the frequency
stepping approach. It was shown that in certain discrete multipath channels,
the swept-carrier technique is superior to the classical approach, both in
terms of error performance and bandwidth efficiency.
An experimental communications system was developed which made
us of largely non-specialised hardware. The successful trial of this system
revealed the suitability of the technique for practical implementation. The
field work also incorporated unusual sounding channel techniques, using chirp
signals, and a TDS-based transducer calibration.
In cases where some level of multipath interference cannot be
avoided the ability to generate and exploit channel estimates from the
swept-carrier signal becomes important. These issues were explored resulting in
a novel receiver design heavily influenced by TDS theory. A computer simulation
of this receiver showed a performance improvement over simpler receivers which
do not incorporate TDS-type channel estimates in the detection process.
Approximate Dynamic Programming with
Adaptive Critics and the Algebraic Perceptron as a Fast Neural Network related
to Support Vector Machines
Thomas Hanselmann
The
This thesis treats two aspects of intelligent control: The first
part is about long-term optimization by approximating dynamic programming and
in the second part a specific class of a fast neural network, related to
support vector machines (SVMs), is considered.
The first part relates to approximate dynamic programming,
especially in the framework of adaptive critic designs (ADCs). Dynamic
programming can be used to find an optimal decision or control policy over a
long period. However, in practice it is difficult, and often impossible, to
calculate a dynamic programming solution, due to the ‘curse of dimensionality’.
The adaptive critic design framework addresses this issue and tries to find a
good solution by approximating the dynamic programming process for a stationary
environment.
In adaptive critic design there are three modules, the plant or
environment to be controlled, a critic to estimate the long-term cost and
action or controller module to produce the decision or control strategy. Even
though there have been many publications on the subject over the past two
decades, there are some points that have had less attention. While most of the
publications address the training of the critic, one of the points that has not
received systematic attention is training of the action module. Normally,
training starts with an arbitrary, hopefully stable, decision policy and its
long-term cost is then estimated by the critic. Often the critic is a neural
network that has to be trained, using a temporal difference and Bellman’s
principle of optimality. Once the critic network has conveyed, a policy
improvement step is carried out by gradient descent to adjust the parameters of
the controller network. Then the critic is retrained again to give the new
long-term cost estimate. However, it would be preferable to focus more on
extremal policies earlier in the training. Therefore, the Calculus of
Variations is investigated to discard the idea of using the Euler equations to
train the actor. However, an adaptive critic formulations for a continuous plant
with a short-term cost as an integral cost density is made and the chain rule
is applied to calculate the total derivative of the short-term cost with
respect to the actor weights. This is different from the discrete systems,
usually used in adaptive crtitics, which are used in conjunction with total
order derivatives. This idea is then extended to second order derivatives such
that
Some other, more minor issues, in the adaptive critic framework
are investigated, such as the influence of discounting factor in the Bellman
equation on total ordered derivatives, the target interpretation in
backpropagation through time as moving and fixed targets, the relation between
simultaneous recurrent networks and dynamic programming is stated and
reinterpreted of the recurrent generalized multilayer perceptron (GMLP) as a
recurrent generalized finite impulse (GFIR-MLP) is made.
Another subject in this area that is investigated, is that of a
hybrid dynamic system, characterized as a continuous plant and a set of basic
feedback controllers, which are used to control the plant by finding a
switching sequence to select one basic controller at a time. The special but
important case is considered when the plant is linear but with some uncertainty
in the state space and in the observation vector, and a quadratic cost
function. This is a form of robust control, where a dynamic programming
solution has to be calculated. Due to the special form of, the recursive
dynamic programming solution can be approximated by a certain form of an
adaptive critic design, sometimes called Q-learning. However, extra care has to
be taken to avoid instability due to the approximation errors and the recursive
procedure of dynamic programming, which tends to considerably amplify errors.
The problem of fast learning with limited data is addressed in
the second part. This area has been investigated for many decades but only
recently started to blossom in full with Vapnik’s statistical learning theory
and a class of algorithms, the so-called support vector machines, which make
use of very high dimensional feature spaces and linear separations therein. In
this thesis, the special case of binary pattern recognition is investigated. An
algorithm, called the algebraic perceptron algorithm is introduced. It extends
the well-known perceptron algorithm to achieve a linear dichotomy in high
dimensional spaces of dimensionality such 350 and which represents
polynomial curves in the input space. This is achieved through inner product
kernels, similar to support vector machines. However, in contrast to SVMs, the
algebraic perceptron will not find an optimal solution to separate the classes.
Nevertheless, it can be optimized towards an optimal solution, if necessary.
Sometimes, especially with many densely placed data points, it can even achieve
a better solution than the theoretically superior optimal support vector
machines whose solutions is often tricky to calculate for large data sets.
There are many interesting geometrical interpretations and
possibilities that can be used to extend the algebraic perceptron. One such
application suggested is to use it as a tool to decompose more complicated
objects into simpler ones. This potential is demonstrated on an artificially
created binary object of overlapping ellipses.
Equalization of Nonlinear Communications
Channels with Practical Large Margin Classifiers
James Ping Young
The
The subject of this thesis is about particular methods that make
solving the problem of communications channel equalization more manageable. The
thesis follows the popular choice of using neural networks for dealing with
nonlinear input-output problems. The two main neural networks investigated were
the Modified Probabilistic Neural Network (MPNN) and the Algebraic Perceptron
(AP).
The MPNN is modelled directly from the Bayesian estimation
technique. When employed to equalise communications channels it was found that
with appropriate settings noise effects can be reduced and that the network
size can be quite lean for some problems. In this thesis, further improvements
to the MPNN method are proposed to make it more suitable for use in
communications environments. First of all, the MPNN is developed to process
complex signals. Secondly, a stochastic gradient method is proposed so that the
MPNN is able to self adapt to changing channel characteristics. Thirdly, a
method which further reduces the size of the network is proposed, because the
size of the network is directly proportional to implementation cost. Lastly,
the MPNN is shown to be able to be tuned to possess the same quality as an
optimal equalizer. The MPNN can serve as a performance benchmark in determining
the usefulness of the other equalizers.
The second focus of the thesis is on the AP. The AP is a binary
kernel classifier. The advantages of the method are its ability to obtain a
solution fast during training and its slimline structure which allows fast
run-time implementation. The downside is that performance is unstable due to
arbitrariness in the construction of its decision boundaries. An improvement to
the method is proposed to reduce the level of arbitrariness. The goal of the
proposed improvement is two fold. It is to increase the margin between the patterns
and the separating hyperplane and at the same time to keep the size of the
network small. The resulting development is given the name Algebraic Perceptron
with Large Margin (APlm). The APlm is compared with the Support Vector Machine
(SVM) and the Sequential Support Vector machine (SVMseq) in the problem of
communications channel equalization. The SVMseq is an approximate
implementation of the SVM. It is, in principle, identical to the more popular
technique named the Kernel Adatron (KA). It is found that the APlm possess many
desirable qualities and is superior in terms of implementation ease,
computational load, robustness and even performance.
In general, it is found that both the APlm and SVMseq can be
well suited to the problem. However, there is room to make the technique even
more versatile. One way is to incorporate the ability to adapt to changing
channels. In the thesis, Decision-directed self adaptation is combined with the
batch training technique to allow the property of large margin to coexist with
self adaptation. In this way the advantage of having a large margin is retained
even when the equalizer adapts. The other development is to structure the
nonlinear classifiers to accommodate decision feedback. By adopting the
decision feedback structure, difficult channels do not have to be solved by
progressively increasing the input dimension of the equalizers. Therefore, the
performance of the equalizer can be improved without requiring a drastic amount
of additional computational load.
