THRESHOLD LOGIC IMPLEMENTATIONS: THE EARLY DAYS

[attachment=16005]


I. INTRODUCTION
Research on neural networks (NNs) goes back sixty years.
The seminal year for the “science of mind” was 1943 when
the paper of McCulloch and Pitts was published [58]. They
introduced the first mathematical model of a neuron
operating in an all-or-none fashion: the threshold logic (TL)
gate (TLG). It computes the sign of the weighted sum of its
inputs: f(X) = sign (Σ i = 1…n wi xi – θ), wi being the synaptic
weight associated to xi, θ the threshold, and n the fan-in.


II. VARIOUS TECHNOLOGIES
In 1951 Marvin Minsky teamed with Dean Edmonds and
designed the first 40-neuron “neurocomputer” Snark [60]. It
was an electro-mechanical implementation using tubes,
motors, and clutches, and successfully modeled the behavior
of a rat searching for food in a maze. In 1957 Frank
Rosenblatt generalized the McCulloch-Pitts neuron
inventing the perceptron [70]. During 1957-1958, Rosenblatt
together with Charles Wightman and others constructed the
Mark I Perceptron having 512 adjustable weights (an 8×8×8
array of potentiometers). Shortly afterwards, Bernard
Widrow together with his students developed the ADALINE
(ADAptive LINear Element) [84]. They used an electrically
adjustable resistor called a memistor, also founding the first
neurocomputer hardware company: Memistor Corporation,
producing neurocomputers during the early to mid 1960s.
For a general overview of the evolution of the field of neural
networks during those times the interested reader should
consult [66], [16].


III. A HISTORICAL PERSPECTIVE
The summing amplifier from 1946 [77] precedes even [58]
by submission date: May 1, 1941. It seems to be the first
hardware implementation of a TLG. The patent details an
“electrical calculating device […] for obtaining the sum of a
plurality of electrical voltages” using “an electrical
amplifier having a feedback. […] by adjustment of the
impedances connected in series with the various sources of
voltage, any one or more of the sources may be, in effect,
multiplied by any desired factor […]” (Fig. 1. (a)). Many
other implementations followed.


IV. CONCLUSIONS
This paper has presented a brief account of the results on
hardware implementations of TL obtained starting from the
forties. It is quite amazing how much effort, ingenuity, and
tenacity has been spent/invested over several decades to
make TL a success, let alone the remarkable diversity of
solutions and the numerous technologies which have been
experimented with. Except a few very early solutions, all the
others have mainly tried to cope with the major drawbacks
of TLGs: power dissipation, reduced noise margins, and
sensitivity to process variations