Presented by
Christopher Edward Davis

---

[attachment=13252]
Abstract
This thesis outlines, discusses, and presents several arti¯cial neural network archi-tectures that are amenable to execution on a Graphical Processing Unit. Traitsare identi¯ed that lead to good or poor performance of the arti¯cial neural networksimulation.’
Introduction
There is nothing more di±cult to take in hand, more perilous to conductor more uncertain in its success than to take the lead in the introductionof a new order of things.Niccolo Machiavelli \The Prince" 1532
1.1 Overview
Arti¯cial neural networks have held the attention of researchers for over 60 yearsbecause of their simplicity, robustness, and similarity to biological structures presentin our own brains. Over the years, researchers have applied neural solutions to many¯elds; from medicine to weather forecasting, autonomous airplane control systemsand simplistic robot controls.Over the past 30 years, graphics cards have become useful and powerful processorsfound in most commercial o® the shelf computers. These computers can range from1Chapter 1. Introduction
console game systems to a commodity desktop, and if applied correctly are currentlycapable of serious computation.A GPU can be de¯ned as a single chip processor used for processing two andthree dimensional transform and lighting operations in the context of computergraphics. These processors are typically moderately parallel and contain severalALUs (Arithmetic Logic Units), several pixel pipelines (around 16), several vertexpipelines (around 6), and fast access to a relatively small amount of on card memory(32 to 512 MB). Modern 3-D graphics cards typically contain one or more of theseGPUs.In this thesis, simulation of arti¯cial neural networks (ANN) on commodity graph-ics processor units (GPU) is explored, measured, and analyzed to provide somegeneral observations about characteristics that make an ANN well or ill-suited tosimulation on the GPU.1.1.1 The ProblemGraphics processing units are being considered for many ¯elds of computation due totheir many attractive traits. One example ¯eld is ANNs. Arti¯cial neural networkso®er some attractive solutions to many real world problems. The ¯eld of arti¯cialneural networks is an area of computation that has been by in large unexplored incomparison to other ¯elds of GPU application. Because of these facts, the marriageof ANNs and the GPU would seem to be a worth-while area of research; to establishor invalidate particular architectures amenability to GPU simulation, to provide afair and unbiased comparison of CPU and GPU execution of these architectures, andto establish some general traits that make ANNs more or less amenable to simulationon a GPU.In this thesis, a brief history of GPUs and Arti¯cial Neural Networks is presented.2Chapter 1. IntroductionIntroductions to both GPUS and ANNs provide a basic overview of the informationneeded to understand the application of ANNs on GPUs. A survey of current GPUand CPU programming languages with ANN supporting features is presented andbrie°y discussed. Finally, a comparison between several ANNs implemented on boththe CPU and the GPU are presented, allowing analysis and generalization about theapplicability of ANNs on GPUs.3