Reconfigurable Streaming Architectures for Embedded Smart Cameras
Abstract
Smart cameras using FPGAs require an automation method
to simplify the design process and to ensure both
computation and memory performance are met.
Reconfigurable logic allows exploration of different
hardware accelerators and memory-hierarchy
configurations based on application needs. This paper
presents a streaming architecture template that is generated
from high level program descriptions. A smart camera
development platform, the software architecture, and
demonstration template are also described.
1. Introduction
Embedded smart cameras using computer vision methods
for video analysis [1,2] can benefit from reconfigurable
FPGA platforms to provide the necessary performance and
flexibility [3]. The levels of integration of modern FPGAs
have advanced to a point where all functions of a complex
System on Chip (SoC) can be mapped onto a single die.
FPGA manufacturers have embedded scalar processor
cores, multipliers, and SRAM memories in order to speedup
commonly used algorithms. They also offer peripherals,
fixed IP functions, and even synthesizable processor cores
for further customization. Architecture designs for smart
camera applications can be configured on the FPGA
platform to better optimize the memory subsystem and
computation structures.
This paper presents a streaming architecture template
which consists of hardware accelerators and memory
subsystem to support the computation and bandwidth
requirements. The design process consists of a stream
programming model with the familiarity of a high level
programming language. The hardware accelerators are
generated from user defined kernels while the streaming
memory subsystem is capable of automatic prefetching and
alignment. Following the design process allows a larger
segment of engineers that may not have expertise in system
architecture and hardware design to prototype on FPGAs.
Since particular hardware structures are abstracted out with
a software-only front end interface, application
development becomes less complicated.
Furthermore, applications related to video analysis are
often limited by bandwidth because of the imbalance
between processor and memory performance [4]. Even
though FPGAs continue to provide larger numbers of
configurable logic blocks that can be mapped to processing
elements to speed up computation, the interconnect delays
and slow memories can become bottlenecks. The
streaming model decouples the descriptions of memory
access sequences from the computation within a kernel,
thus making the customization of each of these two
components (computation and memory access) easier and
more amenable to optimization. The system is then
synthesized for an FPGA in a development kit with
integrated image sensors and peripherals for video
analysis.
The structure of the paper is as follows: Section 2
presents the related work relevant to this paper; Section 3
gives a brief presentation of the system architecture,
stream programming model, and architecture template;
Section 4 describes a smart camera development platform
and the associated software platform; Section 5 concludes
the paper.
2. Related Work
Stream processing is a computational model that
operates on sequences of ordered data (streams) using
computation kernels (filters) [5]. While both industry and
academia have studied the concurrency of computation and
data movement, this streaming model provides a new and
interesting framework that brings together both task and
data level parallelism within the same context. The
programmer explicitly defines the data accesses and
computation separately, thereby exposing concurrency and
locality for the compiler to schedule both in hardware.
The computation and data movement characteristics of
smart camera applications are a good match for the stream
model of computation. Making data movement explicit and
describing which portions of the application can be
computed in parallel enable compilers to optimize data
movement and match it to the available hardware
accelerators.
A number of streaming processor architectures have
been developed over recent years. Examples of stream
processors include RAW [6], Imagine [7], Merrimac [8],
and the RSVP™ architecture [9,10]. Stream processors are
similar to vector processors in their ability to hide latency,
amortize instruction overhead, and expose data parallelism
by operating on large sets of data. However, stream
Reconfigurable Streaming Architectures for Embedded Smart Cameras
Sek M. Chai, Nikolaos Bellas, Greg Kujawa, Tom Ziomek,
Linda Dawson, Tony Scaminaci, Malcolm Dwyer, Dan Linzmeier,
Embedded Systems Research, Motorola Labs,
(sek.chai[at]motorola.com)