Neural Network Hardware

Latest revision: 13 Nov 1998

When implemented in hardware, neural networks can take advantage of their inherent parallelism and run orders of magnitude faster than software simulations. There exist a number of commercial NNW chips and neurocomputers, VME boards, etc., as well as systems built by HEP groups for on-line trigger applications. We list here some of the neural network chips, boards, and systems that are available commercially, are almost commercial, or are just of special interest to those in HEP.

For an overview of hardware neural networks, see the hypertext version or postscript version of the paper ( Lindsey, Lindblad 1994). Also, check out the FAQ on Neural Networks p.7 - Hardware NNW.

Smart Technologies SIG site (formerly Cognizer Almanac) products site lists many companies big and small in NNW's and AI.

Also, DTI Neurocomputing Web has a directory of NNW companies.

See also the publication by Aybay I, Cetinkaya S., and Halici U., "Classification of Neural Network Hardware", Neural Network World, 1/96, pp.11-27 (no web version unfortunately) which has over 90 references.

March 1998 - Online lecture reviewing the status of hardware neural networks at http://www.particle.kth.se/~lindsey/HardwareNNWCourse/home.html.

Note 1: There have been a great number of prototype chips built over the years, typically by university EE groups, and presented at NNW conferences (e.g. see the IJCNN Proceedings). Very few of these, however, have reached commercialization and we have not listed such chips here unless they were specifically targeted for HEP applications.
Note 2: Although we have prices on some of these systems, we generally do not include them here. Prices are subject to considerable change and academic groups can often get substantial discounts anyway. Contact the vendors for current pricing.
Note 3: The information here was obtained from 1993 till current year from various sources, including direct contacts with the vendors, conference proceedings, magazine articles, etc., and should be roughly up to date. However, no effort was made to contact every single vendor to guarantee that all the items here are still available. In fact, some of the smaller companies themselves may no longer exist. Contact the vendors for latest info.
Note 4: * indicates hardware that is no longer available commercially but are still listed here for historical purposes.? -probably not available
but no definitive information yet.

Accurate Automation Corp(AAC): Description: From ACC press clippings: "Accurate Automation's Neural Network Processor (NNP) uses a true multiple-instruction multiple-data (MIMD) architecture capable of running multiple chips in parallel without performance degradation..."
Adaptive Solutions CNAPS: Description: The CNAPS system is a full NNW development system based on the proprietary CNAPS-1064 Digital Parallel Processor chip that has 64 sub-processors operating in a SIMD mode. Each sub-processor has its own 4k bytes of local memory and a fixed point arithmetic unit to perform 1-bit, 8bit, or 16-bit integer arithmetic. Each sub-processor can emulate one or more neurons and multiple chips can be ganged together.
HNC 100 NAP Chip and SNAP Neurocomputer * No longer available: Description: The HNC 100 NAP (Numerical Array Processor) chip contains a 1-D systolic array of 4 arithmetic cells in SIMD ring. The processing is 32 bit floating point. A 17-bit address bus allows maximum of 512Kbytes of off-chip memory for each cell. A single NAP chip can perform 160 32-bit MFLOPS.
IBM ZISC036: Description: The ZISC is a digital chip with 64 8-bit inputs and 36 radial basis function neurons. Multiple chips can be easily cascaded together to create networks of arbitrary size. Input vector V is compared to stored a prototype vector P for each neuron. The 14-bit neuron output (for 16K possible categories) is a distance value calculated according to 2 selectable norms: (1) dist = sum of abs(Vi-Pi), i=1,64; (2) dist = Max((Vi-Pi)), i=1,64. Input is 8-bit serial. It takes 3.5microsecs to load the 64 elements and another 0.5microsecs for the classification signal to appear.
datafactory (formerly INCO) SAND (Simple Applicable Neural Device) Neurochip & NeuroLution PCI Board: Description: "SAND/1 (Simple Applicable Neural Device) is a cascadable, systolic processor array designed for fast processing of neural networks. Feedforward networks, RBF networks and Kohonen feature maps can be mapped onto the neurochip SAND/1. These most common types of neural networks covers about 75% of all important applications. The performance of a single SAND/1 chip is 200 MCPS (million connections per second) due to four parallel 16 bit multipliers and eight 40 bit adders working in one clock cycle. The clock rate of SAND/1 is 50 Mhz.". The chip was developed by the Research Center Karlsruhe and Institute for Microelectronics, Stuttgart (IMS). It is being commercialized by datafactory of Germany.
Innovative Computing Technologies: Description: "Our Feedback Neural Net Chips deliver execution speeds equivalent to six billion connections per second with on-chip learning capability. Unlike its feedforward counterpart, the net can be trained at speeds up to 20 MHz. This speed allows real-time scan binary images for any desired set of templates at video rate. Optimized circuitry delivers this performance with very low power consumption, less than 100mW."; from one of their WWW pages.
Intel 80170NX Electrically Trainable Analog Neural Network (ETANN): Description: Analog neural network with 64 inputs (0-3v), 16 internal biases, and 64 neurons with sigmoidal transfer functions (external gain control available.) Two layer feedforward networks can be implemented with 64 inputs, 64 hidden neurons, and 64 output neurons using the two 80x64 weight matrices. Hidden layer outputs are clocked back through 2nd weight matrix to do the output layer processing. Alternatively, a single 64 layer network with 128 inputs can be implemented using both matrices and clocking in 2 sets of 64 inputs. Weights are stored in non-volatile floating gate synapses (Gilbert analog multipliers). Weights have roughly 6-bit precision.
Irvine Sensors 3DANN: Description: Irvine Sensors has developed a high density FET stacking technology that allows circuits to be built in the vertical as well as; the planar dimensions. Using this stackable circuits, NNWs , which they refer to as 3DANN - 3D Artificial Neural Networm, of very; high density can be built. The company in Sept. 19998 won a US Army contract to "demonstrate the feasibility of an ultra high; density interconnect to enhance its 3D Artificial Neural Network(TM) (3DANN(TM)) technology. The new interconnect is part of; Irvine Sensors' planned progression of technologies intended to eventually lead to a "Silicon Brain", a recognition system conceived to; emulate performance of the human central nervous system." The VIP/Balboa Image Processor uses
Micro Devices MD-1220 * No longer available: Description: The MD-1220 is a CMOS VLSI device with eight internal neural circuits, each processing the synaptic data in parallel. Data after weight * input is stored in accumulators as partial dot products. At the end the of the input frame, the data is applied to a threshold function. The internal accumulator provides 16-bit resolution and therefore the processing or connection rate is 16 clock cycles. Frequency is 20 MHz. Example: 64 neurons processing at 1.25 Mbits/sec * 64 = 80 million connections/sec. Another example: It takes 800 ns to process any synaptic input (or bias). In the interconnect mode all eight inputs + bias are processed before outputting result. This means 7.2 us. In the other mode it takes 12.8 us. So a 3 layer nnw 32 x 32 x 8 takes (32 + 1)*800 for each layer = 79.2 us Threshold functions could be hard limiter, threshold limiter, sigmoid or clamped linear.
MCE MT19003 Neural Instruction Set Processor: Description: A digital processor chip using signed 12-bit internal neuron values, with 16-bit multiplier and 35-bit accumulator. Network input values, bias values, synapse, and neuron values are held in off-chip memory. Network processing is also guided by a given program in off-chip memory using 7 element neural instruction set. Neuron values can be scaled by a transfer function using 4 available tables.
Go to top of page
RC Module Neuroprocessor NM6403: Description: "NM6403 is a high performance microprocessor with super scalar architecture. The architecture includes control unit, address calculation and scalar processing units, node to support vector operations with elements of variable bit length. There are two identical programmable interfaces to work with any memory types as well as two communication ports hardware compatible with those of DSP MS320C4x which permit to build multi-processor systems..." See NM6403 home page.
National Semiconductor NeuFuz/COP8 Microcontrollers: Description: This system uses a combination of neural network and fuzzy logic software to generate code for National's COP8 microcontrollers. Fuzzy logic provides a very flexible and powerful method for control applications. In cases where the "if-then" rules of the system are known, the rules can be expressed in a systematic and straightforward way using the fuzzy logic membership function and fuzzy rule methodology. However, there can be many situations where one doesn't know a priori the "if-then" rules. In such a case, a neural network can be used to learn the rules and then the neural network's "knowledge" can be mapped into the fuzzy rules and membership functions. National has several packages:
Nestor NI1000: Description: This is a network with Radial Basis Function neurons. During learning, prototype vectors are stored under the assumption that they are picked randomly from the parent distribution. Up to 1024 protypes can be stored, with 256 dimensions,5-bits per dimension. Each prototype is then assigned to a given middle layer neuron. This neuron, in turn, is assigned to an output neuron that represents the particular class for that vector (up to 64 classes allowed). All middle layer neurons that correspond to the same class are assigned to the same output neuron. In recall mode, an input vector is compared to each prototype, in parallel, and if the distance between them is above a given threshold, it fires, which in turn fires the corresponding output, or class, neuron.
NeuraLogix (Now Adaptive Logic) NLX110, NLX230, NLX420 * No longer available: Description: NLX110 Eight Channel Pattern Comparator - compares 8 unknown patterns to 1 reference pattern or compares 8 references to 1 unknown pattern. A built in neural net enhances performance. Pattern lengths can be up to 1Mbit. Each channel operates at 20MHz. May be expanded to arrays of 32 wide. Hamming or Euclidian distance metrics.
NeuriCam NC3001 (TOTEM) chip: Description: Developed jointly by IRST and the University of Trento, under a grant by INFN and the European Union, the NC3001 (TOTEM) digital processor uses a SIMD architecture. It has 32 multiply-and-accumulate processors using 2's complement arithmetic. Data is represented by 16 bits for inputs, 8-bit for the weights, and 32 bits for the results. Each internal processor has local storage for up to 128 weights to reduce the I/O bandwidth required at run-time. Multiple chips can be combined to build larger networks.
Oxford Computer Chips and Modules ?: Description: On-board Learning Chip (OBL) is a digital, programmable chip that combines memory and processing in one chip. OBL implements any nnw learning algorithm and transfer function on the chip. A modular design for parallel processing available.
Philips L-Neuro chips: Description: Philips has previously offered the L-Neuro 1.0 chip and has now announced development of the follow-on L-Neuro 2.3 chip.
Sensory Circuits RSC-164 Speech Recognition Chip: Description: "The RSC-164 is a low-cost speech recognition IC designed for use in consumer electronics. It combines an 8-bit processor with neural-net algorithms for high-quality speaker-independent and speaker-dependent speech recognition. The chip also supports voice record/ playback, music synthesis, speech synthsis and system control. The CMOS device includes on-chip RAM, ROM, 16 general-purpose I/O lines, A/D and D/A converters and a 4-MIPS dedicated processor.
Siemens MA-16 chip, SYNAPSE-3 Neurocomputer: Description: The Siemen's MA-16 chip is a fast matrix-matrix multiplier that can be combined to form systolic arrays, i.e. inputs and outputs are passed from one module to another in an assembly line manner. A single module can process 4 patterns, of 16 elements each (16-bit), with 16 neuron values (16-bit) at a rate of 800 multiply/accumulates/sec at 50Mhz. Weights are loaded from off-chip RAM and neuron transfer functions are calculated with off-chip look-up-tables.
Synaptics I10XX Object Recognizer Chip: Description: A customizable chip that includes an area imager, 2 nnw architectures, and a digital interface. It can be used to create object recognizers like characters, patterns or defects, and can process up to 50k images per sec.
UCLi Ltd. pRAM-256 Chip: Description: "The pRAM-256 is a versatile neural network processor with an on-chip learning unit. It offers the flexibility of a software solution with the speed of hardware. Connections between the pRAM neurons are reconfigurable which allows a network's architecture to be modified at any time. The pRAM-256 can complete one pass of the training process, training all 256 pRAMs, in less than 0.25 ms when operating at the maximum clock speed of 33 MHz. Because of the high number of pRAMs supported by the pRAM-256, a typical neural network can be built using a single pRAM Module. Several pRAM Modules can operate in parallel so that larger networks can be built. The pRAM-256 is fabricated using an advanced sub-micron gate array semi-custom technology from GEC Plessey Semiconductors. The use of a 68 pin PGA package allows a compact neural network to be built into existing and future systems. Interfaces to EISA and VME bus systems have been defined." - from the pRAM WWW page.
Other: Description: Miscellaneous commercial hardware

BrainMaker Accelerators: Description: BrainMaker Accelerator - nnw development system that operates at 3 million connections/sec. Includes BrainMaker software plus ISA-bus 20MHZ DSP board based on the TMS320C25 chip.
Current Technology MM32k: Description: The MM32k is a SIMD parallel computer with 32768 bit serial processing elements, each of which has 512 bits of memory. A single chip holds 2048 PE's. All PE's are inteconnected via a switching network. The MM32k is implemented on a single ISA bus PC card. It comes with a host pc programming environment, including a C++ class library which overloads typical arithmetic operators and supports variable precision arithmetic. Comparison of RBF, Kohnonen and BP processing to a i486 66Mhz show speedups by factors of 161, 76 and 336, resp.; to a 150 MHZ Alpha AXP 31, 11, 35,resp. It is suitable for other problems such as image processing. It costs $5000."
HNC Balboa * No longer available: Description: Balboa Developer's system is a hardware and software system that provides up to 80-megaflop performance and 20 nnw architectures. Boards include an AT card using a proprietary NN chip based card and a VME DSP based card.
IBM ZISC/ISA Accelerator for PC: Description: A card with 16 ZISC036 chips has been built for the ISA bus. The chips appear as one ZISC with 576 prototypes. The card comes with a C language library for accessing all the card, training, recall, etc. See the IBM Essonnes Lab ISA Card page.
IBM ZISC PCMCIA card + SIZM Single Inline Module: Description: A card with 3 ZISC036 chips has been built for the PCMCIA or PC Card slot. The chips appear as a single ZISC with 108 prototypes. A Single Inline Module similar to single inline memory modules (SIM)is now available also. On a single module are 6 ZISC chips, and multiple modules can be ganged together. The Inline module is described at http://www.ibm.fr/france/cdlab/sizm.htm.
Mosaic QED Board: Description: This is a general purpose board that can be configured for neural network development. It has digital I/O, 16 analog-to-digital conversion channels, and serial communications. The 3.2x4 inch board includes software support.
Neural Technologies - NT5000 & NT6000: Description: NT5000 is a standalone system with co-processor that is initialized and trained via a connection to a PC but from then on can run independently. The networks can have up to 600/4000 neurons (regular or turbo versions). Internal processing is digital but I/O is analog via DAC's and ADC's.
NeuroDynamX Neural-Accelerators ?: Description: NDX Neural Accelerator XR25 - ISA-bus card with Intel i860 XR RISC processor running at 25MHZ to give up to 22.5M connections/sec and up to two banks of 64MB of RAM.
Nestor ISA, PCI and VME Cards: Description: Nestor now has several cards with the Ni1000 chip:
Rapid Imaging VME Ultima and 0491E1-ISA Cards * No longer available: Description: VME Ultima is a general purpose VME card based on the Intel ETANN chip. Up to 128 inputs, on board learning, analog-in, analog-out real-time processing.
Sundance SMT306 Neural Processing 'C40 TIM: Description: "The SMT306 is industry standard size 2 TIM-40 module conforming to the latest specifications for TIM modules. By using two of the world Neural Instruction Set Processors, NiSPs, in conjunction with the TMS320C40 ('C40) parallel DSP, a balance of computational performance and data transfer bandwidth is achieved.
Telebyte Model 1000 NeuroEngine ?: Description: "The Model 1000 NeuroEngine is a coprocessing board which plugs into a PC and brings high speed neural network processing to the desktop. The board is based upon unique parallel processing concepts developed by AAC. The unit is a multiple instruction multiple data (MIND) stream coprocessor for the PC. The unique MIND parallel processoing architecture takes advantage of sparse topology to obtain high computational efficiency. This single card fits into a PC and operates under DOS. The board is capable of running 140 million connections/s.", Electronic Product News, July/August 1994, Vol.23-No.7/8, p.20.
Vision Harvest NeuroSimulator?: Description: ISA-bus coprocessor board using Intel i860 RISC chip. Runs under NeuroVision software and achieves up to 30M connections/sec.
Ward Systems NeuralBoard?: Description: ISA-bus using 25Mega-flop 50MHZ RISC processor. Under their NeuroWindows NNW software, multiple networks can run on one board and as many as 10 Neuroboards can be installed on one PC.

Bellcore CLNN learning chip: Description: A group at Bellcore has developed a prototype chip with 32 fully interconnected neurons that uses Boltzmann Machine and Mean Field learning algorithms on-chip. The inputs and outputs are analog, while the weights are digital 5-bit. A separate synapse only chip allows for building of multi-chip systems. They have built a workstation co-processor board with 4 chips consisting of 128 neurons and 2000 synapses.
Bellcore cascadable VME Card with 3 ETANN's: Description: A prototype VME card that holds 3 ETANN chips, one per layer of a feed-forward network. Multiple boards can be cascaded together to build very wide structure networks.
Hughes: Description: The M1718 Digital Neural Network from Hughes Semiconductor Products Center is a static CMOS neural network subsystem. It is configured as 1024 (1056) weights supporting 32 8-bit inputs which are fully connected to 32 internal nodes. Each weight is 4 bits wide and multiple devices can be combined to expand the network with respect to inputs and layers. It is claimed that 60,000 patterns per second can be processed independent of network size. Two chips can form
Motorola: Description: A chip was in development but was canceled. Motorola is now in a collaboration with Adaptive Solutions..
NeuroClassifier: Description: A neural net chip aimed especially at HEP applications. The processing through 2 layers is 20nsecs! The chip has 70 analog inputs, 6 hidden and 1 output neuron. The weights are 5-bit digital. All 426 weights can be set in 5ms. The hidden layer neurons have sigmoid transfer functions with user setable gains while the output neuron is a linear function of the weighted hidden neuron outputs. So multiple chips can be ganged together. An external threshold discriminator on the output would be needed to give a binary classification.
Ricoh RN-200: Description: After the presentation of the RN-100 (12 MHz) single neuron chip at Seattle 1991 (Eguci 1991) Richo developed a multi-neuron chip called RN-200 with 16 neurons and 16 synapses for each neuron in fully digital technique. The chip has on-chip learning abilility using a proprietary backprop algorithm. It comes in a 257-pin PGA encapsulation and develops 3.0 W (max).
Univ. of Paderborn Pulse Coded Neural Network Accelerator: Description: To accelerate the performance of Pulse Coded NN (PCNN) simulations, a 5 module system was developed by the Computer Vision Lab of the Dept. of Electrical Engineering at the Univ. of Paderborn, Germany. (The PCNN is more closely related to biological neural systems than typical ANN models and is becoming increasingly popular, especially for artificial vision applications.) The modules reside in a box attached to a host Unix machine. The system consists of communication, initialization, connection, neuron, and learning modules. The digital system was implemented primarily with FPGA's. Hebbian style learning is implemented in the hardware. A single system can hold up to 128k neurons and up to 16M synapses.
Univ. Roma Circuit Labs Cellular Neural Network Chips: Description: The 3x3DPCNN and 6x6DPCNN chips implement the Chua's model of Cellular Neural Network (IEEE Trans. on CAS, Oct. 88) with 9 and 36 cells respectively. Their main feature are:
Other

Bologna Univ. ETANN and MA-16 VME Cards: Description: For the WA-92 Neural Network Demonstration experiment, 2 neural network VME cards were developed: one with an ETANN and one with a MA-16 chip. The MA-16 card was not finished it time for the data taking run but was later finished for benchtesting.
CDF/Michigan ETANN Fastbus Cards: Description: Cards built to implement the CDF neural network triggers. A card holds a single ETANN and can do 2-layer feed-forward processing. ETANN control voltages are also externally setable. The boards are fed via the front panel with 50 analog calorimeter tower signals.
CP-LEAR/Basel ECL Cards and DSP NNW's: Description: A NNW card implemented with descrete ECL components can find tracks in the CP-LEAR detector within 75nsec. Each card receives hitmap information from 5 sectors of the detector spanning 11 layers for a total of 5x11=55 binary inputs. A 55-2-1 multilayer network was trained (off-line) by BP to indicate the presence of a track (out=1) or not (out=0). To To achieve high speeds, both the Input x Wt multiplication and transfer functions are carried out by lookup tables in RAM. The VME cards recieve input from overlapping sectors. 18 cards, covering a quarter of the detector, have been built and tested.
H. Haggerty discrete component network.: Description: For some applications, the weights may never need to be retrained after the patterns are learned. In that case, the NNW circuitry could use fixed weights and avoid the complexity of programmable synapses. Here a test network using analog inputs, fixed resisters for weighting and differential amplifiers, with sigmoidal type transfer functions, was built. The network found track parameters from signals coming directly from a D0 muon chamber. Such networks could be very fast and fairly inexpensive.
RIT ETANN VME Cards: Description: A VME card for the ETANN was built using an ELTEC prototyping card with a 68070 cpu on board. Sets of 64 DAC channels and 64 ADC channels, under the control of the 68070, were added to provide analog inputs and to digitize the outputs of the ETANN. The ETANN sat in a separate piggyback card. The ETANN could both receive inputs from the DAC's, using values passed via the VME bus, or directly in the front panel. Similarly, the outputs could be digitized by the ADC's or passed out a front connector.
RIT ZISC PC/ISA and VME Cards: Description: A PC/ISA Card with 2 of the IBM ZISC chips has been built. A VME card with 4 ZISC's is currently being debugged. The PC card is intended primarily as a way to become familiar with the chips. The ZISC chips themselves are attached to small "piggyback" cards. The ZISC's are electrically very easy to cascade so these piggy-back cards were designed to be stacked to increase the number of available prototype neurons.
Other Go to top of page

Page statistics since May 1996.

NNW in HEP Home Page

Authors: Clark S. Lindsey , Bruce Denby , & Thomas Lindblad

Curator: Clark S. Lindsey (lindsey@particle.kth.se)

Neural Network Hardware

Commercially available NNW Chips and/or Systems

NNW PC accelerators and other cards

Non-Commercial or prototype NNW Chips and/or Systems

HEP/NNW Hardware