Why I'm working on neuromorphic vision chips

The industry way of doing chips

There is a big difference between the way the silicon industry implements computation and the way brains do it. Logic synthesis tools have made it no big deal, technically, to produce chips with millions of transisitors, as long as you have lots of money, are willing to stick to the paradigm of synchronous logic and you have a "big house" behind you to support you with the latest tools and libraries. That means that companies, either established enterprises or startups, are usually conservative in their chosen technology. "Why risk in in a technology that hasn't been shown to work, when we know we can use some ADCs, some DACs, some PLLs and a bunch of synthesized logic to do the job for sure?" Because of the trillion dollars that have been spent on synchronous logic, alternative approaches to computation have a very difficult time making any inroads.

Opportunities

However, despite this seeming dominance of logic, there are excellent opportunities for creative and interesting work on alternatives to this approach for many applications, in particular, in areas like low power chips that process real-world (natural) information--for example, in vision chips. Much of the inspiration for these designs, their organizing principles, and the way we think about them comes from studies of real nervous systems.

Academic successes

People working in this field built many neuromorphic chips that emulated properties of the visual system: for example, my chips for automatic focusing, imagers that compute temporal derivatives of the image, visual motion, and stereo matching. I built the first analog VLSI autofocusing chip based on biological accommodation principles, one of the first silicon retinas with a full set of velocity tuned pixels, and together with Misha Mahowald, the first analog VLSI cooperative stereo matching circuit. Carver Mead and I jointly invented a widely used adaptive photoreceptor circuit, and I invented the bump circuit, which is commonly used to measure similarity or dissimilarity of signals. I have had wonderful time studying how to make good silicon photoreceptors.

These chips are successful in capturing principles of biological vision into silicon and they show many interesting properties that suggest alternatives to the frame-grabber/DSP approach to machine vision, but they have limited practical application. We simply weren't trying to apply this technology, so we weren't constrained enough, by real (practical) applications. I became a bit frustrated with the diffuculty of making something really useful from this approach.

How industry makes chips that work

The reason for this frustration became evident after I worked in silicon valley. It was obvious what was going on--the chips we were making in the lab were much more advanced and ambitious in their intended abilities than the ones that industry was trying to get to market, and we were chasing a much fuzzier intellectual target instead of practical applications. The target was still worthy--don't get me wrong. But intellectual results don't always move to the real world.

Look at the CMOS imager business, for an example. 15 years ago, when we started working on silicon retinas at Caltech, practically no one had even heard of CMOS imagers. CCDs ruled imaging! Here we were, trying to build really advanced, information processing retina chips, while guys like Eric Fossum were simply trying to get images from CMOS technology. I remember thinking (naively) how simple minded these guys were--why work on something as simple as imaging? That's not vision, that's just grabbing snapshots! After I worked on imaging myself at Synaptics and then Foveon, I saw how interesting imaging actually is, and how difficult it is to really get something out to market, and how much it costs, how stringent are the standards when fighting against an enthrenched technology, and how so much depends on factors outside of the technical work itself. Now after 15 years of hard work by a thousand people around the world and probably close to a billion dollars in investment, CMOS imagers are taking over. You see them everywhere, on your computers, in your cellphones, hanging on your neck. But look how hard it was to get from lab samples to products. People were getting decent images from CMOS imagers in the lab nearly 10 years before products actually hit the shelf. And this is just imaging.

Being realistic

Since then, I have changed my view on on what I think is feasible given the entrenched technology for analog and mixed signal chip design. I want to make something out of this neuromorphic approach, but to do that, we have got to prove that we can make something really functional and useful. Industry won't do it. They just don't know how. It's hard enough to find someone how can make you a decent PLL or ADC. Forget about someone who can build you an entirely new system chip using some novel organizing principles and circuits!

I believe that building practical devices that compute like brains compute—and showing that they can be made to really work—is a viable way to start a viable separate line of evolution in computation that is based around principles that are closer to neural wetware than synchronous logic. Realizing fully functional practical devices that are based on analog, collective, continuous-time computation will lead to real progress in this technology, and significant benefit to neuroscience in influencing how neuroscientists think about neural computation.

I am now primarily interested in making devices for specific practical applications that can be demonstrated to work well enough that if a market existed, they could actually be sold. We have the technology in hand to build vision chips with applicability to practical real-world problems whose operational principles are not too different from those used by the nervous system. For examples of these, see my projects. My credos for these designs are KISS (keep it simple, stupid!), develop for practical applications, and productize as much as possible. Learning what it takes to make a few simple neuromorphic vision chips really work well is more productive in the long run than developing many poorly functioning prototypes.