The Future is Heterogenous Systems

Read a very good article on Communications of ACM's August 2011 issue. It is titled Computing Without Processors by Satnam Singh of Microsoft Research.

What is the future? Is it multi core CPUs or alternative computing elements such as general purpose GPU or FPGAs?

1. Multi-core processors: Good for large coarse grained tasks
2. GPUs: Suitable for thousands of fine-grained data-parallel computations
3. FPGAs: Good for Irregular fine-grained tasks with extreme requirements for performance or low energy consumption.

The future is heterogeneous system, which is a combination of all these. The possible deployment of such systems is in the cloud.

Parallel Processing Research Landscape

Currently three top US universities are giving a big push into parallel processing research. Intel and Microsoft are funding this research with more than US $20 million. AMD, HP, NVIDIA  etc are funding about $US 6 million. This is real big money that shows how companies are serious about the problem in parallel processing in multi-core processors.

In 2007, Intel and Microsoft  announced that they would fund a Universal Parallel Computing Research Center (UPCRC). As part of that, two universities were chosen in March 2008 - University of California at Berkeley and University of Illinois at Urbana Champaign. It is a five year long project.

With $10 million for its UPCRC, UCB started ParLab, a multidisciplinary research project exploring the future of parallel processing. ParLab is headed by Prof Kurt Keutzer of EECS Department. (Prof Kurt Keutzer was CTO of Synopsys before he joined UCB in 1998.) Before ParLab was started, UCB published a very good technical report in 2006 titled The Landscape of Parallel Computing Research: A View from Berkeley. I think this report was crucial in UCB getting selected as one of the UPCRC centers. Based on this technical report, the same team then published their research agenda titled The Parallel Computing Laboratory at U.C. Berkeley: A Research Agenda Based on the Berkeley View. This research agenda lists out 13 different project that ParLab would undertake. The ParLab team published their parallel computing views in the Communications of ACM's October 2009 issue. Last month (Feb 2010) ParLab gave an update on their work. They have created a new parallel methodology called Pallas and they were developing a more general parallel framework called Copperhead. Copperhead is being jointly developed with Nvidia. It will work in both Nvidia's CUDA architecture and OpenCL environment. (We'll discuss about CUDA later). ParLab's website didn't reveal much about Pallas and Copperhead.

UIUC also started  Universal Parallel Computing Research Center (UPCRC) with $10 million fund from Intel and Microsoft. They published a whitepaper representing the vision and research agenda of the UPCRC. They're working on multiple projects on Applications, Architecture, Programming and Patterns. Yesterday (March 19th, 2010) UIUC researchers gave an update on their work. They held a UPCRC 2010 Illinois Summit to discuss their ongoing work. They believe that the current shared memory model is fundamentally broken with large number of cores. They revealed about project DeNevo where they defined a new disciplined way of using shared memory. Another interesting project they are working on is defining a determinstic parallel language based on Java and C++. They believe that current techniques of threads and locks will not scale to processors with a dozen or more cores.

Stanford University also started a parallel programming effort named Pervasive Parallelism Laboratory in April 2008. They got US$ 6 million in funding from AMD, HP, IBM, Intel, Nvidia and Sun Microsystem. It is headed by Prof Kunle Olukotun. Prof Kunle Olukotun is considered as the father of Sun's multicore Niagara processor. They are also working on three areas:  Applications, Programming & Software Systems, and Architecture.

These three universities have published a survey paper titled  Ubiquitous Parallel Computing from Berkeley, Illinois and Stanford published in IEEE Micro, March/April issue of 2010.

The parallel processing landscape is very interesting. Many people are doing research to come out with a new parallel programming model to tap into next generation multicore processors.  We will keep track of the research happening in this area.

Why a dedicated blog on parallel processing?

I've just created a new blog titled Multicore-Musings to capture my thoughts from the world of Parallel Processing. I already have a blog titled beberibang. Then why did I choose to create a separate blog on Parallel Processing? There are many reasons behind it. Let me state some of these below:

1. I feel that parallel processing is one of the most important technologies in coming years. Parallel processing has been there since long. But it has become more relevant in recent years because all the microprocessor companies nowadays design and develop processors that by default come with multiple cores. Earlier multi-processors were a luxury; now it's a household item. So why write or use applications that can use only one core?
2. Although multi-core is a hot topic, my experience with many people tells me that it's grossly misunderstood. Not just common men, many techies too confuse between terminologies like multiple-cores, multiple processors, threads, processes etc. I'd like to write in simple languages explaining the technologies behind all these.
3. Parallel processing is an area of immense interest to me. But I don't work directly in this area. One of my goals for creating this blog is to keep my interest in parallel processing alive. I'd like to learn new things in this exciting world of parallel processing and share my knowledge through this blog. (I remember a poem that I read in my school days that says that if you share your knowledge, you become more knowledgeable. It's just opposite to money. If you share your money, you become poorer).
4. To keep a second option in the job market apart from my core field :-)

So join me in this exciting journey to the world of Parallel Processing. We'll explore hardware architecture, APIs, programming language support, parallel algorithms etc through this blog.