The Multicore Advantage

Sun's Most Prolific Inventor Is Causing Problems for Competitors.

Story by Al Riske. Photography by Howard Friedenberg.

6.Sept.05-Marc Tremblay's job is to see the future and, as if that weren't tough enough, get there first.

"All of us who claim we can see 10 years ahead are fooling ourselves," he says - and yet, in 1995, he saw where microprocessor design had to go.

Tremblay, now chief architect of Sun's Scalable Systems Group, is the father of the multicore, multithread Throughput Computing paradigm, and he's seeing his vision materialize in a big way.

"We've been talking to analysts for two or three years about multithreading and multicore. They all were somewhat skeptical in the beginning. Now that Intel and AMD have announced they're doing multicores, they're saying, 'Oh, Sun must be right.' Until the competition does it, it's not clearly the winning path. But that's okay," he says.

It's okay because Sun is still way ahead of the competition with next-generation systems based on the eight-core Niagara processor already in the hands of early access customers in the United States, Europe, and Asia.

Each of Niagara's eight cores can process four software threads, or tasks, simultaneously - all while using no more energy than the average lightbulb (as little as 10 percent of what other processors use).

"What really was the clincher was when we started releasing data from a customer, yet to be named, but a large e-commerce site in Silicon Valley, that has been comparing thousands of Xeon-based machines to a much smaller number of Niagara-based servers," Tremblay says.

"The phenomenal aspect of it is, not only are the Niagara-based servers a lot cheaper just in terms of acquisition costs, but it's the power they save in the datacenter that is mind-boggling. The Gartners of the world that were somewhat interested but not convinced are now changing the way they look at total cost of ownership."

When will Niagara-based systems be widely available? The plan of record is the beginning of 2006, but Tremblay notes that the engineering and operations teams are working hard to beat that.

Meanwhile, the rest of the industry is left to scrambles.

"Now it's a no-brainer -- multicore, multithread is the only way to go -- so everyone is taking their existing core and putting a copy right next to it and either putting those two in the same package or creating a new die with those two. Intel has even gone to the extent of just putting them side by side but they don't really communicate on the die; they communicate on the package," Tremblay says.

Or, simply put, slap-dash efforts to get into the multicore game.

"In the meantime, what I keep telling people is: To truly exploit the value of multicore and multithread, you have to start from scratch. That means a brand new core, a brand new pipeline, and that takes easily four years," Tremblay adds.

"So, fortunately, we started four years ago, through a variety of projects, and we'll see the fruits of that for the next several years, while you're not going to see anything new from the competition until probably the 45-nanometer generation, which is probably another two years away. That gives us a huge window of opportunity."

(A nanometer is one-billionth of a meter. Processors built with a 45-nanometer process will have circuit lines as small as 45 billionths of a meter.)

"In the 90-nanometer generation, which is what people are launching now, you're going to see two cores from the competition versus eight cores from Sun. You're going to see a maximum of four threads from the competition versus thirty-two threads from Sun."

Even at 8 cores and 32 threads per chip, Niagara delivers lower power consumption than anything the competition has to offer. "It truly shows what can be accomplished if you start from scratch," Tremblay says.

Tremblay's work on multicores and multithreading laid the foundation for several projects within Sun's Scalable Systems Group, including Niagara. His "baby," though, is a high-end processor code-named Rock (due out in systems in early 2008). It includes innovations from the past 10 years of his career.

"Marc has led the expeditionary forces into an area of high-performance throughput computing that is really going to cause problems for our competitors," says Sun CTO Greg Papadopoulos, who describes Tremblay as one of the great processor architects of our era.

"He's been consistently out in front of the next wave of architecture."

With 100 patents approved and 60 more pending, Tremblay is Sun's most prolific inventor. He joined the company as a Ph.D. fresh out of UCLA and immediately went to work on a brand new chip architecture - UltraSPARC I.

In 1995, he and Sun cofounder Bill Joy invented MAJC, the Microprocessor Architecture for Java Computing, and the dual-core concept was born.

At the time, they were looking at applications such as speech synthesis, voice recognition, videoconferencing, and 3-D user interfaces - "all the cool stuff that would make your life easier on a desktop" - and noticed a lot of parallelism.

That is to say, tasks that could be processed simultaneously. The upper and lower halves of a videoconference display, for example.

"So just looking at these apps we found out that, boy, it would be a lot better if we had two processors instead of one, on the same chip. There was the birth of the first dual core in the industry, in 1995, which we shipped in 2000. Now in 2005, everybody is talking about that. So another five years later, but 10 years after we started. It just takes a long time," Tremblay says.

MAJC was not the success he had hoped for -- "the mistake we made is that we targeted the desktop" -- but everything learned from MAJC has migrated into other projects.

And, along the way, the concept of multithreading was thrown into the mix.

"In chip multithreading, each core can do multiple things simultaneously. It's the concept of pipelining. If you ask for something that's not in the local memory - it's off chip either in DRAM or disk - the the processor would be idle for hundreds or thousands of cycles, waiting for that data. So instead of leaving the processor idle, we can switch to another search in zero time," Tremblay explains.

"The processor we will be shipping soon can do that four times - run tasks, not concurrently but overlapping in time. That's very powerful because you're using idle resources. Basically, you can have four times as many searches done in the same amount of time for, let's say, 15 percent more area on the chip. The tradeoff there is amazing."

"Predicting needs and then mapping that all the way down to transistors - that's my favorite thing. You know, talking to customers who say, 'We would really need x, y, and z.' Boy, I bet we could invent some new SPARC instructions that would really accelerate this, or some new micro-architecture techniques that would really accelerate that," Tremblay says.

He recalls a meeting two years ago with Nobel prize-winner Lee Hartwell:

"He's been working on cancer since Nixon announced the war on cancer, which was about 30 years ago, and we've basically made zero progress statistically. I said, 'Okay, let's look at your computer requirements.'"

With the Rock processor he is currently working on, Tremblay found a way to speed Hartwell's simulations by a factor of 50. In other words, what used to take a year will take a week. The two talked further and Tremblay found another way to boost performance, this time by a factor of just five, but enough to bring simulation time down to about a day.

His biggest obstacle, he says, is time.

"I have so many ideas, and then it takes so long to implement them. It takes four years basically from concept to revenue release. And it's like, boy, how can we accelerate this? Some of it is like having nine women trying to deliver a child in one month. It doesn't work. Some of it does, though, and it's a matter of percolating it through the team."

The good news: Customers won't have to wait much longer.