The Switch Is On

Changing the Way Data Moves

Story by Al Riske. Photography by Howard Friedenberg.

18.May.07-Hans Eberle doesn't like inflated statements.

He just happens to be working on a new switch that's far simpler and far more efficient than anything available today -- a switch that, one might say, promises to revolutionize the way datacenters operate.

Eberle, who leads the cross-functional team developing and prototyping the switch, simply calls it "a next-generation datacenter switch."

The project started two years ago when his boss asked him to look into building a switch using Proximity Communication, a novel technology under development in Sun Labs.

"His goal was a 10-terabit-per-second switch," Eberle recalls. "To some extent he asked the wrong person, because I'm very grounded and would never promise anything I can't deliver. He kept on talking about the 10-terabit switch and I kept coming back and saying, 'You know, within about two years, what I can build you is a small-scale, four-port switch and the bandwidth will just not be interesting.'"

What was interesting to Eberle was Proximity Communication, which provided the ability to transfer data between chips without wires, at astonishing speeds.

"As a systems person -- and that's how I describe myself -- I'm always looking out for enabling technologies. Proximity Communication is an enabling technology that lets you rethink how you do things," says the soft-spoken engineer.

"If you see one of these new technologies, you don't just try to do things the old way. You have to look at it and try to recognize what the opportunities are."

As with any new technology, however, there would be a number of technical hurdles the team would have to clear, so Eberle decided to start small.

"Actually you wouldn't even build at this scale using this kind of technology, but this was the first time we attempted to build a larger system using Proximity Communication and performance just couldn't be the goal," he says.

So the first mechanical sample is exactly what Eberle promised: a small-scale, four-port switch with underwhelming bandwidth.

What's impressive about the new design is that it proves a better way is now possible.

And it couldn't come at a better time.

In today's datacenters, which often house thousands and thousands of computers, interconnects play an increasingly important role.

Take virtualization, for example. Very popular with datacenter operators, virtualization is basically a layer of abstraction that lets them think in terms of pooling resources -- compute, memory, storage -- and allocating them wherever they're needed most at the moment. Doesn't matter where the resources are physically located. They're all part of the pool.

"But if you add a layer of abstraction, you need some additional bandwidth to make that possible. If your memory suddenly resides on another node, you have to transfer the data," Eberle points out.

"We are so compute-centric. It's always about processors and processor architectures. But I think if you look at the whole system, which you need to do, then moving the data around is probably these days much more critical than actually doing the computation."

Which is where the new switch design comes into play.

With the help of Robert Drost and the VLSI group in Sun Labs, plus folks from Sun's Physical Science Center in San Diego, California (not to mention his own small team), Eberle made good on a detailed two-year plan and now holds in his hands a mechanical sample.

Instead of using a complex, multi-stage, hierarchical design where you actually have to schedule the path through the various switching elements, requiring a fantastic degree of coordination, the new design is a simple, single-stage switch.

"With Proximity Communication we're getting something like two orders of magnitude more I/O bandwidth. What that means is we don't have to make use of hierarchical topologies. We basically can look at this as a flat switch," Eberle says.

Simply put, the new design is cleaner, easier to build, easier to manage, and way faster.

"It also allows us to give some guarantees. If you think about traffic these days, you often have some real-time constraints. Streaming video, for example. So you want to be able to forward data with some bandwidth guarantees. It's much easier to provide that kind of service level in a switch like this than in a multi-stage switch," Eberle says.

"Today, the biggest [single-stage] switch has about 24 ports. We can easily do a 256-port switch and go up to 1000 ports."

This from a man who doesn't like inflated statements and would never promise anything he can't deliver.