The Art of Forecasting

How Data, Diplomacy, and Determination Add Up

By Al Riske

01.May.07-Phillip Yelland remains hopeful.

Over the past seven years, he hasn't been able to solve the problem of demand forecasting, but he has helped the company make incredible strides.

A researcher in Sun Labs, Yelland runs the Management Science Project, where he combines insights into math and human nature and tries to bring a semblance of order to the often chaotic world of business.

"Forecasting is usually a mob scene, a combination of sales and marketing and various other groups who all chip in their two cents," he says. "By statistically processing these forecasts (some of which turn out to be somebody's stretch goal), we've found that we can improve accuracy by 40 to 60 percent."

As a result Sun has been able to save a boatload of cash.

"It doesn't take much to save quite a lot of money in inventory," Yelland says. "It's hard to put a precise number on, but we're talking about millions of dollars."

Yelland, of course, would like to be precise. But with just two people and the occasional intern, his team lacks the resources to follow things all the way down the supply chain to see exactly what's happening.

They don't lack data in other areas, however.

"We get statistics from various data warehouses around the company. We are drowning in data," Yelland says. "Every time a box goes out of the factory, it's scanned by a bar-code reader. Every time somebody downloads software or buys a product online, all the clickstreams are captured. Every time somebody makes a warranty call, that goes into a database. Our machines phone home on a regular basis, so we have terabytes of data that come out of the field, petabytes by now."

Data isn't the problem. Nor is compute power.

"It's quite an exciting time to be doing this sort of stuff because so much more, in just the past 10 years, has become possible and practical. I mean, my daughter, who's two, owns more computing power than I had before I was 30. Every time I step on one of her toys it strikes up a conversation with me," Yelland says. "So, what were hitherto completely impractical approaches in terms of mathematical tractability -- very complicated models and so and so forth -- you can do that now almost on a routine basis."

He notes that technical considerations -- although challenging and never obvious -- proved to be the least of his concerns over the past few years. The biggest obstacle? People.

"There's this notion in economics where you have a simple set of objectives and you capture that in a set of equations. Then you just turn the knobs and you can predict what a person is going to do. That's never the case," Yelland says, by way of example. "People have competing objectives, they have ulterior motives, they have limited understanding, they have creative insights ... they're so much more variable."

Born in Manchester, England, Yelland remembers when the UK tried to implement the economic models of Nobel Prize-winner Milton Friedman in the 1980s.

"When you try to apply these elegant mathematical models to what's actually going on, you find that people are very clever. They find no end of ways to elude proper modeling -- to defy your descriptions, to get around being captured neatly in an equation."

Nobody, Yelland notes, wants to be evaluated on the accuracy of their forecasts.

"What you'll find is that nobody ever measures forecast accuracy. Nobody, in fact, ever produces a 'forecast.' They will tell you they are producing supply-positioning plans or any one of a number of different terms of art that get around the fact that they could be held accountable to a single number," he says.

"The truth is, this is really difficult, so producing a bad forecast is no mark of shame. But for a person who's got kids, who's got a mortgage, and so on and so forth, it can be a kind of hidden threat if you come in and say, 'Well, I can improve what you're doing, using this computer.' It can be a real issue."

Fortunately, Yelland was a student of both math and business and learned the fine art of politics along the way.

"We had to convince people we were not a threat, that we weren't aiming to automate them out of business or supplant what they did, that their input was very important and this was just a way of augmenting their capabilities," he says.

"That was a long process. It took about three years."

Yelland describes himself as a mathematician and computer scientist who went off and got an MBA.

"I was interested in seeing how one could apply mathematical techniques to business problems, particularly forecasting, which is an obvious one. It's very clear that it would be an advantage to a company," he says.

However, he notes, no organization is likely to bring up forecasting on its own.

"If you come in and say, 'I can solve your problems,' people will say, 'We don't have a problem,' because that would mean they weren't doing their jobs," Yelland says. "I was told by an old hand that as a consultant you never solve a problem. You happen to be there when the client solves a problem."

He credits his success to the fact that nobody told him to stop.

"Sun Labs is a very conducive environment. They gave me the benefit of the doubt long enough that I could prove that this was a valuable thing to do and a viable approach to take," he says.

"Once you look at a problem and try to think about it in a systematic fashion, you can usually do something to improve it if not actually solve it. So there's always hope."