Breakthroughs in Management Science

December 15, 2006 - Product managers have pressing questions and they need accurate answers. "How do I predict sales trends for a new product?" "What's the likely uptake in various industries or vertical markets?" "Is my core customer base undergoing a significant change?" "Is the new product expanding our customer base or simply cannibalizing our older products?"

Management science brings a structured approach to answering critical business questions. Also referred to as operations research, quantitative methodologies, and decision support, management science seeks to replace crystal balls, guesswork, and assumptions with hard data and rigorous analysis. To traditionalists who believe that management science is an oxymoron — that product management will always be an art form that combines intuition and a sixth sense about customer buying patterns —Sun Labs researchers Phillip Yelland and Renee Stratulate politely respond, "think again."

"The explosion of data coursing through our corporate networks, combined with inexpensive disk and sophisticated database technologies, gives us the opportunity to find smarter answers to business questions," said Ms. Stratulate, a member of the Sun Labs Operations Research and Management Science project. "What we're doing is applying existing techniques, together with new methodologies that we've developed, to explore the questions and come up with more insightful, meaningful answers."

Clearly, data analysis and forecasting are nothing new in the field of management science; there are plenty of proven quantitative methodologies and statistical analysis tools in use today. Delphi prediction. Conjoint analysis. Judgmental bootstrapping. Structured analogies. Neural nets. Causal models. The list goes on. Three things set the Sun Labs approach apart:

• A focus on exploratory data analysis as opposed to "confirmatory" analysis
• Graphical representation of results for easier interpretation
• Use of sophisticated techniques to ensure that the source data is as accurate and reliable as possible

Exploratory Analysis: Assume Nothing

The projects undertaken by the Sun Labs Operations Research and Management Science team are sponsored by individual product groups within Sun. "Often we're called in to address a specific business question, such as how well a new product can be expected to sell in the telecom market or which sales compensation schemes will be most effective," said Dr. Yelland, who has more than 20 years of experience in IT and has developed advanced econometric modeling tools now being used in Sun's Corporate Supply Management Department. "We sometimes find that our client has already made certain assumptions about the answer. They want someone to confirm that their assumptions are correct; or in the case they're worried a product may not be successful they want someone else to corroborate that their fears are justified.

"So our first job is to get preconceived notions out of their heads and get them to look at this as an exploratory exercise. We've found that once they're open to unexpected results, they're also more open to unconventional approaches to getting meaningful answers."

One such approach employed by Dr. Yelland and his team is a tool that graphically depicts "correspondence analysis," which is a technique used to analyze simple two-way or multi-way tables in which there is some measure of correspondence between the rows and columns.

With graphical software created by Dr. Yelland and his team, the results of correspondence analysis can be displayed in intuitive, easy-to-read plots that clearly show how systems are selling in various industry segments. For example, the tool can highlight the segments in which the new product is underperforming or performing beyond expectations. With this information product managers can quickly spot market problems and prepare further tests to determine whether new features may be needed to bolster sales, or better incentives for sales reps, or more effective marketing campaigns.

Recently, Sun's senior director for volume SPARC servers, Warren Mootrey, engaged Dr. Yelland and Ms. Stratulate to help determine which application spaces Sun's new workgroup servers were occupying. "We had come out with new servers running a new version of our operating system, Solaris 10, and a new technology called Chip Multithreading [CMT]," he said. "So we needed to know which applications our customers were migrating to the new servers so we would know which software vendors we needed to get on board with Solaris 10 first."

According to Mr. Mootrey, the Sun Labs team both confirmed his initial assumptions and took them a step farther. "They improved upon our own research and gave us hard data that made it clear how we could ramp up faster to meet the needs of our customers," he said. "Now instead of approaching the standard list of ISVs I'm going to be able to target the top ISVs from our customers' perspective and offer them incentives to expedite their porting and tuning processes. That's good for our software partners, our customers, and for Sun as well."

Forecasting: The Next Frontier

Sales forecasting has always been a critical element of any business or product plan, but producing accurate forecasts on a consistent basis is a notoriously difficult feat. The results of inaccurate forecasts, unfortunately, are also well-known: stock-outs when product components are unavailable, inventory over-runs when too many parts are laid in.

The Sun Labs Forecasting System — another innovation developed by the Operations Research and Management Science team — aims to improve the quality of the company's forecasts by supplementing the efforts of forecast managers with sophisticated statistical analyses of historical sales trends. The Sun Labs Forecasting System makes extensive use of so-called Bayesian statistical techniques, allowing it to produce forecasts for new products with little or no sales history, and to incorporate judgmental insights into its analyses.

Drawing its data from the company's data warehouses, and delivering its forecast reports weekly using Sun's internal Web, the Forecasting System has been in operation for over two years. Response from Sun's business units has been enthusiastic, and it continues to be consulted by planning staff throughout the company.

Clean Data for Clear Results

The quality of data analysis and forecasting tools is dependent on the quality of the data. Yet all too often, the data available for research purposes is incomplete, inaccessible, or unreliable.

"There are lots of reasons we don't get perfect data for our research," said Dr. Yelland. "Corporations have multiple incompatible systems and data silos; sometimes the data simply isn't collected; sometimes the wrong data is collected; sometimes people get territorial over who has access to their data. Given these realities, we have to do what we can to ensure that the data we have is as accurate and useful as possible."

To that end, the Operations Research and Management Science team is also working on "record linkage," a series of techniques for determining when multiple records are actually referring to the same customer.

"Inconsistencies in data are rife," said Dr. Yelland. "There are keying errors, spelling errors, different naming conventions in different countries, and so on. You can end up with dozens of different ways of referring to the same entity. It's a very common problem but extremely hard to solve. And if you don't solve it, you really can't get to the level of accuracy you'd like in your data analysis and forecasting. And from the customer's perspective, you can't stop a lot of the fraud schemes that are prevalent today."