Project Darkstar: Changing the Game

By radically simplifying online game development and enabling games to scale to thousands of players without a glitch, Sun's Project Darkstar is changing the rules of the game business. Here's a closer look at the inner workings—and jaw-dropping capabilities—of Project Darkstar's core technologies, now freely available for download under open source licensing.

Related information How it works

August 30, 2007 - Developing and deploying online games is not exactly child's play -- particularly if the game needs to scale to tens of thousands of simultaneous users and support players on dozens of different devices. Today you need to be a master of multi-threaded programming, a distributed computing expert, and a transactional wizard all rolled into one. And you'd better have IT expertise and deep pockets for the back-end processing resources that are required.

Project Darkstar is changing all that. By providing a simple programming model and back-end services for building massively scalable online games—and now, by making this technology freely available to the development community through open source—Project Darkstar is making it far easier and less expensive for game developers to create and deploy multi-player online games. Which means that this technology could have huge impact on everyone involved in online gaming—game players, game developers, Sun, and Sun's partners.

Early access versions of the technology are already in the hands of the hands of the Project Darkstar Community, to the tune of more than 19,000 downloads. Project Darkstar is now the focus of a large research project at Sun Labs and it draws on the talents of some of Sun's top scientists and researchers. Simply put, this is big.

Masking the Complexity of Distributed and Multi-threading Programming

The emerging world of multi-core processors, multi-threaded applications, massive parallelism, and virtual environments ought to be great for game players. It means there's plenty of compute power to go along with the network bandwidth to ensure fast, smooth, reliable play. But the new era is tough on game developers. Building a distributed, multithreaded application that can best take advantage of a data center with hundreds or thousands of machines is a complex endeavor. Such programming requires very specialized skills. Even understanding how to test and verify these applications requires deep distributed systems and multi-threading expertise. This in turn can cause delays in time-to-market for new games, and can result in programming errors that cause sluggish game performance or crashes.

Project Darkstar provides a new programming infrastructure for game developers that greatly simplifies coding for multi-threaded environments. With Project Darkstar, the programmer just writes event- handling code; the system does the rest. To the programmer, it appears that the code for the server side of the application is running on a single machine in a single thread—when in fact it is taking full advantage of the inherent parallelism of the Project Darkstar environment. The programmer gets the advantages of multiple threads and multiple machines without having to change from the standard game programming model.

"Games today are like life—they're massively parallel," said Distinguished Engineer Jim Waldo, Project Darkstar Chief Architect. "Games need to take advantage of the parallelism that's available, and that means making it practical for game developers to write and run their games in a multi-threaded environment without having to deal with the complexities of multiple threads."

Multiple Advantages

The core technology behind Project Darkstar holds significant advantages over other technology available to the game industry today, to the benefit of developers and game players alike. Early versions are already demonstrating the value of Project Darkstar's simplified programming model and scalable architecture. And the technology is advancing rapidly toward realization of the platform's full potential, delivering on a vision that enables:

Massive scalability: The distributed architecture of Project Darkstar permits the system to scale in an approximately linear fashion to meet increased demand for compute resources—whether from additional simultaneous users, an increase in the complexity of the application, or both. Either way, load balancing is provided automatically in response to the increased task load. When more capacity is needed, more compute nodes can be added; servers in a compute farm could even be shared across multiple games.

Higher reliability: Even if a server completely fails, that failure should be hidden from the user and not interrupt game play. With this goal in mind, Project Darkstar is designed to enable you to add or remove servers, patch server code, or even roll-back patches all without interrupting players.

Lower development costs: The programming model embodied by Project Darkstar facilitates more rapid development of more robust games. By simplifying programming for the game developer while enabling massive parallelism and transactional integrity on the back end, Project Darkstar-enabled games should be faster, less susceptible to failures, and simpler to maintain and enhance. This translates to shorter development cycles and faster time-to-market.

More efficient use of resources: Project Darkstar's flexible scaling design enables a "shardless" game architecture, meaning that specific hardware need not be assigned to specific regions of game play. Server resources can be dynamically reassigned as the gameplay demands. That means you will not encounter the limitations of sharded architectures, in which a few servers become overloaded while many other servers sit idle. For example, while server utilization rates can be as low as 5 percent with a sharded architecture, the Project Darkstar architecture allows much higher utilization rates. So costs are reduced, capital expenditures can be avoided or delayed, and infrastructure costs can be amortized across multiple stacks of a single game or across multiple different games.

Get Started with Project Darkstar

You can begin using the currently available, single node version of Project Darkstar technology today. Both source and binary code distributions are available under open source license and can be downloaded from the Project Darkstar Community website. The project currently includes the following components:

Project Darkstar Server: the core Project Darkstar technology, currently available in a single-node version, and licensed under the Gnu General Public License, version 2. The Project Darkstar Server download includes documentation and tutorial examples to help developers learn how to write server-side game code.

Project Darkstar Client SDK: client side code for connecting with the Project Darkstar Server, availble under the revised BSD license. This download also includes documentation and tutorial examples to help developers learn how to write Project Darstar game clients.

Why Open Source?

Project Darkstar is just one example of many open source projects at Sun. Sun has open sourced much of its technology, including the Solaris operating system under the OpenSolaris project, the UltraSPARC T1 microprocessor under the OpenSPARC project, and Java technology under the OpenJDK project.

Sun embraces open source because open source is ultimately about choice and higher quality for developers, Sun customers, and users. Sun is committed to open all of its software and participates in countless open source communities. Many of these communities are building cross-platform technologies such as X.org and GNOME, and many, such as OpenSPARC, are seeded by Sun.

Learn about the entire roster of Sun-sponsored open source projects.

Related Links

Podcast: Interview with Jim Waldo, Project Darkstar Chief Architect